Nethserver-awstats needs testers


(Stéphane de Labrusse) #1

I did a new module, I need feedbacks. it is a module for awstats to analyse web access from your virtualhosts (nethserver-virtualhosts). Of course I need real statistics.

For now only NS7

yum install

Then either configure a virtualhost with a wordpress or whatever web application you want, it should be workable out of the box for previous virtualhosts.

you will find your awstats configurations in /etc/awstats/awstats.*.conf For now it is minimalist, we could sort some options more, I wait your Ideas

for example we could ignore statistics from lan

you must wait some minutes before to get statististics (default is 5 minutes), you have a web panel in nethgui to modify the settings and browse the domain statistics.

The web statistics pages are protected for the admin user (default), think to set a password. You have a panel to put more users

(Alessio Fattorini) #2

Ehi please help this man test his new born :baby: @quality_team do you have some time?
If I remember correcty @GG_jr was interested once

(Giacomo Sanchietti) #3

Excellent! I’m planning to upgrade from NS 6 to NS 7.

Great chance to try awstats :wink:

(Stéphane de Labrusse) #4

thank for your interests, you will be surprised by the download statistics. I host a archlinux repository and it is about 5TB (terabyte) for the worse month.

I want to finish this development and I hope to get time to sort a ns6 version.

(Gabriel GHEORGHIU) #5

I’m still interested about AWStats!

And yes, I’m happy because @stephdl did this for us!

I have installed awstats by sdl a few minutes after Stéphane make the announcement.
I have a test server on NS7 with WebTop5, SOGo, Nextcloud and OSTicket installed on it and I will be glad to see statistics for all, if is possible.
Unfortunately, I was very busy and I had no time to “play” with awstats by sdl.

Question: is/will be possible to have statistics also for other web sites/apps which are not hosted on the server on which awstats is installed?

Again, thank you Stéphane! As usually, very good job!

Kind regards,

(Stéphane de Labrusse) #6

Short answer, awstats reads local logs, so if remote logs are copied by scp locally why not.

Concerning other webapp i would develop something based on db property, but for now it is only relied on nethserver-virtualhost

(Stéphane de Labrusse) #7

With the option we could create statistics, but with the many possibilities of name/software/alias/virtualhost, maybe we could to think something flexible by properties (with a text area in the panel).

With something like that we will create an awstats configuration easily.

Create separate config files for each of the selections

OnlyFiles "^/mylottosite"
OnlyFiles "^/mypersonalsite"
OnlyFiles "^/myetcsite"

(Stéphane de Labrusse) #8

testing myself my work, I found that awstats has an issue to sort the different domain name, Indeed all domains are logged to tthe same logs.

@davidep what about if we do one custom log per virtualhost, of course with a log rotation.

I meant about this

(Davide Principi) #9

Seems reasonable…

However the official Apache docs explain why a single log file is good and how to split it by host:

IIRC the default CentOS config uses the vhost log format.

(Stéphane de Labrusse) #10

I must inspect, probably if i just do some minor change in the way how to write the log, probably i might solve my issue. With sme server the domain name starts first for each line and awstats works well for the ibays.

It is just a matter to say what to log, thanks for the pointers.

I’m away for three days, I will look for it later

(Stéphane de Labrusse) #11

I speak something like that

[root@smemirror ~]# tailf /var/log/httpd/access_log - - [05/Jun/2017:22:52:58 +0200] "GET /staging/os/i686/redland-storage-virtuoso-1:1.0.17-5-i686.pkg.tar.xz.sig HTTP/1.1" 404 269 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +" - - [05/Jun/2017:22:52:58 +0200] "GET /NethServer/7/repodata/repomd.xml HTTP/1.1" 200 3001 "-" "urlgrabber/3.10 yum/3.4.3" - - [05/Jun/2017:22:52:59 +0200] "GET /NethServer/7/repodata/7b7cea297883dc8c1ff5029307951a3d1cd1f99e5f190715157f2b5a783b1978-primary.sqlite.bz2 HTTP/1.1" 200 122908 "-" "urlgrabber/3.10 yum/3.4.3" - - [05/Jun/2017:22:53:01 +0200] "GET /staging/os/i686/redland-storage-mysql-1:1.0.17-5-i686.pkg.tar.xz.sig HTTP/1.1" 404 266 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +" - - [05/Jun/2017:22:53:03 +0200] "GET /releases/9.2/smeos/i386/SME/xorg-x11-drv-mach64-6.9.5-1.el6.i686.rpm HTTP/1.1" 200 71664 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +" - - [05/Jun/2017:22:53:04 +0200] "GET /staging/os/i686/qt5-quickcontrols-5.9.0-1-i686.pkg.tar.xz.sig HTTP/1.1" 404 259 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +" - - [05/Jun/2017:22:53:04 +0200] "GET /core/os/x86_64/core.db HTTP/1.1" 200 127082 "-" "curl/7.53.1" - - [05/Jun/2017:22:53:07 +0200] "GET /multilib/os/x86_64/lib32-glibc-2.25-3-x86_64.pkg.tar.xz.sig HTTP/1.1" 200 310 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +"

(Davide Principi) #12

It’s false. We should define a custom log format with %v as first column and apply it to virtual hosts.

I’d like a unique additional log for all virtual hosts and %v.

I wouldn’t go with a global modification: it breaks the current log format.

Completing the module AWSTATS
(Stéphane de Labrusse) #13

Well awstats works with a combined log format, for what I saw it doesn’t work well for all the vhosts in one log.

I don’t know why.

For now I created a custom access log for each vhost, it works well, the error log is not changed.

(Stéphane de Labrusse) #14

yum install

new panel with awstats settings
access logs are specific per virtualhosts

it should be good, I hope

(Gabriel GHEORGHIU) #15


Anyway, besides this:

we need a Legend!

Or, some clues near/below/… every “setting frame”.

(Stéphane de Labrusse) #16

new version, cosmetic changes in the code, thank skitoo

yum install

(Stéphane de Labrusse) #17

Released in my repository.

the dnslookup and the GeoIP are available in the panel
elementList for the templates.

(Stéphane de Labrusse) #18

4 posts were split to a new topic: Awstats: access log of virtualhost is empty

(Stéphane de Labrusse) #19