Nethserver-awstats needs testers

I did a new module, I need feedbacks. it is a module for awstats to analyse web access from your virtualhosts (nethserver-virtualhosts). Of course I need real statistics.

For now only NS7

yum install http://mirror.de-labrusse.fr/NethDev/nethserver-awstats/nethserver-awstats-0.1.0-1.ns7.sdl.noarch.rpm

Then either configure a virtualhost with a wordpress or whatever web application you want, it should be workable out of the box for previous virtualhosts.

you will find your awstats configurations in /etc/awstats/awstats.*.conf For now it is minimalist, we could sort some options more, I wait your Ideas

for example we could ignore statistics from lan

you must wait some minutes before to get statististics (default is 5 minutes), you have a web panel in nethgui to modify the settings and browse the domain statistics.

The web statistics pages are protected for the admin user (default), think to set a password. You have a panel to put more users

7 Likes

Ehi please help this man test his new born :baby: @quality_team do you have some time?
If I remember correcty @GG_jr was interested once

Excellent! I’m planning to upgrade www.nethserver.org from NS 6 to NS 7.

Great chance to try awstats :wink:

1 Like

thank for your interests, you will be surprised by the download statistics. I host a archlinux repository and it is about 5TB (terabyte) for the worse month.

I want to finish this development and I hope to get time to sort a ns6 version.

3 Likes

I’m still interested about AWStats!

And yes, I’m happy because @stephdl did this for us!

I have installed awstats by sdl a few minutes after Stéphane make the announcement.
I have a test server on NS7 with WebTop5, SOGo, Nextcloud and OSTicket installed on it and I will be glad to see statistics for all, if is possible.
Unfortunately, I was very busy and I had no time to “play” with awstats by sdl.

Question: is/will be possible to have statistics also for other web sites/apps which are not hosted on the server on which awstats is installed?

Again, thank you Stéphane! As usually, very good job!

Kind regards,
Gabriel

Short answer, awstats reads local logs, so if remote logs are copied by scp locally why not.

Concerning other webapp i would develop something based on db property, but for now it is only relied on nethserver-virtualhost

1 Like

With the option https://awstats.sourceforge.io/docs/awstats_config.html#OnlyFiles we could create statistics, but with the many possibilities of name/software/alias/virtualhost, maybe we could to think something flexible by properties (with a text area in the panel).

With something like that we will create an awstats configuration easily.

Create separate config files for each of the selections

OnlyFiles "^/mylottosite"
OnlyFiles "^/mypersonalsite"
OnlyFiles "^/myetcsite"

testing myself my work, I found that awstats has an issue to sort the different domain name, Indeed all domains are logged to tthe same logs.

@davidep what about if we do one custom log per virtualhost, of course with a log rotation.

I meant about this https://www.westhost.com/knowledgebase/pages/viewpage.action?pageId=1115032

Seems reasonable…

However the official Apache docs explain why a single log file is good and how to split it by host:

https://httpd.apache.org/docs/2.4/logs.html#virtualhost

IIRC the default CentOS config uses the vhost log format.

I must inspect, probably if i just do some minor change in the way how to write the log, probably i might solve my issue. With sme server the domain name starts first for each line and awstats works well for the ibays.

It is just a matter to say what to log, thanks for the pointers.

I’m away for three days, I will look for it later

1 Like

I speak something like that

[root@smemirror ~]# tailf /var/log/httpd/access_log
archlinux.de-labrusse.fr 141.8.183.13 - - [05/Jun/2017:22:52:58 +0200] "GET /staging/os/i686/redland-storage-virtuoso-1:1.0.17-5-i686.pkg.tar.xz.sig HTTP/1.1" 404 269 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
mirror.de-labrusse.fr 94.21.213.39 - - [05/Jun/2017:22:52:58 +0200] "GET /NethServer/7/repodata/repomd.xml HTTP/1.1" 200 3001 "-" "urlgrabber/3.10 yum/3.4.3"
mirror.de-labrusse.fr 94.21.213.39 - - [05/Jun/2017:22:52:59 +0200] "GET /NethServer/7/repodata/7b7cea297883dc8c1ff5029307951a3d1cd1f99e5f190715157f2b5a783b1978-primary.sqlite.bz2 HTTP/1.1" 200 122908 "-" "urlgrabber/3.10 yum/3.4.3"
archlinux.de-labrusse.fr 141.8.183.13 - - [05/Jun/2017:22:53:01 +0200] "GET /staging/os/i686/redland-storage-mysql-1:1.0.17-5-i686.pkg.tar.xz.sig HTTP/1.1" 404 266 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
smeserver.de-labrusse.fr 51.255.65.55 - - [05/Jun/2017:22:53:03 +0200] "GET /releases/9.2/smeos/i386/SME/xorg-x11-drv-mach64-6.9.5-1.el6.i686.rpm HTTP/1.1" 200 71664 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
archlinux.de-labrusse.fr 141.8.183.13 - - [05/Jun/2017:22:53:04 +0200] "GET /staging/os/i686/qt5-quickcontrols-5.9.0-1-i686.pkg.tar.xz.sig HTTP/1.1" 404 259 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
archlinux.de-labrusse.fr 177.96.149.228 - - [05/Jun/2017:22:53:04 +0200] "GET /core/os/x86_64/core.db HTTP/1.1" 200 127082 "-" "curl/7.53.1"
archlinux.de-labrusse.fr 141.8.183.13 - - [05/Jun/2017:22:53:07 +0200] "GET /multilib/os/x86_64/lib32-glibc-2.25-3-x86_64.pkg.tar.xz.sig HTTP/1.1" 200 310 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"

It’s false. We should define a custom log format with %v as first column and apply it to virtual hosts.

I’d like a unique additional log for all virtual hosts and %v.

I wouldn’t go with a global modification: it breaks the current log format.

Well awstats works with a combined log format, for what I saw it doesn’t work well for all the vhosts in one log.

I don’t know why.

For now I created a custom access log for each vhost, it works well, the error log is not changed.

1 Like

yum install http://mirror.de-labrusse.fr/NethDev/nethserver-awstats/nethserver-awstats-0.1.2-1.ns7.sdl.noarch.rpm

new panel with awstats settings
access logs are specific per virtualhosts

it should be good, I hope

2 Likes

Nice!

Anyway, besides this:

http://www.awstats.org/docs/awstats_config.html

we need a Legend!

Or, some clues near/below/… every “setting frame”.

new version, cosmetic changes in the code, thank skitoo

yum install http://mirror.de-labrusse.fr/NethDev/nethserver-awstats/nethserver-awstats-0.1.3-1.ns7.sdl.noarch.rpm

Released in my repository.

the dnslookup and the GeoIP are available in the panel
elementList for the templates.

1 Like

4 posts were split to a new topic: Awstats: access log of virtualhost is empty