Fail2ban does huge logs

Support
,
#1

NethServer Version: 7.4
Module: fail2ban

I was checking my logs tonight and noticed my fail2ban log was almost a gig. Just from a day.

fail2ban.filterpoll [31356]: ERROR Unable to get stat on /var/log/httpd/ssl_error_log-20171007 because of: [Errno 2] No such file or directory: ‘/var/log/httpd/ssl_error_log-20171007’

fail2ban.filterpoll [31356]: WARNING Too many errors. Setting the jail idle

It does this hundreds of thousands of times, for the jail, all idle.

fail2ban.filter [31356]: WARNING Error decoding line from ‘/var/log/sogo/sogo.log-20171024.gz’ with ‘UTF-8’. Consider setting logencoding=utf-8 (or another appropriate encoding) for this jail. Continuing to process line ignoring invalid characters: ‘\x1f\x8b\x08\x00\x81\xec\xeeY\x00\x03\xed]ko\xdc6\x16\xfd\xde_A\x18X\xb4\xc5n8|J$\xfb\xc0z\xebIj \xb5\x8d\xb1\xbb\xc0"\x08\n’

Thats a SOGO thing.

Any hints as to the issue?

Edit: The issue seems to be that the logfile isnt rotated fast enough (or at all???) and the jail stops. Seems to be only an issue with the httpd jail(?) but from what I can tell it is a fail2ban issue upstream so maybe nothing we can do about it. Maybe we need to investigate if it is actually doing its job because failing to block a ddos or a hack attempt could be dangerous.

1 Like
#2

Is it happening with latest updates?
https://community.nethserver.org/t/fail2ban-doesn-t-ban-webinterface/8094?u=dnutan

#3

possibly :frowning:

the problem is not only upstream but also in Nethserver. For example the logs used by nethserver for apache are not access.log or error.log but like you see ‘ssl_error_log-20171007’ when the logrotate is triggered.

At first fail2ban expect to find something in access.log or error.log but they are empty, like we find in Fail2Ban doesn´t ban webinterface
Therefore I tried to add a wildcard to the fail2ban regex (’*’) and reload the fail2ban configuration when the fail2ban logs are rotated…but I suspect that maybe it doesn’t work

to solve you jail problem do : fail2ban-client reload
look after in the fail2ban log if the jail started well

before I would be interested by the content of

ll /var/log/sogo/
ll /var/log/fail2ban
ll /var/log/httpd

and the version of nethserver-fail2ban

rpm -qa nethserver-fail2ban

#4

Also as of this morning, after about 10 hours the fail2ban log is 400 megs…filled with the same errors.

Version : nethserver-fail2ban-0.1.25-1.ns7.sdl.noarch

SOGO

[details=Summary]-rw-r–r-- 1 sogo sogo 7530 Oct 26 08:39 sogo.log
-rw-r–r-- 1 sogo sogo 2596 Oct 20 03:30 sogo.log-20171020.gz
-rw-r–r-- 1 sogo sogo 1652 Oct 21 03:15 sogo.log-20171021.gz
-rw-r–r-- 1 sogo sogo 1419 Oct 22 03:46 sogo.log-20171022.gz
-rw-r–r-- 1 sogo sogo 1728 Oct 23 03:17 sogo.log-20171023.gz
-rw-r–r-- 1 sogo sogo 2393 Oct 24 03:32 sogo.log-20171024.gz
-rw-r–r-- 1 sogo sogo 1598 Oct 25 03:46 sogo.log-20171025.gz
-rw-r–r-- 1 sogo sogo 24319 Oct 26 03:09 sogo.log-20171026
[/details]

FAIL2BAN

fail2ban.log (There is no subfolder for fail2ban it is located /var/log/fail2ban.log

HTTPD

[details=Summary]-rw-r–r-- 1 root root 4784046 Oct 26 08:41 access_log
-rw-r–r-- 1 root root 23505 Oct 26 03:09 error_log
-rw-r–r-- 1 root root 11911 Oct 26 08:38 ssl_access_log
-rw-r–r-- 1 root root 0 Oct 26 03:09 ssl_error_log
-rw-r–r-- 1 root root 580 Oct 25 08:26 ssl_error_log-20171026
-rw-r–r-- 1 root root 14903 Oct 26 08:38 ssl_request_log
[/details]

#5

question concerning your errors in fail2ban log, it concerns only httpd and sogo. have you already reported what it occurs precisely. I mean that I read two lines of errors concerning sogo and httpd.

Eventually could you please paste a gift of the fail2ban log

just in case, does the command fail2ban-client reload solved your issue please ?

#6

A reload does seem to resolve part of the issue, its been half an hour and log file is blank except for

[details=Summary]2017-10-26 09:39:49,039 fail2ban.filter [31629]: WARNING Error decoding line from ‘/var/log/sogo/sogo.log-20171025.gz’ with ‘UTF-8’. Consider setting logencoding=utf-8 (or another appropriate encoding) for this jail. Continuing to process line ignoring invalid characters: ‘\x1f\x8b\x08\x00EA\xf0Y\x00\x03\xdd\ko\xdb6\x14\xfd\xbe_A\x04\x18\xd0\x02\x0b\xcd\xa7\xf8\xe8\x03\x0b\xd64+\xd6\xc5\x81\x93}\n’
[/details]

1 Like
#7
#8

well time to check if this issue continue, for your or others…

it was an old logs that fail2ban was looking for -> ssl_error_log-20171007

please could you monitor this a couple of weeks

#9

ok posted a gist, is there a better way to report errors in logs? Id be happy to monitor logs, but I dont want to open new topics when I find errors if Im clogging the forum :wink:

#10

go and shoot :smiley:

#11

ok whenever I change a jail or even change a setting in fail2ban I still get all those sogo errors, requiring a reload.

#12

The sogo error is not related to a lack of log. I have the feeling that sogo wrote some ‘binary’ things in logs
Does it is just one line in fail2ban log or another tsunami of informations like for httpd

1 Like
#13

Weird. So I do a reload of the service about once a day, overnight it starts the error again and the next morning I reload the service. It goes about 18 hours before it starts filling the log again.

#14

sogo or httpd errors.

can you past the error please

#15

Im sorry, still sogo.

Edit: As soon as I post its another log :smile:

#16

can you post the content

ll /var/log/fail2ban.log*

#17

that gist was it…

#18

I don’t have a cristal ball you know :slight_smile:
If you don’t answer, it might be hard for me :slight_smile:

#19

the gist refers to the missed fail2ban log, nothing related to sogo

#20

Yes I said it was another log :smiley: Its been all sogo but when I got in to it this morning and cleared the log it started fail2ban log :slight_smile: