How to analyze who triggers fail2ban

Regarding this thread we should try collect know how to analyze which client triggers fail2ban.
In my mind that summarize know how about:

  1. Some reproducible strategies to find the root causes of bans
  2. most common commands for analysis of the relevant log files
  3. prevention against the most common scenarios to lock yourself out.

I will progressively consolidate the answers here in the first posting.

Use Case #1
1. Architecture

  • root server within an external data center

  • 2 network interfaces with WAN-IP (RED) and dummy LAN-IP (Green)

  • installed IPS, Thread Shield

  • own LAN with more than 30 clients behind a DSL router with dynamically changed IP address they use Nethsever for…

    • Mail
    • authentication for some Nethserver provided applications like FileServer, DokuWiki, NextCloud, Webmail, Wordpress, VPN (Road Warrior)
      2. fail2ban configuration
  • active jails:

    • apache-auth
    • apache-badbots
    • apache-botsearch
    • apache-fakegooglebot
    • apache-modsecurity
    • apache-nohome
    • apache-noscript
    • apache-overflows
    • apache-scan
    • apache-shellshock
    • dovecot
    • dovecot-nethserver
    • ejabberd-auth
    • httpd-admin
    • mysqld-auth
    • nextcloud-auth
    • openvpn
    • pam-generic
    • pam-generic-nethserver
    • phpmyadmin
    • postfix
    • postfix-ddos
    • postfix-sasl-abuse
    • recidive
    • rspamd
    • sieve
    • sogo-auth
    • sshd
    • vsftpd
  • whitelisting: yes

  • static ban time

  • log level:INFO

  • Number of attempts: 3

  • time span: 10 Minutes

  • ban time: 15 Minutes

3. fai2ban behavior

  • fail2ban bans my own LAN because one of my clients hammers against fail2ban. That’s why analysis and administration is totaly blocked
  • Assumption 1: Typo within mail client logon data
  • Assumption 2: user changed his own password via cockpit and the credentials within mail clients become invalid

4. work around:

  • connect administrator client via mobile phone to internet to access to the root server
  • whitelisting the WAN IP of the DSL-Router (unfortunately each time if DSL-Router changes his IP-address)
  • preventive renewal of the mail login data for all clients, as it was not possible to identify the triggering client quickly

5. Analysis Procedure:

  • Identify banned IP and whitelist and unblock it if it is identical to your own WAN IP
  • Determination of the applicable jail / service and trigger time
  • Determination of the applicable local client
  • tbd

6. Analysis commands:
6.1 Analysis of triggerd jails
# cat /var/log/fail2ban.log | grep "Ban my.dynamic.wan.ip"

6.1 Analysis of current postfix/smtpd jail

# fail2ban-regex /var/log/maillog /etc/fail2ban/filter.d/postfix.conf --print-all-matched
# fail2ban-regex /var/log/maillog /etc/fail2ban/filter.d/postfix-ddos.conf --print-all-matched

6.2 Analysis of earlier postfix/smtpd jail

# fail2ban-regex /var/log/maillog-yyyymmdd /etc/fail2ban/filter.d/postfix.conf --print-all-matched
# fail2ban-regex /var/log/maillog-yyyymmdd /etc/fail2ban/filter.d/postfix-ddos.conf --print-all-matched

Matrix of log files corresponding to jails

|       Jail                |       Logs                    |
| apache-auth               | /var/log/httpd/*error_log     |    
| apache-badbots            | /var/log/httpd/access_log     |
| apache-botsearch          | /var/log/httpd/*error_log     |
| apache-fakegooglebot      | /var/log/httpd/access_log     |
| apache-modsecurity        | /var/log/httpd/*error_log     |
| apache-nohome             | /var/log/httpd/*error_log     |
| apache-noscript           | /var/log/httpd/*error_log     | 
| apache-overflows          | /var/log/httpd/*error_log     |
| apache-scan               | /var/log/httpd/*error_log.    |
| apache-shellshock         | /var/log/httpd/*error_log     |
| dovecot                   | /var/log/imap                 |
| dovecot-nethserver        | /var/log/imap                 | 
| ejabberd-auth             | /var/log/ejabberd/ejabberd.log|
| httpd-admin               |/var/log/httpd-admin/access_log|
| mysqld-auth               | /var/log/mariadb/mariadb.log  |
| nextcloud-auth            |/var/lib/nethserver/nextcloud/nextcloud.log |
| openvpn                   | /var/log/openvpn/openvpn.log  |
| pam-generic               | /var/log/secure               |
| pam-generic-nethserver    | /var/log/secure               |
| phpmyadmin                | /var/log/httpd/access_log     |
| postfix                   | /var/log/maillog              |
| postfix/smtp              | /var/log/maillog              |
| postfix-ddos              | /var/log/maillog              |
| postfix-sasl-abuse        | /var/log/maillog              |
| recidive                  | /var/log/fail2ban.log         |
| rspamd                    |/var/log/httpd-admin/access_log|
| sieve                     | %(dovecot_log)s               |
| sogo-auth                 | /var/log/sogo/sogo.log        |
| sshd                      | %(sshd_log)s                  |
| vsftpd                    | /var/log/vsftpd.log           |
| webtop                    |/var/log/webtop/webtop_auth.log|

Please correct and add with your suggestions.

4 Likes

I would add some feed for thoughts

  • If the remote gateway is a static IP, add it to the whitelist
  • Create a vpn tunnel and add the remote local IP to the well know network (cc @pagaille)
  • If the server is to authenticate users, try the incremental ban time, better than the recidive where people goes to jail for a long time.
    FindTime: 900,BanTime:60, RecidiveBan:incremental

In order to find what client has triggered the jail, the idea is

  • Go to fail2ban.log and find the jail
  • use fail2ban-regex with the relevant log (written in jail.local) and the relevant filter (same as the jail name)
1 Like

I could imagine a script that looked onto an (external) dns (maybe dynamic dns), find out the (daily) ip adress to set this ip onto the fail2ban whitelist for a predefined time or until the dns points to an other ip.

IMHO ddclient: https://wiki.nethserver.org/doku.php?id=module:ddclient.
I use it to get the current dynamic address on my DSL router for the Nethserer VPN.

1 Like

Question: does the remote gateway (wan-ip) are whitelisted inside your fail2ban

At me, yes. But whitelisting is obsolete when my DSL router changes its dynamic IP address.

I always refused to add a domain name to the whitelist but I am quite sure it could work … Sometime a developer does not reach the need of sysadmin

:-?

I’m having a hard time understanding how it could help.

Do you mean the following mechanism I practice for VPN dial-up to my DSL router?

  1. my DNS provider determines via ddclient my current dyamic address at the gateway.
  2. my DNS provider assigns this as A-record to a subdomain like vpn.mydomain.tld
  3. I whithelist the subdomain vpn.mydomain.tld with the current dynamically assigned IP address

I mean in /etc/fail2ban/jail.local add to

ignoreip = 127.0.0.1 stephdl.dynamic.ip.provider.net

restart fail2ban then

2021-04-20 11:45:40,422 fail2ban.filter [26521]: INFO [sogo-auth] Ignore 90.1.xxx.xxx by dns

Then your dynamic IP will be ignored by fail2ban but the UI will create an error because the validation accept only IP or network

@giacomo @filippo_carletti what do you think if we could accept dynamic domain inside the UI of fail2ban

Like this we could be sure to be not banned when you do not have a static IP ?

fail2ban accepts hostnames, I don’t know why we limited the interface to IPs only. :slight_smile:

5 Likes

At the start I even refused to add CIDR :smiley:

@capote it is coming

https://github.com/NethServer/nethserver-fail2ban/pull/113

With this you can add FQDN or hostname

1 Like

This is the same as I thought. The difference is that I don’t need a separate subdomain like dynip.mydomain.de because I have one for VPN dialup called vpn.mydomain.de.

One big request…
Could you please integrate the ability to comment on the whitelisted IPs, CIDR or domains so that everyone can document the reason for white listening, as…

#Admins LAN
stephdl.dynamic.ip.provider.ne
#uptimerobot 1
72.55.136.192/28
#anything else
11.22.33.44

inside textarea I think it is not possible

Too bad, I thought it would be as simple as in any script. But I don’t understand any of it either.

web development is never easy, as well how to understand the good UI. Imagine comments inside a textarea, after all everything is possible I could create modals to fill the whitelist with a table like for the unban, but please go to github and check how many hundred line of code I will add for something that is a hole in the security…what withlist is :smiley:

1 Like

Can anyone contribute additional relationships between triggered jail and log files to be analyzed (where more in-depth analysis would be needed)??

everything can be found in /etc/fail2ban/jail.local

Unrelated to fail2ban but a quick & dirty script to get the changing IP address of a domain (lets say of another of your publicly reachable hosts) could be something like this:

#!/bin/bash
#
# Update host object IP address (to be used with firewall rules, i.e. for public domain)
# (Requires: dig, ipcalc, e-smith commands)
#

# get IP address
ip=$( /usr/bin/basename $( /usr/bin/dig +short domain.tld A | tr '\n' '/' ) 2>&- )

# validate IPv4 address
if ! /usr/bin/ipcalc -c "$ip"; then exit; fi

# get stored IP address (if any) # hard-coded value
host='hostnamex'
oldip=$( /usr/sbin/e-smith/db hosts getprop $host IpAddress )

# set firewall object to the new IP address
if [ "$ip" = "$oldip" ]
then
   exit
else
   /usr/sbin/e-smith/db hosts set $host host IpAddress $ip
   /usr/sbin/e-smith/signal-event firewall-adjust
fi

For instance to be used with a cron job to update the host ip address used on some firewall rules.
Probably there are better ways to do it but there’s that.

1 Like