Fail2ban Log : 'ipset v7.1: Hash is full, cannot add more elements'

stephdl · February 16, 2020, 7:35am

config show fail2ban

I never understood why sysadmin never trusted fail2ban, it bans all the time you need, three bans after and you go to recidive for 15 days…there is one guilty here… Me

stephdl · February 16, 2020, 7:44am

We have changed the ban engine, we store the IP list to ipset, it is really fast but it is limited to 2^16 records per set. Before we used a list in a json file, it was long to read it and not efficient

So the recidive perpetual is still workable until you reach 65500 IP inside

stephdl · February 16, 2020, 7:53am

Hum… We could increase the maxelem when we create the set

I think we have something to do here

stephdl · February 16, 2020, 7:59am

In the meanwhile it complains about the hashfile, so we need maybe to increase it, probably not related to the number of IP

stephdl · February 16, 2020, 8:07am

When we create the set maybe we need to increase the hash

Could you try a fix ?

ipset destroy f2b-recidive
ipset create f2b-recidive -exist hash:ip hashsize 32768 maxelem 80000 timeout 0

rmk · February 16, 2020, 2:11pm

Thanks… I tried to ipset destroy f2b-recidive
getting error: ipset v7.1: Set cannot be destroyed: it is in use by a kernel component
Also tried stopping the fail2ban module before running the ipset command; still getting the same error above.

stephdl · February 16, 2020, 2:47pm

Try first

shorewall stop

Create the set

And restart shorewall

shorewall start

rmk · February 16, 2020, 3:32pm

Thanks… that seems to work…
Made the changed
Name: f2b-recidive
Type: hash:ip
Revision: 4
Header: family inet hashsize 32768 maxelem 80000 timeout 0
Size in memory: 120
References: 1
Number of entries: 0
Members:

Will monitor it for now… much appreciate the quick reply !
Have a good weekend.

stephdl · February 16, 2020, 4:38pm

Have a good holiday

stephdl · February 16, 2020, 4:39pm

stephdl · February 16, 2020, 4:49pm

In the meanwhile if shorewall restart, it will overwrite your changes

Maybe you could change the line below with your settings

github.com

NethServer/nethserver-fail2ban/blob/master/root/etc/e-smith/templates/etc/shorewall/initdone/20fail2ban#L11


{
    # create the ipset jail in iptables
    # no rules if status disabled
    use NethServer::Fail2Ban;
    return "" if ($fail2ban{'status'} ne 'enabled');
    my $bantime= $fail2ban{'BanTime'} || '1800';
    my $perpetual = $fail2ban{'Recidive_Perpetual'} || 'disabled';
    foreach (NethServer::Fail2Ban::listAllJails()) {
    
        if (($_ eq 'recidive') && ($perpetual eq 'enabled')) {
            $OUT .= "system(\"/usr/sbin/ipset -quiet -exist create f2b-recidive hash:ip timeout 0 \");\n";
        } elsif ($_ eq 'recidive') {
            # max ban time for ipset is 2147483 seconds
            $OUT .= "system(\"/usr/sbin/ipset -quiet -exist create f2b-recidive hash:ip timeout 1209600 \");\n";
        } else {
            $OUT .= "system(\"/usr/sbin/ipset -quiet -exist create f2b-$_ hash:ip timeout $bantime \");\n";
        }
    }
    $OUT .= '1;';
}

rmk · February 16, 2020, 7:10pm

Oh woo… so beautiful. Love it ! Thanks for sharing …
View from where I’m

rmk · February 16, 2020, 7:11pm

Thank you…
Done it ! Appreciate the support …I think this will fix it !

Ideas for Future : Is it possible somehow, track those repeating IPs that keep offending into the recidive filter, such that the BanTime for those IPs will proportionally increased by users’ settings ?

stephdl · March 4, 2020, 3:44pm

Hi mates we still have a rpm to be verified…thank in advance

rmk · March 5, 2020, 1:34pm

Thanks so much Stephane… will surely to give this a try…

dnutan · March 5, 2020, 1:57pm

A post was split to a new topic: Another app is currently holding the yum lock

stephdl · March 5, 2020, 2:56pm

Please have a go it is a trivial QA

CptCharlesG · June 12, 2020, 11:05pm

Hello,

I am having a very similar problem after turning on some firehol lists in the Threat Shield.

Cron e-mail:

ipset v7.1: Error in line 131073: Hash is full, cannot add more elements
[WARNING] Can’t load bl-firehol_abusers_30d ipset

(Maybe should open another topic, but felt like it is a better place).

stephdl · June 13, 2020, 8:18am

Yes new topic, this one is related to fail2ban

EM_3 · February 5, 2021, 4:41pm

Hi sir, what does it mean? (axelem 80000 timeout 0) because i wanted to add brazil in my country ban list , but it will comsume a lot of memory.

https://www.ipdeny.com/ipblocks/data/countries/br.zone