Fail2ban Log : 'ipset v7.1: Hash is full, cannot add more elements'

Bug
,
1

Upon checking the fail2ban.log … I found several ERROR Warning that says ‘ipset v7.1: Hash is full, cannot add more elements’

Does this mean … that fail2ban not able to add more banned IPs ? Could this be a BUG ?

2020-02-15 06:49:02,382 fail2ban.utils [2362]: ERROR 7ff681bbde70 – stderr: ‘ipset v7.1: Hash is full, cannot add more elements’
2020-02-15 06:49:02,383 fail2ban.utils [2362]: ERROR 7ff681bbde70 – returned 1
2020-02-15 06:49:02,383 fail2ban.actions [2362]: ERROR Failed to execute ban jail ‘recidive’ action ‘shorewall-ipset-proto6’ info ‘ActionInfo({‘ip’: ‘177.226.240.11’, ‘fid’: <function at 0x7ff681bebc80>, ‘family’: ‘inet4’, ‘raw-ticket’: <function at 0x7ff681bee1b8>})’: Error banning 177.226.240.11
2020-02-15 06:49:45,612 fail2ban.actions [2362]: NOTICE [recidive] Ban 143.0.165.34
2020-02-15 06:49:45,632 fail2ban.utils [2362]: ERROR 7ff6800ce030 – exec: ipset add f2b-recidive 143.0.165.34 timeout 0 -exist
2020-02-15 06:49:45,632 fail2ban.utils [2362]: ERROR 7ff6800ce030 – stderr: ‘ipset v7.1: Hash is full, cannot add more elements’
2020-02-15 06:49:45,632 fail2ban.utils [2362]: ERROR 7ff6800ce030 – returned 1
2020-02-15 06:49:45,633 fail2ban.actions [2362]: ERROR Failed to execute ban jail ‘recidive’ action ‘shorewall-ipset-proto6’ info ‘ActionInfo({‘ip’: ‘143.0.165.34’, ‘fid’: <function at 0x7ff681bebc80>, ‘family’: ‘inet4’, ‘raw-ticket’: <function at 0x7ff681bee1b8>})’: Error banning 143.0.165.34
2020-02-15 06:50:27,515 fail2ban.actions [2362]: NOTICE [recidive] Ban 223.191.52.48
2020-02-15 06:50:27,531 fail2ban.utils [2362]: ERROR 7ff681473090 – exec: ipset add f2b-recidive 223.191.52.48 timeout 0 -exist
2020-02-15 06:50:27,533 fail2ban.utils [2362]: ERROR 7ff681473090 – stderr: ‘ipset v7.1: Hash is full, cannot add more elements’
2020-02-15 06:50:27,533 fail2ban.utils [2362]: ERROR 7ff681473090 – returned 1
2020-02-15 06:50:27,533 fail2ban.actions [2362]: ERROR Failed to execute ban jail ‘recidive’ action ‘shorewall-ipset-proto6’ info ‘ActionInfo({‘ip’: ‘223.191.52.48’, ‘fid’: <function at 0x7ff681bebc80>, ‘family’: ‘inet4’, ‘raw-ticket’: <function at 0x7ff681bee1b8>})’: Error banning 223.191.52.48

1 Like
2

You have probably set a recidive perpetual and ipset is limited to 65 000 records…

Do I am right ?

3

ipset -L f2b-recidive

What is the output, it should output the ban number

4

Oh yes I did… Is the new update no longer allowed perpetual ?
I was getting so much spam from certain IPs… I normally set the recidive to perpetual in the older version… and manually flush the banned IPs when I see those SPAMMERS settled down… Thanks for the update… now I know

5

There are only 28 Banned Entries… that seems quite low… normally I used to get up to about 2,000 entries or so…before I flush the banned IPs.
Unfortunately I keep getting the same bad spam IPs all the time and I was able in the last version to enable RBL to block most of those spammers.

Here are the Infor from the recidive output.

Name: f2b-recidive
Type: hash:ip
Revision: 4
Header: family inet hashsize 1024 maxelem 65536 timeout 0
Size in memory: 2712
References: 1
Number of entries: 28

6

config show fail2ban

I never understood why sysadmin never trusted fail2ban, it bans all the time you need, three bans after and you go to recidive for 15 days…there is one guilty here… Me :slight_smile:

7

We have changed the ban engine, we store the IP list to ipset, it is really fast but it is limited to 2^16 records per set. Before we used a list in a json file, it was long to read it and not efficient

So the recidive perpetual is still workable until you reach 65500 IP inside

1 Like
8

Hum… We could increase the maxelem when we create the set

I think we have something to do here

1 Like
9

In the meanwhile it complains about the hashfile, so we need maybe to increase it, probably not related to the number of IP

11

When we create the set maybe we need to increase the hash

Could you try a fix ?

ipset destroy f2b-recidive
ipset create f2b-recidive -exist hash:ip hashsize 32768 maxelem 80000 timeout 0

12

Thanks… I tried to ipset destroy f2b-recidive
getting error: ipset v7.1: Set cannot be destroyed: it is in use by a kernel component
Also tried stopping the fail2ban module before running the ipset command; still getting the same error above.

13

Try first

shorewall stop

Create the set

And restart shorewall

shorewall start

14

Thanks… that seems to work…
Made the changed
Name: f2b-recidive
Type: hash:ip
Revision: 4
Header: family inet hashsize 32768 maxelem 80000 timeout 0
Size in memory: 120
References: 1
Number of entries: 0
Members:

Will monitor it for now… much appreciate the quick reply !
Have a good weekend.

15

Have a good holiday :slight_smile:

1 Like
16

37
2020-02-16 17.38.37.jpg3120×4160 1.43 MB

1 Like
17

In the meanwhile if shorewall restart, it will overwrite your changes

Maybe you could change the line below with your settings

1 Like
18

Oh woo… so beautiful. Love it ! Thanks for sharing …
View from where I’m

IMG_1527
IMG_1527.jpg4845×1930 2.5 MB

Snapseed
Snapseed.jpg3311×1479 2.79 MB

1 Like
19

Thank you…
Done it ! Appreciate the support …I think this will fix it !

Ideas for Future : Is it possible somehow, track those repeating IPs that keep offending into the recidive filter, such that the BanTime for those IPs will proportionally increased by users’ settings ?

21

Hi mates we still have a rpm to be verified…thank in advance

1 Like
22

Thanks so much Stephane… will surely to give this a try…

1 Like