Threat shield: Hash is full, cannot add more elements

Support
#1

Hello,

After turning on some firehol blacklists, I get the following Cron error email:

ipset v7.1: Error in line 131073: Hash is full, cannot add more elements
[WARNING] Can’t load bl-firehol_abusers_30d ipset

Everything is the latest version.

How could I increase the hashsize and and maxelem for threat shield?

Thanks!

Edit: disabling bl-firehol_abusers_30d the other lists can fit the limits, so possibly it will stay this way for a while…

#2

Each set can actually have max 131072 elements: https://github.com/NethServer/nethserver-blacklist/blob/master/root/usr/share/nethserver-blacklist/load-ipsets#L111

For now the limit is hard-coded. Extending such limit, could eventually bring to performance loss.

Also bear in mind that many ipsets has duplicated IPs. I think that you did the right choice: do not use extremely large ipsets.

IP"table"s full?
#3

Well, currently using Firehol lvl 1-2-3, abusers_1d (and testing webclient/server but it looks like those blocks some service update IPs too).

#4

What you folks should consider for Threat Shield is to add an ipset for each list (and automaticall break up larger lists into multiple ipsets), this way you avoid the maxlen issues with hashtable size, which I think it tied to a specific ipset unless I am mistaken.