NethServer Version: latest
Module: unknown
I started getting a bunch of mail like this since yesterday:
Cron sleep $(( ( RANDOM % 60 ) )); /usr/share/nethserver-blacklist/download ipsets
[ERROR] Can’t update blacklist repository: fetch failed
…any idea where to look and what is messed up?
Honestly… Yes. Bolding the first hit i received reading the messages.
So… You have a Threat shield module installed. And at least a list is willing to be downloaded.
Please, consider to share a bit more about your current setup, otherwise trying to help would be like fishing during the night. Blindfolded 
I understand that threat shield is doing this.
I only use the IP blacklist part.
I haven’t touched this for months, yet only yesterday it started popping this.
I have FireHOL level 1 and 2.
Why it fails?
Other hints coming from the first post…
You can do a little log scrubbing using the time of your email message, for find if there’s reported any kind of error about fetching the rules.
Moreover, IDK if the retrieve of the rules can be scheduled and how often.
Doing that more than twice a week seems to me… Excessive, because before put an IP in that kind of list should be done if “enough” data is collected that that connection is doing something wrong.
I mean…
I could try to login as a lot of asian and russian IP addresses try to do on a lot of system. But before classify my connection as evil, i mean… At least 10.000 tries should be raised in a short amount of time (1 day). Don’t quote me on that, please, it’s only my opinion and i am not figuring myself as a security expert.
I didn’t make the schedule. The installation itself did it.
So if it is set too often and why, is not a matter of me configuring it wrong.
I know you are trying to help, by I am talking about a module that is enabled for months and since yesterday started doing this.
Behavior could be dependent not only from the module, the lists chosen, but also on the availability of the servers providing lists.
Therefore, maybe for months something outside your setup did not change and yesterday did.
Obviously this not help easing the situation o make go away the undesired message, but maybe on this NethServer theres not much you can do.
How to figure it out if something can be done or not? Via the logs. Lists are downloaded via http, which has got a lot of errorcodes telling why you can’t have the file the installation asked for.
(adding TLS don’t change the protocol answers)
For telling about a possible comparable scenario…
This installation was not able to retrieve updates. Other three installations on the same site were able. At first glance, I considered as cause of issue the mirrors system, blocking connections to my setup “suddenly”.
That was not the real cause of the issue: the firewall appliance in front of all the installation was limiting the number of connection for that installation (via IPsec monitor quite different devices on other 5 sites) so…
Then if I read you correctly, there is a new bug.
If the threat filter has a few preset lists to use and those change download location or are removed altogether, it is a matter of the people maintaining this “app” (as NS calls them), to update those lists and kill the dead links.
In that case I wonder how nobody else reported this. People don’t use threat filter app?
it is a git repository, could you try to remove it then launch again the /usr/share/nethserver-blacklist/download ipsets
rm -f /usr/share/nethserver-blacklist/ipsets
/usr/share/nethserver-blacklist/download ipsetsDoesn’t allow me to delete because it has 4 folders inside.
Should I really delete everything?
Use -rf
Maybe you have a conflict with the remote and the local git repository hence you cannot fetch it
Just a hint
Did the delete, the fetch still failed (immediately)…
rm -rf /usr/share/nethserver-blacklist/ipsets
I said I did the delete.
Did you delete the folder /usr/share/nethserver-blacklist/ipsets
What is the output of
ll /usr/share/nethserver-blacklist/
total 24
-rwxr-xr-x 1 root root 3051 Oct 18 10:05 download
-rwxr-xr-x 1 root root 1476 Oct 18 10:05 geoip
drwxr-xr-x 3 root root 36 Oct 22 22:38 ipsets
-rwxr-xr-x 1 root root 3475 Oct 18 10:05 load-dnss
-rwxr-xr-x 1 root root 2970 Oct 18 10:05 load-geoips
-rwxr-xr-x 1 root root 2982 Oct 18 10:05 load-ipsets
-rwxr-xr-x 1 root root 2936 Oct 18 10:05 search-ip
ipsets is empty
you cannot do a valid git clone http://… if the folder ipsets still exists, remove the full folder and not only the content
Thanks I’ll try now then, as I got 40 mail in the last few hours about the fail.
EDIT: This seems to work (manually, it pulled the content). I will see how the schedule works now…
Sometimes an IP (lets say a github IP) is included on some of the fireHOL blacklists and then this kind of errors could happen.