Thread Shield: Updating and Enabling of IP-Backlist failed

capote · March 7, 2021, 5:59pm

Thank you for your hint. But I use google-DNS permanently.
Sincerely, Marko

capote · March 8, 2021, 11:03am

Anyone else has any ideas?

How can I completely uninstall Thread Shield?

Sincerely, Marko

filippo_carletti · March 8, 2021, 3:11pm

Run:

/usr/share/nethserver-blacklist/download --debug ipsets

capote · March 8, 2021, 3:31pm


[root@ns-srv01 ~]# /usr/share/nethserver-blacklist/download --debug ipsets

[DEBUG] Cloning repository Cloning into ‘/usr/share/nethserver-blacklist/–debug’… remote: Enumerating objects: 1470, done. remote: Counting objects: 100% (1470/1470), done. remote: Compressing objects: 100% (826/826), done. remote: Total 1470 (delta 650), reused 1381 (delta 643), pack-reused 0 Receiving objects: 100% (1470/1470), 25.26 MiB | 4.72 MiB/s, done. Resolving deltas: 100% (650/650), done. [root@ns-srv01 ~]#

The error persists.

capote · March 8, 2021, 4:01pm

asked again: How can I completely uninstall Thread Shield?

capote · March 9, 2021, 9:46am

@filippo_carletti and cc: @mrmarkuz

Hello,
is there really no way to completely remove thread shield and set it up from scratch?
best regards, Marko

pike · March 9, 2021, 10:20am

It’s a module… remove it as usual. For deleting configuration o reset it, i don’t know if there’s any way. @capote are you trying to emulate cron?

capote · March 9, 2021, 10:21am

Yes I did, same result.

mrmarkuz · March 9, 2021, 11:44am

I’d remove nethserver-blacklist and dependencies.
Check what’s going to be removed before uninstalling.

yum autoremove nethserver-blacklist

Maybe you need to manually remove the cron job in /etc/cron.d/nethserver-blacklist

stephdl · March 9, 2021, 11:44am

Remove also /usr/share/nethserver-blacklist

capote · March 9, 2021, 12:03pm

that was the trick - Thank you

Juan_Martin · March 10, 2021, 5:48am

Hi, I started to have the same problem yesterday: after weeks without any problem, suddenly I started to receive the following emails:
[ERROR] Can't update blacklist repository: fetch failed

I’m using firehol/block-list ipsets and it seems that at some point an IP address from github (140.82.121.4) was included in some of the ipsets (specifically blocklist_net_ua and firehol_level4).

To solve it, from nethserver terminal I did:
ipset add bl-whitelist 140.82.121.4
/usr/share/nethserver-blacklist/download ipsets

I had to do that from the terminal as adding the IP address to the whitelist from the web GUI and even removing the category complained about the blacklist repository, maybe because it was trying to download the ipsets before applying changes.

After that, I finally added github IP address to the whitelist via web GUI and everyting worked fine.

jookk · March 11, 2021, 9:25pm

/usr/share/nethserver-blacklist/download --debug ipsets
DEBUG] Pulling changes
DEBUG] Repository have been updated: reloading --debug
/usr/share/nethserver-blacklist/download: line 123: /usr/share/nethserver-blacklist/load---debug: No such file or directory

Using blacklist from:
firehol git

Ip blacklist is not updating, currently its old for about 5 days.
Trhread Shield v 1.1.7

filippo_carletti · March 12, 2021, 9:17am

[EDITED]

Sorry, my bad, the --debug option must be the last option.
The right command is /usr/share/nethserver-blacklist/download dnss --debug.

[DEBUG] Pulling changes
[DEBUG] Repository have been updated: reloading ipsets
[DEBUG] Resetting ipset bl-whitelist
[DEBUG] Resetting ipset bl-alienvault_reputation
[DEBUG] Resetting ipset bl-ISO_country_code_za
[DEBUG] Creating global whitelist
[DEBUG] Creating ipset bl-alienvault_reputation
[DEBUG] Creating ipset bl-ISO_country_code_za

You could also use bash -x.
Here’s the output of a working download for reference:

[root@ns7-com nethserver-blacklist]# bash -x /usr/share/nethserver-blacklist/download ipsets
+ TYPE=ipsets
+ PROP=blacklist
+ case $TYPE in
+ PROP=blacklist
++ /sbin/e-smith/config getprop blacklist Url
+ URL=https://github.com/firehol/blocklist-ipsets.git
++ /sbin/e-smith/config getprop subscription SystemId
+ SYSTEM_ID=
++ /sbin/e-smith/config getprop subscription Secret
+ SYSTEM_SECRET=
+ DEST_DIR=/usr/share/nethserver-blacklist/ipsets
+ DEBUG=0
++ getopt -o d --long debug -- ipsets
+ options=' -- '\''ipsets'\'''
+ '[' 0 -eq 0 ']'
+ eval set -- ' -- '\''ipsets'\'''
++ set -- -- ipsets
+ true
+ case "$1" in
+ shift
+ break
+ '[' -z https://github.com/firehol/blocklist-ipsets.git ']'
+ mkdir -p /usr/share/nethserver-blacklist/ipsets
++ echo https://github.com/firehol/blocklist-ipsets.git
++ grep ://
++ sed '-es,^\(.*://\).*,\1,g'
+ proto=https://
++ echo github.com/firehol/blocklist-ipsets.git
+ url=github.com/firehol/blocklist-ipsets.git
+ auth=
+ [[ ! -z '' ]]
+ quiet=
+ '[' 0 -eq 0 ']'
+ quiet=' --quiet '
+ '[' '!' -d /usr/share/nethserver-blacklist/ipsets/.git ']'
+ opts='--git-dir=/usr/share/nethserver-blacklist/ipsets/.git --work-tree=/usr/share/nethserver-blacklist/ipsets'
+ debug 'Pulling changes'
+ '[' 0 -eq 1 ']'
+ git --git-dir=/usr/share/nethserver-blacklist/ipsets/.git --work-tree=/usr/share/nethserver-blacklist/ipsets fetch --all
+ '[' 0 -gt 0 ']'
+ git --git-dir=/usr/share/nethserver-blacklist/ipsets/.git --work-tree=/usr/share/nethserver-blacklist/ipsets reset --hard origin/master
+ '[' 0 -eq 0 ']'
+ debug_flag=
+ '[' 0 -eq 1 ']'
+ debug 'Repository have been updated: reloading ipsets'
+ '[' 0 -eq 1 ']'
+ exec /usr/share/nethserver-blacklist/load-ipsets --reload
[root@ns7-com nethserver-blacklist]# echo $?
0

jookk · March 12, 2021, 12:20pm

/usr/share/nethserver-blacklist/download ipsets --debug
[DEBUG] Pulling changes
[DEBUG] Repository have been updated: reloading ipsets
[DEBUG] Resetting ipset bl-whitelist
[DEBUG] Resetting ipset bl-blocklist_de_apache
[DEBUG] Resetting ipset bl-blocklist_de_bots
[DEBUG] Resetting ipset bl-blocklist_de_bruteforce
[DEBUG] Resetting ipset bl-blocklist_de_imap
[DEBUG] Resetting ipset bl-blocklist_de_mail
[DEBUG] Resetting ipset bl-dshield_top_1000
[DEBUG] Resetting ipset bl-firehol_abusers_1d
[DEBUG] Resetting ipset bl-spamhaus_drop
[DEBUG] Resetting ipset bl-spamhaus_edrop
[DEBUG] Creating global whitelist
[DEBUG] Creating ipset bl-blocklist_de_apache
[DEBUG] Creating ipset bl-blocklist_de_bots
[DEBUG] Creating ipset bl-blocklist_de_bruteforce
[DEBUG] Creating ipset bl-blocklist_de_imap
[DEBUG] Creating ipset bl-blocklist_de_mail
[DEBUG] Creating ipset bl-dshield_top_1000
[DEBUG] Creating ipset bl-firehol_abusers_1d
[DEBUG] Creating ipset bl-spamhaus_drop
[DEBUG] Creating ipset bl-spamhaus_edrop

But, at web gui still showing last update was 5 days ago.

capote · March 12, 2021, 3:56pm

if the Maintainer of the list does not provide anything new, there can be no new list.

thorsten · March 12, 2021, 4:09pm

@mrmarkuz

Exactly my problem - I think there is a larger bug in the firewall

mrmarkuz · March 12, 2021, 4:14pm

Does the solution work for you?

filippo_carletti · March 12, 2021, 4:25pm

Nothing to be worried about, it’s a cosmetic problem of the Firehol repository.
Even if the list are updated, the date is wrong. You can see it by yourself:

# cd /usr/share/nethserver-blacklist/ipsets
# git log
commit 9fa77f0574c0bf5fe8860915eb85c77b053047f3
Author: Costa Tsaousis <costa@tsaousis.gr>
Date:   Sun Mar 7 06:30:15 2021 +0000

thorsten · March 12, 2021, 4:33pm

Nope - no change still the same if the MAC validator is turned on. However I deleted /etc/cron.d/nethserver-blacklist after I counter-checkt that nethserver-blacklist is not installed

(yum autoremove nethserver-blacklist without results)

but still:

grafik

Still the same problem - everywhere - I really start getting frustrated - does not matter what I do - I get the errors you have already seen.