[SOLVED] Suricata has failed -- "too much Netfilter queue registered"

Bug
1

Just noticed that about two days ago Suricata had failed, and doesn’t want to run again. I am not sure what broke, but it would appear an update I did two days is the culprit.
The last suricata block entry was on 2020-10-05 21:06, and the update was performed at 2020-10-05 21:12. Since then no enteries are logged, so I assume the service failed after the update.

The error shows up as:

Oct 07 10:11:28 nethserver.triggeredstudios.com systemd[1]: Starting Suricata Intrusion Detection Service...
Oct 07 10:11:28 nethserver.triggeredstudios.com systemd[1]: Started Suricata Intrusion Detection Service.
Oct 07 10:11:28 nethserver.triggeredstudios.com suricata[15979]: 7/10/2020 -- 10:11:28 - <Error> - [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - too much Netfilter queue registered (16)
Oct 07 10:11:28 nethserver.triggeredstudios.com systemd[1]: suricata.service: main process exited, code=exited, status=1/FAILURE
Oct 07 10:11:28 nethserver.triggeredstudios.com systemd[1]: Unit suricata.service entered failed state.
Oct 07 10:11:28 nethserver.triggeredstudios.com systemd[1]: suricata.service failed.

I tried updating Nethserver completely and doing a full reboot, and also tried disabling all Suricata rulesets and restarting Suricata, but I always get the same error. My Google-fu has come up broke.

Everything else seems to be working, and Suricata is the only failed service.

Yum history output from upgrade that seems to have broke things:

Loaded plugins: changelog, fastestmirror, nethserver_events
Transaction ID : 13
Begin time     : Mon Oct  5 21:12:41 2020
Begin rpmdb    : 1053:f5142f7c7980526bf0da45861864b8edddacc5a7
End time       :            21:14:12 2020 (91 seconds)
End rpmdb      : 1053:8ac703cb6265b5d3b56cac9eb4dcf2db6bde758f
User           : root <root>
Return-Code    : Success
Command Line   : update
Transaction performed with:
    Installed     rpm-4.11.3-43.el7.x86_64                        @anaconda
    Installed     yum-3.4.3-167.el7.centos.noarch                 @anaconda
    Installed     yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch @ce-updates
Packages Altered:
    Updated     arp-scan-1.9.2-1.el7.x86_64                          @nethserver
    Update               1.9.7-3.el7.x86_64                          @epel
    Updated     ndpi-3.2.0-2841.x86_64                               @ntop
    Update           3.2.0-2861.x86_64                               @ntop
    Updated     nethserver-cockpit-1.7.7-1.ns7.noarch                @nethserver-updates
    Update                         1.7.8-1.ns7.noarch                @nethserver-updates
    Updated     nethserver-cockpit-lib-1.7.7-1.ns7.noarch            @nethserver-updates
    Update                             1.7.8-1.ns7.noarch            @nethserver-updates
    Updated     nethserver-fail2ban-1.5.0-1.ns7.noarch               @nethserver-updates
    Update                          1.5.2-1.ns7.noarch               @nethserver-updates
    Updated     nethserver-firewall-base-3.11.2-1.ns7.noarch         @nethserver-updates
    Update                               3.12.2-1.ns7.noarch         @nethserver-updates
    Updated     nethserver-firewall-base-ui-3.11.2-1.ns7.noarch      @nethserver-updates
    Update                                  3.12.2-1.ns7.noarch      @nethserver-updates
    Updated     nethserver-mail-common-2.17.2-1.ns7.noarch           @nethserver-updates
    Update                             2.17.3-1.ns7.noarch           @nethserver-updates
    Updated     nethserver-mail-filter-2.17.2-1.ns7.noarch           @nethserver-updates
    Update                             2.17.3-1.ns7.noarch           @nethserver-updates
    Updated     nethserver-mail-getmail-2.17.2-1.ns7.noarch          @nethserver-updates
    Update                              2.17.3-1.ns7.noarch          @nethserver-updates
    Updated     nethserver-mail-imapsync-2.17.2-1.ns7.noarch         @nethserver-updates
    Update                               2.17.3-1.ns7.noarch         @nethserver-updates
    Updated     nethserver-mail-server-2.17.2-1.ns7.noarch           @nethserver-updates
    Update                             2.17.3-1.ns7.noarch           @nethserver-updates
    Updated     nethserver-mail-smarthost-2.17.2-1.ns7.noarch        @nethserver-updates
    Update                                2.17.3-1.ns7.noarch        @nethserver-updates
    Updated     nethserver-mattermost-1.5.1-1.ns7.x86_64             @nethserver-updates
    Update                            1.5.2-1.ns7.x86_64             @nethserver-updates
    Updated     nethserver-openssh-1.6.0-1.ns7.noarch                @nethserver-updates
    Update                         1.7.0-1.ns7.noarch                @nethserver-updates
    Updated     nethserver-pulledpork-2.1.6-1.ns7.noarch             @nethserver-updates
    Obsoleting  nethserver-pulledpork-2.1.7-1.ns7.noarch             @nethserver-updates
    Updated     nethserver-stephdl-1.1.1-1.ns7.sdl.noarch            @/nethserver-stephdl-1.1.1-1.ns7.sdl.noarch
    Update                         1.1.2-1.ns7.noarch                @stephdl
    Updated     nethserver-subscription-3.6.4-1.ns7.noarch           @nethserver-updates
    Update                              3.6.5-1.ns7.noarch           @nethserver-updates
    Updated     nethserver-subscription-inventory-3.6.4-1.ns7.x86_64 @nethserver-updates
    Update                                        3.6.5-1.ns7.x86_64 @nethserver-updates
    Updated     nethserver-subscription-ui-3.6.4-1.ns7.noarch        @nethserver-updates
    Update                                 3.6.5-1.ns7.noarch        @nethserver-updates
    Updated     nethserver-suricata-2.1.2-1.ns7.noarch               @nethserver-updates
    Update                          2.2.0-1.ns7.noarch               @nethserver-updates
    Updated     ntopng-4.0.200923-9285.x86_64                        @ntop
    Update             4.0.201005-9285.x86_64                        @ntop
    Updated     ntopng-data-4.0.200923-9285.noarch                   @ntop-noarch
    Update                  4.0.201005-9285.noarch                   @ntop-noarch
    Updated     olefy-1.2.0-1.ns7.x86_64                             @nethserver-updates
    Update            1.2.1-1.ns7.x86_64                             @nethserver-updates
    Updated     pfring-7.6.0-3176.x86_64                             @ntop
    Update             7.6.0-3203.x86_64                             @ntop
    Obsoleted   pulledpork-0.7.4-1.el7.noarch                        @epel
    Dep-Install pulledpork7-0.7.3-7.ns7.noarch                       @nethserver-updates
    Updated     radcli-1.2.11-1.el7.x86_64                           @epel
    Update             1.2.12-1.el7.x86_64                           @epel
Scriptlet output:
   1 Received an unexpected HTTP status code of 401 from https://updates.maxmind.com/app/update_secure?db_md5=00000000000000000000000000000000&challenge_md5=44bd2679ddfb43e2533f6e0a59126fae&user_id=0&edition_id=GeoLite2-Country:
   2 Invalid account ID
   3
   4 geoipupdate 2.5.0
   5 Opened License file /etc/GeoIP.conf
   6 AccountID 0
   7 LicenseKey 000000000000
   8 Insert edition_id GeoLite2-Country
   9 Insert edition_id GeoLite2-City
  10 Read in license key /etc/GeoIP.conf
  11 Number of edition IDs 2
  12 url: https://updates.maxmind.com/app/update_getfilename?product_id=GeoLite2-Country
  13 md5hex_digest: 00000000000000000000000000000000
  14 url: https://updates.maxmind.com/app/update_getipaddr
  15 Client IP address: 71.214.94.211
  16 md5hex_digest2 (challenge): 44bd2679ddfb43e2533f6e0a59126fae
  17 url: https://updates.maxmind.com/app/update_secure?db_md5=00000000000000000000000000000000&challenge_md5=44bd2679ddfb43e2533f6e0a59126fae&user_id=0&edition_id=GeoLite2-Country
  18
  19 To use geolocation in ntop products, follow the instructions at
  20 https://github.com/ntop/ntopng/blob/dev/doc/README.geolocation.md
  21 warning: /etc/pulledpork/pulledpork.conf created as /etc/pulledpork/pulledpork.conf.rpmnew
  22 Setting up redis auto startup
  23 Note: Forwarding request to 'systemctl enable redis.service'.
  24 Creating link under /usr/local/bin
history info
2 Likes
2

I can now confirm that rolling back changes with “yum rollback” has fixed the suricata issue, so there must be an issue with either the new pulledpork package, or possibly the renaming of “pulledpork” to “pulledpork7” is causing issues somewhere.

1 Like
3

Can you confirm that you have a 16 cores cpu?
I may need the output of the nproc command.

1 Like
4

I am currently running Nethserver in a VM with 24 cores added. Output of nproc is indeed 24.

The actual server has 2x 6 core 12 thread xeons.

5

Thank you for reporting, we already have a fix in place: https://github.com/NethServer/dev/issues/6297

Would you mind to test it? Just execute:

yum --enablerepo=nethserver-testing update nethserver-suricata

After the update, suricata should be up&running with 16 queues.

1 Like
6

After updating to the testing package I can confirm that the problem is now fixed, Suricata runs normally.

2 Likes
7

Thank you for the feedback!

Fix released :wink: