Shorewall issues

NethServer Version: 7.8.2003
Module: shorewall

I have had various issues with Shorewall in the past. Usually it was just small annoyances (like shorewall spam in the console), but now things have gotten worse. Each week I run the updates in the software center. It seems that any time there is a network-related update, I will lose all connectivity to my server (Cockpit UI, legacy UI, and SSH). This requires me to connect in through KVM and clear out the shorewall rules in order to access the server again.

I have done nothing very special to my setup. I have only used the UI options available, so I don’t understand how this could happen.

The main “special” thing I have done is to add a green interface through my cloud provider so that the machine sees another local network. I have this using DHCP because every time I try to do it with a static IP everything breaks.

I’ve taken a look at my shorewall rules, and this is what I have. Any help to figure out how to prevent my network from being lost after updates and get rid of shell spam would be highly appreciated!

# ================= DO NOT MODIFY THIS FILE =================
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at NethServer official site: https://www.nethserver.org
#
#
#
# Shorewall version 4 - Rules File
#
# For information on the settings in this file, type "man shorewall-rules"
#
# The manpage is also online at
# http://www.shorewall.net/manpages/shorewall-rules.html
#
######################################################################################################################################################################################
#ACTION         SOURCE          DEST            PROTO   DEST    SOURCE          ORIGINAL        RATE            USER/   MARK    CONNLIMIT       TIME         HEADERS         SWITCH
#                                                       PORT    PORT(S)         DEST            LIMIT           GROUP
#SECTION ALL
#
#
# SECTION ALL
#
?SECTION ALL
 
#
# SECTION ESTABLISHED
#
?SECTION ESTABLISHED
 
#
# SECTION RELATED
#
?SECTION RELATED
 
 
#
# SECTION NEW
#
?SECTION NEW
 
#
# Accept Ping from the "bad" net zone.
#
Ping/ACCEPT   net             $FW
#
#  Make ping work bi-directionally between the dmz, net, Firewall and local zone
#  (assumes that the loc-> net policy is ACCEPT).
#
Ping/ACCEPT    loc            $FW
 
#
#       Accept DNS connections from the firewall to the Internet
#
DNS/ACCEPT      $FW     net
#
# 50docker
#
?COMMENT docker-jitsi-meet_jicofo_1
 
?COMMENT docker-jitsi-meet_prosody_1
 
?COMMENT docker-jitsi-meet_web_1
DNAT loc dock::80 tcp 8000 - -
DNAT loc dock::443 tcp 8443 - -
 
?COMMENT docker-jitsi-meet_jvb_1
DNAT loc dock::10000 udp 10000 - -
DNAT loc dock::4443 tcp 4443 - -
 
?COMMENT portainer
 
 
 
#
# 50pf -- PORT FORWARDING
#
 
#
# PF    &eth0:3282 -> 192.168.1.2  
#
?COMMENT Golem 1 from net
DNAT:none       net     ovpn:192.168.1.2        tcp,udp 3282    -       &eth0
#
# PF    &eth0:40102:40103 -> 192.168.1.2  
#
?COMMENT Golem 2-3 from net
DNAT:none       net     ovpn:192.168.1.2        tcp,udp 40102:40103     -       &eth0
 
 
#
# 60rules
#
 
#
# 65aqua Accept ping from aqua
#
 
Ping/ACCEPT    aqua            $FW
 
 
#
# 65aqua -- Rules for Docker containers
#
 
?COMMENT aqua
ACCEPT  aqua    $FW     tcp     3306
 
# Rule for docker jitsiLdap
ACCEPT  aqua    $FW     tcp     389
ACCEPT  aqua    $FW     tcp     636
 
#
# 60cockpit
#
?COMMENT cockpit
ACCEPT  loc     $FW     tcp     9090
ACCEPT  net     $FW     tcp     9090
 
#
#       Service: Jitsi Access: red,green
#
?COMMENT Jitsi
ACCEPT  net     $FW     tcp     8443
ACCEPT  loc     $FW     tcp     8443
?COMMENT Jitsi
ACCEPT  net     $FW     tcp     4443
ACCEPT  loc     $FW     tcp     4443
?COMMENT Jitsi
ACCEPT  net     $FW     tcp     8000
ACCEPT  loc     $FW     tcp     8000
?COMMENT Jitsi
ACCEPT  net     $FW     udp     10000
ACCEPT  loc     $FW     udp     10000
#
#       Service: chronyd Access: green
#
?COMMENT chronyd
ACCEPT  loc     $FW     udp     123
#
#       Service: collectd Access: NONE
#
#
#       Service: dnsmasq Access: green
#
?COMMENT dnsmasq
ACCEPT  loc     $FW     tcp     53
?COMMENT dnsmasq
ACCEPT  loc     $FW     udp     53
?COMMENT dnsmasq
ACCEPT  loc     $FW     udp     67
?COMMENT dnsmasq
ACCEPT  loc     $FW     udp     69
#
#       Service: docker Access: NONE
#
#
#       Service: dovecot Access: green,red
#
?COMMENT dovecot
ACCEPT  loc     $FW     tcp     110
ACCEPT  net     $FW     tcp     110
?COMMENT dovecot
ACCEPT  loc     $FW     tcp     143
ACCEPT  net     $FW     tcp     143
?COMMENT dovecot
ACCEPT  loc     $FW     tcp     4190
ACCEPT  net     $FW     tcp     4190
?COMMENT dovecot
ACCEPT  loc     $FW     tcp     993
ACCEPT  net     $FW     tcp     993
?COMMENT dovecot
ACCEPT  loc     $FW     tcp     995
ACCEPT  net     $FW     tcp     995
#
#       Service: ejabberd Access: green,red
#
?COMMENT ejabberd
ACCEPT  loc     $FW     tcp     5222
ACCEPT  net     $FW     tcp     5222
?COMMENT ejabberd
ACCEPT  loc     $FW     tcp     5223
ACCEPT  net     $FW     tcp     5223
?COMMENT ejabberd
ACCEPT  loc     $FW     tcp     5269
ACCEPT  net     $FW     tcp     5269
?COMMENT ejabberd
ACCEPT  loc     $FW     tcp     5280
ACCEPT  net     $FW     tcp     5280
?COMMENT ejabberd
ACCEPT  loc     $FW     tcp     5443
ACCEPT  net     $FW     tcp     5443
?COMMENT ejabberd
ACCEPT  loc     $FW     tcp     5444
ACCEPT  net     $FW     tcp     5444
#
#       Service: evebox Access: NONE
#
#
#       Service: fail2ban Access: NONE
#
#
#       Service: httpd Access: green,red
#
?COMMENT httpd
ACCEPT  loc     $FW     tcp     80
ACCEPT  net     $FW     tcp     80
?COMMENT httpd
ACCEPT  loc     $FW     tcp     443
ACCEPT  net     $FW     tcp     443
#
#       Service: httpd-admin Access: green,red
#
?COMMENT httpd-admin
ACCEPT  loc     $FW     tcp     980
ACCEPT  net     $FW     tcp     980
#
#       Service: ipsec Access: NONE
#
#
#       Service: loolwsd Access: NONE
#
#
#       Service: lsm Access: NONE
#
#
#       Service: mysqld Access: NONE
#
#
#       Service: nms Access: NONE
#
#
#       Service: olefy Access: NONE
#
#
#       Service: opendkim Access: NONE
#
#
#       Service: postfix Access: green,red
#
?COMMENT postfix
ACCEPT  loc     $FW     tcp     25
ACCEPT  net     $FW     tcp     25
?COMMENT postfix
ACCEPT  loc     $FW     tcp     465
ACCEPT  net     $FW     tcp     465
?COMMENT postfix
ACCEPT  loc     $FW     tcp     587
ACCEPT  net     $FW     tcp     587
#
#       Service: rh-php72-php-fpm Access: NONE
#
#
#       Service: rh-php73-php-fpm Access: NONE
#
#
#       Service: rspamd Access: NONE
#
#
#       Service: rsyslog Access: NONE
#
#
#       Service: shorewall Access: NONE
#
#
#       Service: slapd Access: green
#
?COMMENT slapd
ACCEPT  loc     $FW     tcp     389
?COMMENT slapd
ACCEPT  loc     $FW     tcp     636
#
#       Service: smartd Access: NONE
#
#
#       Service: sshd Access: green,red
#
?COMMENT sshd
ACCEPT  loc     $FW     tcp     22221
ACCEPT  net     $FW     tcp     22221
#
#       Service: sssd Access: NONE
#
#
#       Service: suricata Access: NONE
#
#
#       Service: unbound Access: NONE
#
#
#       Service: yum-cron Access: NONE
#
 
#
# 90dns_blue
#
 
 
#
# 90openvpn-tunnels
#

Which provider/VPS do you use?

I recommend to setup one green interface with the VPS public IP first.

From the documentation:

If the server is installed on a public VPS (Virtual Private Server), it should must be configured with a green interface. All critical services should be closed using Network services panel.

You may use red and green interface If you can configure public and private interface over your VPS/cloud provider or you use a virtual interface.

I am using Hetzner. I have already set up an additional Green interface, but it’s not using the Public IP. I’m confused, I thought RED interface was for internet facing (public IP) that I would want to block most things, and GREEN is for internal network (private IP) for services within Neth or on the same network as Neth that may be communicating with each other.

Here’s my configuration (with public IP blanked out of course)

This is true, if you use 2 interfaces (red and green).
Simplest configuration for a VPS is using one green interface with public IP and disable unwanted access in services.

When the server has only one interface, this interface must have green role.
(Source)

EDIT:

I recommend to make eth0 green and then release role of eth1.

Thanks. I’ll give this a try. I guess the Red / Green is more useful if I have multiple servers talking to each other?

** Edit **
Tried to set eth0 to green and got this error:
image

It can be useful on a VPS if you need VPN or samba AD. In these cases you may use the virtual interface.

Next to using it on a VPS you may use NethServer as firewall/gateway/all-in-one-box at home/company too.

What happens if you try 255.255.255.0?

I don’t use hetzner but I found a config howto for CentOS so it may work with NethServer. I didn’t test.

You may set the netmask in a terminal as a workaround:

db networks setprop eth0 netmask 255.255.255.255

It does not like it one bit, and server is now down… :frowning:

I think I want to keep the Red Green because I want the ability to do VPN sometimes. Can you advise how to reset my interface back to Red from the command line, and how to fix the original shorewall issue?

You may try network recovery:

network-recovery

To set the role of a interface:

db networks setprop eth0 role red

Reload network config:

signal-event interface-update

See the documentation for more details.

Seems hetzner VPS uses a special network configuration:

I think it is related to using more network interfaces in the same network.

Thanks. I found the snippet in that Networking documentation that explains how to reset by deleting the interfaces from the DB, deleting any startup scripts, and then restarting the network. When I do that, the server is accessible again and I don’t see any shorewall spam in the console.

Once I run the command to set eth0 to red again, I start getting shorewall spam and nothing is accessible. Even doing shorewall clear in this case doesn’t seem to help the system become available again… So it seems that something in the network configuration is really messed up and the only way to make it work is to get Neth to not touch it by deleting all its configs.

How can I start fresh?

I don’t know hetzner but maybe you have a backup?
I assume you just need to fire up a new centos image and reinstall NethServer.

But if it works with green and public IP I’d work with that configuration. If you really need a second interface (red and green), please follow these steps.

Sorry for my quick last post. Was in a bit of a panic mode. I’ll give this a try and see if I can get things working reliably with a single WAN interface set to green.

I assume that what you mentioned above, as well as what the other Hetzner post is talking about, is that the Hetzner “private network” has a slightly strange configuration.

Once I get things working well with the Green WAN I will follow up with them about the specific configuration of this private network. If there is nothing conclusive from them then I will use the dummy interface as you mentioned.

As always, thanks for your support! :slight_smile:

1 Like

After lots of server restarts, I have things almost working at a minimal level. Here’s what I did:

  • Followed the instructions here to reset the interfaces in the db
  • rebooted the server
  • ran shorewall clear

at this point, I could now access Neth again, but my network was not configured.

I thought I saw in the logs somewhere that some network bridges were still set up for eth1 (the network interface I removed)

Then I did this:

  • Removed Docker
  • Removed VPN
  • Removed Firewall
  • Added my eth0 as a Green DHCP (it never worked when setting it as static)

Now I can access my server and it starts normally, but I notice that shorewall is failing to start:

-- Unit shorewall.service has begun starting up.
Jul 26 20:46:41 neth.mydomain.com shorewall[5552]: Compiling using Shorewall 5.1.10.2...
Jul 26 20:46:41 neth.mydomain.com shorewall[5552]: Processing /etc/shorewall/params ...
Jul 26 20:46:41 neth.mydomain.com shorewall[5552]: Processing /etc/shorewall/shorewall.conf...
Jul 26 20:46:41 neth.mydomain.com shorewall[5552]: Loading Modules...
Jul 26 20:46:42 neth.mydomain.com kernel: nf_log: can't load ipt_ULOG, conflicting nfnetlink_log already loaded
Jul 26 20:46:42 neth.mydomain.com kernel: ipt_ULOG: ULOG: fail to register logger.
Jul 26 20:46:42 neth.mydomain.com shorewall[5552]: ERROR: Unable to find tcstart file /etc/shorewall/modules (EOF)
Jul 26 20:46:42 neth.mydomain.com root[5875]: ERROR:Shorewall start failed
Jul 26 20:46:42 neth.mydomain.com systemd[1]: shorewall.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Jul 26 20:46:42 neth.mydomain.com systemd[1]: Failed to start Shorewall IPv4 firewall.
-- Subject: Unit shorewall.service has failed

I haven’t done any custom configurations to Shorewall, and I think I have now removed all the applications that have done significant changes. So it comes back to my original question: How do I reset the shorewall configuration?

It seems there are issues with shorewall modules. Maybe the system has a customized kernel?

You may check your kernel version with uname -r

[root@testserver ~]# uname -r
3.10.0-1127.13.1.el7.x86_64

Did you update the system?
See also Firewall not start

Mine shows the same, but I did add the dummy module per the instructions. I guess that the firewall needs to be told about this as the other article mentions?

No, this is only needed if you use another kernel.
I’d try to make it work with one interface first. If everythings ok, add the dummy virtual interface.

I’ve removed the dummy module and still getting the same error. I searched google and I see a gist from you with the same error message… So how did you fix it? :slight_smile:

Ah, but this isn’t an error. I see the error a few lines down:
ERROR: Unable to find tcstart file /etc/shorewall/modules (EOF)

This gist was about ndpi, did you setup firewall rules?

I didn’t set up any firewall rules except for a port forward and whatever the VPN app sets up. I uninstalled both the Firewall app and the VPN app, so I assumed they would clean up after themselves. Is there some additional clean-up I need to do?

Please check if there are still entries in the db:

db fwrules show
db portforward show

To debug shorewall and hopefully find out more about the error (see also Shorewall docs):

shorewall restart -T

Reconfigure firewall for production after debugging:

signal-event firewall-adjust

As regards the static IP issue: (Warning! This may break your network again.)
Just go through the steps as described here.

If it doesn’t work you could try this one:

db networks setprop eth0 bootproto static onboot yes userctl no ipaddr <YOUR IPv4 ADDRESS> netmask 255.255.255.255 dns1 213.133.98.98 dns2 213.133.99.99 ipv6addr <IPv6 FROM SUBNET LIKE 2a01:4f8:0::1>/64 ipv6init yes ipv6_defaultgw fe80::1%eth0 ipv6_defaultdev eth0 role green

You need to replace <IPv4 ADDRESS> and <IPv6 FROM SUBNET LIKE 2a01:4f8:0::1> with the IPs from your configuration:

To get the actual network config:

ip a

Apply config:

signal-event interface-update