Problem with dns - nethserver get same ip on eth0 and br0

Hi,
(sorry, my englisch is not so good)

my nethserver installation has an unexpected behaivior (in diffrence to similar installations). He has a red and a green interface. the green interface is bridged to the green br0 because of the ndsc on this installation.

The nethserver should be the first dns in my (green) network since few days but he was unreliable. The green ip was ping- and connectable all the time but often there was no answer from the nethserver dns. As reason i found the existence of a ip that exists twice at a time in my network (?). But where the doubled ip comes from? He ist from the nethserver itself!

In the network section from admin panel the eth0 interface has no ip (as it should be) and the br0 bridge has the local ip 192.168.X.2. The ndsc has the ip 192.168.X.3 in the same subnet. The “interface” br0 is shown in my router with his static ip but the eth0 interface is also shown in my router with the same ip (but diffrent mac). So the ip exists twice in one subnet and makes some trouble…

in the router config or with ifconfig i can see that the mac from br0 and eth0 are different. the br0 mac is identical with the mac from vb-nsdc. The mac from eth0 interface is the “real” mac from the network card (a virtual network adapter from prvmox ve).

i have compared my configuration with two other (but similar) nethserver configurations (on different virtual hosts) but i have not found a misconfiguration nor logical knowledge.

In the first other configuration the br0 has the same mac as the connected eth0 interface. In the second other configuration the br0 has the same mac as the nsdc (as in my case).
but in contrast to my nethserver both other configurations works fine!

I have tried to remove, delete, reattached and reconfigured the br0 and eth0 intefaces but the behavior ist always the same. One time i have found the ndsc was connected to eth0 and not to br0 but to correct this makes no difference. I have changed the mac from eth0 interface (change the mac from virtual network adapter) but the problem is the same. the ip appears twice.
Then i have changed the mac from br0 with “ip link set br0 address xyz”. Afterwards the ip wasnt double anymore but the problem thar the dns is anwering unreliable is the same.

Any Ideas? What is the “normal” behavior when bridging a interface? Which mac is normally used to create the bridge or the vb-nsdc? Any suggestion to fix the Problem?

Regards yummiweb

@support_team, any ideas?

Just some ideas:

Is an IP-adress defined in system/network in proxmox ve?
What mac-adress defined in options of vm in proxmox ve?
What kind of nic is defined in proxmox ve? I got best experiences with VirtIO.

Who is your DHCP? Your Router or your NS? What router do you use?

Please have a look at:

db networks show 
brctl show
ifconfig

If the settings in networks db are all correct, you can do signal-event interface-update. This will call a script to set all interfaces to the values defined in networks db.

Normal behaviour of a bridge AFAIK is, that the br0 has the IP-address and the joined interfaces have non. That logical, because it’s wanted that all interfaces are reachable on the same IP. The mac-addresses of the members are normally different. But virtual adapters like tapX can “adopt” the mac of bridge.

As I said, just some ideas. I don’t know what you already did of this all.

1 Like

hi,

thank you for helping.
if something is unclear i can better explain this in german.

i have tried something:
i have changed the mac for the virtio nic in proxmox-options.
i have reconfigured all network cards, bridges and settings in nethserver.
i have soft-changed the mac for eth0 and/or br0 in in centos (ip link set)
i have removed and added all bridges in centos.

there are no difference at all.
the mac from the br0 will “reset” to “nsdc” mac after the vb-nsdc is/was reactivated.
(after restart or some other netserver signal routine)

on time i have reconfigured ALL network settings as described here:
https://docs.nethserver.org/projects/nethserver-devel/en/latest/nethserver-base.html#reset-network-configuration

the strangeness with the mac behavior (that my internet router shows the same ip on eth0 and br0 mac) persist but maybe this is not the cause for my actual problem with the nethserver dns (in opposite to my previos post).

my main problem is that the nethserver dns is not responding (after a while) if a green AND a “red” interface is configured. without a red interface the dns works as espected. (independent from the mac behavior and that my internet router shows the same ip on eth0 and br0 mac)

if i configure one or more nic (or bridges) as “green” the dns works and i can use the br0-ip, the nsdc-ip or some other “green” ip from nethserver as dns.

but if i configure a “red” interface additional or instead the behavior is strange.
first the dns will responding from br0-ip , nsdc-ip AND from the “red” ip (if i permit this in service settings) but after some time (not measured exactly) all clients in my network (all in green network) receiving no answers from dns.

this persists until (some times but not sure) the client gets a new dhcl lease from router (no matter of the ip is new or the same). but after some time again the clients will no get answers from dns.

sometimes the dns works fine despite from the existence of a red interface for example after reconfiguring the eth1 role from green to red but after a restart the problem is the same.

ALL the time the ip(s) from dns response to ping command.

so why i use the red interface and not the green only?
i need the red interface because i wanna use openvpn-roadwarrior in routed mode (tun) and the vpn-clients can only connect to other devices in the green network if a red interface is used.

so, here are my settings for now: (“x,y,z” represents anonymized places)

network:

dsl-router (fritz-box) for internet connection and as dhcp-server (and something)
his ip is: 192.168.x.99 (the gateway)
the router has an 4 internally switch-ports (unmanaged)
the router was already restarted, reconfigured etc…

netgear network switch (unmanaged)
tp-link network switch (unmanaged)
(both already restarted)

proxmox ve host (up to date and restarted):
the proxmox ve host has one nic and a soft-bridge.
the nic has no ip.
the bridge (vmbr0) ip is (manual): 192.168.x.9
the mac of the soft-bridge is identical with the nic:
xx:xx:xx:ab:e0:71

proxmox ve guest:

the nethserver guest has two nic and both are virtio.
the first nic (net0) is defined as: xx:xx:xx:2f:00:02 (in nethserver eth0)
the second nic (net1) is defined as: xx:xx:xx:2f:00:10 (in nethserver eth1)

nethserver vm at running:

the first nic (eth0) has no ip as is bridged to br0.
the eth0 role is “bridged”
the eth0 mac is: xx:xx:xx:2f:00:02
the bridge br0 has the ip: 192.168.x.2
the br0 mac is (shown in ipconfig): xx:xx:xx:28:7a:17
the br0 role is “green”

the bridge mac is identical as the mac from vb-nsdc:
(shown in ipconfig): xx:xx:xx:28:7a:17

the second nic (eth1) has the ip: 192.168.x.10
the eth1 role is “red”
the eth1 mac is: xx:xx:xx:2f:00:10

nethserver database settings:

db networks show:
br0=bridge
FwInBandwidth=
FwOutBandwidth=
bootproto=none
gateway=192.168.x.99
ipaddr=192.168.x.2
netmask=255.255.255.0
nslabel=
role=green
eth0=ethernet
bridge=br0
role=bridged
eth1=ethernet
FwInBandwidth=
FwOutBandwidth=
bootproto=none
gateway=192.168.x.99
ipaddr=192.168.x.10
netmask=255.255.255.0
nslabel=
role=red
ppp0=xdsl-disabled
AuthType=auto
FwInBandwidth=
FwOutBandwidth=
Password=
name=PPPoE
provider=xDSL provider
role=red
user=
red1=provider
interface=eth1
weight=1

brctl show:

bridge name bridge id STP enabled interfaces
br0 8000.xxxxxx287a17 no eth0
vb-nsdc

ifconfig:

br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.X.2 netmask 255.255.255.0 broadcast 192.168.x.255
inet6 fe80::xxxx:xxxx:xx28:7a17 prefixlen 64 scopeid 0x20
ether xx:xx:xx:28:7a:17 txqueuelen 1000 (Ethernet)
RX packets 632112 bytes 45590844 (43.4 MiB)
RX errors 0 dropped 530062 overruns 0 frame 0
TX packets 871 bytes 50504 (49.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::xxxx:xxxx:xx2f:2 prefixlen 64 scopeid 0x20
ether xx:xx:xx:2f:00:02 txqueuelen 1000 (Ethernet)
RX packets 922822 bytes 74880776 (71.4 MiB)
RX errors 0 dropped 47 overruns 0 frame 0
TX packets 19940 bytes 7988382 (7.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.x.10 netmask 255.255.255.0 broadcast 192.168.18.255
inet6 fe80::xxxx:xxxx:xx2f:10 prefixlen 64 scopeid 0x20
ether xx:xx:xx:2f:00:10 txqueuelen 1000 (Ethernet)
RX packets 1445846 bytes 341905813 (326.0 MiB)
RX errors 0 dropped 795202 overruns 0 frame 0
TX packets 577653 bytes 650535073 (620.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 1000 (Local Loopback)
RX packets 148872 bytes 19740324 (18.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 148872 bytes 19740324 (18.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
tunrw: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1500
inet 192.168.y.1 netmask 255.255.255.255 destination 192.168.y.2
inet6 fe80::xxxx:xxxx:xx06:4a03 prefixlen 64 scopeid 0x20
unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 100 (UNSPEC)
RX packets 144 bytes 17412 (17.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 159 bytes 106535 (104.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vb-nsdc: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::xxxx:xxxx:xx28:7a17 prefixlen 64 scopeid 0x20
ether xx:xx:xx:28:7a:17 txqueuelen 1000 (Ethernet)
RX packets 16908 bytes 6892620 (6.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 833629 bytes 60786822 (57.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

(edit=some characters)

IIUC you have only one nic.
Is the red interface in a Vlan? I don’t think so, cause you only have unmanaged switches.
So the firewall is not physically present between red and green interface.
I think you created a ARP-Roulette. Please have a look at this (2nd part):
http://shorewall.net/FoolsFirewall.html

Maybe this how-to to create a dummy interface for systems that only have one nic can help you:
https://wiki.nethserver.org/doku.php?id=virtual_network_interface&s[]=dummy

hi ralf,

thank yo for helping me.

ideed the actual configuration with one nic and one subnet is suboptimal in terms of security.
and maybe therefore comes problems with lower network/packet levels. and yes, with a problem Its always recommended to eliminate other problems in network. but ideed two other installations with (nearby) same configuration running without problems.

but i think that isnt all. the dns problem is (aside from security issues) my only problem in this configuration. all other services are o.k. and reachebale from locale network and internet (depending on the configuration). i think a problem on lower network levels wouldnt distinguish between dns and other services like sharing for example. Interestingly the dns problem seems to be a local problem.

what is the difference to configure a virtual interface in nethserver in opposite so configure a second virtio interface in the proxmox ve host? in my opinion a second virtio interface was the better way.

indeed i dont need a red interface. this configuration is for open-vpn-roadwarrior routing in my local network only.
is there an other option to bring netserver in a “routing-mode” so vpn guests can have access to other local maschines?

regards yummiweb

hi yummiweb,

you marked post 4 as solution. Did you solve your problem?

Yes, for vpn you need a red interface, AFAIK.
The dummy interface was ment as a way to try. I can’t tell what’s the exat difference between a second virtio and the dummy is. But if I have a problem I can’t solve and there is a second way, I try it.
When I began with nethserver some time ago, I had a server with 2 nic connected to the same switch (fools firewall) and I had very unreliable internetconnection and dns problems. But endeed this was with NS6.

BR Ralf