Odd network behaviour

Bit of an odd one - any thoughts or suggestions appreciated!

I have repeatedly built nethservers with most modules installed - they have repeatedly opeated for about 2 or 3 weeks, after which point they can only be accessed from the local network (they are sat on the lan of a BT router). Each time this has happened it has not been as a result of any change or trigger by me

For background the Nethserver is built as a guest partition (kvm) on a Ubuntu 1604 host installed on a DL380G5. In its latest build I have ensured that there is a seperate red and green interface and only the red interface has been presented to the internet router

Ive taken a paranoid approach and reinstalled the dl380 from scratch - any suggestions on further checks i can do would be appreciated - this fels like im being toyed with!

So you put the Nethserver red WAN Port to the Internet Router LAN Port? If you only use one interface because you are behind a router, it would be better to use the green(LAN) interface and not to have a red interface at all. But you may try disabling firewall on Nethserver for testing(shorewall clear). Can you ping outside from your Nethserver?

Thinking loud:
So the Nethserver KVM guest is working properly and you can connect from local network.
When the VM is working then I assume that the Ubuntu KVM host does his job too.
When the problem occurs, try the following:
Restarting guest vm, router, vm host one after each other and see if one of the steps solves the problem to exclude a device.

General checks:
Changing dynamic ip/ddns service problems(https://www.whatismyip.com/, compare with router settings, check ddns service)
internet problems(ping ip, ping fqdn, etc.)

Check router:
Can you reach the router or another lan device from wan device via wan, if you open ports? If not, the router or provider has the problem.
You may try making the nethserver to the dmz host, to port forward everything to your nethserver, if reachable then, check port forwarding rules or maybe router problem
Check ubuntu host:
Can you reach the ubuntu host from wan side? if yes, then Ubuntu with KVM has the problem.
Disable firewall on Ubuntu host, ifup all interfaces, try again

Some discussions about loosing connectivity, hanging KVM and troubleshooting Ubuntu KVM:

https://help.ubuntu.com/community/KVM/Networking

Cool decision, I always try to safe the systems ending up in reinstalling after hours…

1 Like

thanks for input Markuz - response (so far!)
solves the problem to exclude a device.
=> im a reboot man - these steps have been taken and changed nothing!

General checks:
Changing dynamic ip/ddns service problems(https://www.whatismyip.com/1, compare with router settings, check ddns service)
internet problems(ping ip, ping fqdn, etc.)
=> i have a set of static ips - strangely, when these servers have gone wrong they have allowd me to see the http test screen but all other services disappear

Check router:
Can you reach the router or another lan device from wan device via wan, if you open ports? If not, the router or provider has the problem.
=> im working remotely most of the time and, against my beter judgement, have a laptop on the lan side of the router that i can vnc on to (as a last resort only - this machine is NEVER used to access the nethserver until it becomes unavailable via internet) - from this device i can confirm that the router is up and all its static ips/port forwarding stuff is ok. i am also able to logon to the nethserver admin etc and i recall that email services would work also (despite none of this being available via internet)
You may try making the nethserver to the dmz host, to port forward everything to your nethserver, if reachable then, check port forwarding rules or maybe router problem
Check ubuntu host:
Can you reach the ubuntu host from wan side? if yes, then Ubuntu with KVM has the problem.
=> previously i could access the ubuntu host from the VNC laptop. i have tightened this up and positioned the ubuntu host so that it is now only available on the nethserver green network (which i access via a vpn) - to avoid rug being pulled from under it i have a second nethserver on a seperate dl580 host via a seperate static ip which shares this green network at its back end (some dream of enteprise class resiliance going on here!)
Disable firewall on Ubuntu host, ifup all interfaces, try again
=> i hadnt thought of this - in its current incarnation the nethserver guest has dedicated physical network interface for each of red and green so i would hope the hosts firewall isnt intefering!

1 Like

I have a server running Ubuntu + qemu/kvm and on that server I have pfSense as firewall/router and linuxschools as the server for all services (Samba4 AD accountprovider, fileserver, DHCP, DNS, nextcloud, printserver etc…)
I deliberately only configure the green interface on KVM host. Only pFSense has RED interface configured and I VT-d-ed this interface so ONLY pFSense can use it. The Linuxschools server only has GREEN interface configured.

I know, Linuxschools + pFSense is not the same as NethServer, but it rather close in terms of functionality. This setup is running stable for over 18months now. (the KVM host is not directly connected to the internet and even has never been updated during those 18+ months)
This all is behind a ISP router so there is no external IP address on any interface used.
(sometimes doubleNAT is not that bad)

You can opt to mimic this setup: Use any distro with KVM and setup 1 instance of NS stricktly as Gateway. VT-d the RED interface to the NS Gateway instance and use the GREEN interface for NS Gateway, NS services and KVM host.

If your server is somewhat decent (CPU, Memory) I can very much recommend to use proxmox as VM manager. It has some very neat features. I have it currently running on my home server (Xeon D-1521, 16GB memory) and I just love it.

1 Like