Can't access containers outside of Nethserver

NethServer Version: 7.8.2003
Module: Active Directory

I’m experiencing this issue with the active directory support, which creates nsdc container.

STR:

  • starting with clean install of 7.8.2003
  • run initial wizard on old UI
  • run updates on new UI
  • reboot
  • install active directory
    • click users in new UI
    • choose active directory
    • choose create new
    • set IP to free address in MGMT vlan.
  • ping nethserver (expected ping / got ping)
  • ping ad.domain.com (expected ping / no ping)

My system is virtualised (QEMU/KVM) and I have read the remarks about virtualising bridge and promisc mode in manual. However, I am using virtual ethernet devices as my host device has SR-IOV compliant network adaptors.

I have tried a multitude of configurations:

conf1: using libvirt to set the vlan tag of the virtual ethernet device given to nethserver causes failure on installation of active directory components (nsdc i think), because I think there’s an unhandled error when enabling vlan filtering.

conf2: remove libvirt vlan tag. before installation of active directory, create eth.100, create br0 on eth.100.

conf3: remove libvirt vlan tag. before installation of active directory, create br0, enable vlan filtering, install active directory, configure vb-nsdc with pvid and vlan 100.

conf4: set the hosts PF connection to switch from trunk to vlan100, no libvirt vlan tag, install from fresh again (no vlan anything - so purely managed in physical switch)

in conf2-4 the installation is successful and with slight exception to conf3 (which I really struggled to figure out) I could also ping the AD form nethserver, but have never been able to ping the AD from any other machine. I have other devices in MGMT vlan which I can ping, and I can ping nethserver so don’t believe it’s routing issue.

I suspect I either need to do nasty manipulation of the internal switch of the hosts SR-IOV network adaptor OR something wrong in nethserver/my configurtion?

Shorewall/Firewall - I have tried shorewall clear to drop all rules and open everything - but no progress.

I’m not using nethserver as an edge/internet router. I would just like a domain controller/dns/dhcp/ntp running internally. I’m using untangle on the edge.

Any help would be greatly appreciated.

@umbramalison

Hi Andrew

Welcome to the NethServer forum!

I manage about 20-30 virtualized NethServers for as much clients (SME companies), all are running in KVM virtualization and are running the AD in Nethserver. Some clients have 3 sites and about 50 users in all, all managed from one, well available NethServer.

I’ve never had ANY issues either in setting up AD or the virtualization. I know about the promiscious mode needed for NethServers AD, but never even had to bother about it…

But then, I’m using the free, open source Proxmox underneath, which offers (all for free, no paid enterprise mode with more functions!):

  • KVM full virtualization
  • LXC Linux Contailners
  • Live Backups for all guest OS.
  • Clustering with FAST migration (90 secs!)
  • Full HA High Availability clusters (min 3 nodes)
  • Out of the box, easy ZFS installation
  • Out of the box, CEPH installation
  • Very stable! Maybe one reboot / year needed!
  • Updates of Proxmox I do live, while people are working!
  • Promicious mode active and VERY functional by default!
  • Up to 4096 virtual switches inside the Proxmox hosts.
  • full vLAN supported, but not used if not needed!

A lot of the people here, especially a lot of devs use this same Proxmox system.

Even if you do want to license it, that’s fairly simple and cheap, compared to it’s competitors:
ca 100$ / year / CPU-Socket. Only used CPU sockets need a license, not for any number of cores.
My SME clients all use the licensed version, with “tested” updates & community support!
We’ve NEVER needed any support!

Why not try this out (20 minutes, that’s all!) and save yourself all of those time wasting tweaks trying to get something to work? Proxmox just works! I’ve installed it on several different hardware, from HP Proliants to Mac Minis!

I’m preparing a use cases doc, it’s still WIP, but you may have a peek, and see if your system can compare:
https://wiki.nethserver.org/doku.php?id=userguide:nethserver_and_proxmox

If you need help with Proxmox, you’ll find plenty of help / tips here in this forum, but not for a home done KVM…
There’s just too many variables to that home built solution, and most here have realized that whatever they build, it can’t compare to the ease and stability Poxmox offers…

Note: None of my clients or friends have the firewall on NethServer, they all use a separate box, some friends even have the firewall, OPNsense running in Proxmox, it’s that stable!

My 2 cents
Andy

1 Like

Thanks Andy,

I did review Proxmox before my current incarnation, and I also reviewed it 7 years ago before my previous setup.

I’m more familiar with QEMU/KVM and libvirt and atleast see my self as more of a ‘power user’ and was an aspiring developer at one stage. QEMU/KVM is what proxmox uses + containers.
I actually don’t feel Proxmox adds value (to me) beyond fedora + cockpit (similar already to nethserver) + openzfs (there even exists a cockpit plugin for zfs), and cockpit with the right plugins is very competitive. You are right, there are aspects that Proxmox provides which I won’t have (HA stuff for instance) but I don’t believe anything that I need in my setup.

I was also put off by the licencing scenario of Proxmox, and didn’t want to be hacking the solution around which was the first impression I got with regards to disabling the licence nag screen.

But what I do need is samba and dc (since moving ZFS from virtual FreeNAS to bare metal) and my intention is for nethserver to fill that void. If Proxmox did that then I would be interested as it could reduce the number of VM’s I need. The E5 1650v3 is beginning to feel dated compared to epyc and ryzens in today’s market… so tempting.

If successful i’ll have my entire setup based on cockpit and linked to each other and centos/fedora, which helps with the maintenance having all my systems being similar.

So I really want to resolve this issue. I think my best approach now, especially since I won’t be using Proxmox is to see if I can recreate the issue on bare metal and take the virtualisation out of the discussion.

Although I would still really like to hear from someone using SR-IOV’s - maybe there is something additional that needs to be done. I have just been tinkering with the FDB, putting the MAC of the NSDC container into FDB of the host network device. This is something you have to do if you run a bridge on the PF as the VF’s can’t route to the bridges tun/tap devices until the PF knows their mac addresses IIRC.

It didn’t help - but I am just making wild guesses at the moment!

thanks again,

@umbramalison

Hi Andrew

I do not quite understand exactly WHAT you’re put off by concerning Proxmox licensing, it’s the same as NethServer, besides that one Nag message…

As said, you do not have to license Proxmox. You even get updates faster.
And the ONLY issue is the one nag message after login.

Proxmox has moved very fast forwards in the last years!

You’re not the first power user who spent days / weeks trying to get their own solution working.
Interestingly, it’s always networking! I have already helped 4 users on this forum stuck with the same / similiar issues as you are having now - NethServer / AD and networking. All 4 a now happy Proxmox users - and DO not regret letting Proxmox take over!

If you still want to try, no probs with that! If you still want to try using Proxmox, I’m still around, you can PM me anytime!

Good Luck!

My 2 cents
Andy

Did you try to ping the IP or only the name?

No, Proxmox didn’t do it. It is only for virtualization.

Thanks for your reply.

It’s not a DNS issue. I perhaps should have covered that. IP address or hostname the results were the same.

Thanks again.

I have a working system, i’m just trying to add a domain controller. There is already enough stuff going on that starting again with another solution just to solve what will probably be a silly mistake on my part is not worth it and isn’t what I am going to do.

I appreciate your help and suggestion to use proxmox, but I’d much prefer to fix this problem and learn from it.

1 Like

Ok so I’m reasonably confident that the problem I have is simply a routing issue.

As I mentioned before I have a firewall which I am keeping (untangle) which was also the router, dhcp and dns.

I’m moving at least dhcp, dns to nethserver which will run behind the firewall.

I’d rather do intervlan routing (some VLAN must not touch internet, untrusted ip cameras etc) on nethserver because being virtualised it can benefit from best network bandwidth. I am also reasonably sure it’s good practice to keep internal traffic away from firewall, certainly not to run internal traffic through it.

The reason why I can’t ping nsdc container is simply because my gateway was set to the firewall.

if i set the gateway to itself (nethserver) then my network can ping nsdc container but can’t ping internet.

if i double nat i can ping it all, but then untangle is a bit of a waste as it can’t see the traffic source/destinations clearly.

So i’m looking for a solution without double nat.

I’ve not spent my usual time researching this bit yet. but I wanted to update you on it.

@Andy_Wismer, I actually also found you had a very similar problem before -> NethServer AD Routing

i’m unsure though that I need to change anything in the nsdc container, I think i just need to manage clients routing tables with either the route to internet, or route to nsdc container.

thanks again, hopefully my issue makes more sense now.

1 Like

I’m not sure, but it could help to take a second network card to nethserver. One is connected as red interface to your firewall, the other as green interface to your local network. Set nethserver as gateway for your clients and the firewall as gateway for nethserver.

Hope this helps.

Thanks Micheal.

I think what you describe is double nat solution which I did try and can verify is a valid solution.

The problem I have with going down that route is that it obfuscates all internal traffic from the firewall. From the firewall you just see one computer doing a lot of things.

Untangle can do a lot more, but it needs to be able to separate the traffic out. I hope that makes sense.

I think I need to learn how to set routes up for clients, but it’s not in the UI. you can add routes in the network section, but these don’t go the clients. So i’m not sure how they are to be used.

Yes you are right.
Can you tell me why the container has to be in a different subnet? Both, nethserver and the container should be connectable from your clients, or not? So I thinks it’s not a security reason, or am I wrong?

Hmm.
I was under the impression that because the domain controller is the keys to the kingdom so to speak that it shouldn’t be accessible to everyone.

By having it in its own vlan I was expecting to then create rules that facilitate authentication and ad only. Eg, maybe wouldn’t let anyone SSH to it.

Can’t recall if I read that in neth documentation or samba wiki…

@umbramalison

Hi

It’s neither Samba nor Nethserver:

Actually, Microsoft always had even guest (readonly) access to NetLogon Share, needed otherwise an admin could not join a PC to the Domain / AD.

My 2 cents
Andy

right, correct you are!

I found the reference

brianwhelton

Mace

brianwhelton Verified ProfessionalAug 12, 2018 at 10:16 PM

Whelton Network Solutions is an IT service provider.

For the DCs to be able to communicate with users PCs and laptop, they just need IP connectivity, so sure, in the same IP segment/VLAN they will work, in another VLAN, as long as IP routing allows communication, it will also work. But just because something will work, doesn’t always make it the best solution for the job in hand.

So the question really is, is it sensible to have a DC, effectively the keys to the kingdom, in the user facing VLAN where any host has full access to it on every available and open port when it’s negligibly easy to put a few barriers in the way for zero hareware/software cost and a tiny amount of time? Layers 1-4 would be effectively unguarded. In my view that’s not an acceptable risk, indeed, no server or other shared resource (such as printers) should be in the same broadcast domain as a user device as the fact it’s shared means it has full access to everything.

Sure you could use Windows firewall or some other vendor host based firewall, but i would like to rely upon them, maybe configure them also as another layer.

By installing the server in a different VLAN you can instantly restrict what ports devices from the user VLAN can communicate with the server on, this also provides a perimeter where should a device in the user VLAN is attempting to get to the server on a port or ports that couldn’t possibly be accidental, you have what I refer to as an IoS, Indicator of Shenanigans.

If a users machine was compromised or someone could join the network via a wireless AP, the effort of getting Domain Admin is a lot easier and could be done without any potential warnings, and once someone is DA they can create new DAs and then they own the environment. If you own the environment, you own everything on it.

I appreciate that is just one persons opinion. I thought it was a well reasoned opinion, such that I later recalled it to be officially in documentation - i do apologise. The human leaks (flight of the navigator!)

Definitely can’t be taking recommendations from Microsoft though right?!

Anyway if that is the issue, then I don’t have to retain the separate VLAN. I do however still believe it be wise to keep management interfaces to a separate VLAN, and CCTV to separate VLAN.

If I don’t resolve these types of issues, how will I ever manage intervlan communication?

I installed the basic firewall too, but I can’t see how to control inter network communication. Got these zones (green, red etc) completely useless to be because I need to categorise based on the network and that’s always going to be 1-1 relationship with any category (colour).

Untangle rules just says wan or not wan, if a particular non-wan then specify the interface for the rule.

I think these are the problem I always face with solutions like Nethserver (and dare I say proxmox) it’s so difficult to provide intuitive interfaces that still let you do everything. I go these routes to make life easier, and then spend more time trying to make them work as they were possibly not intended to. I might just be better off with a bare bones centos/fedora install! but that is a lot of work too :slight_smile:

@umbramalison

Hi

I’d just like to inform you, that I do have 25-30 odd NethServers on the Internet…
Ports 980 AND 9090 are open. Fail2Ban takes care of security VERY well, five years plus on most of these servers, and no one’s successful except in adding to the fail2ban list… :slight_smile:

These ports are open so that users can change their (AD) Password…

I may be even more paranoid than you, but I know my networks are well secured!
(FYI: Fail2Ban is not the only “Gotcha”!)…

I’d still suggest using Proxmox… :slight_smile:
-> Fail2ban is a simple apt install away…

vLans makes a lot of sense, eg one for VoIP, as you say CCTV is also another one, but also for guests (Not just WLan…) or in schools, with Wall sockets protected additionally with vLans…

My 2 cents
Andy

Thanks @Andy_Wismer

How and where are you defining your gateway, routing, ACL, cross interface|network|vlan rules?

all on nethserver?

I’d still suggest using Proxmox… :slight_smile:

thought you might!

@umbramalison

Hi

I define everything to do with access, IPs, etx all on my firewall, an OPNsense box.
I use these as hardware, sometimes as virtual on Proxmox.

DHCP for clients and VPN stuff is also handled by the OPNsense, I use both OpenVPN and IPsec.

DNS is done by OPNsense AND by my NethServer. (OPNsense uses Unbound, but other options are available, like Bind).

This way my NethServer does what NethServer does, and AD. But never issues with networking!

Also if anything goes wrong when testing stuff, and I need to reboot or restore (fast with Proxmox) I STILL have Internet access, to eg look up the problem.

OPNsense is a professional grade firewall, even, hardware available, Enterprise class. Like PFsense, but no licensiing issues!

My 2 cents
Andy

2 posts were split to a new topic: Link the firewall of OPNsense to the AD in Nethserver