Problem configuring NethServer as gateway in Proxmox

Support
,
1

Continuing from:

2

Hello Marko

As I’m a heavy Proxmox user, maybe I can help you with your Proxmox Interface issues.

Is the Proxmox running at home, or at a Provider (hosted)?

Most likely are two issues:
Proxmox prefers a connected NIC, at a hosted environment, usually only one is actually connected (=active).
The second is a misconfiguration of the 2nd NIC in Proxmox, often seen when using DSL


My 2 cents
Andy

3

I’m using it at home on my own server.

During installation only one port is connected to the router. The second I would connect to my switch after installing NS as Gateway.
If I understand right both should be connected during installation?

4

Hi

Not necessarily. The important one is the LAN connection of the Proxmox, the one you’ll use to configure Proxmox. We’ll call this NIC1 (eth0 on the hardware, depending which NICs were connected when, and to where
) Note that Proxmox does not need connection to the gateway when booting, important later on


If your hardware has two interfaces, then it should be easily possible


As I understand it, the second Interface (NIC2) is connected to your Router. Is this a real Internet router (Using some form of NAT), or a Bridge (For cable or DSL)?

This Network shown here is something similiar as you have, it’s running at a friends home. He has a VDSL Router, now running as a DSL-Bridge, meaning the whole PPPoE is passed through to the Firewall (ABHO-OPNsense).

This OPNsense is a VM, running in Proxmox and is configured as the first VM to start up.
The HP Microserver has two NICs, the NIC1 is connected to the LAN using IP 192.168.209.61.
The second NIC in Proxmox is configured as a NIC (NIC2), active but no TCP/IP configuration. This is used to pass PPPoE thru to the Firewall, here OPNsense.

My friend does have a NethServer running. But in your case, think of the OPNsense box as your NethServer. It also has two connections, one to the LAN, the other to the Internet and it is acting as firewall


Note that my friend also uses Synology, but as DiskStorage and Backup for Proxmox.

38
Bildschirmfoto 2020-02-28 um 01.30.38.JPG1142×910 126 KB

How the firewall “sees” it’s NICs (one connected to Proxmox NIC1, the other to Proxmox NIC2).

50
Bildschirmfoto 2020-02-28 um 01.32.50.jpg1104×324 84.9 KB

This is what the Interfaces look like in Proxmox:

49
Bildschirmfoto 2020-02-28 um 01.42.49.JPG2164×418 105 KB

The trick is simply editing the config file directly, using nano or whatever you prefer:

nano /etc/network/interfaces

This is what it should look like:

30
Bildschirmfoto 2020-02-28 um 01.43.30.JPG1248×988 169 KB

And this is what the Proxmox VM allocation looks like for the virtual Firewall (in your case your NethServer).

37
Bildschirmfoto 2020-02-28 um 01.46.37.JPG1204×564 114 KB

You’d need to adapt the used LAN adresses, here 192.168.209.61 to whatever is used in your LAN


Also important: Proxmox uses a LAN Gateway, which is not available when Proxmox is booting. It’s the LAN IP of the OPNsense Firewall (Your Nethserver), which is the first to boot up when Proxmox starts.

Hope this helps
Andy

5

Thank you @Andy_Wismer for your awesome description!

Now I tried to reproduce it on my Server 2


auto lo
iface lo inet loopback

iface enp2s0 inet manual

iface enp3s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.3.200
        netmask 255.255.255.0
        gateway 192.168.3.1
        bridge_ports enp3s0
        bridge_stp off
        bridge_fd 0

auto vmbr1
iface vmbr1 inet static
        bridge_ports enp2s0
        bridgs_stp off
        bridge_fd 0

The Server has internet connection. But I’m bit irritated, that the interface state inside NS is different to your show case. Possibly because I have not yet installed the virtual NS and have not yet connected enp2s0 to my router?

image
image.png1059×137 20.9 KB

My current Architecture:

  1. Internet-/DSL-Router: 192.168.2.1/24

    • currently connected via switch to my dedicated Nethserver on Server 1
    • later planned to connect to enp2s0 of my Server 2 (Promox)

  2. Server 1: dedicated NS-Gateway 192.168.3.1/24 (GREEN/LAN) with DHCP and DNS + 192.168.2.5 (RED/WAN)
    Server 1 shout be substituted by Server 2 with Promox and a virtualised Nethserver as Gateway

  3. all Clients (Diskstation, Workstations, Laptops, Mobile Phones
),connected via switch with Server 1 (dedicated NS): 192.168.3.2-192.168.3.254)

  4. for Installation purposes Server 2 (Promox) with enps20 (not connected) and enp3s0 connected via switch to the gateway. This server is pingable from other clients and has internet connection.

My planned Architecture:

  1. Internet-/DSL-Router: 192.168.2.1
    • connected to enp2s0 of my Server 2 (Promox)
    • connection to Server 1 disconnected
  2. Server 2:
    • enps2s0 with 192.168.2.2 connected to my DSL-Router
    • enp3s0 connected to my switch (LAN-side)
    • virtual NS-Gateway with DHCP and DNS
  3. all Clients (Diskstation, Workstations, Laptops, Mobile Phones
),connected via switch with Server 1 (dedicated NS): 192.168.3.2-192.168.3.254)
  4. Server 2 (Promox) with enps20 (connected to DSl-Router) and enp3s0 (connected to the switch).

The next step should be the installation of my virtual Nethserver as Gateway.

My Question:

  1. Is the current Network Interface configuration and my planned architecture well done or wrong?
  2. Is my assumption correct that before installing the virtual net server, I need to turn off the dedicated (server 1) and connect enp2s0 to the router?
  3. Can I migrate my DHCP-, Firewall- and Webproxy - Configuration from Server 1 to the new virtual Nethserver?

Thank you very much!

ps: Maybe it makes sense if an admin here moves this topic to a new thread.

6

Hi

I’m on the road right now, but will give you feedback in ca. 1 hour


As to the network state in the proxmox interface. Set them all as active & automatic (in the interface), then reboot your proxmox and look again. Should be better.

Andy

7

Hi

I’ve had time to look over your current and planned architecture.

Current is clear and more or less “standard”, using NethServer as a hardware router and server.
Planned: As far as stated ok, but could be optimized with a few more steps


To answer your questions:

  1. see above (and below).
  2. Yes, there can only be one default gateway in a network at any time. Same goes for DHCP servers, MS switches of it’s DHCP Server Service if another active DHCP server on the LAN is detected.
  3. You can “migrate” your config, but I’d suggest having two browsers open, one to the old server, one to the new, and recreate the config by hand. The learning effect is much higher! :slight_smile:
    Another advantage is at the moment, there are not to many rules (yet). This gives you a good chance to document it, and review each rule if really necessary.

At this moment, I’d suggest fix the proxmox like stated in the previous post (active / auto NICs), reboot.
The next logical step would be setting up your new NethServer as firewall inside Proxmox.

For Proxmox (some basics):
Use XFS file system, if you’re not a guru with ZFS/CEPH

Use .qcow2 as disk file (if possible). raw may be a mite faster, but wastes a lot of time & space with backups

Have a NAS (I use Synology, as seen above) for your backups. I use shares like:

  • PVE_Container
  • PVE_ContainerTemplate
  • PVE_DiskImage
  • PVE_ISOImage
  • PVE_VZDumpBackup

on the NAS, inside Proxmox they’re mounted as Container, ContainerTemplate (all without the PVE_) using NFS.

40
Bildschirmfoto 2020-02-28 um 17.15.40.JPG2812×576 215 KB

59
Bildschirmfoto 2020-02-28 um 17.15.59.JPG1202×452 48.2 KB

Two images showing the NAS attached to Proxmox with NFS.

Here’s the Synology side of things (Sry this is in german):

40
Bildschirmfoto 2020-02-28 um 17.18.40.JPG1722×1136 133 KB

52
Bildschirmfoto 2020-02-28 um 17.18.52.JPG1108×960 128 KB

Using Proxmox as a possibility for Backup and fast recovery is one of your primary goals - like for most of us using Proxmox
 :slight_smile:

Storage & Backups are defined in Proxmox at the Datacenter level.
Here is a sample of backups:
Note: The amount of backups (generations) is defined in the storage register (Max Backups) shown above. I use 7 generations. You can define the same storage twice (two different names), one with 7 generations, one with say 3, for less critical stuff.

13
Bildschirmfoto 2020-02-28 um 17.16.13.JPG2254×316 67.7 KB

Play with the backups (Using your freshly installed NethServer as test candidate), try a manually triggered one, and see how fast things go. restore it with a different VMID (Proxmox VM Identifier), to see that everything works! (Don’t have both running at the same time, you’ll get a classical IP conflict!)

Optimizing the “planned” state of your home network


If you anyway have two hardware servers around, and that’s ok, why not use BOTH as proxmox, and run everything inside of Proxmox?

Once you’ve seen how fast and reliable a live backup works with Proxmox and an external NAS, you’ll never feel ok running anything native on hardware, with no independent option to make a backup or snapshot ANY time you want to! Running anything on native on hardware means that system has to provide a reliable backup and restore option.

But that usually also means part of that system must be up and running - to be able to start a restore

Proxmox backups can start on any KVM capable hardware, even some Synologys provide KVM

That’s flexibility!

Andy

8

But this is exactly my current hurdle.
If I try to activate enp2s0, I would enter:
IP: 192.168.2.2/24
Gateway: 192.168.2.1 ← at this point comes a message, that enp3s0 has already defined a gateway

And how should I activate vmbr1? I would like to assign the IP 192.168.3.1 for the GREEN-Side of the future Nethserver Firewall.

9

Don’t put ANY IP information on that NIC.

A NIC (Network Interface Card) can and usually nowadays transports TCP/IP, so everyone thinks I need to put in an IP adress, or it’ll get one from DHCP.

However, think back a few years, or maybe ten, twenty or thirty


There was a time most NICs worldwide used NetBEUI (Microsoft pre TCP/IP Network Protocoll) or IPX (Novell). Times of Windows 95, Windows 98 or Win98SE
 Mainframes were using even baser stuff


We want to use this Interface for the Firewall’s Internet Connection. Proxmox just “passes” the Message, without “touching” or changing it at all - it IS called a Bridge (VMBR) after all!

See my example from 2-3 posts earlier, vmbr1 (as mine is called) has no tcp/ip info.

You can only set it to auto in the Interface, after that it NEEDs to be connected (To anything, even an empty powered switch) - and a reboot - to show up as active. That leaves you the liberty to switch the current gateway / Internet Connection only when you’re ready


:slight_smile:

See here:

28
Bildschirmfoto 2020-02-28 um 17.53.28.JPG1202×598 43 KB

13
Bildschirmfoto 2020-02-28 um 17.53.13.JPG2246×1024 148 KB

10

set “Autostart”
 that’s really tooo easy. Normally I do that, but in your 1st. screenshot Autostart is remarked as OFF. Thats why


image
image.png1081×152 21.6 KB

Now I will try to install nethserver


11

Good!

Make a note of the NICs (vmbr0 should be the first one listed (by the NethServer installation programm) , that would point to your LAN for NethServer, the other one (vmbr1) would become your “RED” WAN.

Your commenting of the NICs is good!

:slight_smile:

Good installing!

:slight_smile:

12

PS:

Small Afterthought:

It’s always a good idea to plan and document your Network.
Since I can’t post an Excel file here, I can give you the basics:

This is a generic network, as a planning base. The default gateway will always be 1. The first printer will have 31. Nethserver will have 20, unless running as gateway/firewall, then it uses 1 (or even both). At home, I don’t have any POS (Point of Sales) devices, but this is a generic IP scheme, to be adapted as needed.

57
Bildschirmfoto 2020-02-28 um 18.37.57.JPG1578×1000 353 KB

Later, it looks like this. (MAC-Adr are removed, these are for DHCP reservations).

14
Bildschirmfoto 2020-02-28 um 18.43.14.JPG1508×928 218 KB

This document is my “Master” when it comes to IPs/MAC-Adresses

A copy / paste from NethServers DHCP Page works well
 :slight_smile:

1 Like
13

Too late :slight_smile:
I installed Nethserver now: the NIC inside the installation script was completely different

image
image718×322 41 KB

The IP 192.168.2.100 came via DHCP from my router
Now I cannot configure my network inside nethserver
only one interface

image
image839×170 15.2 KB

that’s all.
I will try again with the newly arranged bridges in Proxmox.

14

Sh*t happens
 :slight_smile:
But Proxmox makes a second try quick


And you saw that our configuration of Proxmox for the Internet side was correct: Your internet router gave that IP, Proxmox didn’t touch that!

:slight_smile:

15

Actually, for the installation, I’d temporarily deactivate the RED Interface in Proxmox.
-> Have only the one pointing to the LAN available and active for the installation.
Less errors!

16

Nethserver VW is not respondig for shut down and purging
reboot of the Proxmox needs a long time: 3 Jobs are hanging

1 of 3 "A job is running for PVE API Deamon (x min x sec / y min y sec)
2 of 3 "A job is running for Self Monitoring 
SMART (x min x sec / y min y sec)
3 of 3 "A job is running for PVE status Deamon (x min x sec / y min y sec)

After a while (time out)

"A job is running for Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling (x min x sec / y min y sec)

17

That may indicate a hardware problem

Disk or controller

18

ok, I will install Proxmox again
 Last installation attempts I didn’t get such messages

UFF


19

I’d check the system / bios (HP Server?) if the RAID says it’s OK or not


20

I’m using ZOTAC ZBOX-CI327NANO, no RAID only SSD.