Proxmox and vlan aware?

Chat
#1

Hello everyone, who uses proxmox I would have to ask:

I activated in a cluster of 3 nodes the vlan aware option, an option that as I read allows the management of VM vlans between them on the cluster structure. I tried and everything works well only when the vlan tags e.g. 200 or other value are only and exclusively inserted into the VMs of the same node. If I try to give the same tag to another node in the same cluster it doesn’t work. I would like to point out that this option (unless I’m wrong or you were misinformed) should work regardless of any switch. I just have a doubt, having a DLINK des 1100-24 switch and not managed, you need to have a switch that supports vlan even if as written before you do not need to configure anything? Thank you

#2

@france

Hi

I did have a three node cluster (for a client) with:

  • LAN (Bonded)
  • Storage
  • Cluster

I created a vmbr99, and a vLAN on all nodes with vlan99.
The vlan99 is for nodes NOT on the normal LAN to be able to communicate with each other, a sort of DMZ for certain hosts.
Because of Live Migration, the vLAN had to be available on all nodes.

This was with Proxmox 6.x - and it was working.

I haven’t tried it yet in Proxmox 7.x…

My 2 cents
Andy

#3

Thanks Andy, but what I wanted to ask you is enough to activate vlan aware eg. on the 3 VMBR0 and after you can use the vlan tags on the VMs I want to activate? Is something missing? Because at the moment they only work if the VMs are on the same node, but if I try to activate a tag on another VM e.g. Node1 doesn’t work or you don’t see all VMs tagged e.g. 200.

#4

No, this is not enough.

Somewhere you DO need to specify the vLAN itself.

What seems to work (If my memory is not playing tricks on me) is set a vlan on the NIC.
Then make a Bridge of the vlan → vmbr99
This can be done on ALL nodes.
VMs can use the vmbr99 Bridge for primary or secondary connectivita (2nd LAN)…

Bildschirmfoto 2021-11-08 um 14.39.02
Bildschirmfoto 2021-11-08 um 14.39.021206×570 82.9 KB

Bildschirmfoto 2021-11-08 um 14.39.34
Bildschirmfoto 2021-11-08 um 14.39.341202×510 50.8 KB

A managed switch is NOT needed.
Last year I did this at home, without having a managed switch (I have one now!) - just so my VM OPNsense could get Internet directly from my OPNsense firewall, also equipped with a vlan99 (With DHCP!) as a pseudo Internet Provider…

Hope this helps!

My 2 cents
Andy

Docs:

https://pve.proxmox.com/wiki/Network_Configuration#_vlan_802_1q

From the Proxmox forum, but with good infos…

This one’s in german, but Deepl translation ought to help:

All three have good infos

#5

Mhmhm, I get a headache. I should study well what you showed me. So I understand that in addition to the option of enabling vlans is necessary to create bridge and linux vlan in order to make everything work. I had watched a video explaining how to do it, but they had little focused on the fact that to make everything work they had configured gns3 to virtualize the network.

1 Like
#6

Andy, I performed a test:

I created a vlan on my pfsense on interface > Vlans with 200 tags

I created and assigned a network class e.g. 192.168.4.2 /24 to the new interface that refers to vlan 200.

Always on the same node I gave the 200 tag to a VM, and the pfsense with the VM can be seen. The fact remains that in order to manage everything on the entire cluster, from as you wrote to me you need to create bridges and linux vlan to allow the cluster to rotate like frame vlan. Correct?

vlan1

vlan2
vlan21204×470 59.8 KB

vlan3
vlan31880×400 42.8 KB

Moreover, even if the operation is partial because the creation of node interfaces is missing, can it be said that this is functional but limited only to one node?

#7

Hi

Yes

You do not need vLANs to manage all nodes in a cluster:

Bildschirmfoto 2021-11-08 um 16.01.28
Bildschirmfoto 2021-11-08 um 16.01.282866×1112 315 KB

All nodes in a cluster are equal - there is NO “master”.
As such, you can login to the WebGUI and administrate ANY Proxmox in your cluster!

In the image, i am logged in to PVE3, but I am administrating PVE7… :slight_smile:

1 Like
#8

Yes Andy I already know this, there is no precise hierarchy, all the nodes are master! Thank you as always.

#9

hi Andy, I tried to put your vlan aware indications into practice. I double-checked everything but the VMs don’t want to communicate between one node and the other. can I double check other parameters? Thanks as always

#10

Hi

Out of office now, but I’ll check and reply later…

My 2 cents
Andy

1 Like
#11

God forbid! I just wanted to inform you that I have tried your configuration which in my opinion seems correct. Thanks as always .

#12

Hi Andy, did you manage to get more info? Thank you

#13

@france

Hi

I did try it on 3 machines. Configured it on OPNsense and 2 Proxmox servers.

Using numbered or unnumbered links (No IP on the vmbr99) I wasn’t able to connect.

A managed switch or unmanaged makes no difference, only “tagging” a switch ports can make a difference.
But vLANs will work with unmanaged switches, eg from Host 1 to Host 2…

Needs more research, but I assume it has to do with the change in NICs that came with Debian 11…

My 2 cents
Andy

Bildschirmfoto 2021-11-10 um 14.25.37
Bildschirmfoto 2021-11-10 um 14.25.372868×1124 311 KB

#14

So the problem remains limited or vlan only on the same way. Eli that I’ve seen around and that many users use bonding that in my opinion I don’t understand the context. On the proxmox documentation refer to the numbering e.g. ens18 in ens18.100 vlan

#15

vLans in Proxmox can either be set using NAME.99 or ens01.99
Using these options usually fill out the vLAN field correctly.

However - I tried more or less all options using VMs or Proxmox and I can’t get a single connection, when a few months ago it was working… (I need this mostly for my test-OPNsense and VM OPNsense, sometimes a NethServer I’m testing with VPN…).

I will get it working, Proxmox yesterday and today delivered alltogether 3 sets of updates (For both Proxmox PVE and PBS)…
Either Proxmox/Debian has a bug (Will be fixed) or I have a hardware issue or something similiar…

:slight_smile:

My 2 cents
Andy

#16

Yes, I thought about this too, can bugs? I looked and searched on the network but it seems that the operations to be done are only these, to make it short even only the vmbr0 that you use for lan header with the vlan aware enabled flag should pass all the vlans and as we have seen it works, but only on the same node. It becomes a problem when vlan aware is activated on the nodes remaining in the cluster and on the same e.g. vmbr0, VMs will not be able to communicate with other vms of the nodes of the same cluster.

However, a few hours ago I updated the system.

#17

The larger the company / institution, the more likely vLANs are in use.
Also a lot of small users / labs use vLANs, simply as their switch doesn’t have enough slots…

:slight_smile:

#18

@france

Hi

I can confirm my vLANs at home are working.
For testing I have an existing vLAN on my OPNsense, connected to my Home-LAN as vLan99.
This provides DHCP for 192.168.29.0/24…
This was working before - with Proxmox also.

Bildschirmfoto 2021-11-10 um 23.23.54
Bildschirmfoto 2021-11-10 um 23.23.542844×1456 243 KB

I configured an additional vLAN on my MACbooks Thunderbolt Ethernet:

Bildschirmfoto 2021-11-10 um 23.23.25
Bildschirmfoto 2021-11-10 um 23.23.251526×1114 175 KB

As you can see, my Macbooks vLAN get’s a correct IP via DHCP (192.168.29.0/24), my normal LAN at home uses 192.168.31.0/24, also DHCP.

My Macbook allows me to “isolate” the vLAN issue to Proxmox alone.

Now to see if I can configure any form of vLAN on Proxmox that get’s a correct IP via vLAN99…

Note: My Macbook is hooked up to the unmanaged switch, same as my OPNsense. As both LAN and vLAN reach my Macbook correctly (Both have DHCP and a correct IP for that network) I can assume the Issue is NOT with my Hardware or Switches. I get the IP also on my managed switch, on any “untagged” port as is to be expected.

Removed on my Proxmox3 all vLAN configurations, and tried a configuration of a VM with the correct vLAN tag vlan99 - no connection… So it seems that somewhere inside Proxmoxes internal networking the vLAN Tag is lost.

Bildschirmfoto 2021-11-10 um 23.47.59
Bildschirmfoto 2021-11-10 um 23.47.591166×606 73.6 KB

More tests needed…

It seems even inside a Proxmox, Proxmox can’t ping anything anything, not even a VM also using the same vLAN with correct networking applied.

I also was able to test today on a clients Proxmox 6.x, there the same configuration works. ???

More research needed…

My 2 cents
Andy

1 Like
#19

Hi Andy, in fact from what you wrote to me I had no doubt that everything worked with the version of cluster 6.0, unlike the 7 that seems to want to work only if the configurations of the vlan remain on the same node and are not rotated on the other nodes. From your configuration that is very similar to mine, I attach screenshots that show how it works only at the single node level.

at
at1228×499 97.5 KB

Schermata 2021-11-11 alle 06.33.10
fw
fw2196×448 94.7 KB

As you can see, on the pfsense the vlan 200 cond dhcp is active, and in the cluster test environment a hundreds that you receive from the dhcp of the pfsense tagged 200 the ip parameters. He can ping, in short, it also works correctly with regard to routing to LAN and the internet. BUT ONLY ON A KNOT !!
I can’t understand, maybe a bug? As soon as you have information about it, let me know Andy because it is important. Thanks Anyway, for the tests you carried out.

1 Like
Nethserver samba issue
#20

@france

Hi

I’ve posted the issue on Proxmox Forum:

There are others with a same / similiar issue…

Andy