Hello folks,
i’m testing the latest release of Nethserver to replace my pfsense router, but i’ve got an issue with the pppoe speed.
Let me explain, atm i’ve both nethserver and pfsense virtualized on a proxmox host, i’ve assigned 2 virtual bridge, one with the connection coming from my fritzbox and the other going to the switch for the green interface.
Everything looks fine, but the internet speed seems to be stuck at around 140Mbit/s (more or less).
It’s just a vm with 6 core (xeon 2ghz under the hood) and 4gb ram, with virtio network card.
The same machine don’t have this issue with pfsense.
If i turn on the QoS (just with the default high and low class no additional rules), every speed get lower, around 110mbit after a spike of 120/130, but with a torrent download after a while get capped to max 150Mbit/s.
if i let my fritzbox do the pppoe, setting the red interface as static private ip, every problem disapper, i can reach the max cap of the bandwith ( around 190Mbit/s ), both with QoS enabled or Disabled.
Anyone can give me an hint on what could be the reason to this performance drop ? Thanks
AFAIK QoS is meant to don’t boost up everything, but keep data flow smoothly according to rules.
And also QoS chunks some CPU power, even PPPoE does, also adding a bit of network overhead due to encapsulation.
How many vCores does your Proxmox can use?
Does the pCore increase occupation when using PPPoE or QoS? Are you reserving any CPU share to this host?
I’ve noticed the pppoe process taking up to 75% of a single core while speedtest was running, i’ve increased from 6 to 12 vcore but the process keep using only a single core it seems.
The host has 2 CPU Intel Xeon E5-2620 @ 2.00GHz, with a total of 24 vcore, it’s a homelab server, and basically, it’s idle while testing ( total cpu usage of the host around 2% while not speedtesting ), the only other vm on atm it’s the other “router” with pfsense, wich not show the pppoe issue.
Because it’s not multithreaded, so load cannot be split among cores. At least, AFAIK.
Maybe Fritz!Box RISC SoC handles load better for that task.
Have public IP address on RED is quite nice for some things, but there’s no downside into using PPPoE done by Fritz!Box and do a nice port forwarding.
Does your connection has a public static ip?
Also, distro have different goals.
PfSense is pure networking oriented. Nethserver is quite closer to a multifunction server. Which can also be network gateway with some power-features.
idd, that’s why i wanted to test nethserver, and use it as replacement for gateway ad and mail, but atm i guess i’ve to stick with pfsense until this problem is fixed.
Ye, but i was testing NS not just for me but, for my customers, for those small business that now have 3/4 phisical appliance/server (router, utm, dc ), i was planning to replace with a single server with NS installed bare metal, to act as router, gateway, mail server and so on.
I know it’s a single point of failure, but sometimes, for example on a small office with few client they don’t wanna spend money on multiple appliance / server
Ye, i had that issue back in time, but if i remember right it was a totally different issue, i mean, it took ages to just open the login screen, i guess we’re facing here another type of issue, related to pppoe, the offload issue present on pfsense, where higly noticeable, here i see the issue only if my ISP provide > 150Mbit of bandwith.
Ofc, if i let my ISP router do the PPPoE the problem disapper, and the red eth interface configured as static private ip, can reach 190Mbit/s, no matter if QoS it’s enabled or not.
Changing the interface from VirtIO to E1000, didn’t make noticeable difference, still the PPPoE process eat like 80/90% of 1 vCPU.
Yes, GRO and GSO can be disabled, using the ethtool command or the relevant configuration files.
An example command to make settings permanent on NethServer could be (change enpXXX):
db networks setprop enpXXX ethtool_opts "\"-K \${DEVICE} tso off\""
Experiment with “tso”, “gro” and “gso”. You could set them all.
So, i’ve switch back the eth0 to pppoe and used ethtool to disable hardware offload, but the problem still exist.
it doesn’t matter if the tso gro or gso is enable or disabled in my case, the pppoe process eats up to 90% of a single core if i reach around 150Mbit/s.
as you can see i can barely hit 6MB/s, and the core skyrocket
The tests were made on a clean installation, with only 1 client connected, since the pppoe process seems to eat a lot of cpu resources, i guess that the speed performance will be even worst in a real world scenario with a lot of services and clients connected
Maybe the PPPoE process remain stuck into one core and it’s not so efficient in managing low frequencies?
This could explain the lowering performance into a smaller/less powerful device like the APU2e4.
Fritzbox AFAIK they use ARM SoC, so that’s a whole another story. @filippo_carletti there’s any available way to switch for a multithread/core PPPoE Encapsulation process?