NethServer Version: NS8
Module: Internal OpenLDAP / guacamole / wordpress / Lets Encrypt
This is a VM of NS8 on TrueNAS. For some reason I am hitting 96% mem usage and 96% swap usage.
Any thoughts what is causing it?
Thanks,
-SF-
Hi @Shadowfire
I suspect TrueNAS…
What specific version are you running? Scale? / Core?
My 2 cents
Andy
TrueNAS: TrueNAS-SCALE-23.10.2
VM: Debian GNU/Linux 12 (bookworm)
I was hoping that would not be the issue as the TrueNAS has ample cpu, mem and hdd space.
I don’t normally use TrueNAS for VM purposes, but I saw quite a few people running VM successfully on there setups, so I figured I would try on my side.
-SF-
TrueNAS is known for Storage, certainly not for running VMs, although it can do that. But how good?
I am running Debian 12.5 easily on a low powered Proxmox box… 
I am also running the same setup on a much larger, faster HPE server equipped with NVMEs…
Low powered:
My 2 cents
Andy
Yes, TrueNAS is definitely known for storage. I am very happy using it for the destination of backups.
Curious, If your able to elaborate… how many items are running on that NS8 deployment?
Do you have or run TrueNAS under your Proxmox setups. If you do. What is your best practice for running it on Proxmox and how do you handle the storage for TrueNAS?
Thanks,
-SF-
I have used TrueNAS in the past for my clients, but at the time wasn’t too happy with the performance. I only ever used TrueNAS as Storage (eg generic NAS), never for anything Proxmox specific, except maybe as additional Backup to NAS after PBS Backup.
All my Proxmox setups use PBS (Proxmox Backup Server) for VM backups, as I still have to see anything beating PBS. And I run PBS on a very low powered Odroid box (H3+), costing without disks about 300 USD!. I use 2x 16 TB Seagate Exos disks for PBS, and a small NVME (250 GB) for PBS system.
Running TrueNAS inside Proxmox can work, but is IMHO not really worth the effort. I get much better performance running OpenMediaVault if I need a VM NAS (Which I do occasionally use!).
Hardware summary
Disks used
ZFS
Compression / Deduplication
And here is OpenMediaVault, running on almost identical hardware.
This is again my small Home Proxmox:
This uses a max of 120W, normal use much less (less heat / power costs!)
My 2 cents
Andy
On the disk usage pic you show - it looks like your main proxmox drive is sitting on 1x2TB Samsung SSD 990 Pro drive.
I am guessing the 2x16TB zpool has the 2TB snapshot images backed up on it?
-SF-
Not quite, but close…
The PBS with 2x 16TB disks is one at a clients, not mine at home. But it’s also at my clients home, for offsite backups. Their main PBS at the office is the exactly identical box / hardware. It easily handles simultaneous backups from 2 big Proxmox PVE boxes (HPE / Supermicro 19").
The main NethServer here is 1.4 TB big. A normal - incremental backup takes about 2 miinutes for this VM!
On my Home Proxmox the main local storage is on the 2 TB NVME, as you presume. But I also use “Shared Storage” (OpenMediaVault NAS running on an Odroid H3+), connected by a dedcated 2.5 GB/S network.
Depends if I need volume or speed…
My 2 cents
Andy
Ameridroid has these boxes in the US, I’ve ordered also from them!
@Andy_Wismer
FYI: Last saturday I ordered a new Odroid H4+ directly at Hardkernel. Yesterday I already received confirmation from the transporter that this package will arrive on June 3rd (to the Netherlands), so that will be very fast. There is even a new H4 Ultra.
2 Likes
I’ll probably order 1-2 H4 Ultras this weekend…
The H4+ / H4 Ultra can only handle 48 GB RAM vs 64 GB RAM of the H3+.
Makes the H4+ / H4 Ultra more destined for Storage, the H3+ has use as low powered Proxmox PVE.
See here for full details:
2 Likes
@Andy_Wismer if the VM is hitting its memory limits, how should that be caused by the hypervisor?
By the way, both Truenas Scale and proxmox are using the same underlying kvm technology in the Linux kernel for virtualization. So I would expect similar behavior.
I think the reason is within the NS8 VM, unless it is configured with insufficient memory on the Truenas side. @Shadowfire how much memory and swap did you assign to the NS8 VN?
So when I built the VM, I install debain 12 as the main OS for the VM, I setup the memory at 16 GB because my TrueNAS system has 96GB in all.
I then ran the installation procedure on the debain OS. I believe it built the size of the swap during installation with install.sh script.
Once I saw the load on mem and swap I bumped the VM memory for NS8 VM to 32GB because I saw it had a load on mem and swap.
Then I posted since I was unable to think of anything else to do to help VM.
-SF-
Hi
by this logic, a Ferrari and a Volkswagen Beetle are both using the same underlying technology for motorization (A combustion engine), both have the same 4 wheel basis, and the motor is in the back.
So you would expect similiar behaviour?
TrueNAS builds Storage. They only hopped late on the virtualization train, after more or less dumping their BSD basis.
Proxmox has been building HyperVisors long, long before.
And yes, I have been using TrueNAS, and it’s predecessor FreeNAS since it was forked from m0n0wall. It’s quite OK as Storage, provided you don’t skimp on the hardware or actually use their hardware.
But neither as Storage nor in the Hypervisor scene have I seen them being real bringers of new ideas, solutions or real movers. They’re there, that’s about it. They do contribute to ZFS/Linux, but they’re not the top contributer there either. They were once good in pushing BSD, but those times are long past.
As to the RAM usage. I have running NS8, fully productive, for paying clients, running with Debian 12.5 on Proxmox (Very different Hypervisor Power & Speed). They all use either 8 or 16 GB RAM at the moment. My clients are in the range of 2-50 users.
So it’s not the RAM of the VM configuration on TrueNAS nor RAM on TrueNAS hardware itself…
The used file system can also have issues, I use BtrFS now for NS8 / Debian, after starting with XFS (NS7 uses XFS).
What FS are you using in Debian12?
And just for the record:
I do admit being a fan of Proxmox, they made me move on from VMWare, which I’ve been using since before 2000! I moved on 2014-2015 after testing Proxmox. I have used XEN also in the past, but not really satisfied with it either - then Citrix bought Xen. 
I used to support TrueNAS a lot more, while it was BSD based. Since it became a more or less “off the mill” Linux wannabe EL (It still isn’t, IMHO), I don’t see any real advantage in TrueNAS…
Note:
I am neither paid by Proxmox nor by any other Open Source (or ANY) company, entity or anything, in case anyone has ever wondered.
→ If I ever endorse anything, it’s only because that has fulfilled my (high) expectations in it.
In open source, I do specifically endorse:
- NethServer
- Proxmox PVE & PBS
- OPNsense
- Home Assistant
but also:
- FOG Project
and others…
My 2 cents
Andy