Hi Andrea
I don’t have much time this week to testing, so i’m asking if you and the other guys on the forum and nethesis team @davidep @giacomo have found which is the best procedure/configuration for install Nethsecurity on Proxmox VE on ZFS Storage?
Perhaps it would also be useful for everyone once the procedure for zfs has been found, to update the official nethsecurity documentation.
I’ve never tried NethSecurity on proxmox. Let’s if @francio87 has some ideas on the issue.
What can I say, it that I develop the system on KVM with an underlying ext4 system and I never had this kind of issue even if the machine has been killed.
So it seems to me really a problem of the virtualization system rather then NethSecurity itself.
I have conducted some tests on an old Proxmox 7.2 node with ZFS storage, where I have a VM for my home assistant.
root@pvens8:~# pveversion -v
proxmox-ve: 7.2-1 (running kernel: 5.15.64-1-pve)
pve-manager: 7.2-11 (running version: 7.2-11/b76d3178)
pve-kernel-5.15: 7.2-13
pve-kernel-helper: 7.2-13
...
...
At the moment, I can’t upgrade Proxmox to version 8.x to make the test on the new version, however, the creation of the machine and the import process are successful, and it starts up correctly, maybe it’s just an issue of Proxmox 8.x
(The other nodes that I have available do not use ZFS, but rather LVM on hardware RAID controller.)
This is the history of the proxmox node:
324 qm create 401 --name "Nsec-test" --ostype l26 --cores 1 --memory 1024 --net0 virtio,bridge=vmbr0,firewall=0 --net1 virtio,bridge=vmbr1,firewall=0 --scsihw virtio-scsi-pci
325 wget "https://updates.nethsecurity.nethserver.org/dev/8-23.05.3-ns.0.0.2-beta2-33-g10180b4/targets/x86/64/nethsecurity-8-23.05.3-ns.0.0.2-beta2-33-g10180b4-x86-64-generic-squashfs-combined-efi.img.gz"
326 gunzip nethsecurity-8-23.05.3-ns.0.0.2-beta2-33-g10180b4-x86-64-generic-squashfs-combined-efi.img.gz
327 ls
328 qm importdisk 401 nethsecurity-8-23.05.3-ns.0.0.2-beta2-33-g10180b4-x86-64-generic-squashfs-combined-efi.img local-zfs
329 qm set 401 --scsi0 "local-zfs:vm-401-disk-0"
330 qm set 401 --boot order=scsi0
331 qm start 401
The machine start without issue :
Well, in the end I did what I usually advise against … I performed an in-place upgrade of Proxmox to version 8.1.10 
The old VM started correctly and also created a new one, and this one also had no problem.
root@pvens8:~# pveversion
pve-manager/8.1.10/4b06efb5db453f29 (running kernel: 6.5.13-3-pve)
qm create 501 --name "Nsec-post-upgrade" --ostype l26 --cores 1 --memory 1024 --net0 virtio,bridge=vmbr0,firewall=0 --net1 virtio,bridge=vmbr1,firewall=0 --scsihw virtio-scsi-pci
qm importdisk 501 nethsecurity-8-23.05.3-ns.0.0.2-beta2-33-g10180b4-x86-64-generic-squashfs-combined-efi.img local-zfs
qm set 501 --scsi0 "local-zfs:vm-501-disk-0"
qm set 501 --boot order=scsi0
qm start 501
Hi @francio87
I create the vm with this command:
qm create 401 --name "fw01" --ostype l26 --cores 1 --memory 1024 --net0 virtio,bridge=vmbr0,firewall=0 --net1 virtio,bridge=vmbr1,firewall=0 --scsihw virtio-scsi-pci
wget "https://updates.nethsecurity.nethserver.org/dev/8-23.05.3-ns.0.0.2-beta2-33-g10180b4/targets/x86/64/nethsecurity-8-23.05.3-ns.0.0.2-beta2-33-g10180b4-x86-64-generic-squashfs-combined-efi.img.gz"
gunzip nethsecurity-8-23.05.3-ns.0.0.2-beta2-33-g10180b4-x86-64-generic-squashfs-combined-efi.img.gz
qm importdisk 401 nethsecurity-8-23.05.3-ns.0.0.2-beta2-33-g10180b4-x86-64-generic-squashfs-combined-efi.img local-zfs
qm set 401 --scsi0 "local-zfs:vm-401-disk-0"
qm set 401 --boot order=scsi0
qm start 401
But when i start same problem:
P.S i noticed that you download a different version of the image:
wget "https://updates.nethsecurity.nethserver.org/dev/8-23.05.3-ns.0.0.2-beta2-33-g10180b4/targets/x86/64/nethsecurity-8-23.05.3-ns.0.0.2-beta2-33-g10180b4-x86-64-generic-squashfs-combined-efi.img.gz"
On the official nethsecurity documentation:
wget 'https://updates.nethsecurity.nethserver.org/stable/8-23.05.2-ns.0.0.2-beta2/targets/x86/64/nethsecurity-8-23.05.2-ns.0.0.2-beta2-x86-64-generic-squashfs-combined-efi.img.gz'
This is the version of the documentation:
nethsecurity-8-23.05.2-ns.0.0.2-beta2-33
This is the version you used:
nethsecurity-8-23.05.3-ns.0.0.2-beta2-33
Anyway i try with both image but still the error “No bootable device”
This is my proxmox version:
root@pve01:~# pveversion -v
proxmox-ve: 8.1.0 (running kernel: 6.5.13-1-pve)
pve-manager: 8.1.4 (running version: 8.1.4/ec5affc9e41f1d79)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.5.13-1-pve-signed: 6.5.13-1
proxmox-kernel-6.5: 6.5.13-1
proxmox-kernel-6.5.11-4-pve-signed: 6.5.11-4
P.S i noticed that you download a different version of the image:
I usually just use the latest dev image avaible here > Download — NethSecurity documentation
Regarding the issues with Proxmox, I don’t have many other ideas. Have you perhaps tried installing it on another node?
The steps you have followed seem correct to me
No but @sarz4fun have done many test with nethsecurity on proxmox ve with zfs, and if i remember correctly he has the same problem i have, and it fix doing attach 2 disk on the vm.
But in your case @francio87 as you written above, you followed the procedure as described on the nethsecurity official documentation, and the vm booted normally…
Hi
With ZFS, a common issue is simply a “not quite enough” CPU, RAM issues. But mainly spped issues (timeout issues) due to a low powered CPU / RAM / hardware.
On what hardware is your Proxmox running?
If you say 32 GB RAM, quad core, those are also probably the issues.
And - a quad core on a 3 GHz CPU is a different animal from a 1 GHz CPU.
Take into account IO issues…
Running on NVME will probably have no issues, on SSD or HD you will probably heve issues.
My 2 cents
Andy
Hi install ProxmoxVE inside Virtualbox and install on ZFS (Raid1) and follow the procedure as described on nethsecurity documentation it boot up at 1st try:
UPDATE
As written in previous post above, i do the command to import disk two times, and so i have two scsi device, scsi0(not boot) and scsi1(boot nethsec correctly), so i procede with detach scsi0 and remove it and then detach scsi1 and re-add as scsi0 device, so this is possible solution for who have the same problem.
Thanks 
