Install Nethserver on CentOS 7 armhf?

[fausp (fpausp)]

I run Proxmox 6.4-4 on a Raspberry Pi 4 (8GB) and would like to know if it is possible to install Nethserver on it?

I have installed a CentOS 7 Container made from rootfs.tar.xz

# CentOS 7
root@pimox:~# mkdir -p /opt/lxc-templates/centos-7
root@pimox:~# cd /opt/lxc-templates/centos-7
root@pimox:~# wget https://us.images.linuxcontainers.org/images/centos/7/armhf/default/20210530_07:08/rootfs.tar.xz
root@pimox:~# tar xvf rootfs.tar.xz
root@pimox:~# rm rootfs.tar.xz
root@pimox:~# tar cvfz /var/lib/vz/template/cache/centos_7_armhf_2021.05.30.tar.gz *

You can find the howto (in german) to install proxmox on a raspberry pi here

[mark_nl (Mark Verlinde)]

hi @fausp have to investigate was not aware Proxmox can run on a RPI.

This being said; I use armv7hf (arm32-bit) VM’s using qemu/libvirt on a Raspberry PI aarch64 (arm64-bit) for a while now…
From this experience can tell the virtualization for arm-32bit on arm-64bit is at low cost (does not take a lot of resources to run a 32bit vm on 64bit), as a matter of fact it still runs faster than virtualization on x86_64 throwing 24 cores at it…

Thanx for the pointer

[Andy_Wismer (André Wismer)]

@fausp
@mark_nl

Hi

Also Thanks for the pointer!
As almost always in virtualization: The greatest performance hit is alwas having to “emulate” a different CPU. Trying for example to emulate a PowerPC CPU on an Intel compatible platform was horribly slow. The otherway around was a bit faster.
Also emulating a RISC CPU was very slow.

But running native ARM stuff on a Raspberry (With ARM!) - a neat idea!
Needs more testing for me to define limits, really usable combinations, etc, but still a good idea!.

My 2 cents
Andy

[fausp (fpausp)]

Hi folks,

This is also brand new for me too, lets see what benefit we can get.
Would love to see a howto install Nethserver under Proxmox-ARM64…

[mark_nl (Mark Verlinde)]

Setting up an arm32 bit VM is not as straight forward as arm64 bit or other architectures.
Core cause of this linux-arm 32bit originates from the embedded space which lack’s an unified boot loader ( such as grub(2) ) . However quite recently there have been a lot of improvements in the arm-linux-kernel in this area most notably at the arm-linux-efi stub…

With this we still not out of the woods to boot an centos-7-armv7hl VM because of the implementation of secure boot in elX. It’s a long story: it boils down to the arm-kernel which is not well suited to be loaded as an portable executable (PE) needed to chain-load signed (trusted) binaries.

About an year ago had a proof of concept using a close to mainline build of grub2. Did this for mageia with a little bit of work this can be ported to Centos el7/8 .

To boot a arm32 bit VM using (grub2) uefi as an bootloader the host needs qemu-system-arm, qemu-efi-arm and ofcuase an image with grub2-efi or ISO …

I suggest to try one of the fedora arm images from here: https://www.kraxel.org/repos/images/
It are qcow images do not know Proxmox can handle this virtual-disk format.

[Andy_Wismer (André Wismer)]

@mark_nl

Hi Mark

Proxmox can handle the format .qcow2 out of the box, it’s used as standard eg on NFS, which has no built in Snapshot function like ZFS does… (On ZFS, Proxmox uses .raw…).

My 2 cents
Andy

[fausp (fpausp)]

@mark_nl

I use armv7hf (arm32-bit) VM’s using qemu/libvirt on a Raspberry PI aarch64

Can we use this as the base OS and install Nethserver on it?

[mark_nl (Mark Verlinde)]

@fausp

Can we use this as the base OS and install Nethserver on it?

To my knowledge it is not possible to run a 32 bit container on a 64 bit system, it has to be a vm with a kernel.

To give it another try you may install centos 7 aarch64 from the official minimal ISO.
However keep in mind epel-7 for aarch64 is not maintained since end 2019, that why from the Nethserver side we only keep the NS repositories up to date but do not release a aarch64 version.

[fausp (fpausp)]

@mark_nl

To give it another try you may install centos 7 aarch64 from the official minimal ISO

OK, with some little tweaks it is up and running:

Edit: Installing Nethserver ATM with:

# Update the system:
yum --enablerepo=* clean all && yum update -y

# Optional Install nano and nmap:
yum install nano nmap -y

# Enable NethServer software repositories with this command:
yum install -y http://mirror.nethserver.org/nethserver/nethserver-release-7.rpm

# To install the base system, run:
nethserver-install

# Reboot:
reboot

[mark_nl (Mark Verlinde)]

nice , please keep the unmaintained epel-7 warning in mind

My first go installing pimox did not succeed will try agian.

@fausp

OK, with some little tweaks

Can you be a bit more specific?

[fausp (fpausp)]

@mark_nl

Can you be a bit more specific?

To configure the VM is a bit tricky:

And default it would boot CentOS:

How can I change it to Nethserver?

[fausp (fpausp)]

Maybe GRUB_DEFAULT in /etc/default/grub doesnt work:

[mark_nl (Mark Verlinde)]

@fausp

How can I change it to Nethserver?

Have no clue, you may manually create a new grub.cfg by running run (as root)
grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg

[fausp (fpausp)]

@mark_nl

Have no clue, you may manually create a new grub.cfg by running run (as root)

Didnt work… How far are you atm, is the proxpi running?

[fausp (fpausp)]

@fausp

Didnt work…

OK it works with:
nano /boot/efi/EFI/centos/grub.cfg

[mark_nl (Mark Verlinde)]

@fausp

How far are you atm, is the proxpi running?

It’s up and running, with your directions of setting up the VM it’s installing centos aarch64.

I have a clue why grub does not update the kernel entry properly, most likely because during nethserver-install we enable the centos-altarch sig kernel repository. From there it pulls in the kernel from the newer 5.4.x branch.
Should de solved by editing /etc/sysconfig/kernel to resemble this:

# UPDATEDEFAULT specifies if new-kernel-pkg should make

# new kernels the default

UPDATEDEFAULT=yes

# DEFAULTKERNEL specifies the default kernel package type

DEFAULTKERNEL=kernel

then grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg EDIT: (confirmed) works as expected

[fausp (fpausp)]

@mark_nl

Should de solved by editing /etc/sysconfig/kernel

OK, thank you for this…

I did some tests in the meantime:

1. Nethserver virtual HD running on the SDCard gives me best performance…
2. Nethserver virtual HD running on a USB 3.1 Stick, formated with ZFS… 2nd best
3. Nethserver virtual HD running on NFS (Backup Server), what gives me 3rd best performance…

Edit: with ZFS it would be ease to have 2 USB-Sticks formated as RaidZ for redundancy.

I would like to run also OPNsense and Windows 10 (Windows10_InsiderPreview_Client_ARM64_en-us_21354.vhdx) embedded on the proxpi, any suggestions?

[mark_nl (Mark Verlinde)]

@fausp

I did some tests in the meantime:

1. Nethserver virtual HD running on the SDCard gives me best performance…
2. Nethserver virtual HD running on a USB 3.1 Stick, formated with ZFS… 2nd best
3. Nethserver virtual HD running on NFS (Backup Server), what gives me 3rd best performance…

Running (and booting) from an USB Attached SCSI (uas) ssd turned out to be the fastest in other setups. Doubt is ZSF is an viable set up on these low resources. However it is amazing it can… just because it is possible is part of the fun

@fausp

I would like to run also OPNsense and Windows 10 (Windows10_InsiderPreview_Client_ARM64_en-us_21354.vhdx) embedded on the proxpi, any suggestions?

Not familiar with Proxmox nor OPNsense nor windows IOT stuff, you probably found these links yourself…
https://www.yrzr.tk/opnsense-images-for-aarch64/#1-introduction

Converting a Hyper-V vhdx for use with KVM or Proxmox VE

[fausp (fpausp)]

@mark_nl

Not familiar with Proxmox nor OPNsense nor windows IOT stuff, you probably found these links yourself…

Cool, thank you for the hint…

[mark_nl (Mark Verlinde)]

@fausp thanks again for this topic … it is pretty cool

Going to give the creator of pimox a few thumbs up too.