Howto install Pimox 7 on a Raspberry Pi 4 to host NethServer 7 and Fedora 34 as VMs

This howto describes the installation of Pimox 7 - Proxmox 7 on a Raspberry Pi 4. I think it could be useful for Nethserver ARM development to have more instances/snapshots of development and testing VMs.

Thanks to @fausp and @mark_nl for the great work in Install Nethserver on CentOS 7 armhf?

Install Pimox 7

Get debian bullseye daily image from https://raspi.debian.net/daily-images/ and flash to SD card:

wget https://raspi.debian.net/daily/raspi_4_bullseye.img.xz
xzcat raspi_4_bullseye.img.xz | dd of=/dev/mmcblk0 bs=64k oflag=dsync status=progress

In my case I used a SATA to USB adapter to use an SSD, I booted with SD card:

xzcat raspi_4_bullseye.img.xz | dd of=/dev/sda bs=64k oflag=dsync status=progress

For Windows you may use Etcher or Raspberry Imager to flash the SD card.

(Re)boot the Pi and login as root - there’s no password

# Enable SSH root login
nano /etc/ssh/sshd_config
add Permitrootlogin yes
systemctl restart sshd

# set root password
passwd

# update system
apt update;apt full-upgrade -y

# install needed packages
apt install git curl screen wget -y

# set interfaces
# I used IP address 192.168.1.30 with gateway 192.168.1.11
# and hostname pimox
echo "auto eth0
iface eth0 inet static
address 192.168.1.30/24
gateway 192.168.1.11" > /etc/network/interfaces.d/eth0

# set hostname and hosts
echo 'pimox' > /etc/hostname
echo '192.168.1.30	pimox' >> /etc/hosts

reboot

Use screen or tmux to make the installation process survive the network reconfiguration.

screen
curl https://raw.githubusercontent.com/pimox/pimox7/master/pimox.sh | sh

Use default settings, just press Enter

The SSH connection will be lost but you can reconnect to SSH session and reattach with screen -d;screen -r` to see the process finishing.

Install Nethserver armhf VM on the Pimox7

@mark_nl provides a qcow2 image, download and extract it on the pimox:

wget https://github.com/markVnl/nethserver-createimg/releases/download/dev-qemu-efi/Nethserver-7.9.2009-Final-Qemu-Efi-armhfp.qcow2.xz
xz -T 0 -k -d Nethserver-7.9.2009-Final-Qemu-Efi-armhfp.qcow2.xz

Create a new VM (in my case with VM ID 100) in UI and name it, select no media and no firewall.
Detach and remove the hard disk of the VM in the VM Hardware view.

Import NethServer qcow disk to VM 100 to local_lvm storage in qcow2 format to make snapshots available:

qm importdisk 100 Nethserver-7.9.2009-Final-Qemu-Efi-armhfp.qcow2 local --format qcow2

Go to VM Hardware and double click the Hard Disk and “Add” it.

Open /etc/pve/qemu-server/100.conf (config file of VM with ID 100) and add/edit arch, bios and boot.
arch: arm is needed for armhfp 32 bit.

arch: arm
bios: ovmf
boot: order=scsi0
cores: 1
memory: 2048
name: NethServerArmhfp
net0: virtio=16:F1:1A:C6:E4:A7,bridge=vmbr0
numa: 0
ostype: l26
scsi0: local:vm-100-disk-0.qcow2,size=4G
scsihw: virtio-scsi-pci
smbios1: uuid=c6fa6feb-dac1-4f94-80dd-eccd47139027
sockets: 1

Start the VM.

Login as root with password Nethesis,1234

Get IP address and force NethServer install:

ip a
touch /var/spool/first-boot
reboot

After 2 minutes the NethServer configuration should be finished, you may check with journalctl -f

When you see “Startup finished in … 2min 8.671s” the NethServer VM should be ready at https://<VMIP>:9090

Install Fedora 34 VM as a base for Neth ARM development

# Get a Fedora 34 armhfp image
wget https://download.fedoraproject.org/pub/fedora/linux/releases/34/Spins/armhfp/images/Fedora-Minimal-34-1.2.armhfp.raw.xz

# Extract
xz -T 0 -k -d Fedora-Minimal-34-1.2.armhfp.raw.xz

# Convert from raw to qcow2 to make snapshots work
qemu-img convert -f raw -O qcow2 Fedora-Minimal-34-1.2.armhfp.raw Fedora-Minimal-34-1.2.armhfp.qcow2

# Import the disk.
# You need to prepare a VM first, see NethServer installation above
qm importdisk 101 Fedora-Minimal-34-1.2.armhfp.qcow2 local --format qcow2

Start the VM in the Proxmox console view, after boot you may set the root password or other settings.

Please follow Arm development setup for NS7 (how to get started) to setup an ARM development VM on Fedora 34.

Hints

Set root pw in image

# Needed for virt-customize
apt install libguestfs-tools

# Set root password to Nethesis,1234
virt-customize -a <image>.qcow2 --root-password "password:Nethesis,1234"
9 Likes

Going to work on grub2-efi and grubby on el7 a bit;

As it stands now grubby assumes UEFI can not run on arm-32bit, and we proved the opposite… :grinning:
So have to think of a “trick” and add a bash script in /etc/kernel/postinst.d/ to update grub2.cfg in /boot/efi/EFI/centos upon a kernel update… :thinking:

4 Likes

As always,I appreciate your work :slightly_smiling_face:. Thank your for that!

Just one thing, do you know how to sync the time correctly? It looks like it works different as on the big brother… Time sync on Proxmox

It seems chrony is used now, even on new proxmox 7 installations:

Time Synchronization: New installations will install chrony as the default NTP daemon, because the design limitations of systemd-timesync make it problematic for server use. Users upgrading from a system using systemd-timesyncd, should manually install either chrony, ntp or openntpd.

systemctl restart chrony

OK, looks good now… :+1:

Does someone know, can we run windows 10 on the pi?

@fausp

The ARM Version Preview should run, Parallels is touting that it runs “Windows 10 ARM Preview” on Mac with M1 CPU, an ARM Variant by Apple…

They also sell licences for Windows ARM…

My 2 cents
Andy

2 Likes

Imported the VHDX and try to run it from NFS-Storage because the SD is not that big… ATM I am waiting… and waiting… and waiting… :grinning:

Is it possible to add such a Pimox installation as a node to a “normal” Proxmox cluster?
Sincerely, Marko

Yes, It shoud be possible to join the pimox-7 node to a Proxmox-7 cluster…

3 Likes

Hello Markus @mrmarkuz,

I have followed all the above steps very carefully for multiple times. However, I always ended up with a message saying, “Guest has not initialized the display (yet).” It will remain like this forever. And the CPU usage with 1 CPU core assigned is over 100%. So I thought it probably needed more CPU cores. I then assigned 2 cores to it. It’s now at a little above 50%. However, it’s still stuck with the same message as above.

Please see attached screenshot.

Any suggestions, please?

IIRC this happens when the VM settings are wrong:

Are there errors in the proxmox logs?

1 Like

No, there was no error at all. See this screenshot, please.

Forgot to attached the 100.conf. Here it is:

arch: arm
bios: ovmf
boot: order=scsi0
cores: 2
efidisk0: local:100/vm-100-disk-1.qcow2,efitype=4m,pre-enrolled-keys=1,size=64M
memory: 2048
name: NethServerArmhfp
net0: virtio=F6:F4:0F:BC:4F:30,bridge=vmbr0
numa: 0
ostype: l26
scsi0: local:100/vm-100-disk-0.qcow2,size=4G
scsihw: virtio-scsi-pci
smbios1: uuid=f1f462a3-5ede-4a7b-bf10-ab731d82a37d
sockets: 1

1 Like

I guess you need to remove the EFI disk. It can be done via web UI.

Oh I forgot to mention that it was not there in the first place. Only when I started the VM in CLI with “qm start 100”, it prompted me that EFI doesn not exist and it will created a temporary EFI in a temp’ish file name. I thought it was the reason that the VM couldn’t start or at least one of hte reasons. So I went ahead and created an EFI disk in the web UI. And when I started the VM with CLI again, it was clean without any prompt at all.

I’m going to install another NethServer ARM VM and report…

You may try to cleanup the VM with qm cleanup or recreate a new VM:

There’s also an issue about that error:

EDIT:

I could reproduce the issue with following version:

root@pimox:~# pveversion
pve-manager/7.0-13/d36ac0ea (running kernel: 5.10.0-9-arm64)

I tried the following but without success:

I have tried re-creating the VM many times. I have even started with the blank SD card for four times by now. None worked, unfortunately.

Changing the VM CPU to host instead of default kvm64 enables the console but the VMs don’t boot.

I tried the alternative install on RaspiOS64 and got the same error.

aarch64 images still work, I tested opensuse tumbleweed so it seems only armhf images are affected.

I was not able to find the NethServer image at opensuse tumbleweed, was there a specific image for NethServer on this page?

It’s not a Nethserver image, it’s another linux distribution just for testing ARM64, I used this image.

For Nethserver we need working ARM32 boot. The problem is known at pimox side so I hope it will be fixed soon.