Testers needed nethserver-arm img

mark_nl · October 11, 2018, 8:07pm

I think you making it to hard for yourself. If you stay within the default configurations offered you do not have to do any thing, it just does the job. i.e if you install a mailserver mails gets backed-up, if you install roundcubemail the database gets backed up, if you make a fileshare data gets backed up… However everything is customizable

rgmhtt · October 11, 2018, 8:41pm

In my current setup, all data is backed up nightly via a number of rsync crontab jobs to a 2TB USB drive. Don’t have one, yet, for testing on this server.

And saturday night, this is repeated but to a network attached C7-arm server with a 4TB USB drive. The plan is to move this to my neighbor’s for off-site backup…

davidep · October 11, 2018, 9:59pm

They are quite different. Both are present in AD by default. Apart from that, a user is always member of domain users: the membership is granted implicitly when the account is created. For this reason that “magic” group is not listed.

Domain admins are similar to any “normal” group. Membership must be granted explicitly. Members of that group are granted special domain privileges too.

rgmhtt · October 11, 2018, 11:40pm

Oh, I know the difference. The point is that ‘domain users’ is not showing up under the groups tab, nor selectable for adding a use. I can, perhaps, see why if all users are always a member of this group.

Just not what I expected.

Moving on.

And still cannot connect to accounts provider from the Status, Domain Accounts.

mrmarkuz · October 12, 2018, 12:55am

Does it work from command line:

account-provider-test dump

Can you list the users?

/usr/libexec/nethserver/list-users

rgmhtt · October 12, 2018, 12:15pm

{
   "BindDN" : "ldapservice@HOME.HTT-CONSULT.COM",
   "LdapURI" : "ldaps://nsdc-homebase.home.htt-consult.com",
   "DiscoverDcType" : "ldapuri",
   "StartTls" : "",
   "port" : 636,
   "host" : "nsdc-homebase.home.htt-consult.com",
   "isAD" : "1",
   "isLdap" : "",
   "UserDN" : "dc=home,dc=htt-consult,dc=com",
   "GroupDN" : "dc=home,dc=htt-consult,dc=com",
   "BindPassword" : "_XXXXXXXXXXXXXXXXXXX",
   "BaseDN" : "dc=home,dc=htt-consult,dc=com",
   "LdapUriDn" : "ldap:///dc%3Dhome%2Cdc%3Dhtt-consult%2Cdc%3Dcom"
}

I broke this up into a line per user instead of the one long line it produced.

{"administrator@htt-consult.com":{"locked":1,"gecos":"Administrator","expired":0,"groups":[],"shell":"/usr/libexec/openssh/sftp-server","new":0,"expires":"yes"}
,"admin@htt-consult.com":{"locked":1,"gecos":"NethServer Administrator","expired":0,"groups":[],"shell":"/usr/libexec/openssh/sftp-server","new":1,"expires":"yes"}
,"abba@htt-consult.com":{"locked":0,"gecos":"Abba","expired":0,"groups":[],"shell":"/bin/bash","new":0,"expires":"no"}
,"rgm@htt-consult.com":{"locked":0,"gecos":"Robert Moskowitz","expired":0,"groups":[],"shell":"/bin/bash","new":0,"expires":"no"}}

I thought that the users would be created in the AD zone of home.htt-consult.com, not the ‘root’ zone of htt-consult.com

But the web tool still cannot connect to get status.

mark_nl · October 12, 2018, 7:36pm

For some reason the centos PI kernel defaults to power-save scaling governor. On our RPI images a systemd service “cat’s” ondemand in each scaling governor of all 4 cpu’s.

I do not have hardware with (centos) kernel support for CPUFreq, hence /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor is absent.

@rgmhtt is it present on Allwinner A20 SOC’s ?
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

Or are there other means to check the scaling governor and/or current CPU frequency?

rgmhtt · October 12, 2018, 7:55pm

Blank if I know anything about cpufreq! It freqs me out trying to understand this stuff since the 80286!

Anyway according to

https://linux-sunxi.org/Mainlining_Effort

A20 has had CPUfreq since kernel 4.0

mark_nl · October 12, 2018, 8:03pm

the 286 is a failure and intel even admits this: around minute 13 Microprocessor Marketing Wars

Is /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor present or not?

rgmhtt · October 12, 2018, 8:48pm

Yes, quite the failure…

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

ondemand

mark_nl · October 12, 2018, 8:58pm

Thanx, there seems no need to correct it non PI devices!

mark_nl · October 17, 2018, 11:44am

The images for the Raspberry PI and (other) u-boot based SBC’s are updated to reflect changes in partitioning and swap as discussed here.

Nethserver-7.5.1804-Devel-RaspberryPi-img.raw.xz
Nethserver-7.5.1804-Devel-Generic-img.raw.xz

Changes:

Partitioning (1) Boot (768MB) and (2) RootFS (2560MB), file system for Boot on RPI is FAT32 and Generic ext4.
Small change under the hood: first partition begins @4096 to give more room for (extra) early-stage bootloaders. (unfortunately this is hard-coded in the appliance-tools, hence all images have this change)
Swap is provided by Comressed RAM (zram-swap).
Package file was missing needed by nethserver-duc, it’s included in the nethserver-arm comps group now. (odd thing was functionality does not seem to affected by this bug)
uboot-images-armv7 is installed on Generic image. This way u-boot for your board can be picked up from the just flashed sd-card making it possible to prepare the card on non Centos/Fedora systems.

One odd issue I can’t explain is the size of the compressed images RPI ~ 280MB vs 480MB for Generic. Cause is xz has an lower compression rate for the latter despite the same settings/command…

Short write up to prepare/ flash-uboot for the Generic image:

export sdcard="/dev/sdX"

As always: Be sure you got the right device (/dev/…) pointing to your sd-card

xzcat Nethserver-7.5.1804-Devel-Generic-img.raw.xz | sudo dd of=${sdcard} status=progress bs=4M && sudo sync

Mount your sd-card, mountpoint (/mnt) is arbitrary:
sudo mount ${sdcard}2 /mnt

Find the available boardmodels:
ls /mnt/usr/share/uboot/

(in this writeup orangepi_plus2e is assumed)
export boardmodel="orangepi_plus2e"

Find name of u-boot file:
ls /mnt/usr/share/uboot/${boardmodel}

(turns-out out for orangepi_plus2e it is u-boot-sunxi-with-spl.bin)
export uboot="u-boot-sunxi-with-spl.bin"

Flash u-boot to sd-card:

sudo dd if=/mnt/usr/share/uboot/${boardmodel}/${uboot} of=${sdcard} bs=1024 seek=8 conv=fsync,notrunc

Un mount sd-card:
sudo umount /mnt

giacomo · October 18, 2018, 7:36am

I tried the Devel image using an emulated ARM machine with QEMU, I followed this Fedora documentation

I found a couple of issues:

at boot the, the process blocks because the system can’t mount the /boot partition because it doesn’t recognize the vfat filesystem. To fix the problem: enter the dracut rescue console and comment the boot partition inside the fstab. I didn’t spent much time on this, but I couldn’t find a way to mount the vfat partition. Even if the kernel should have the support (grep vfat /usr/lib/modules/4.14.52-v7.1.el7/modules.builtin).
firewalld and NetworkManager were running, I had to disable both using systemctl
swap is on a partition and no zram block device has been created

Probably next week I will give it another try.

Thank you for work!!

mark_nl · October 18, 2018, 8:00am

This is very strange i will check the links to the images because the generic image should not have any fat partition. None of the Nethserver images have firewalld enabled and Network Manager is not installed at all.

Strange…

EDIT: (links) to Nethserver images are good…

Very interesting experiment, which could lead to builds on travis cli.

mrmarkuz · October 19, 2018, 9:12pm

I tested the new image on a raspberry and it works as expected.

It’s faster now, isn’t it?

localhost systemd: Startup finished in 1.929s (kernel) + 3min 25.699s (userspace) = 3min 27.629s

I got some warnings/errors in /var/log/messages:

[root@raspi ~]# cat /var/log/messages | grep -i "warn\|fail\|error"
Jan  1 00:00:03 localhost kernel: WARN::dwc_otg_hcd_init:1046: FIQ DMA bounce buffers: virt = 0xbad04000 dma = 0xfad04000 len=9024
Jan  1 00:00:03 localhost kernel: WARN::hcd_init_fiq:459: FIQ on core 1 at 0x80619654
Jan  1 00:00:03 localhost kernel: WARN::hcd_init_fiq:460: FIQ ASM at 0x806199ac length 36
Jan  1 00:00:03 localhost kernel: WARN::hcd_init_fiq:486: MPHI regs_base at 0xf0006000
Jan  1 00:00:05 localhost kernel: random: 7 urandom warning(s) missed due to ratelimiting
Jan  1 00:00:11 localhost systemd-tmpfiles: Failed to create directory or subvolume "/var/lock/ppp": No such file or directory
Jan  1 00:00:11 localhost systemd: systemd-tmpfiles-setup.service: main process exited, code=exited, status=1/FAILURE
Jan  1 00:00:11 localhost systemd: Failed to start Create Volatile Files and Directories.
Jan  1 00:00:11 localhost systemd: Unit systemd-tmpfiles-setup.service entered failed state.
Jan  1 00:00:11 localhost systemd: systemd-tmpfiles-setup.service failed.
Jan  1 00:00:22 localhost kdumpctl: Starting kdump: [FAILED]
Jan  1 00:00:22 localhost systemd: kdump.service: main process exited, code=exited, status=1/FAILURE
Jan  1 00:00:22 localhost systemd: Failed to start Crash recovery kernel arming.
Jan  1 00:00:22 localhost systemd: Unit kdump.service entered failed state.
Jan  1 00:00:22 localhost systemd: kdump.service failed.
Jan  1 00:00:25 localhost esmith::event[713]: Warning: Migration of certificates failed fatally: Can't call method "get_all" on an undefined value at /usr/share/perl5/vendor_perl/esmith/templates.pm line 825.
Jan  1 00:00:25 localhost esmith::event[713]: Warning: Migration of fwservices failed fatally: Can't call method "get_all" on an undefined value at /usr/share/perl5/vendor_perl/esmith/templates.pm line 825.
Oct 19 20:31:31 localhost esmith::event[713]: Warning: Migration of fwrules failed fatally: Can't call method "get_all" on an undefined value at /usr/share/perl5/vendor_perl/esmith/templates.pm line 825.
Oct 19 20:31:31 localhost esmith::event[713]: Warning: Migration of hosts failed fatally: Can't call method "get_all" on an undefined value at /usr/share/perl5/vendor_perl/esmith/templates.pm line 825.
Oct 19 20:31:31 localhost esmith::event[713]: Warning: Migration of accounts failed fatally: Can't call method "get_all" on an undefined value at /usr/share/perl5/vendor_perl/esmith/templates.pm line 825.
Oct 19 20:31:31 localhost esmith::event[713]: Warning: Migration of portforward failed fatally: Can't call method "get_all" on an undefined value at /usr/share/perl5/vendor_perl/esmith/templates.pm line 825.
Oct 19 20:31:31 localhost esmith::event[713]: Warning: Migration of routes failed fatally: Can't call method "get_all" on an undefined value at /usr/share/perl5/vendor_perl/esmith/templates.pm line 825.
Oct 19 20:31:32 localhost /etc/e-smith/events/system-init/S00initialize-default-databases[725]: /var/lib/nethserver/db/configuration: NEW passwordstrength=configuration|MaxPassAge|180|MinPassAge|0|PassExpires|no|PassWarning|7|Users|strong
Oct 19 20:32:56 localhost esmith::event[1940]: WARNING in /etc/e-smith/templates//etc/sssd/sssd.conf/00template_vars: Use of uninitialized value $domainName in uc at /usr/share/perl5/vendor_perl/NethServer/SSSD.pm line 397.
Oct 19 20:32:56 localhost esmith::event[1940]: WARNING: Template processing succeeded for //etc/sssd/sssd.conf: 1 fragment generated warnings
Oct 19 20:32:57 localhost esmith::event[1940]: WARNING in /etc/e-smith/templates//etc/samba/smb.conf/10base: Use of uninitialized value $domainName in uc at /usr/share/perl5/vendor_perl/NethServer/SSSD.pm line 397.
Oct 19 20:32:57 localhost esmith::event[1940]: WARNING: Template processing succeeded for //etc/samba/smb.conf: 1 fragment generated warnings

mark_nl · October 20, 2018, 12:11pm

Thank you! (as always) a thorough report which makes it better for the future!

According to a quick search the (early stage) Fast Interrupt (FIQ) warnings can be ignored.

I see urandom running out of entropy all the time while the kernel still runs in the initramfs. And because it occurs during this stage installing haveged does not help…

Can not reproduce this
@mrmarkuz does systemctl status systemd-tmpfiles-setup.service give any clue’s?

With both images the kdump.sercice fails for different reasons. On NS x86_64 the service is disabed by default. I’m considering to even mask ( systemctl mask kdump ) the service on armhfp. tagging this as Bug (thanks)

mark_nl · October 21, 2018, 7:16pm

Announcement:

The devel repository is moved to another location, again kindly provided by @mrmarkuz.

Please update your installs to make this change effective

yum update

-or-

yum install https://mrmarkuz.goip.de/mirror/nethserver-arm/7.5.1804/nethserver-release-7-arm.noarch.rpm

For fresh installs: Please download new images who have this change in place:

Nethserver-7.5.1804-Devel-RaspberryPi-img.raw.xz
Nethserver-7.5.1804-Devel-Generic-img.raw.xz

Tip: take a look at the two bash scripts in the nethserver-arm-dev issue tracker if you want to “curl” them from the command line.

other change:
The above discused bug regarding systemd: kdump.service failed is solved by masking this service

dz00te · October 22, 2018, 4:11pm

finally i had some free time… i have done a rebuild for arm if you want to test it (on aarch64, install is ok… backup to test )

if you have some time to test it:

http://mirror.framassa.org/nethserver7-arm/other/

giacomo · October 23, 2018, 7:17am

Wow!!

mark_nl · November 9, 2018, 9:09pm

Here is a qcow image for qemu emulation/virtualization of armhfp on x86_64 with a large address space lpae kernel included.

On Fedora 28 (can not get it to work on Centos) it is possible to emulate armhfp. On my hardware it’s dead slow (boot takes about 70 sec…) but is seems to work.

sudo dnf install qemu-system-arm virt-install libvirt libvirt-python libguestfs-tools
(above is deducted from bash_history ; not sure if I installed more in the past )

Download the image and unpack it on a convenient place
xz -vdk -T 0 Centos-Qemu-lpae-armhfp.qcow2.xz

Extract the kernel from the image:
virt-builder --get-kernel Centos-Qemu-lpae-armhfp.qcow2

Then create the emulated virtual machine:

virt-install  --name centos7_armhfp  --memory 2048 --vcpus 2 \
--boot kernel=vmlinuz-4.14.78-201.el7.armv7hl+lpae,\
initrd=initramfs-4.14.78-201.el7.armv7hl+lpae.img,\
kernel_args="console=ttyAMA0 rw root=LABEL=_/ rootwait" \
--disk Centos-Qemu-lpae-armhfp.qcow2 \
--import  --arch armv7l  --machine virt

It should boot (as said it’s dead slow…)

login: root
passwd: centos

useful commands:

^] > leave the console of the VM (=“CNTRL + ]”, like telnet… )

virsh destroy centos7_armhfp > stop the VM
virsh edit centos7_armhfp > edit libvirt configuration of VM
virsh start centos7_armhfp > start the VM
virsh console centos7_armhfp > get a console for the started VM
virsh undefine centos7_armhfp > delete/remove the VM

Not sure if it has any useful purpose