I’ve now tried this on “bare metal” rather than a VM, and here is much more of a step-by-step walkthrough. Very little (if any) of this is original to me; my sources are linked above.
As a big-picture overview, this procedure will install CentOS to one partition on a disk, create a ZFS pool on another partition, copy the CentOS installation to that ZFS pool, and configure Grub to boot from that pool.
Edit: It should be obvious, but don't do this on a production system, or one you expect to immediately put into production. It's working for me as far as I can see, but it will need plenty of testing to confirm that it's going to be stable and not interfere with anything that Neth does with the system.
You’ll start by doing a minimal install of CentOS 7.4. In the CentOS installer, set up custom partitioning. Tell it to use standard partitioning, not LVM. The first partition will be xfs, with a mountpoint of /mnt/for-zfs. Size it as the total size of your disk minus about 15 GB. The second partition will be ext4, with a mountpoint of /. Size it to take the remaining space. The installer will warn you about the lack of a swap partition, but this isn’t a problem—we’ll add that later. The remainder of the installer settings can be left at defaults, and/or set to your preference. Complete the installation and reboot.
On reboot, you’ll find that your network isn’t active, because CentOS inexplicably doesn’t activate network interfaces by default. To activate your network, you’ll first need to determine the name of your network interface. Do this by running
nmcli d. Having determined the interface name (for example,
enp0s25), bring it up by running
Now that the network is active, determine your IP address by running
ip addr. Make a note of this; you’ll need it later. You’ll also be able to use
yum to install your favorite text editor. Install
rsync as well; we’ll use it later.
yum install nano rsync.
Now you can configure your network interface to automatically start on boot. Edit
enp0s25 with your interface name), find the line that says “ONBOOT=no”, and change that to “ONBOOT=yes”. The network interface will now start up automatically on boot.
You’ll now need to edit your SSH configuration to allow root logins with password. Edit
/etc/ssh/sshd_config. On line 38, uncomment the line that says
PermitRootLogin yes. Then restart the SSH service:
service sshd restart. You should now be able to connect to this system using SSH, at the IP address you noted above.
yum update to bring your installed packages up to date.
Unmount the for-zfs partition:
umount /mnt/for-zfs. Edit
/etc/fstab to remove the listing for that partition.
Install the ZFSonLinux repository:
yum install http://download.zfsonlinux.org/epel/zfs-release.el7_4.noarch.rpm.
/etc/yum.repos.d/zfs.repo. In the [zfs] block, set
enabled to 0. In the [zfs-kmod] block, set
enabled to 1.
Then install ZFS itself:
yum install zfs zfs-dracut.
dd if=/dev/urandom of=/etc/hostid bs=4 count=1
Activate the ZFS kernel module:
Check your disk IDs:
ls /dev/disk/by-id/* -la
Now it’s time to create the pool. This command will specify a number of ZFS feature flags, to ensure that Grub will be able to boot from the pool. Note that the page I used as my main source for these instructions includes the
copies=2 option. I’m not including it in this command, as I’ll be creating a two-disk mirror shortly.
zpool create -f -d -o feature@async_destroy=enabled -o feature@empty_bpobj=enabled -o feature@lz4_compress=enabled -o ashift=12 -O compression=lz4 -O acltype=posixacl -O xattr=sa -O utf8only=on -O atime=off -O relatime=on rpool /dev/disk/by-id/ata-Hitachi_HDS723020BLA642_MN1210F338JYMD-part1
Turn on the autoexpand property, so the pool will grow as disks are added or replaced:
zpool set autoexpand=on rpool
You’ll now want to create the root filesystem:
zfs create rpool/ROOT.
Create a tmp directory, and mount the root directory there:
mount --bind / /mnt/tmp
Then use rsync to copy the data:
rsync -avPX /mnt/tmp/. /rpool/ROOT/.
Once that finishes, edit
/etc/fstab to remove the root partition, and
You’ll next create some symbolic links that Grub needs:
ln -s /dev/disk/by-id/* . -i.
Mount proc, sys, and dev into the ZFS pool:
for dir in proc sys dev;do mount --bind /$dir /rpool/ROOT/$dir;done.
Now change root into the new root filesystem:
Create the Grub configuration:
grub2-mkconfig -o /boot/grub2/grub.cfg.
In the output of
grub2-mkconfig, note the complete version number of the first linux/initrd images.
Remove the ZFS cache:
Create the initramfs files:
dracut -f -v /boot/initramfs-$(uname -r).img $(uname -r);
dracut -f -v /boot/initramfs-3.10.0-693.5.2.el7.x86_64.img 3.10.0-693.5.2.el7.x86_64 In the second command here, if the version number does not match the version number you noted above, change the version number in the command (both places) to match.
Install Grub on your disk:
grub2-install --boot-directory=/boot /dev/sda. If that completes with no errors,
exit from the new root.
dev from the new root:
for dir in proc sys dev;do umount /rpool/ROOT/$dir;done. Reboot the system and cross your fingers.
If you’re lucky, your system will reboot without issue.
Login via SSH again. It’s time to delete the original root partition. Run
fdisk /dev/sda. Enter p (to print the partition table), d (to delete a partition), 2 (partition #2), p (to show the table again, making sure it still has the large partition #1), w (to write the partition table and quit).
Reboot the system again.
Log in once more. Run
zpool status to confirm that your pool is in good shape.
Now you can begin to install Neth:
yum install http://mirror.nethserver.org/nethserver/nethserver-release-7.rpm. Then begin the installation:
When the installation completes, you’ll have Nethserver installed and running on a single-disk ZFS pool. If that’s your objective, you can stop here, though you’d probably be better off adding a swap partition to your disk in the space previously occupied by your old root partition. The next post will discuss adding disks to create a mirrored volume.