Upgrade paths to ns7

v7

(Alessio Fattorini) #21

Maybe other can chime in here like our brand new @ambassadors_group


(Davide Principi) #22

This is now on testing! /cc @quality_team

https://github.com/NethServer/dev/issues/5196


(Davide Principi) #23

Upgrade ns6 to ns7: backup/restore

Let’s start with it: restore a backup set of ns6 on ns7.

Luckily, there are more similarites than differences. Most of the modules are 1-1 upgrade. I’d like to discuss about how to cope with differences!

The File server/Shared folders module received the biggest changes, and present the following issues:

  1. Authenticated access to shared folders. If the account provider is LDAP, authentication cannot work: Active Directory is required.

  2. Web access to shared folders. In ns7, web access is provided by a new package, nethserver-virtualhosts. Shared folders are accessible only as Samba shares (also SFTP and NFS soon - yes, this is another story).

This morning, I was discussing those points with @filippo_carletti and @giacomo.

About 1, a possible strategy could be:

  • retain nethserver-directory as ns6 (LDAP provider) after restore

  • If needed, perform an additional, separate, step: upgrade to nethserver-dc

The upgrade is technically possible because the ns6 LDAP backup contains the Samba SAM DB data.


About 2, there could be different solutions, depending on how a particular shared folder is used. Possible scenarios that come to mind:

  1. The shared folder was the virtual host web root: http://vhost.tld

  2. The shared folder was a path in a particular virtual host http://vhost.tld/ibayaliasname

  3. The shared folder was available from ANY virtual host (global): http://192.168.1.1/ibayaliasname, http://vhost.tld/ibayaliasname

  4. HTTP Basic Authentication was required

  5. Custom filesystem permissions are set

  6. (…did I miss something?)

It’s hard to catch them all. Let’s see some possible approaches:

I feel the “Upgrade to virtual host” action would cover a lot of upgrade requirements. It’s missing scenario 3. Because of it, a generated .conf file is still necessary.

Are we missing some other cases? What do you think? /cc @dev_team @quality_team @Lewis @Mario_Lanno @telekomiker (@alefattorini, please help me on notifying other people interested on this!)


[6 Mar] What are you working on this week?
(Marc) #24

/cc @Alejandro_Guerra @lorentedford @petralemoisson @ArchParks


(Alessio Fattorini) #25

@EddieA @mark_nl @robb @Lewis @phucktank @clinton


(Giacomo Sanchietti) #26

We just implemented the described behavior of 1, 2 and 3.
Shared folder available from ANY virtual host (global) are migrated to a new special virtual host named default.


(Davide Principi) #27

The restore procedure now supports a ns6 backup set! This long awaited feature has been implemented and is now on testing!! /cc @quality_team

To test it, enable the nethserver-testing repo in /etc/yum.repos.d/NethServer.repo

Then follow the procedure described in the test case here:


Upgrading NS 6.8 on live system to 7
Accounts provider - Service Pack 1
(Alessio Fattorini) #28

Is there anyone that have tested the new procedure? We’d like to gather as many data as possible.


#29

yes i do a quick test in past week-end, on my mail server+ file server
but i have no notes written :frowning:
i remember two problem:

  1. restore of apps seemed ok, but it seems to have some preblem on mail:
    mail was restored in /var/lib/nethserver/vmail/user
    instead of /var/lib/nethserver/vmail/user@test.net
    so reading mail from roundcube give me only emty mailboxes

  2. i use fetchmail on 6.8, where there was the General Tab to disable fetchmail,
    with getmail ther is no such option, so at the end of restore all the
    account in getmail was fully enabled. But i wanted to check if the restore was all good before
    re-enable getmail… in short: i miss an option to quickly disable getmail…

i hope to have some time soon to retest well the procedure


(Alessio Fattorini) #30

I guess that @nrauso (and not only…) would be interested in an enhancement like that


(Davide Principi) #31

The ns6 upgrade and sme8 migration procedures have been released.

Now it’s time for the live update! Did anyone try the deprecated upstream tool?

I hope it still works for a minimal set of upstream packages! I’d go with this procedure

  • pre-backup-config event
  • remove all nethserver packages. Only centos-minimal packages are preserved
  • run upgrade assistant
  • cross fingers
  • proceed with post-restore events

As usual, no UI at first.


Upgrading NS 6.8 on live system to 7
(jack) #32

will also transfer the virtual machine/webvirtmgr?


(Davide Principi) #33

The webvirtmgr package has been moved to nethforge repo in ns7: I don’t have any plan for it at the moment. Maybe @giacomo has more info about it!


(Giacomo Sanchietti) #34

Virtual machines are not migrated since they are not part of standard backup.

I suppose you can safely copy libvirt configuration and virtual disks, then restart everything from webvirtmgr.


(Davide Principi) #35

Today I ran a live upgrade of a ns6 system to ns7. The upstream procedure works with minor tweaks:

https://github.com/upgrades-migrations

This is just an attempt, but results are encouraging!

The live upgrade strategy can follow the previous ones:

  • reduce ns6 packages to a minimal set
  • live upgrade
  • restore-config

Upgrading NS 6.8 on live system to 7
CentOS upgrade tool outcomes
(Davide Principi) #36

I have an interesting update for this topic :sunny:

I prepared a couple of RPMs for NethServer 6 that perform a live upgrade to NethServer 7, based on the upstream procedure.

The procedure runs the following steps:

  1. downloads all required RPMs
  2. at next reboot runs the RPMs upgrade transaction, then reboots
  3. completes by running post-restore-config and post-restore-data events

The upgrade procedure returns a NethServer 7 installation, retaining

  • grub boot loader installation and configuration
  • previous partitioning, RAID and (ext4) file systems
  • LDAP accounts provider - if present

All the installed RPMs are upgraded/removed to conform to a NethServer 7 installation. Access to Enterprise repositories is preserved.

I tested it on 2 VMs and 2 bare metal machines. Both UEFI and BIOS firmwares were tested. During my tests the procedure has failed because:

  • not enough disk space was available (but recovered safely before starting the RPM transaction)
  • the old grub package was removed (fixed now)

Individual modules may still have issues, but I think it is stable enough to be ALPHA.

I didn’t test encrypted file system, but should work.


NETHSERVER 6 UPGRADE PROCEDURE – DO NOT USE IN PRODUCTION!!!

In ns6 install the upgrade tool /cc @quality_team :

yum --enablerepo=nethserver-testing install nethserver-upgrade-tool

Run the event and reboot

signal-event nethserver-upgrade-tool-prepare

RPMs are downloaded – it takes some time. If everything is OK reboot: the upgrade starts automatically:

:crossed_fingers: if everything is ok you’ll get your NethServer back, upgraded to 7!


Next steps:

  • Write the docs
  • Fix issues for individual packages
  • Enforce disk space check before rebooting
  • User interface to choose the account provider before starting the procedure

Further information:

https://github.com/NethServer/nethserver-upgrade-tool/blob/master/README.rst

Code repositories:

Roadmap to public Beta

https://github.com/NethServer/dev/issues/5564


(Alessio Fattorini) #37

Looks a great job… i hope it would be a great way to replace a lot of 6s with 7