NethServer Hotsync [Needs Testing]

@wbilger I’ve reproduced your issue with IO error. We’ll try to find a solution soon, but it’s probably related to this https://bugzilla.samba.org/show_bug.cgi?id=7809

There are some updates, checkout github wiki for new installation instructions, and last bleeding edge/testing packages can be found in autobuild repository

About debugging, I had a deep debugging and I took some notes:

Increase rsync verbosity
on slave /etc/rsyncd.conf
add
max verbosity = 10

on master add -vvv
to rsync command in /usr/sbin/hotsync
The more ‘v’ you add, the more verbosity

write hotsync stderror to standard output
edit /usr/sbin/hotsync
and add cat /tmp/hotsync.log before the last line rm -f /tmp/hotsync.log

copy list of includes and excludes
add in /usr/sbin/hotsync
cp $INCLUDE_FILE /tmp/INCLUDE_FILE cp $EXCLUDE_FILE /tmp/EXCLUDE_FILE

Do you want me to test any of that, or wait for a fix?
Right now, I am in the middle of building 2 fresh machines, to see if I still get the IO errors (I do).
The sync just finished, I have a couple questions;

  1. should the slave machine be up to date with updates from the software center, or will they propogate?
  2. should all the same packages be installed (webtop, nextcloud, etc).

Here is what I experienced from running hotsync from a machine with webtop, nextcloud, roundcube, getmail, and connected to a zentyal domain to a slave which was a fresh install of NS.

  1. restore of configuration went mostly well, IP, fqdn, Zentyal join all worked.
  2. Webtop, NextCloud, Roundcube, getmail, all installed.
  3. But, WebTop worked, but NextCloud and RoundCube did not. Nextcloud just gave a blank white screen. Roundcube gave a database error.
  4. An uninstall and reinstall of roundcube, and it is running.
  5. An uninstall and reinstall of NextCloud produces an Internal Server Error.
    If I try to run an occ scan, it gives a ‘PDOException’ with message ‘SQLSTATE[42S02]: Base table or view not found: 1146 Table ‘nextcloud.oc.appconfig’ doesn’t exist’ in /usr/share/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnections.php:104

wait, probably today we’ll have a testing package

once every hour, slave installs (or update) all nethserver- packages

no, use a clear installation, do the NS first configuration wizard and don’t do anything else

I think that there’s an issue on mysql restore, I’ll try to spot it.

Ok, great, thanks, will you notify here?
I am all in for this module, it is EXACTLY what I need, please let me know if there is anything I can test to help.

3 Likes

Yes, sure

I have tested nextcloud and work for me.
Here is a list of packages to be tested by everyone wants to help :thumbsup:

1 Like

last package in nethserver-testing reository: nethserver-hotsync-1.0.0-1.46.g80dbaf5.ns7.noarch.rpm

1 Like

@Sebastian is also developing an interface, but I think that you’ll see it next week :slight_smile:

1 Like

Ill need to set up another server for this :smiley: looks good!

2 Likes

If you lend a hand, you would be more than welcome!

A new interface for nethserver-hotsync in nethserver-cockpit is ready for testing:

GitHub Repository nethserver-cockpit-hotsync: GitHub - NethServer/nethserver-cockpit-hotsync

Install

First install nethserver-cockpit:

After install nethserver-cockpit-hotsync:

yum install http://packages.nethserver.org/nethserver/7.4.1708/testing/x86_64/Packages/nethserver-cockpit-hotsync-0.1.3-1.ns7.noarch.rpm

Now NethServer Hotsync is configurable in NethServer Cockpit, Applications.

8 Likes

Awesome, I will look at the new interface in cockpit right away.

@Stll0 I have been away for weekend, but first tests this morning, and with the new version of hotsync, the IO errors are gone, and now file deletions work as expected. Great news! I will test a restore today.

1 Like

Give this man a like!!! :thumbsup:

3 Likes

I have the nethserver-cockpit-hotsync up and running looks good. Is there plans to add any functionality to it, like sync times? I assume so, if that functionality is added to hotsync.
FYI, I had to log out of cockpit, and log back in, as until then, the NethServer Hotsync settings showed as ‘not available’, not sure if that is normal behaviour or not.
Should or can cockpit/hotsync be added to the slave to aid in role changing, or should slave be left a fresh install as suggested by @Stll0? Although, once an initial promotion to master is done, it would have all of the modules at that point once set back to slave.

2 Likes

Thanks for your help :clap:
If sync time will be added to hotsync it will be visible in cockpit interface too.
At the momento after new cockpit module installation recommend to run systemctl restart cockpit and all wil be fine.
Nethserver-hotsync-cockpit and nethserver-cockpit can be installed on master and slave because they do not conflict with nethserver-hotsync. Configuring, if you select role Master the information to be entered will be different than if you select role Slave so you can configure nethserver-hotsync completely using the interface both in master and slave.

1 Like

I’ve created the relative issue 5401

2 Likes

We have a new testing package :tada:

Call to arms
If you want to test if your favourite module works with HotSync, there’s a lot of packages to check. Look at the issue

@lucag nethserver-webtop5 is waiting for you. I should have fixed the problem you reported
@mrchiao @mrmarkuz I’ve tried nethserver-freepbx and seems to work, but I’m biased :innocent:
@wbilger I think that issue with nextcloud and roundcube should be resolved, but I didn’t test it
@Jclendineng @jjimenez let me know if it works now with your setup (and what modules you confirm to be tested)

3 Likes

I’ve tried a couple promotions of slave to master, here is what I found.
First, what does command “# hotsync-slave”, as I’ve noticed if I don’t run it it works.

  • If I shut down master, and just run config restore on slave, everything seems to be good, I can then login to cockpit on new ‘master’ and disable hotsync, so I don’t keep getting error messages that can’t find slave.
    EDIT… Also, would it be possible or a good idea, to disable hotsync on the promotion to master, to not get the error messages, as typical scenario would be to use backup until proper ‘Master’ is fixed, so would not have a ‘slave’ of promoted master.
    Also, what is the command to disable Hotsync on master, without uninstalling.

  • If I shut down master, and hotsync-slave, and then then run config restore on slave, cockpit is there, but hotsync-cockpit is not, and if I add it, it says already installed. If I remove and re-install, it is still not there.
    Both methods see the following issues in WebTop, and Nextcloud, tested with most recend update.

  1. Is it necessary to run signal-event nethserver-hotsync-update after upgrading hotsync?
  2. Can we continue from existing slave, or should we try a fresh install and full hotsync?

Webtop Issues (still present with newest Hotsync update)

  • email flags are all there, except memo (may not be a flag). Memo icon is there on email, but there is not memo data, and after some time (refresh?), memo icon is gone
    -Calendar items not synced
  • Contacts not synced
  • Tasks not synced

NextCloud Issues (still present with newest Hotsync update)

  • installed Apps are not installed (if you install the App after, App data is present though)

@wblinger sorry, I wasn’t clear before: to put again in production you now have to launch signal-event nethserver-hotsync-restore

hotsync-slave is executed every hour, it’s useful to give it by hand only in testing environment. It take the list of NethServer packages installed on master and install (or update) them without launching configuration scripts.

@Sebastian maybe hotsync-cockpit is in backup package list:
tar -xf /var/lib/nethserver/backup/backup-config.tar.xz -C / var/lib/nethserver/backup/package-list
cat /var/lib/nethserver/backup/package-list

nethserver-hotsync-update exent is executed automatically after package installation/update. Every nethserver package launch signal-event $NAME-update

You can use slave as new master (I’m not sure if I got the question)

try with signal-event nethserver-hotsync-restoreinstead of restoring config

1 Like

I confirm that by executing the signal-event nethserver-hotsync-restore on the SLAVE server I find all the webtop data without problems.

2 Likes