Can't run a SOGo Backup

[mrmarkuz (Markus Neuberger)]

You’re welcome!

[pagaille (Matthieu Gaillet)]

Please note that nethserver’s backup module seems to take care of the Sogo backups. I’ve seen that in the logs, there are some includes too.

[stephdl (Stéphane de Labrusse)]

yes, I did not see it before because the backup script push to /dev/null the error message, but each night the backup is done in /var/lib/sogo/backup. We have tested the restoration and it should be workable but of course we can test it again

[stephdl (Stéphane de Labrusse)]

Just tested it again and all is workable for me

install nethserver-automysqlbackup and do a backup

automysqlbackup

drop the mysql database of sogo

mysql
drop database sogo;

build it again

signal-event nethserver-sogo-update

then restore all settings and data (calaendar and contact)

sogo-restore-user /var/lib/sogo/backups/sogo-2018-01-04_0030 -A

We can see the warning, it is workable even if for each user we see it one time. It is true we know the settings in ldap (samba4 and ldap) are not good, mainly when we want to set the imap acl (mailbox, calendar, contact) because the sAMAccountName is used and not the userPrincipalName (for example stephane and not stephane@de-labrusse.fr)

My concern is if we adjust this correct setting, all records made in mysql are lost because they are done with the key ‘stephane’ and not with the full address mail.

In fact this should be tested parameter per parameter, probably it is not hard to find.

[m.traeumner (Michael Träumner)]

@stephdl

Jan 03 22:50:07 sogod [26991]: <0x0x55b8b4e81e40[LDAPSource]> <NSException: 0x55b8b5ad7450> NAME:LDAPException REASON:operation bind failed: Invalid credentials (0x31) INFO:{“error_code” = 49; login = “samaccountname=stephane,dc=ad,dc=de-labrusse,dc=fr”; }
2018-01-03 22:50:07.876 sogod[26991:26991] ERROR(-[NGLdapSearchResultEnumerator nextObject]): does not support result references yet …

I have the same at my logs, for me it’s a problem with caldavsynchronizer. If you enter a mailadress at the synchronization profile you get these errors.
For more details you can also have a look at the following thread:

Sogo LDAP bind error
Do you use caldavsynchronizer?

[stephdl (Stéphane de Labrusse)]

I don’t use caldavsynchronizer, just lightning and sogo connector, also K9 with a carddav application

[mark_nl (Mark Verlinde)]

could someone try it the old fashioned way?

With the some_test_dir of your choice and location (can be in var/tmp)

mkdir  /root/some_test_dir
chown sogo:sogo /root/some_test_dir
su -s '/bin/bash' -c '/usr/sbin/sogo-tool backup /root/some_test_dir ALL' - sogo

Executing it like this sogo-tool does not have to read sogo.conf

[mrmarkuz (Markus Neuberger)]

Didn’t work for me in /root/somedir

[root@testserver ~]# chown sogo:sogo /root/some_test_dir [root@testserver ~]# su -s '/bin/bash' -c '/usr/sbin/sogo-tool backup /root/some_test_dir ALL' - sogo 2018-01-04 13:50:39.192 sogo-tool[4822:4822] an error occured during directory creation

so I used /tmp:

[root@testserver ~]# su - sogo -s '/bin/bash' -c '/usr/sbin/sogo-tool backup /tmp/sgbkp ALL' <0x0x22a8840[SOGoCache]> Cache cleanup interval set every 300.000000 seconds <0x0x22a8840[SOGoCache]> Using host(s) '127.0.0.1' as server(s)

Backup file is written.

[mark_nl (Mark Verlinde)]

@mrmarkuz

[root@testserver ~]# su - sogo -s ‘/bin/bash’ -c ‘/usr/sbin/sogo-tool backup /tmp/sgbkp ALL’

@mrmarkuz

Backup file is written.

Oke we are getting somewhere!

[mrmarkuz (Markus Neuberger)]

@mark_nl

Oke we are getting somewhere!

The backup file is written, even when the error occurs, and has working content. I think we can ignore this error as regards the backup function. It seems to occur randomly, I couldn’t find a pattern:

sogo-tool[30089:30089] ERROR(-[NGLdapSearchResultEnumerator nextObject]): does not support result references yet ..

@kis2006

it does write the user file that contains the sogo settings for each user yes.i will look at some alternatives if I cant get the sogo-tool going.

Did you find a solution that works for you? As @stephdl tested again, sogo-tool should work as expected. The error doesn’t affect the backups.

[kis2006 (kieran)]

As far as I am aware, the sogo-tool is supposed to include the data as well as config files. The output I get is a config file for the user (sogo settings) and that’s it. No backup of the database or emails or calendar.

[root@testserver ~]# su - sogo -s ‘/bin/bash’ -c '/usr/sbin/sogo-tool backup /tmp/sgbkp ALL

This gives me the same “does not support result references yet” error and the output in /tmp/sgbkp is only small config files.

[mrmarkuz (Markus Neuberger)]

I’d recommend to use the built-in cron job backup described here. Calendar events and contacts are included in backup:

Nethserver-sogo, needs test

I would like to implement the sogo backups by a cron each night. what is saved: user configuration calendar events contacts where: /var/lib/sogo/backups I want to backup this folder by nethserver-backup-data cons, pro, I’m listening you

https://wiki.nethserver.org/doku.php?id=sogo&s[]=sogo#backup_sogo

To backup mysql db you may use this module:

https://wiki.nethserver.org/doku.php?id=module:automysqlbackup

The mails may be backed up with nethserver-backup.

http://docs.nethserver.org/en/v7/backup.html?highlight=Data

[kis2006 (kieran)]

OK, thanks
Just to clarify
1, (filter rules, specific settings, events, contacts) are contained within the User file and can be saved with the sogo-tool anywhere.
2, (emails) are contained in the /var/vmail/mydomain.com/user1 folder
3, (SQL database) presumable contains the relationships between a user and the above.

Is it possible to save only 1 and 2, then re-create the server manually restoring the data over the top at the end?
Could I not just back the whole drive up every night? Using some sort of incremental backup (to avoiding writing 500gb everytime)

From the backup I really just want to save users emails and calendar events nightly to an external drive (for disaster recovery). We are only talking 20 users, so rebuilding accounts and settings is not that bad.

[mrmarkuz (Markus Neuberger)]

@kis2006

Could I not just back the whole drive up every night?

Yes, that’s possible. With automysqlbackup to save your databases you may just use the Nethserver backup data function, which does schedule an incremental backup.

Another approach would be some kind of virtualization(proxmox, vmware) to run your Nethserver and just backup/snapshot the VM image.

@kis2006

We are only talking 20 users, so rebuilding accounts and settings is not that bad.

That’s right but why setting it up again when having it all in one incremental backup.

EDIT:

Hotsync backup may also be a solution for you to reduce disaster recovery time to nearly zero, maybe next to data backup:

NethServer Hotsync [Needs Testing]

HotSync is a set of scripts that leverage NethServer backup to copy a snapshot of a NethServer over another one every 15 minutes. In case of a disaster, it’s possible to promote slave to master in a few minutes. We call it “Poor men HA”. It was used few times in production for the enterprise, and AFAIK it works quite well. If you want to take a look at it, here you can find a draft of documentation, including new features that we’d like to implement. You can try current version installing it …

[stephdl (Stéphane de Labrusse)]

@kis2006

1, (filter rules, specific settings, events, contacts) are contained within the User file and can be saved with the sogo-tool anywhere.
2, (emails) are contained in the /var/vmail/mydomain.com/user1 folder
3, (SQL database) presumable contains the relationships between a user and the above.

1. calendar, contacts, personal settings are saved by sogo-tool and the cron job each night (you can specify when you want to backup them also)

2. email are saved by the nethserver-backup-data if installed

3. all settings, calendars, contacts are stored in the mysql database you can save it by nethserver-automysqlbackup or by nethserver-backup-data.

the nethserver-hotsync looks promised

[kis2006 (kieran)]

Thanks again everyone for the help.
The VM snapshot sounds great - wish I was on a VM with this install, too much hassle to change right now.
I like the hotsync, but it is overkill for me on this one, I really want to keep it simple (for ongoing maintenance - when I’ve forgotten how I set I up).

Is this a workable backup in theory. (I realise there are tools/scripts for these)
1, Backup the sogo user files
2, Backup the /var/vmail/mydomain.com/

Restore
3, Reload server (same version as the backup)
4, Recreate users and server settings as before
5, Overwrite the sogo user files and the var/vmail/mydomain.com/

[stephdl (Stéphane de Labrusse)]

why simply not use the backup way of nethserver -> install nethserver-backup-data

it handles the installation of missed softwares and of course the restoration of your configuration.

Keep things simple

Of course a backup strategy is valid, only if you try the restoration time to time…After I like to get the belt and the suspenders, therefore my email are also saved by rsync and my database by automysqlbackup

[kis2006 (kieran)]

Many thanks, will try a restore tomorrow.

[kis2006 (kieran)]

Thanks everyone for the help.
The nethserver backup works great, The belt and braces approach from above is the one I am going for.
Restore worked for me too.

[kis2006 (kieran)]

Just one last question…
After restoring the backup all the data is as it should be apart from when I use SOGo webmail to copy to an email to one of the folders I created before the backup.

Here is what works…
I can copy from the folder I created
I can copy to and from all other folders
Shared calendars seem to work ok as do address books

Here is what I have tried…
Used a sub-folder under the Inbox
Used a folder inline with the Inbox
Redoing the backup from scratch
Separately copying the mail folder across
Checking the permissions on the mail folders