Set up Duplicati to back up your Neth Server

backup
v7

(Dan) #1

I’d mentioned Duplicati in the “multiple backup” thread as a possible alternative backend for the backup system. It has a few handy features, some of which I’d mentioned in the other thread, but the biggest to me is the ability to very easily set up AES256-encrypted backups to a number of cloud storage providers through a pretty simple web UI. It’s pretty straightforward to install on Neth, too:

yum install yum-utils
rpm --import "http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF"
yum-config-manager --add-repo http://download.mono-project.com/repo/centos7/
yum install mono-devel libappindicator https://updates.duplicati.com/beta/duplicati-2.0.3.3-2.0.3.3_beta_20180402.noarch.rpm

echo "[Unit]
Description=Duplicati Backup software
[Service]
ExecStart=/usr/bin/mono /usr/lib/duplicati/Duplicati.Server.exe --webservice-interface=any
Restart=on-failure
RestartSec=30
[Install]
WantedBy=multi-user.target" > /etc/systemd/system/duplicati.service

systemctl enable --now duplicati

config set fw_duplicati service status enabled TCPPort 8200 access green
signal-event firewall-adjust

Then browse to http://yourIP:8200 for the Duplicati web UI. You can set a password there if you like.

Setting up a backup is easy. First click Add Backup from the menu:


Select Configure a new backup, then Next. On the next screen, give the backup a name (whatever you like), select the encryption type, and enter or generate a password/phrase:

On the next screen, enter your backup destination. Here, I’ll select Google Drive. Enter the path on your desired destination, then click on the blue AuthID link:


That will bring up the following window; click the blue button:

Enter your login credentials, and then you’ll be asked if you want to allow Duplicati access to your Google Drive account. Click the blue Allow button. A long AuthID will be generated, and will populate back into Duplicati’s Destination page. Click the blue Test Connection button. If the connection was successful, click Next to select what to back up.

Here you can select the source material from the tree, or just enter paths one at a time using the Add path button. Click Next when done.

The next page is the scheduler. Select when you want backups to run, then click Next.

On the next page, you’ll set options. The upload volume size and retention are self-explanatory. The Advanced Options allow you a lot of control over the backup job; one way you can use that is to signal events (say, signal-event pre-backup-data before starting, and signal-event post-backup-data on completion, use the run-script-before and run-script-after options, respectively).


Click Save, and your job will be saved.

After you’ve set up the job through the GUI, it will give you the option to export it as a command line. That will look kind of like this:

 mono /usr/lib/duplicati/Duplicati.CommandLine.exe backup "googledrive://duplicati/phabricator/?authid=(redacted)" /var/repo/ /var/log/ /root/ /var/lib/nethserver/ /var/lib/collectd/ /var/www/html/ /usr/share/nextcloud/config/config.php --backup-name="Server backup" --dbpath=/root/.config/Duplicati/WBNGWVABWS.sqlite --encryption-module=aes --compression-module=zip --dblock-size=50mb --passphrase="(redacted)" --retention-policy="1W:1D,4W:1W,12M:1M" --run-script-before="signal-event pre-backup-data" --run-script-after="signal-event post-backup-data" --disable-module=console-password-input 

(Markus Neuberger) #2

Great work! :+1: Another nice backup manageable via web UI. :sunglasses:

Going to test…

Minor issue: yum-utils providing yum-config-manager is not installed by default.

I tried SSH backup via command line export to another Nethserver and it worked as expected.


(Dan) #3

Ah, I must have previously installed it on this system and not remembered it. I’ll update.

The open question, IMO, is how we’d make this comparable to the built-in backup/restore. I’m pretty sure, by backing up everything in /etc/backup-data.d/*.include, we get all the same data, but there are probably other steps needed to get the configuration in a usable way.

Compared to the standard system, Duplicati lags in that it won’t handle SMB/NFS mounts. It can certainly back up to those locations, and maybe you could mount/unmount the shares using the run-script-before and run-script-after options, but it would be an added step. The big win in my book is how easy it is to set up the cloud backup (while it’s still encrypted, so Google et al can’t read your data).


(Markus Neuberger) #4

I did a quick research and it seems that duplicati doesn’t support include files like the ones in /etc/backup-data.d/. Duplicati has the include switch and filters so we need a script converting the Nethserver backup includes and other paths (ie. /etc/, /var/lib/nethserver) to a duplicati command line.

I don’t know how far to go with integration as Duplicati would work completely on it’s own with own time schedule which could be an advantage. On the other hand with a Nethserver managed duplicati there’s no backup overlap and Nethserver backup is designed to be able to integrate other backups…

I have to think about it :thinking:


(Dan) #5

Well, it’d be neat to have it as a plug-in replacement/addition to the system, but I was thinking more of matching the workflow on restore. Find/restore/download the config backup, upload that to the Neth box. Then install Duplicati and restore the rest of the data, followed by signal-event post-restore-data. To make that process work, we might need to set up a separate config backup job.


(Markus Neuberger) #6

Maybe it’s enough to just include /var/lib/nethserver/backup/backup-config.tar.xz to the duplicati backup?


(Dan) #7

If that’s all it is, simple enough: restore that file out of the backup, upload, continue. Maybe not super-elegant, but I don’t think it’s any less elegant than the current system in that regard.

Duplicati’s web UI will also let you download the configuration for a particular backup job as a JSON file (optionally encrypted). So the disaster recovery workflow would look like:

  • Install Neth
  • Install Duplicati
  • Upload the backup configuration to Duplicati
  • Restore /var/lib/nethserver/backup/backup-config.tar.xz
  • Upload to Neth
  • Restore the rest of the backup
  • signal-event post-restore-data

In place of step 3, you can reconfigure the backup job in Duplicati, as long as you have a record of your encryption passphrase.


(Markus Neuberger) #8

Great, it’s another “VPS disaster recovery/migration in the cloud” possibility!
@oneitonitram, did you already migrate your VPS?


(Nitram Oneito) #9

no, @mrmarkuz I have not yet migrated my vps.

still looking for a more efficient method, that I wont loose some records in the process, especially emails. since this is a production server.


(Nitram Oneito) #10

For those who have tested duplicate, at the moment, what does it backup on nethserver, is it everything, or is it some items. and what happens In the process of restore to another nethserver instance.

ill be trying the whole process on my nethserver staging environment and see.
otherwise the feedback regarding will help.


(Dan) #11

It backs up whatever you tell it to back up. The built-in backup-data system backs up the directories specified in /etc/backup-data.d/*.include, so if you include all those paths in your Duplicati backup you’ll get the same data.

I still think hotsync will be the most efficient method for you, even if you will need to reconfigure the network at the command line.


(Alessio Fattorini) #12

That’s amazing! Thanks for sharing.
Are you the the Howtosmaster?