I still have some questions. I guess my question is regarding to Duplicity. I decided to create this new topic.
The first question is regarding to the use of multiple USB disks. Does it work out of the box?
I have noticed that the next backup fails when I remove the back-up files from the USB disk (/mnt/backup). I have worked around this by removing the folder in [ /var/lib/nethserver/backup/duplicity/ ]. The backup works again, by executing the [ /sbin/e-smith/backup-data ] command.
I am currently re-running the first back-up to reproduce the above. I just want to verify if the behaviour is consequent (I am using this product for the first time).
What I would like to know is if there is anyone who is using the backup as is (out of the box) with multiple USB hard disks. Is the above behaviour normal? Or am I dealing with a coincidence? Please bear with me - as the backup process takes several hours (I can test it myself but it takes long - some help of an experienced user would be very much appreciated).
The second question is with regards to compression ratio. I have noticed that the files produced by Duplicity are tar.gz volumes. What is the best way to increase the compression ratio? I guess I could pass the environment variable “GZIP=-9” system wide - but what is best practice? How am I supposed to accomplish this on a system running Nethserver?
By the way: is there a reason to use volumes of 250M? Is it possible to increase these to eg 1000M?
I am hoping to use the backup module as is with multiple USB hard disks. I am willing to add a hard disk and use rsync if required… but I prefer an out of the box experience.
I’m not sure that using multiple disk with incremental backups could work. I prefer use a single disk as destination and copy its content to multiple usb disks.
Lower values are helpful when the backup is restarted (too hard to explain for me, please read duplicity documentation).
It can be changed: config setprop backup-data VolSize=1000
Your next backup will have bigger slices.
I don’t know how to control compression ratio and a quick search didn’t find more details.
I have connected two USB3 drives to the Nethserver.
1x 4TB disk with a red chassis (this one remains connected to the server permanently)
1x 3TB disk - we have 5 of these which will be changed every working day
#!/bin/bash
####################################################################################################
# WHAT DISK?
# We required two mounted disks: SDC and SDD
# The 4.00 TB disk has to be mounted to /mnt/backup
# The 3.00 TB disk has to be mounted to /mnt/sync
dmesg | grep "sdc" | grep "3.00" | grep "TB" || backup="/dev/sdc1"
dmesg | grep "sdc" | grep "4.00" | grep "TB" || sync="/dev/sdc1"
dmesg | grep "sdd" | grep "3.00" | grep "TB" || backup="/dev/sdd1"
dmesg | grep "sdd" | grep "4.00" | grep "TB" || sync="/dev/sdd1"
####################################################################################################
# MOUNT SYNC DISK
# This is the disk which should be changed on working days by the backup administrator.
if [ $(mount | grep -c /mnt/sync) != 1 ]
then
mount $sync /mnt/sync || exit 1
echo "/mnt/sync is now mounted"
else
echo "/mnt/sync already mounted"
fi
####################################################################################################
# MOUNT BACKUP DISK
# This is the disk which is connected to the server permanently
if [ $(mount | grep -c /mnt/backup) != 1 ]
then
mount $backup /mnt/backup || exit 1
echo "/mnt/backup is now mounted"
else
echo "/mnt/backup already mounted"
fi
#!/bin/bash
####################################################################################################
# MOUNT DISKS
# We will run the pre-backup.sh script as this script is only used to mount the disks
/opt/backup/pre-backup-data.sh
####################################################################################################
# SYNC DIRS
#timeStamp
myTimeStamp=$(date -d "today" +"%Y-%m-%d-%Hh%M")
#check log dir
if [ ! -d "/var/log/rsync" ]; then
mkdir "/var/log/rsync"
fi
echo "Sync /mnt/backup to /mnt/sync"
rsync -a --log-file="/var/log/rsync/backup-data-$myTimeStamp.log" --delete --exclude="lost+found" /mnt/backup /mnt/sync
if [[ $? -gt 0 ]]
then
#error
echo "Er is een serieus probleem met de synchronisatie van de backup op SERVER1! Er dient direct actie ondernomen te worden om dit op te lossen." > /tmp/rsync-mail.txt
echo "" >> /tmp/rsync-mail.txt
echo "Log file:" >> /tmp/rsync-mail.txt
cat /var/log/rsync/backup-data-$myTimeStamp.log >> /tmp/rsync-mail.txt
mail -r admin@ourcustomer.inet -c support@yeahright.inet -s "Our Customer: RSync SERVER1 Failure /mnt/backup to /mnt/sync" maillog@ourcustomer.inet < /tmp/rsync-mail.txt
else
#success
echo "De synchronisatie van de backup op SERVER1 is gelukt. De USB disk mag omgewisseld worden. Alvast bedankt en een fijne dag toegewenst." > /tmp/rsync-mail.txt
echo "" >> /tmp/rsync-mail.txt
echo "Log file:" >> /tmp/rsync-mail.txt
cat /var/log/rsync/backup-data-$myTimeStamp.log >> /tmp/rsync-mail.txt
mail -r admin@ourcustomer.inet -s "Our Customer: RSync SERVER1 Success /mnt/backup to /mnt/sync" maillog@ourcustomer.inet < /tmp/rsync-mail.txt
fi
####################################################################################################
# UMOUNT SYNC DISK
# This is the disk which should be changed by the backup admin every working day
if [ $(mount | grep -c /mnt/sync) != 1 ]
then
echo "/mnt/sync is not mounted"
else
umount /mnt/sync || exit 1
echo "/mnt/sync is now unmounted"
fi
I have changed the email addresses and the name of our customer (and so forth).
I am hoping the above is helpful for others who are struggling with back-ups.
No. I changed over to a NAS as a backup target. Besides that I setup USB copy on the NAS. The customer is able to copy the NAS-backup to USB disks during office hours by just connecting a USB hard disk drive (to the NAS). This works fine as the NAS-backup runs during the night and is done before office hours. This way the backup works much easier.