Incremental Backup to USB drive / changing drives

[tstalzer]

NethServer Version: 6
Module: Backup

Hello

I have a question regarding the backup module.

I would like to configure a backup scheme for my client with 3 USB harddrives

1 drive stays at home
1 drive is on the road between office and home
1 drive is connected to the server

On friday morning, the customer takes the oldest drive from home to the office and and connects it to the server. The previous connected drive will be taken home. (He now has two drives at home). The next week, he will do the same.

With this scheme, one backup is always at home (worst case - 2 weeks old).

I would like to configure backup for an incremental backup every evening with a full backup on saturday morning.
What happens when a system tries to do an incremental backup to a backup which is already 2 weeks old? This could happen, if the customer either forgets to swap the drives or plugs the wrong one in.

Kind regards
Thomas

[bwdjames]

It depends on how the backup module recognizes which files needs to be needs to be backed up.

If the backup module is clever enough to pickup the timestamp of the last backup on the USB disk and generates a list of files that has changed since then - then there is no issue as it will backup 2 weeks worth of data.

If the backup module works off when the last incremental/full backup job ran, then it could be an issue as it could miss any files that changed on days 8-14 for a best case scenario or longer for a worst case scenario if the customer does this for more than 1 week running and this can be very easily done as well.

For this sort of situation, the sneaky little bastard in me kind of takes control and I connect up a 4th USB hard disk which stays permanently connected to the server and have rsync running as a daily cron which backs up the data to the 4th hard disk. This essentially gives a daily full backup of the data.
Now if you really wanted to get ultra sneaky and cleaver (like I have done for my home and small business) and the broadband connection is fast enough to support this and there is no internet data cap issues or big financial constraints on the backup scenario, you could open an AWS (Amazon Web Services) account and create an S3 bucket (which is essentially online file storage area) and do an AWS equivalent of a rsync of your files to the S3 bucket.
If you turn on Version Control on the AWS S3 bucket and do the rsync daily, you get to have both full and incremental backups done on a daily basis, thanks to the Version Control feature.

The only real thing that gets in the way of this working is the internet connection is down for an extended period of time (assuming the rsync cron is setup correctly and proven to be working).

[tstalzer]

It depends on how the backup module recognizes which files needs to be needs to be backed up.

That is basically my question.
However, backing up online is also an interesting option.

Thanks

Thomas

[filippo_carletti]

I connect up a 4th USB hard disk

This is what I do, too.
I use NethServer standard backup to a local usb disk and rsync its content to another disk I usually keep at home that I bring to the office once a week.
The rsync starts when I plug the usb disk thanks to an udev rule.

[bwdjames]

@filippo_carletti That udev rule is a good idea, I have been wanting to do that but haven’t been able to figure out how to do it yet.

Do you have a working example that you are able share?

[filippo_carletti]

Here’s the header of my script, I hope it’s enough to get the idea.

# filippo
# fork code taken from USB auto Backup by Marc Quinton
#
# rsync to usb disk on connect
#
# udevadm info -a -p $(udevadm info -q path -n /dev/sdXX)
# cat /etc/udev/rules.d/50-backup.rules
# KERNEL=="sd?1", ACTION=="add", ATTRS{serial}=="serialXXXX", ATTRS{manufacturer}=="SanDisk Corporation", ATTRS{product}=="Cruzer Contour*", RUN+="/usr/local/bin/usb-auto-backup --fork %k"


if [ "$1" == "--fork" ]; then
	# When run from udev, we fork to return udev rules and wait for mounts.
	shift # remove $1 (--fork)
	bash $0 $* &  # run in backgroud (fork)
	exit 0
else

	# now, we are in the forked process ; just sleep some time waiting for mounts.
	sleep 2

[bwdjames]

Thanks @filippo_carletti, I’ll give that a try.