Help hacking backup script for rsync

Support
#1

I am running NS7.5. I have some huge share that change rarely or, at least, in very low percentage (let’s say 99% of files are exactly the same from ages).
I have excluded it from the duplicity backup and I would like to append a rsync that syncs the ibay folder in the same samba share as the duplicity backup (let’s say if myserver is the normal backup, myserver_ibay should be synced) and possibly add a line to the backup log if something goes wrong in rsync.
Actually this simple bash script does the job, but it would be great to append the sync to normal backup:

mkdir -p /mnt/ibay_backup
mount -t cifs -o username=bkuser,password=mypassword //192.168.0.157/backup /mnt/ibay_backup
rsync -avzr --delete /var/lib/nethserver/ibay /mnt/ibay_backup
umount /mnt/ibay_backup

Thanks for any hint, I am super noob in scripting

#2

Did you see the new backup inlcuding rsync? Maybe scripting is not necessary…

EDIT:

Oh, I noticed you like to rsync to cifs, this seems not possible with the new rsync backup as it uses symlinks. See Backup-data: multiple schedule and backends

2 Likes
#3

Thanks for pointing this out, I will check if my NAS supports NFS.
Can you just point me to the docs on new Rsync backup? I couldn’t find it in the options.

#4

Here are the docs:

http://docs.nethserver.org/en/latest/backup.html#rsync
http://docs.nethserver.org/projects/nethserver-devel/en/latest/nethserver-backup-data.html#nethserver-rsync

#5

Thank you! I am trying via sftp because my NAS has terrible NFS support.

#6

You’re welcome. Please share your testing experience here.

1 Like
#7

May I just some more noobish question (I am really newbie and I don’t want to mess up things in the official test post).
I tried to set up a backup for 3:00pm to test it. And nothing happened. This is config show:

mybackup1=rsync
BackupTime=30 15 * * *
Notify=always
NotifyFrom=backup@mydomain.net
NotifyTo=backup@mydomain.net
SftpDirectory=/i-data/9b906e16/server/ns07_rsync
SftpHost=192.168.0.157
SftpPort=22
SftpUser=root
VFSType=sftp
status=enabled

What could be wrong? Where are logs stored for this kind of backup? How can I launch manually this kind of backup?

#8

Backup was set at 15:30

You can look at /var/log/messages, and also some other logs under /var/log with name containing backup (i.e. /var/log/last-backup.log, /var/log/backup-data.log), if any.

backup-data -b mybackup1
1 Like
#9

Thanks all for your help: I really appreciated and learnt a lot.
Unfortunately I discovered my (cheap) NAS has no sftp support and very poor NFS support. So I go back to the original question.
How can I append the sh script above to the backup process or, at least, schedule it through the template system?

#10

From the top of my head I think you could add an action for the post-backup-data event.
Put your script in /etc/e-smith/events/actions/ with executable permission. Give it a descriptive name.
Then link the action to the post-backup-data event:

cd /etc/e-smith/events/post-backup-data/
ln -s ../actions/yourscriptname S90yourscriptname

Replacing yourscriptname as appropriate.
Haven’t tested this but think it will work (unless you find a permission issue).

2 Likes
File mode of data backup files
#11

Thank you! I Ended up with this one line script:

rsync -azv --delete --stats --log-file=/mnt/backup/$(date “+%Y-%m-%d_%H-%M-%S”)_ibay_sync.log /var/lib/nethserver/ibay /mnt/backup

Linked as:

ln -s …/actions/ibay-rsync-mnt-backup S79ibay-rsync-mnt-backup

Being S80umount-cifs I use the original /mnt/backup and I don’t have to mount it manually.

I have permission issues being samba on the NAS very basic, but it is no problem for me as it is only storage for Windows files.

Aiming to perfection: 1) can i append the log file to backup mail? 2) can I tell rsync to ignore permission issues so that the mail marks “SUCCESS”

#12

Just giving some ideas, not sure they’ll work…

What happens if your script prints the log to stdout when the backup is called through backup-data-wrapper?

You may try to append the outcome of rsync to the log generated by backup-data. Take a look at logs and wrapper section of the manual, and the hooks.

Another option would be to call mail or sendmail command from your script to send the log report in a new e-mail message (so not in the same e-mail message you use to get):

Don’t know, you can try with -q (or --quiet) option to suppress non-error messages.
A more drastic option would be to force the exit code (exit 0) of your script.

1 Like
#13

Problem 2 solved thanks to your inspiration: instead of forcing 0, I grab the exit code before rsync and I exit with that code

#!/bin/bash
GetErrorCode=$?
rsync -az --delete --stats --log-file=/mnt/backup/$(date “+%Y-%m-%d_%H-%M-%S”)_ibay_sync.log >/var/lib/nethserver/ibay /mnt/backup
exit $GetErrorCode

Now I will work on the mail.

1 Like
#14

This is my final version: thanks for precious help.

#!/bin/bash
#Script to sync local “ibay” folder with remote samba server
#Build 2018.08.03 - Daniele Lolli (UncleDan)

#Save last error code
GetErrorCode=$?

#Variables
backup=“ibay_rsync”
source="/var/lib/nethserver/ibay"
log="/var/log/last-ibay-sync.log"
hostname=$(hostname -f)
subject=“Backup $backup [$hostname]: LOG”
sender="backup@mydomain.com"
recipient="backup@mydomain.com"

#Do rsync

GetStartTime=$(date “+%Y-%m-%d %H:%M:%S”)
rsync -az --delete --stats --log-file=$log $source /mnt/backup
GetEndTime=$(date “+%Y-%m-%d %H:%M:%S”)

#Send mail
output="===== Report for “$backup” backup =====

Backup started at “$GetStartTime”
Backup ended at “$GetEndTime”

--------------[ Backup Log ]---------------------
“$(cat $log)”
-------------------------------------------------"

cat <<EOF | /sbin/sendmail -t -f $sender
To: $recipient
From: $sender
Subject: $subject
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=“utf-8”
$output
Log file: $log
EOF

exit $GetErrorCode

#Esecuzione automatica post backup
#Help hacking backup script for rsync

#dnutan (Marc)
#From the top of my head I think you could add an action for the post-backup-data event.
#Put your script in /etc/e-smith/events/actions/ with executable permission. Give it a descriptive name.
#Then link the action to the post-backup-data event:
#cd /etc/e-smith/events/post-backup-data/
#ln -s …/actions/ibay-rsync-mnt-backup S79ibay-rsync-mnt-backup
#Haven’t tested this but think it will work (unless you find a permission issue).