I’m setting up File Shares for various use cases. I can see in my /var/lib/nethserver/ibay/ folder each of the file shares I’ve setup. I’ll be backing up to S3 using Restic.
What I want to create is a backup task for each file share or folder in /ibay/ where each backup task will have their own start date and time and their retention. Where can I configure the specific folder in /var/lib/nethserver/ibay/ for each backup task.
For example:
Folder in /ibay/ called accountant:
Schedule backup of this folder nightly at midnight and cleanup backups older than 364 days
Folder in /ibay/ called db:
Schedule backup of this folder at 2am daily and cleanup backups older than 3 days.
Secondly I see that the scheduled backup in Restic won’t allow me to change the days to remove old backups between 1 day and 7 days. How can I add in more options here to give me the option to cleanup backups older than 2 or 3 or 4 days?
To confirm, I create a backup through Nethserver GUI called accountant and I setup this backup for the time and retention I want. Then from the terminal I create a file in /etc/backup-data/accountant.include. This file will include the path - /var/lib/nethserver/ibay/accountant. This will backup all inside the /accountant folder.
What do I put into the /etc/backup-data/accountant.exclude folder? Do I add in every single folder from the configuration to ensure nothing else is backed up?
I’ve been testing backups using includes files for each backup I’ve created. I have two that I’m testing out. Here are the details:
My Nethserver is version 7.8.2003 (fully updated today)
I’ve created two shared folders on my Nethserver. Shared Folder “Accounting” and Shared Folder “DB BKUP”
I’ve created two backup schedules on Nethserver Backup both going to the same S3 bucket for each shared folder. Accounting backup starts at Midnight and DB BKUP starts at 3am. Both backups run daily.
My Accounting backup scheduled job runs without errors each day.
My DB BKUP backup scheduled job fails with the following error:
Backup: DB BKUP
Backup started at 2020-05-27 3:00:00
Pre backup scripts status: SUCCESS
Fatal: create key in repository at s3:s3.symm.cloud/files failed: repository master key and config already initialized
Fatal: wrong password or no key found
Backup failed
Action ‘backup-data-restic DB BKUP’: FAIL
Backup status: FAIL
Like I mentioned above my Accounting backup goes to the same S3 bucket and has no Fatal Errors. I’ve walked through my configuration setup of the DB BKUP job and confirmed through the Test connection that I can successfully connect to my S3 bucket using the S3 Access Key and S3 Server Key with no problems.
Do you have any insight on where else I can look for the reason why the Nethserver Backup task for my DB BKUP fails with “wrong password or no key found”?
Is the problem that I can’t send two backup jobs to the same S3 bucket?
Thank you @mrmarkuz for this information. I wasn’t aware of this restriction/requirement. Is this a Nethserver requirement or something within S3 that won’t allow backups to go to the same bucket?
On my S3 I’ve added a folder to my bucket as you’ve suggested - bucket/dbbkup.
But when I test my connection from Nethserver S3 I get an error:
Configuration error. Check configuration properties and retry.
So it seems that I cannot add folders inside buckets from my Nethserver backup. Is this a Nethserver requirement to not allow all backups go all go into the same bucket?
I will also ask my S3 provider for comment but want to make sure I understand how Nethserver requires S3 configurations to be setup.
I’ve done more testing on backups to my S3. I cant’ schedule two backup jobs from Nethserver backup to the same bucket (even if the scheduled jobs are not running at the same time). I receive the error I’ve identified above. But I can backup to two separate buckets on my S3 from Nethserver backup. So although I have a solution and can use two buckets, is this a Nethserver requirement to not allow S3 backups using Restic to go into the same bucket? I’ve asked my S3 provider and they have not put any restriction on S3 to not allow this.
In earlier versions restic used a directory restic to put the snapshots in but I did not find a possibility to set it.
In NethServer one restic backup schedule is bound to one repository (storage backend). I think this was done to not complicate things.
If you can create buckets without big effort, I’d leave it like that. If not, we’ll find a solution.
EDIT:
Maybe it just works if you use the same password for both restic backups in /var/lib/nethserver/secret/restic_* instead of the autogenerated ones?
You may need to remove existing backups in the buckets first.