[solved] Data-Backup to USB-disk needs the same space at local file-system for "security-copy"

Support
,
#2

Hi @tscheepy
if u have an USB backup, backup engine do a backup copy into a local folder /mnt/backup-backup-data/nethserver before sync it with USB device.
You can decide to change your backup support and use a CIFS or NFS disk (like QNAP).

If you can’t change backup support I suggest you to clean USB device and use restic as backup engine and take 1 week of backup.
To clean USB device you can refer to this doc: http://docs.nethserver.org/en/v7/backup.html#usb-disk-configuration reading configuration for restic engine.
Then you have to clean /mnt/backup-backup-data: you can use rm -rf /mnt/backup-backup-data to remove backup folder.
Apply new restic engine using: db backups settype backup-data restic and then signal-event nethserver-backup-data-save

But I don’t believe that it solve the problem because you have not enough space to complete a backup.
If you don’t need to backup all data you can try to use exclusions (http://docs.nethserver.org/en/v7/backup.html#exclusion), but you can always try to see behavior differences between duplicity and restic.

Best practice would be a Network disk with restic configured. Remember that backup partition on disk has to be cleaned before use restic that is more fast than duplicity with lot of quantity of data.

P.s. If you have a router that permit to share media you can share an USB disk from router, but NethServer will be connect to it by CIFS or NFS (it depends by router model). In CIFS or NFS configuration NethServer doesn’t copy backup to a local folder: if the disk isn’t online backup fails.

1 Like
#3

Okay, thanks! Is this new to the version I have installed or is it the same way my backups were stored before the update to 7.6.1810?

Do you mean that restic also copies into a local folder first, before syncing with an external storage?

#4

Yes, I think it was the same before update.

Yes, I don’t believe that it depends by backup engine, but it depends by backup support.
You can always try.

#5
  • pre-backup-data event: prepare the system (eg. dump of mysql tables)
  • /etc/e-smith/events/actions/backup-data-<program> action: execute the backup This actions must implement full/incremental logic and should also take care to mount and umount the destination
  • post-backup-data : cleanup. Actions in this event can also implement retention policies

(http://docs.nethserver.org/projects/nethserver-devel/en/v7/nethserver-backup-data.html#backup)

Is writing a copy of the full backup to the local file-system part of the “pre-backup-data event”?

#6

No. Pre-backup data prepare the backup (e.g. mysql dump). Backup-data copy on USB via local folder.

#7

Is there a way to stop a working backup-engine from the command line? Now the system does a backup to the local file system and I don’t want to delete the created duplicity-files manually anymore.

#8

Backup is launch from Server Manager or CLI?
If from CLI use CTRL + C.
If is launch from Server Manager kill relative process.or reboot the server.

Update me with what do you decide to do relative to my post #2

#9

It was enabled through server-manager. Meanwhile I disabled it through server-manager, but there is still a working backup. I can see the created files in /mnt/backup-backup-data/nethserver at the local file system.

#10

See previous answer.

#11

I think a share USB disk from router seems to be a good solution for me.
But before I deal with this I try to read and understand the docs related to backup using usb storage better. I wonder why I cannot find any information about the needed space at the local file system, although the backup destination is an external storage. I think this is an very important information. It means: The more you data grows the more space at the local file system is needed. That is a problem for everybody who holds a root-partition that is big enough to store the nethserver system files and not more.

#12

Before I change my backup-setup, I will repeat the steps preparing a disk for usb storage as explained in the document above

#13

@tscheepy Prepare the disk for restic and set it as backup engine: it’s faster than duplicity.

#14

Duplicity seems not to be the problem. When I run
duplicity /var/lib/nethserver file:///mnt/backup/nethserver
from the CLI it does exactly what I want. My Backup is stored at the external storage without any kind of storing the backup at the local file system before. Is this possible using the backup module through server-manager, too?

#15

How I say, I think the “problem” is not duplicity or restic, but backup-data command that do a “secure copy” before write it into USB device.
I don’t suggest to modify backup-data because every update will overwrite your modify.
If you can’t use any other suggested method you can decide to set cron commands like this

and use it to backup all you need. You need to backup your data folders (including eventually nextcloud and other apps) and also config backup that you can find into /var/lib/nethserver/backup
Disabling backup-data and using crontab you’ll bypass the problem, but it’s not a suggested method!

#16

Okay, then I won’t go this way.

I will try a network solution via router, because there is no “secure copy” to the file-system, as you told me.
Till I get this working, I must do it from CLI.
Cron is new to me. Could you give me a link to a useful document concerning cron, please.

As I could read here “all relevant files are saved under /var/lib/nethserver/ directory”: Mails, Shared Folders and even Nextcloud-files.

And even system configuration backup is stored under /var/lib/nethserver.

Is this documented anywhere? I think it’s important to know before planning a backup-setup.

#17

You can simply search on google “Crontab” and “Crontab CentOS 7”.

Yes, /var/lib/nethserver should be enough.

As you can see the behavior is that files will be copied also into /mnt/backup-backup-data before sync to USB device. You can also refer to this topic.

Regards.

#18

Yes, that’s true without any doubt. And without your help I would not have been able to find that out. What I mean is that this behaviour could be explained here, too, because it’s very important to know, obviously.

Thank you.

1 Like
#19

Could you do a pull request at github?

#20


I think dev_team has solved this issue.

1 Like
#21

It would be nice, if the backup-engine could check the space at the local file-system before writing the security-copy. If there is not enough space on it, it could give an error-message and ask the user if it’s okay when the backup is written to the external USB-disk directly without copying to the local file-system before or the user could have the opportunity to disable the security-copy completely.
The solution above seems to be useful in the case, that there is no external disk to store the backup. As it does not avoid writing backup-data to local file-system, if there is not enough space, it is not a solution for my problem, isn’t it? Maybe I’m wrong.