Backup-data: multiple schedule and backends


(Giacomo Sanchietti) #41

We can’t remove it, we must preserve backward compatibility with thousands servers.
But we can hide the single backup when we will implement a new UI using cockpit.

I agree, but this is the most flexible implementation. We will hide also this complexity.

Already removed in this commit: https://github.com/NethServer/nethserver-rsync/commit/2da9d79eff92f5dc0ee92a4e11b6b85a74487a8e
(Even if upstream is implementing it, but there are some bugs).

I forgot to remove it from the README, just done.

No one is. It’s bad hack for a problem without a simple solution.

No is not: the index is incremental, but still it’s way too slow!

This is true from the command line, but not from the UI which currently needs to create a tree.

This is an error inside the cron.d template, already fixed. Thank you for reporting!


(Michael Kicks) #42

question time…
Restic seems most interesting engine. Is there any plan to integrate a REST server into NethServer?


(Giacomo Sanchietti) #43

Not officially for now, but you can find the instructions here: https://github.com/NethServer/nethserver-restic/blob/master/README.rst


(Matthieu Gaillet) #44

Hi @giacomo :slight_smile:

Okay. Still not sure to see the backward compatibily issue (looks easier to migrate old settings to fit the new backup system) but I trust you.

Don’t forget the documentation.

Remember we already share some thoughts about this :

We could also use some Lucene on the fly indexing if we really want the UI be super responsive instead of building a file list when loading the UI page. I’m not sure it’s worth the trouble : someone wanting to restore some files will easily accept to wait some seconds when the list of files is being built.

My pleasure !


(Giacomo Sanchietti) #45

I just merged nethserver-restic and nethserver-rsync inside nethserver-backup-data package.
restic binary has been moved to a restic rpm which can be eventually switched for non-x86_64 architectures.

If you already installed the packages from testing, you need to execute the following:

yum --enablerepo=nethserver-testing clean all
rpm -e nethserver-restic nethserver-rsync --nodeps
yum --enablerepo=nethserver-testing update \*backup\*

(Héctor Pérez) #46

Ok, this is a work for my new virtualization environment :smiley: (an old CompaqCQ45 with 8Gbytes of ram), so hands on!!!


(Marc) #47

test case 2(a-c): OK

  • Program=restic
  • VFSType=cifs and nfs

test case 3(a-c):

  • Program=rsync
  • VFSType=cifs

Result: FAILED

  • Backup is done
  • Can list backup files (backup-data-list)
  • Cannot restore files: from server-manager it says the folders were restored but they were not. From CLI reports symlink problem.
results of restore-data command
# restore-data 
Restore started at 2018-07-21 20:16:03
Event pre-restore-data: SUCCESS
rsync: change_dir "/mnt/backup/server/latest" failed: No such file or directory (2)

Number of files: 0
Number of created files: 0
Number of deleted files: 0
Number of regular files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 20
Total bytes received: 12

sent 20 bytes  received 12 bytes  64.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]
Action '/etc/e-smith/events/actions/restore-data-rsync': SUCCESS

Cause of failure:
As reported and explained before, symlink error:

rsync_tmbackup: Backup completed without errors.
ln: failed to create symbolic link ‘/mnt/backup/server/latest’: Operation not supported
Action 'backup-data-rsync ': SUCCESS

test case 3(a-c):

  • Program=rsync
  • VFSType=nfs

Result: FAILED
Many chown failures (could be my test nfs server):

rsync: chown "/mnt/backup/server/2018-07-21-214239/root" failed: Operation not permitted (1)

Using single backup with any engine /var/log/last-backup.log is not created.
Log also shows Requested path not found:

esmith::event[4719]: Action: /etc/e-smith/events/pre-backup-data/S20nethserver-backup-config-predatabackup SUCCESS [1.203213]
esmith::event[4719]: Requested path not found

Many problems using webdav on single backup mode with any engine. I suspect the webdav server. Reports of open files exceeding max cache size, problems removing locks, or duplicity’s remote manifest not matching local one.