Nethserver-cockpit Backup module don't take existing backup settings into account

NethServer Version: 7.6
Module: cockpit / backup

Hi there, title says all : it looks like the Backup module into cockpit isn’t aware of existing backup settings, even if it uses the new backup modules (restic / rsync / duplicity based)

For example I’ve a machine with that configuration

# config show backup-data
backup-data=configuration
BackupTime=03:00
CleanupOlderThan=28D
FullDay=0
IncludeLogs=disabled
NFSHost=ftpback-xxxxxx.ovh.net
NFSShare=/export/ftpbackup/nsxxxxxxxxxxxxxx.eu
Notify=always
NotifyFrom=
NotifyTo=me@somewhere.com
Program=restic
Prune=6
SMBHost=
SMBLogin=
SMBPassword=
SMBShare=
Type=incremental
USBLabel=
VFSType=nfs
VolSize=250
WebDAVLogin=
WebDAVPassword=
WebDAVUrl=
status=enabled

But cockpit warns that I should configure “at least one data backup”.

Worse, I once ended up with two concurrent backup settings running altogether.

What do you think ?

Matt

It’s not really like this: if you configured some multiple backups from command line, everything will show up in the UI.

Cockpit doesn’t take care only of old backup-data and this is by design.
We are planning to automatic port the configuration of backup-data into the multiple backups, but it requires some hours of analysis and then much more hours of implementation and testing.
I will work on this later in the new year.

This shouldn’t be really a problem except for the load of the machine.

Hi,

I installed the new server management module for 7.6. It reported that there was no data backup available, but it was configured and run without errors. I disabled the data backup config on the 7.5 module and enabled on 7.6 with the same settings. This solved the problem (last night backup was OK).

regards
Thorsten

2 Likes

Perfect. I understand it’s an alpha stage and that it will be improved later.

Could lead to free space exhausted on the target disk.

1 Like

@pagaille @thorsten the feature is ready for testing, please help us!

If you try anyone of the test case, please report it here :wink:

4 Likes

Will do ! :slight_smile: Thanks Giacomo !

2 Likes

On a system where I configured a ’new’ backup from command line a long time ago, if I enable a backup on the old server manager it doesn’t get listed in cockpit. Is this normal ? I only see my rsync based "backup-data” configuration, not the legacy one.

Maybe is it because I choose the name ‘backup-data’ which happens to be the one used by the Server Manager ?

Lastly I’m afraid to click on the restore button on Cockpit because last time I did that it started a restore out of the blue without asking anything ! I expected a folder or file selection dialog and found this particularly misleading :slight_smile:

Backup burned completely down on my end - doesn’t see me external USB disk anymore - I try to reformat. Also no backup was running last week.

In the night to today, a backup (new interface, USB disk) was created, see logfile:

Backup: my_test_backup
Backup started at 2019-01-17 01:05:01
Pre backup scripts status: SUCCESS
--------------[ Backup Statistics ]--------------
StartTime 1547683863.69 (Thu Jan 17 01:11:03 2019)
EndTime 1547696076.16 (Thu Jan 17 04:34:36 2019)
ElapsedTime 12212.48 (3 hours 23 minutes 32.48 seconds)
SourceFiles 109149
SourceFileSize 202768528510 (189 GB)
NewFiles 109149
NewFileSize 202768528510 (189 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 109149
RawDeltaSize 202757440721 (189 GB)
TotalDestinationSizeChange 181032860132 (169 GB)
Errors 0
-------------------------------------------------

Action 'backup-data-duplicity my_test_backup': SUCCESS
Post backup scripts status: SUCCESS
Backup status: SUCCESS
Backup ended at 2019-01-17 04:38:03
Time elapsed: 3 hours, 33 minutes, 2 seconds

However restore was not possible: Clicking to restore leads to some unknown process.

I was expecting some kind of selecting the files / directory to be restored.
Using the old interface, it was possible to see / find / select some directory to be restored:

As you can see, the restore seems to be sucessfull, but it was not - no folder was created.

I hope this description helps. I can offer to show system setup by teamviewer on the evening.

Best regards
Thorsten

I’m not sure i fully understand your config. Could you please paste the content of these commands (make sure to hide the passwords before pasting)?

  • config show backup-data
  • db backups show

Yes probably: backup-data is kind of reserved special key. If a backup is named like this, it is visible from the old Server Manager.

There is no file/directory selection, the restore button will restore the entire system.
But I agree with you: a further confirmation is needed. Added to the todo list :wink:

Is this relative to the backup package from testing?

Do you mean the restore from cockpit?

I know, probably we will not have something like this for a long time.
We need to find more efficient and maintainable ways to implement the selective restore feature.

So the restore doesn’t work anymore after migration using the testing package. Am I right?
In this case, can you post the output of above commands?

1 Like
# config show backup-data
backup-data=configuration
    IncludeLogs=enabled
# db backups show
backup-data=rsync
    BackupTime=0 23 * * *
    CleanupOlderThan=14D
    FullDay=1
    NFSHost=
    NFSShare=
    Notify=error
    NotifyFrom=
    NotifyTo=root@localhost
    SMBHost=
    SMBLogin=
    SMBPassword=
    SMBShare=
    Type=incremental
    USBLabel=external
    VFSType=usb
    VolSize=250
    WebDAVLogin=
    WebDAVPassword=
    WebDAVUrl=
    status=disabled

@giacomo, @thorsten, I also tried to use the restore function from the old server manager, it sees my rsynced usb drive, it says it restored the chosen folder or file to /original/folder_2019xxxx in green when I click restore, but nothing really happens, the file or folder isn’t created.

The configuration seems good but the backup is disabled.

I will look into the restore function next week (that is really a nightmare!)

OK, same as here.
In fact, without the change to selectively restore folders, I do not dare to switch over to the backup in the new environment. Backup is a basic / essential function. I returned to old backup for safety purposes. And it was quite hard to mange. I had to reformat my USB backup drive - which was quite difficult.
New backup does this quite well, while an ext3 / ext4 formatted partition was not accepted by the old module.

Besides the fact that the disk is empty, old backup sees earliert backups which of course I can not access… I hope that everything will be back on duty after 30 days …

1 Like

Should be fixed now, to install the fix:

yum --enablerepo=nethserver-testing clean all
yum --enablerepo=nethserver-testing install nethserver-restore-data

Thank you for your patience!

Working here ! :slight_smile:

3 Likes

Thank you Matthieu!

Also @stephdl is doing a tremendous work on this issue and already nailed various bugs!
With the help from @davidep we should be ready to release the update in few days :wink:

5 Likes

Found a bug.

Tried to initialise a new local disk using cockpit (regular one, not from testing repo). The disk is 3To, no partitions, just a MBR.

logs :

Jan 31 16:47:59 appserver cockpit-bridge: Checking that no-one is using this disk right now ...                                       │
Jan 31 16:47:59 appserver kernel: sdb:                                                                                                │
Jan 31 16:47:59 appserver cockpit-bridge: OK                                                                                          │
Jan 31 16:47:59 appserver cockpit-bridge: Warning: partition 1 has size 24.0 TB (24004459741184 bytes),                               │
Jan 31 16:47:59 appserver cockpit-bridge: which is larger than the 17592186040320 bytes limit imposed                                 │
Jan 31 16:47:59 appserver cockpit-bridge: by the DOS partition table for 4096-byte sectors                                            │
Jan 31 16:47:59 appserver cockpit-bridge: sfdisk: I don't like these partitions - nothing changed.                                    │
Jan 31 16:47:59 appserver cockpit-bridge: (If you really want this, use the --force option.)   

Screen cap :

1 Like

Think I found another one.

When creating a SFTP backup, I left the password field empty so that sftp would the ssh public key (feature supported as far as I can see when reading the field tooltip).

However, when clicking “test”, the page complains that the field is empty !

Correction. Meanwhile I found the explanation by reading the doc : http://docs.nethserver.org/projects/nethserver-devel/en/latest/nethserver-backup-data.html#sftp

Maybe the tooltip should state that “first connexion must be done by password. Subsequent will use public key” or something.

2 Likes

Should be fixed by this:

4 Likes