SFTP backup fails

capote · September 28, 2021, 8:31am

NethServer Version: 7.9
I have been using a SFTP backup to another server (an additional simple hosting package from my provider, not a different Nethserver) for a year now.
Since a few days this backup fails.


Pre backup scripts status: SUCCESS
subprocess ssh: ssh: connect to host 11.222.33.44 port 22: Connection timed out
Fatal: create repository at sftp:ftpuser@11.222.33.44 :/bak/ns-serv01 failed: unable to start the sftp session, error: EOF

I did not change the credentials, but a few days earlier the provider changed the certificate.

What can I do to fix the problem?

Sincerely, Marko

mrmarkuz · September 28, 2021, 9:09am

It seems the backup can’t connect to port 22…

May I ask which provider you use? Is the cert valid?

Does it work with another SSH/SFTP client?

You could try to create a testbackup to another directory for testing.

capote · September 28, 2021, 9:17am

The same address and port is via ftp program reachable.

Is the cert valid?

Yes freshly renewed on Sept. 16.

You could try to create a testbackup to another directory for testing.

No, same errors.

It seems to me that there is a relation between the occurrence of the errors and the renewal of the certificate on the backup server

I have also tested new credentials:

via SFTP client access is possible.
via Nethserver the access fails

mrmarkuz · September 28, 2021, 10:14am

Did you try to ssh to the backup provider from the Neth CLI?

ssh ftpuser@11.222.33.44

This looks wrong, shouldn’t it be sftp://ftpuser@11.222.33.44 ?

capote · September 28, 2021, 11:36am

yes, it works

This looks wrong, shouldn’t it be sftp://ftpuser@11.222.33.44 ?

It was quoted from the logfile. i checked it again… its the right quotation.

mrmarkuz · September 28, 2021, 1:57pm

Do you use restic or rsync backup?

The backup commands are in /etc/e-smith/events/actions/backup-data-*. Maybe try on CLI with verbose switch to see more error details

If you like, you can pm me credentials of your provider so I can check it…

capote · September 28, 2021, 5:00pm

Hello MArkus, thank you for your offer. I may come back to this.

In the meantime …

I have deleted the backup job completely and created a new rsync one. At least the configuration passesd the check and the job could be saved. Shortly after the start, however, the abort.
I have deleted the backup job completely and created a new restic one. Now the job runs.

Thank you very much, Marko

capote · September 28, 2021, 5:13pm

for the sake of completeness…
Backup: ns-serv01-data-bak


Backup started at 2021-09-28 15:34:42
Pre backup scripts status: SUCCESS
df: ‘’: No such file or directory
-: df: command not found
rsync_tmbackup: No previous backup - creating new one.
rsync_tmbackup: Creating destination ftpuser@11.222.33.44:/bak/ns-serv01/2021-09-28-153517
rsync_tmbackup: Starting backup…
rsync_tmbackup: From: /
rsync_tmbackup: To: ftpuser@11.222.33.44:/bak/ns-serv01/2021-09-28-153517/
/usr/bin/rsync_tmbackup: line 547: -D --numeric-ids --links --hard-links --one-file-system --times --recursive --perms --owner --group --stats --human-readable --files-from=/tmp/03GkdURiGD --exclude-from=/tmp/oc0_wr9z3N =-D --numeric-ids --links --hard-links --one-file-system --times --recursive --perms --owner --group --stats --human-readable --files-from=/tmp/03GkdURiGD --exclude-from=/tmp/oc0_wr9z3N --compress: No such file or directory
rsync_tmbackup: Running command:
rsync_tmbackup: rsync -e ‘ssh -p 22 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null’ -D --numeric-ids --links --hard-links --one-file-system --times --recursive --perms --owner --group --stats --human-readable --files-from=/tmp/03GkdURiGD --exclude-from=/tmp/oc0_wr9z3N --log-file ‘/root/.rsync_tmbackup/2021-09-28-153518.log’ – ‘/’ ‘ftpuser@11.222.33.44:/bak/ns-serv01/2021-09-28-153517/’
-: rsync: command not found
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: remote command not found (code 127) at io.c(226) [sender=3.1.2]
rsync_tmbackup: [ERROR] Rsync reported an error. Run this command for more details: grep -E ‘rsync:|rsync error:’ ‘/root/.rsync_tmbackup/2021-09-28-153518.log’
Backup failed
Action ‘backup-data-rsync ns-serv01-data-bak’: FAIL

Backup status: FAIL

mrmarkuz · September 28, 2021, 7:07pm

Do df and rsync exist on remote side? You may check by ssh into the backup server and just enter the commands to see if they exist.

capote · September 28, 2021, 8:04pm

in deed… Wer lesen kann ist klar im Vortei -)

bash-4.4$ df bash: df: command not found bash-4.4$ rsync bash: rsync: command not found

But it doesn’t explain why the restic backup worked for a year, then it didn’t and now it works again after a new installation.

mrmarkuz · September 28, 2021, 8:15pm

From the documentation:

Connection to remote host uses a specific public key. A password is needed only once to copy the
public key to the remote host. SSH client configuration is added to /etc/ssh/sshd_config .

I guess the key changed and the connection wasn’t trusted anymore. After new setup the SSH connection was renewed and now it works again…

capote · September 29, 2021, 1:33pm

Is it possible to re-trust w/o uninstallation?

mrmarkuz · September 29, 2021, 2:27pm

On the backup host you should find the authorized keys ssh configuration of the sftpuser in ~/.ssh/authorized_keys. In this file you may remove the line with sftpuser@yournethserver at the end.
Then reenter the password in the backup UI and start the backup once.
It works when you can login via ssh from your Nethserver to you backup host as sftpuser without password.

If a backup immediately stops, it could also be permissions issue, on the backup host you need to set write permissions for sftpuser for the SFTP directory.

capote · October 5, 2021, 12:58pm

Unfortunately, this curious behavior occurred again.
I cleared the ~/.ssh/authorized_keys completely. The reentering the password on backup GUI didn’t help.

write permissions for sftpuser for the SFTP directory are set correctly, unchanged.

mrmarkuz · October 5, 2021, 2:01pm

Did you check the logfile?

capote · October 6, 2021, 6:51am

The rsync log file is not valid, because I used finally restic backup. A restic log file I cannot find.

The strange thing is, that it worked until now without any problems, but now there is no connection, even if I delete the bakup job and create a new one.

SSH from my local console is possible without any problems. SSH from the Nethserver web console does not establish a connection.

capote · October 6, 2021, 8:43am

you can’t think as stupid as reality is - I have whitelisted the IP for the backup destination in thread shield, now the backup job is started and runs. The connection to the destination is established.