NethServer Version: NS8 2.8.2
Module: core
HP Microserver: Debian 12 and rotating disks… i.e slooooow.
I am testing the disaster recovery, but hit an error while trying to restore the Samba app.
I am using local storage for backup, a separate disk that is mounted as described in the admin manual. Backups have been working for a few days, and now i decided to replace the system disk and try a disaster recovery to another disk.
This seemed to work until i reached the point where i want to restore the Samba app.
Loki and Traefik restored just fine.
I manually tried to restore the files from the backup (samba restic repo.) and this is possible.
Below is the trace back i get.
Trace:
{“context”:{“action”:“restore-module”,“data”:{“node”:1,“path”:“samba/02e6ba65-b0e7-47b7-b543-2aec56b0fcd3”,“replace”:false,“repository”:“86d1a8ac-ef89-557a-8e19-8582ab86b7c4”,“snapshot”:“”},“extra”:{“completion”:{“extraTextParams”:[“node”],“i18nString”:“backup.instance_restored_to_node”,“outputTextParams”:[“module_id”]},“description”:“Restoring samba to Node 1”,“node”:“Node 1”,“title”:“Restore app”},“id”:“7584146a-8ee1-4807-be32-11a5c95eac33”,“parent”:“”,“queue”:“cluster/tasks”,“timestamp”:“2024-06-07T20:59:10.613685237Z”,“user”:“admin”},“status”:“aborted”,“progress”:0,“subTasks”:,“validated”:false,“result”:{“error”:“restic dump latest state/environment\nrclone: 2024/06/07 20:59:13 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 479.963567ms: unexpected EOF\nrclone: 2024/06/07 20:59:13 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 763.643566ms: unexpected EOF\nrclone: 2024/06/07 20:59:14 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 1.055919327s: unexpected EOF\nrclone: 2024/06/07 20:59:15 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 1.44527535s: unexpected EOF\nrclone: 2024/06/07 20:59:17 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 1.380758948s: unexpected EOF\nrclone: 2024/06/07 20:59:18 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 4.800885413s: unexpected EOF\nrclone: 2024/06/07 20:59:23 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 5.330041297s: unexpected EOF\nrclone: 2024/06/07 20:59:28 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 4.782222165s: unexpected EOF\nrclone: 2024/06/07 20:59:33 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 12.274319597s: unexpected EOF\nrclone: 2024/06/07 20:59:45 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 25.355794145s: unexpected EOF\nrclone: 2024/06/07 21:00:11 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nrclone: 2024/06/07 21:00:11 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 351.260609ms: unexpected EOF\nrclone: 2024/06/07 21:00:11 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 691.487789ms: unexpected EOF\nrclone: 2024/06/07 21:00:12 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 1.624208822s: unexpected EOF\nrclone: 2024/06/07 21:00:13 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 1.084583223s: unexpected EOF\nrclone: 2024/06/07 21:00:14 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 3.291357022s: unexpected EOF\nrclone: 2024/06/07 21:00:18 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 4.341660396s: unexpected EOF\nrclone: 2024/06/07 21:00:22 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 5.025249489s: unexpected EOF\nrclone: 2024/06/07 21:00:27 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 9.070132186s: unexpected EOF\nrclone: 2024/06/07 21:00:36 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 17.526606356s: unexpected EOF\nrclone: 2024/06/07 21:00:54 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 16.568317904s: unexpected EOF\nrclone: 2024/06/07 21:01:10 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nrclone: 2024/06/07 21:01:10 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 512.304663ms: unexpected EOF\nrclone: 2024/06/07 21:01:11 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 969.280114ms: unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 833.298915ms: unexpected EOF\nrclone: 2024/06/07 21:01:12 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 1.305641245s: unexpected EOF\nrclone: 2024/06/07 21:01:13 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nrclone: 2024/06/07 21:01:14 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 1.550922467s: unexpected EOF\nrclone: 2024/06/07 21:01:16 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 3.46617364s: unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 7.119309638s: unexpected EOF\nrclone: 2024/06/07 21:01:19 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 8.76536578s: unexpected EOF\nrclone: 2024/06/07 21:01:26 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 7.551778491s: unexpected EOF\nrclone: 2024/06/07 21:01:35 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nrclone: 2024/06/07 21:01:42 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nLoad(<lock/1d7c26825f>, 0, 0) returned error, retrying after 12.018186311s: unexpected EOF\nrclone: 2024/06/07 21:01:54 ERROR : locks/1d7c26825f8174150a99eb3a0dbe7738b237309a7a5bd7353400de8c4747184e: Didn’t finish writing GET request (wrote 0/145 bytes): unexpected EOF\nFatal: unable to create lock in backend: unexpected EOF\nTraceback (most recent call last):\n File "/var/lib/nethserver/cluster/actions/restore-module/50restore_module", line 46, in \n agent.run_restic(rdb, repository, path, ["–workdir=/srv"], ["dump", snapshot, "state/environment"], text=True, encoding=‘utf-8’, stdout=fenv, check=True)\n File "/usr/local/agent/pypkg/agent/init.py", line 255, in run_restic\n return subprocess.run(podman_cmd, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib/python3.11/subprocess.py", line 571, in run\n raise CalledProcessError(retcode, process.args,\nsubprocess.CalledProcessError: Command ‘[‘podman’, ‘run’, ‘-i’, ‘–rm’, ‘–name=restic-cluster-5177’, ‘–privileged’, ‘–network=host’, ‘–volume=restic-cache:/var/cache/restic’, ‘–log-driver=none’, ‘-e’, ‘RESTIC_PASSWORD’, ‘-e’, ‘RESTIC_CACHE_DIR’, ‘-e’, ‘RESTIC_REPOSITORY’, ‘-e’, ‘RCLONE_WEBDAV_URL’, ‘–workdir=/srv’, ‘Package restic · GitHub’, ‘dump’, ‘latest’, ‘state/environment’]’ returned non-zero exit status 1.\n”,“exit_code”:1,“file”:“task/cluster/7584146a-8ee1-4807-be32-11a5c95eac33”,“output”:“”}}
Br,
/erik
