Restic Backup Save Data Error

stevieb413 · April 16, 2023, 3:49pm

NethServer Version: 7.9
Module: backup
Hey guys,

I’m trying to do a restic backup to a local hard drive however I continue to get the following errors, I have tried repartitioning and formating the local hard drive and the backup still fails any help with this issue would be greatly appreciated.

Backup: bkvol041623
Backup started at 2023-04-16 11:04:05
Pre backup scripts status: SUCCESS
created restic repository 6b9b5ef1e9 at /mnt/backup-bkvol041623

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
created new cache in /var/lib/nethserver/backup/restic/bkvol041623
no parent snapshot found, will read all files
Save(<data/f2753419db>) returned error, retrying after 552.330144ms: Sync: sync /mnt/backup-bkvol041623/data/f2/f2753419dbfba95ecfdde3d7f88ded914cce27bc5527c94b52155ca7ee9718ec: input/output error
Save(<data/86399b2fc6>) returned error, retrying after 720.254544ms: Sync: sync /mnt/backup-bkvol041623/data/86/86399b2fc6675c43e3d365792749e3dda7a0caf45a45ec32c7dc16c806fd34a8: input/output error
Save(<data/f2753419db>) returned error, retrying after 873.42004ms: OpenFile: open /mnt/backup-bkvol041623/data/f2/f2753419dbfba95ecfdde3d7f88ded914cce27bc5527c94b52155ca7ee9718ec: input/output error
Save(<data/86399b2fc6>) returned error, retrying after 703.28564ms: OpenFile: open /mnt/backup-bkvol041623/data/86/86399b2fc6675c43e3d365792749e3dda7a0caf45a45ec32c7dc16c806fd34a8: input/output error
Save(<data/86399b2fc6>) returned error, retrying after 1.040217184s: OpenFile: open /mnt/backup-bkvol041623/data/86/86399b2fc6675c43e3d365792749e3dda7a0caf45a45ec32c7dc16c806fd34a8: input/output error
Save(<data/f2753419db>) returned error, retrying after 1.335175957s: OpenFile: open /mnt/backup-bkvol041623/data/f2/f2753419dbfba95ecfdde3d7f88ded914cce27bc5527c94b52155ca7ee9718ec: input/output error
Save(<data/86399b2fc6>) returned error, retrying after 954.512469ms: OpenFile: open /mnt/backup-bkvol041623/data/86/86399b2fc6675c43e3d365792749e3dda7a0caf45a45ec32c7dc16c806fd34a8: input/output error
Save(<data/f2753419db>) returned error, retrying after 1.107876242s: OpenFile: open /mnt/backup-bkvol041623/data/f2/f2753419dbfba95ecfdde3d7f88ded914cce27bc5527c94b52155ca7ee9718ec: input/output error
Save(<data/86399b2fc6>) returned error, retrying after 1.511079094s: OpenFile: open /mnt/backup-bkvol041623/data/86/86399b2fc6675c43e3d365792749e3dda7a0caf45a45ec32c7dc16c806fd34a8: input/output error
Save(<data/f2753419db>) returned error, retrying after 2.027308147s: OpenFile: open /mnt/backup-bkvol041623/data/f2/f2753419dbfba95ecfdde3d7f88ded914cce27bc5527c94b52155ca7ee9718ec: input/output error
Save(<data/86399b2fc6>) returned error, retrying after 3.854635449s: OpenFile: open /mnt/backup-bkvol041623/data/86/86399b2fc6675c43e3d365792749e3dda7a0caf45a45ec32c7dc16c806fd34a8: input/output error
Save(<data/f2753419db>) returned error, retrying after 4.987726727s: OpenFile: open /mnt/backup-bkvol041623/data/f2/f2753419dbfba95ecfdde3d7f88ded914cce27bc5527c94b52155ca7ee9718ec: input/output error
Save(<data/86399b2fc6>) returned error, retrying after 4.067955962s: OpenFile: open /mnt/backup-bkvol041623/data/86/86399b2fc6675c43e3d365792749e3dda7a0caf45a45ec32c7dc16c806fd34a8: input/output error
Save(<data/f2753419db>) returned error, retrying after 5.015617898s: OpenFile: open /mnt/backup-bkvol041623/data/f2/f2753419dbfba95ecfdde3d7f88ded914cce27bc5527c94b52155ca7ee9718ec: input/output error
Save(<data/86399b2fc6>) returned error, retrying after 6.988645419s: OpenFile: open /mnt/backup-bkvol041623/data/86/86399b2fc6675c43e3d365792749e3dda7a0caf45a45ec32c7dc16c806fd34a8: input/output error
Save(<data/f2753419db>) returned error, retrying after 8.277195667s: OpenFile: open /mnt/backup-bkvol041623/data/f2/f2753419dbfba95ecfdde3d7f88ded914cce27bc5527c94b52155ca7ee9718ec: input/output error
Save(<data/86399b2fc6>) returned error, retrying after 10.034154425s: OpenFile: open /mnt/backup-bkvol041623/data/86/86399b2fc6675c43e3d365792749e3dda7a0caf45a45ec32c7dc16c806fd34a8: input/output error
Save(<data/f2753419db>) returned error, retrying after 10.163166574s: OpenFile: open /mnt/backup-bkvol041623/data/f2/f2753419dbfba95ecfdde3d7f88ded914cce27bc5527c94b52155ca7ee9718ec: input/output error
Save(<data/86399b2fc6>) returned error, retrying after 22.663987965s: OpenFile: open /mnt/backup-bkvol041623/data/86/86399b2fc6675c43e3d365792749e3dda7a0caf45a45ec32c7dc16c806fd34a8: input/output error
Save(<data/f2753419db>) returned error, retrying after 13.811796615s: OpenFile: open /mnt/backup-bkvol041623/data/f2/f2753419dbfba95ecfdde3d7f88ded914cce27bc5527c94b52155ca7ee9718ec: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 351.593438ms: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 645.653563ms: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 1.204507436s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 2.299204301s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 2.007570431s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 3.026422858s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 7.133794867s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 6.0363136s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 17.496021524s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 23.002952479s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
error while unlocking: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output errorFatal: unable to save snapshot: OpenFile: open /mnt/backup-bkvol041623/data/f2/f2753419dbfba95ecfdde3d7f88ded914cce27bc5527c94b52155ca7ee9718ec: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 511.910153ms: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 396.227312ms: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 740.619312ms: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 1.868490179s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 3.734205348s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 2.200112976s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 6.235277092s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 4.776550251s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 15.275143199s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Remove(<lock/604cdf39a9>) returned error, retrying after 15.406612236s: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
error in cleanup handler: Chmod: chmod /mnt/backup-bkvol041623/locks/604cdf39a96e1774034292e7ef62f70d1b03f7cc2a8a9122e370323ba6fd3ec9: input/output error
Backup failed
Action 'backup-data-restic bkvol041623': FAIL
Backup status: FAIL

Andy_Wismer · April 16, 2023, 4:28pm

Hi @stevieb413

As you should know, a local disk is never a good idea for backups - it may be a different disk, but is on the same host it’s backing up and probably also using the same controller chip…
(But that’s a different issue, some people are specialists in doing things a “special” way…)

But no information from you side WHAT you tried repartitioning and reformating the disk with…
Dis you use FAT?
Or maybe even ext4?
Did you mount that disk, and where?

You are aware, that NethServer uses XFS as file system?

I don’t use restic, I only use duplicity, and then to a NAS as a different host.
But I do know that NethServer does NOT like USB Backup disks mounted, it mounts them itself!

Maybe the formatting or mounting are issues!
Wasn’t restic to be able to use rsync to a different host - emphasis on the different host…?

My 2 cents
Andy

stevieb413 · April 16, 2023, 6:29pm

I do know local disk backups are not Ideal, I have a 32tb Truenas server in which I do daily restic bakups too. I have two sata hot swap-able drive bays that I was hoping I could use one of them to backup once a week and remove the the 2tb ssd drive for an off site backup. I have formated the drives XFS and ext4 I left the mounting option set to default.

Andy_Wismer · April 16, 2023, 7:20pm

Would be easier to have your NAS (I also use TrueNAS in some cases!) rsync your backups daily to another NAS offsite…
This evades any issues of incorrectly unmounted USB drives, but also issues with USB changing, if another device is plugged in…
And, your Backups are daily offsite, no matter if you forget them!

→ No hardware changes on running servers, not even plug-in of USB!
And: All my servers are virtualized on Proxmox, even those in the “Cloud”…
1st Backup: Proxmox PBS (And NAS!)
2nd Backup: NethServers Backup
3rd Backup My rsync job
All three land offsite, using different methods…
I do prefer it this way…

One Backup too many is always better than one backup too little!

My 2 cents
Andy

stevieb413 · April 16, 2023, 8:31pm

Thanks, I think thats what I will do build another NAS server and take it home and rsync to it for my off site backup.

Andy_Wismer · April 17, 2023, 6:10pm

@stevieb413

Don’t forget that doing a backup creates a lot of load on the NethServer, considering a backup was already made.

Rsync uses much less energy and CPU resources synching two sites than your server recreating another backup…

So you’re saving money, energy and getting more from your hardware, be it simplicity, reliability, etc…

My 2 cents
Andy