Pre Backup Failure

kiemosan · February 19, 2019, 8:51am

NethServer Version: 7.6
Module: backup

Hi,

Since I have another backup in place at the VM level I have not worried about this but now have some time to fix it.

For a while my internal backups have failed every day. I have this in the email report:

Backup: system
Backup started at 2019-02-19 08:29:10
Event pre-backup-data: FAIL
Backup status: FAIL

I search logs for “Pre-backup-data” and find the following:

Feb 19 08:29:11 remote esmith::event[31011]: Event: pre-backup-data system Feb 19 08:29:24 remote esmith::event[31011]: Action: /etc/e-smith/events/pre-backup-data/S20nethserver-backup-config-predatabackup SUCCESS [13.296056] Feb 19 08:30:43 remote esmith::event[31011]: Action: /etc/e-smith/events/pre-backup-data/S20nethserver-restore-data-duc-index SUCCESS [78.537928] Feb 19 08:30:45 remote esmith::event[31011]: Action: /etc/e-smith/events/pre-backup-data/S50mysql-dump-tables SUCCESS [2.201328] Feb 19 08:31:33 remote esmith::event[31011]: Action: /etc/e-smith/events/pre-backup-data/S50nethserver-ibays-dump-acls FAILED: 1 [48.222584] Feb 19 08:31:35 remote esmith::event[31011]: Action: /etc/e-smith/events/pre-backup-data/S80nethserver-sogo-backup-users SUCCESS [1.691798] Feb 19 08:31:35 remote esmith::event[31011]: Event: pre-backup-data FAILED

The key failure seems to be:

Action: /etc/e-smith/events/pre-backup-data/S50nethserver-ibays-dump-acls FAILED

but I don’t know where to go after this to find and fix the problem?

Can anyone point me in the right direction?

Thanks

K

federico.ballarini · February 19, 2019, 9:02am

Backup never works or after a specific day stop working?

kiemosan · February 19, 2019, 9:11am

Honestly I can’t remember. I think it worked one or two times?

federico.ballarini · February 19, 2019, 9:24am

Could you post extended /var/log/messages?

kiemosan · February 19, 2019, 9:33am

Feb 19 08:28:41 remote /sbin/e-smith/db[30871]: /var/lib/nethserver/db/backups: DELETE backup-data=duplicity|BackupTime|30 14 * * *|CleanupOlderThan|7D|FullDay|1|NFSHost||NFSShare||Notify|always|NotifyFrom|backup@itwerx.co.uk|NotifyTo|john.savidge@itwerx.co.u|SMBHost|192.168.1.5|SMBLogin|administrator|SMBPassword|Hellonasty?2189|SMBShare|Neth_Backup|Type|incremental|USBLabel||VFSType|cifs|VolSize|250|WebDAVLogin||WebDAVPassword||WebDAVUrl||status|enabled 
Feb 19 08:28:41 remote esmith::event[30872]: Event: nethserver-backup-data-save 
Feb 19 08:28:42 remote esmith::event[30872]: expanding /etc/backup-data.d/nethserver-backup-data.include 
Feb 19 08:28:42 remote esmith::event[30872]: expanding /etc/cron.d/backup-data 
Feb 19 08:28:42 remote esmith::event[30872]: expanding /etc/davfs2/davfs2.conf 
Feb 19 08:28:42 remote esmith::event[30872]: expanding /etc/davfs2/secrets 
Feb 19 08:28:42 remote esmith::event[30872]: expanding /etc/ssh/ssh_config 
Feb 19 08:28:42 remote esmith::event[30872]: Action: /etc/e-smith/events/actions/generic_template_expand SUCCESS [0.613761] 
Feb 19 08:28:42 remote esmith::event[30872]: Action: /etc/e-smith/events/nethserver-backup-data-save/S40nethserver-backup-data-ssh SUCCESS [0.003008] 
Feb 19 08:28:42 remote esmith::event[30872]: Event: nethserver-backup-data-save SUCCESS
Feb 19 08:29:11 remote esmith::event[31011]: Event: pre-backup-data system 
Feb 19 08:29:11 remote esmith::event[31011]: ===== Report for configuration backup ===== 
Feb 19 08:29:11 remote esmith::event[31011]: Backup started at 2019-02-19 08:29:11 
Feb 19 08:29:11 remote esmith::event[31014]: Event: pre-backup-config 
Feb 19 08:29:11 remote esmith::event[31014]: expanding /etc/backup-config.d/nethserver-sssd.include 
Feb 19 08:29:12 remote esmith::event[31014]: WARNING in /etc/e-smith/templates//etc/backup-config.d/nethserver-sssd.include/20kerberos: Use of uninitialized value in string ne at /etc/e-smith/templates//etc/backup-config.d/nethserver-sssd.include/20kerberos line 5. 
Feb 19 08:29:12 remote esmith::event[31014]: WARNING: Template processing succeeded for //etc/backup-config.d/nethserver-sssd.include: 1 fragment generated warnings 
Feb 19 08:29:12 remote esmith::event[31014]: at /etc/e-smith/events/actions/generic_template_expand line 64. 
Feb 19 08:29:12 remote esmith::event[31014]: Action: /etc/e-smith/events/actions/generic_template_expand SUCCESS [0.313992] 
Feb 19 08:29:12 remote esmith::event[31014]: Action: /etc/e-smith/events/pre-backup-config/S40nethserver-dc-pre-backup SUCCESS [0.719403] 
Feb 19 08:29:12 remote esmith::event[31014]: Action: /etc/e-smith/events/pre-backup-config/S40nethserver-mail-shrmbx-cfgbackup SUCCESS [0.047059] 
Feb 19 08:29:13 remote esmith::event[31014]: Action: /etc/e-smith/events/pre-backup-config/S40nethserver-sssd-backup-tdb SUCCESS [0.076505]

giacomo · February 19, 2019, 1:05pm

Try to execute the following and post the output, let’s try to figure out where is failing:

bash -x  /etc/e-smith/events/pre-backup-data/S50nethserver-ibays-dump-acls

kiemosan · February 19, 2019, 2:10pm

I don’t remember installing Plex…

[root@remote ~]# bash -x /etc/e-smith/events/pre-backup-data/S50nethserver-ibays-dump-acls

cd /var/lib/nethserver/ibay/
getfacl -R -P --skip-base .

getfacl: ./General/Plex Media Server/Metadata/Albums/c/8335a34385a190bc8c197da1fa9de6aa11e3c5b.bundle/Contents/_combined/posters: Structure needs cleaning

getfacl: ./General/Plex Media Server/Metadata/Albums/c/8335a34385a190bc8c197da1fa9de6aa11e3c5b.bundle/Contents/_combined/tracks/1b5d7197b1317f734c47f7aa13c9db486466508e: Structure needs cleaning

getfacl: ./General/Plex Media Server/Metadata/Albums/c/8a3142bc9a3f320ff00379003ee0c4a144bc445.bundle/Contents: Structure needs cleaning

getfacl: ./General/Plex Media Server/Metadata/Albums/c/8baa9e6c7d0c7b9843cc7e63f28146987e2e91e.bundle/Contents/com.plexapp.agents.localmedia/posters: Structure needs cleaning

getfacl: ./General/Plex Media Server/Plug-in Support/Caches/com.plexapp.agents.thetvdb/HTTP.system/ec/bc7828946dd3f32b03a52dddf79a3f2cc2333e_attributes: Structure needs cleaning

getfacl: ./General/Plex Media Server/Plug-in Support/Caches/com.plexapp.agents.thetvdb/HTTP.system/ec/e6697007bb804a9e1e2ad8032c078f05ccce45_attributes: Structure needs cleaning

getfacl: ./General/Plex Media Server/Plug-in Support/Caches/com.plexapp.agents.thetvdb/HTTP.system/ec/fd0878fb9defc5e17935bbee6ba1cddc7bd023_attributes: Structure needs cleaning

getfacl: ./General/Plex Media Server/Plug-in Support/Caches/com.plexapp.agents.thetvdb/HTTP.system/f0: Structure needs cleaning

filippo_carletti · February 19, 2019, 3:05pm

Your file system is corrupt. You can try to fix it with a reboot (xfs repairs on mount) or following any guide on the net (https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/storage_administration_guide/xfsrepair).

xfs corruption is extremely rare, you should check your hardware.

kiemosan · February 20, 2019, 8:41am

Thank you, I will investigate!