New problem with production server re; missing module / backups

Support
,
1

NethServer Version: 7.4
Module: ?

Opened the gui to see;

NSmissingmodules.jpg761×339 39.2 KB

This is v7.4, note pic shows 7.3 but it was updated to 7.4 last week, it’s just not showing it.

Messages log shows an error during last nights backup, though the backup, apparently, successfully completed.

I guess it goes without saying that because this is a production server, I’m a little concerned.

Anyone have any input?

Oct  3 21:00:02 server7c esmith::event[12910]: Event: pre-backup-data
Oct  3 21:00:02 server7c esmith::event[12913]: Event: pre-backup-config
Oct  3 21:00:02 server7c esmith::event[12913]: expanding /etc/backup-config.d/nethserver-sssd.include
Oct  3 21:00:03 server7c dbus[682]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service'
Oct  3 21:00:03 server7c dbus-daemon: dbus[682]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service'
Oct  3 21:00:03 server7c systemd: Starting Time & Date Service...
Oct  3 21:00:03 server7c dbus[682]: [system] Successfully activated service 'org.freedesktop.timedate1'
Oct  3 21:00:03 server7c dbus-daemon: dbus[682]: [system] Successfully activated service 'org.freedesktop.timedate1'
Oct  3 21:00:03 server7c systemd: Started Time & Date Service.
Oct  3 21:00:03 server7c esmith::event[12913]: Action: /etc/e-smith/events/actions/generic_template_expand SUCCESS [0.596119]
Oct  3 21:00:05 server7c esmith::event[12913]: Action: /etc/e-smith/events/pre-backup-config/S40nethserver-dc-pre-backup SUCCESS [2.450877]
Oct  3 21:00:05 server7c esmith::event[12913]: Action: /etc/e-smith/events/pre-backup-config/S40nethserver-sssd-backup-tdb SUCCESS [0.051204]
Oct  3 21:00:05 server7c esmith::event[12913]: error: rpmdb: BDB0113 Thread/process 2691/140578306205696 failed: BDB1507 Thread died in Berkeley DB library
Oct  3 21:00:05 server7c esmith::event[12913]: error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
Oct  3 21:00:05 server7c esmith::event[12913]: error: cannot open Packages index using db5 -  (-30973)
Oct  3 21:00:05 server7c esmith::event[12913]: error: cannot open Packages database in /var/lib/rpm
Oct  3 21:00:05 server7c esmith::event[12913]: error: rpmdb: BDB0113 Thread/process 2691/140578306205696 failed: BDB1507 Thread died in Berkeley DB library

…

Oct  3 21:00:06 server7c esmith::event[12913]: error: cannot open Packages database in /var/lib/rpm
Oct  3 21:00:06 server7c esmith::event[12913]: error: rpmdb: BDB0113 Thread/process 2691/140578306205696 failed: BDB1507 Thread died in Berkeley DB library
Oct  3 21:00:06 server7c esmith::event[12913]: error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
Oct  3 21:00:06 server7c esmith::event[12913]: error: cannot open Packages database in /var/lib/rpm
Oct  3 21:00:06 server7c esmith::event[12913]: Action: /etc/e-smith/events/pre-backup-config/S50nethserver-backup-config-list-packages SUCCESS [0.303938]
Oct  3 21:00:06 server7c esmith::event[12913]: Event: pre-backup-config SUCCESS
Oct  3 21:00:15 server7c esmith::event[13219]: Event: post-backup-config
Oct  3 21:00:16 server7c esmith::event[13219]: Action: /etc/e-smith/events/post-backup-config/S40nethserver-dc-post-backup SUCCESS [0.07204]
Oct  3 21:00:16 server7c esmith::event[13219]: Event: post-backup-config SUCCESS
Oct  3 21:00:16 server7c esmith::event[12910]: Action: /etc/e-smith/events/pre-backup-data/S20nethserver-backup-config-predatabackup SUCCESS [13.824295]
Oct  3 21:00:32 server7c esmith::event[12910]: Requested path not found
Oct  3 21:00:32 server7c esmith::event[12910]: Action: /etc/e-smith/events/pre-backup-data/S20nethserver-restore-data-duc-index SUCCESS [16.767872]
Oct  3 21:00:35 server7c esmith::event[12910]: Action: /etc/e-smith/events/pre-backup-data/S50mysql-dump-tables SUCCESS [2.499959]
Oct  3 21:00:35 server7c esmith::event[12910]: Action: /etc/e-smith/events/pre-backup-data/S50nethserver-ibays-dump-acls SUCCESS [0.102939]
Oct  3 21:00:35 server7c kernel: Key type dns_resolver registered
Oct  3 21:00:35 server7c kernel: Key type cifs.spnego registered
Oct  3 21:00:35 server7c kernel: Key type cifs.idmap registered
Oct  3 21:00:35 server7c esmith::event[12910]: Action: /etc/e-smith/events/pre-backup-data/S70mount-cifs SUCCESS [0.450621]
Oct  3 21:00:36 server7c esmith::event[12910]: Action: /etc/e-smith/events/pre-backup-data/S70mount-nfs SUCCESS [0.249375]
Oct  3 21:00:36 server7c esmith::event[12910]: Action: /etc/e-smith/events/pre-backup-data/S70mount-usb SUCCESS [0.250161]
Oct  3 21:00:36 server7c esmith::event[12910]: Action: /etc/e-smith/events/pre-backup-data/S70mount-webdav SUCCESS [0.245629]
Oct  3 21:00:36 server7c esmith::event[12910]: Event: pre-backup-data SUCCESS
Oct  3 21:01:01 server7c systemd: Started Session 26 of user root.
Oct  3 21:01:01 server7c systemd: Starting Session 26 of user root.
Oct  3 21:01:07 server7c esmith::event[13406]: Event: post-backup-data
Oct  3 21:01:08 server7c esmith::event[13406]: Action: /etc/e-smith/events/post-backup-data/S20nethserver-backup-config-push2history SUCCESS [1.603882]
Oct  3 21:01:09 server7c esmith::event[13406]: Action: /etc/e-smith/events/post-backup-data/S20nethserver-backup-config-push2mount SUCCESS [0.576697]
Oct  3 21:01:09 server7c esmith::event[13406]: Action: /etc/e-smith/events/post-backup-data/S50mysql-delete-dumped-tables SUCCESS [0.054116]
Oct  3 21:01:12 server7c esmith::event[13406]: Action: /etc/e-smith/events/post-backup-data/S70cleanup-data-duplicity SUCCESS [3.233301]
Oct  3 21:01:13 server7c esmith::event[13406]: Action: /etc/e-smith/events/post-backup-data/S70cleanup-restore-index SUCCESS [0.329935]
Oct  3 21:01:13 server7c esmith::event[13406]: Action: /etc/e-smith/events/post-backup-data/S70disk-usage SUCCESS [0.285745]
Oct  3 21:01:13 server7c esmith::event[13406]: Action: /etc/e-smith/events/post-backup-data/S80umount-cifs SUCCESS [0.452341]
Oct  3 21:01:14 server7c esmith::event[13406]: Action: /etc/e-smith/events/post-backup-data/S80umount-nfs SUCCESS [0.246001]
Oct  3 21:01:14 server7c esmith::event[13406]: Action: /etc/e-smith/events/post-backup-data/S80umount-usb SUCCESS [0.24882]
Oct  3 21:01:14 server7c esmith::event[13406]: Action: /etc/e-smith/events/post-backup-data/S80umount-webdav SUCCESS [0.250565]
Oct  3 21:01:14 server7c esmith::event[13406]: Event: post-backup-data SUCCESS
2

I’m not even sure if the errors are related.

3

Ummmm… crap.

[root@server7c ~]# yum update
error: rpmdb: BDB0113 Thread/process 2691/140578306205696 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 -  (-30973)
error: cannot open Packages database in /var/lib/rpm
CRITICAL:yum.main:

Error: rpmdb open failed
[root@server7c ~]# yum clean all
error: rpmdb: BDB0113 Thread/process 2691/140578306205696 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 -  (-30973)
error: cannot open Packages database in /var/lib/rpm
CRITICAL:yum.main:

Error: rpmdb open failed
4

rm -f /var/lib/rpm/__db*
rpm --rebuilddb
yum clean all

From:
https://community.nethserver.org/t/yum-rpmdb-failed/7876

2 Likes
5

@filippo_carletti Actually, I was just going to do that, but, I shut it down to get a clean snapshot and after bringing it back up, and running yum update at the cli, it cleaned itself up. So dnsmasq is now updated.

6

So, when I click the “install missing modules” from the gui error above… I get;

and then…

Oct  5 10:07:52 server7c pkgaction[2938]: remove:
Oct  5 10:07:52 server7c pkgaction[2938]: update:
Oct  5 10:07:52 server7c pkgaction[2938]: install:
Oct  5 10:07:52 server7c pkgaction[2938]: [InstallError] No package(s) available to install
7

Have same problem for at least 2 weeks now.
Appreciate help/ inputs

1 Like
9

AFAIK, the Restore missing modules is a false alarm.
It should be displayed only after a restore. We probably hit a corner case, but it’s hard to tell from here.
I think that it is linked to the rpm db corruption because we need to query the rpm db to know what packages to install.
Let me go deeper tomorrow.

10

@fasttech could you check if this update fixes both your problems?

11

@filippo_carletti Good morning, I logged into the server this morning to a clean gui. No error banner, no errors in the logs I could find and last night’s backup was clean…

12

Same with me- very strange.

13

This issue returned today.
Hopefullly, this weekend I can pull this machine from production and get it updated to backup config 2.0.3.