[SOLVED] Backup fails to dump rh-mariadb105 database

After doing a bit of research it appears a new service has been added after the upgrade, “rh-mariadb105-mariadb@nextcloud”. I don’t remember seeing this service here prior to the upgrade, I can’t start the service, presumably because there is already a MariaDB service running on the server.

It’s causing major issues with doing nightly backups. They fail with the following in the error log:

Jun 24 00:02:57 network esmith::event[2398]: Event: pre-backup-data mainbackup
Jun 24 00:03:07 network esmith::event[2398]: Action: /etc/e-smith/events/pre-backup-data/S20nethserver-backup-config-predatabackup SUCCESS [10.120327]
Jun 24 00:03:08 network esmith::event[2398]: Action: /etc/e-smith/events/pre-backup-data/S50mysql-dump-tables SUCCESS [1.160179]
Jun 24 00:03:08 network esmith::event[2398]: Action: /etc/e-smith/events/pre-backup-data/S50nethserver-ibays-dump-acls SUCCESS [0.004412]
Jun 24 00:03:08 network esmith::event[2398]: Action: /etc/e-smith/events/pre-backup-data/S60nethserver-nextcloud-rh-mariadb105-dump-database FAILED: 2 [0.013348]
Jun 24 00:03:11 network esmith::event[2398]: Action: /etc/e-smith/events/pre-backup-data/S80nethserver-sogo-backup-users SUCCESS [2.930666]
Jun 24 00:03:11 network esmith::event[2398]: Event: pre-backup-data FAILED

EDIT: I was able to temporarily fix the issue and at least take a backup by going into the file (/etc/e-smith/events/pre-backup-data/S60nethserver-nextcloud-rh-mariadb105-dump-database), and commenting out it’s contents.

While not ideal, I was still able to grab a backup. Looking at the logs from the previous backup (June 23 2021), that backup task does not exist, as you can see below.

Jun 23 00:00:03 network esmith::event[3518]: Event: pre-backup-data mainbackup
Jun 23 00:00:13 network esmith::event[3518]: Action: /etc/e-smith/events/pre-backup-data/S20nethserver-backup-config-predatabackup SUCCESS [10.115935]
Jun 23 00:00:14 network esmith::event[3518]: Action: /etc/e-smith/events/pre-backup-data/S50mysql-dump-tables SUCCESS [1.141623]
Jun 23 00:00:14 network esmith::event[3518]: Action: /etc/e-smith/events/pre-backup-data/S50nethserver-ibays-dump-acls SUCCESS [0.004579]
Jun 23 00:00:21 network esmith::event[3518]: Action: /etc/e-smith/events/pre-backup-data/S80nethserver-sogo-backup-users SUCCESS [6.72923]
Jun 23 00:00:21 network esmith::event[3518]: Event: pre-backup-data SUCCESS

Yes, that’s because NC21 now requires a newer version of MySQL/MariaDB. It should be able to run alongside the previous version. But if it won’t start, that’s going to break Nextcloud completely (not to mention the problem you’re seeing with your backups).

What does the service status says (errors)?

Please delete the above post (as it contains the server’s hostname, which for security we want redacted)

 Redirecting to /bin/systemctl status rh-mariadb105-mariadb@nextcloud.service
    ● rh-mariadb105-mariadb@nextcloud.service - MariaDB 10.5 database server
       Loaded: loaded (/etc/systemd/system/rh-mariadb105-mariadb@.service; disabled; vendor preset: disabled)
      Drop-In: /etc/systemd/system/rh-mariadb105-mariadb@nextcloud.service.d
               └─nethserver.conf
       Active: failed (Result: exit-code) since Thu 2021-06-24 11:55:45 EDT; 3s ago
         Docs: man:mysqld(8)
               https://mariadb.com/kb/en/library/systemd/
      Process: 23090 ExecStartPre=/usr/bin/scl enable $RH_MARIADB105_SCLS_ENABLED -- /opt/rh/rh-mariadb105/root/usr/libexec/mysql-check-socket --defaults-group-suffix=.%I (code=exited, status=1/FAILURE)
      Process: 23081 ExecStartPre=/usr/bin/scl enable $RH_MARIADB105_SCLS_ENABLED -- /usr/bin/scl_enabled rh-mariadb105 (code=exited, status=0/SUCCESS)

    Jun 24 11:55:45 network systemd[1]: Starting MariaDB 10.5 database server...
    Jun 24 11:55:45 network scl[23090]: Socket file /var/lib/mysql/mysql.sock exists.
    Jun 24 11:55:45 network scl[23090]: Is another MySQL daemon already running with the same unix socket?
    Jun 24 11:55:45 network scl[23090]: Please, stop the process using the socket /var/lib/mysql/mysql.s...vice.
    Jun 24 11:55:45 network systemd[1]: rh-mariadb105-mariadb@nextcloud.service: control process exited,...tus=1
    Jun 24 11:55:45 network systemd[1]: Failed to start MariaDB 10.5 database server.
    Jun 24 11:55:45 network systemd[1]: Unit rh-mariadb105-mariadb@nextcloud.service entered failed state.
    Jun 24 11:55:45 network systemd[1]: rh-mariadb105-mariadb@nextcloud.service failed.
    Hint: Some lines were ellipsized, use -l to show in full.

I’d think it should be set to use a different socket file. @stephdl?

1 Like

what is the issue please @danb35

The rh-mariadb105-mariadb@nextcloud service is failing to start, because it sees an existing /var/lib/mysql/mysql.sock (which I assume is from the “normal” mysql for the system). This seems to be causing a variety of brokenness as described further up-topic.

log are at /var/log/rh-mariadb105/nextcloud.log

pid and socket are at /var/run/rh-mariadb105-mariadb/

It should not interfere with mysql

systemctl restart rh-mariadb105-mariadb@nextcloud.service

Then check in /var/log/rh-mariadb105/nextcloud.log what is wrong

Perhaps, but look at the status output:

    Jun 24 11:55:45 network scl[23090]: Socket file /var/lib/mysql/mysql.sock exists.
    Jun 24 11:55:45 network scl[23090]: Is another MySQL daemon already running with the same unix socket?
    Jun 24 11:55:45 network scl[23090]: Please, stop the process using the socket /var/lib/mysql/mysql.s...vice.
    Jun 24 11:55:45 network systemd[1]: rh-mariadb105-mariadb@nextcloud.service: control process exited,...tus=1
    Jun 24 11:55:45 network systemd[1]: Failed to start MariaDB 10.5 database server.
1 Like

we need the log output, maybe also to reconfigure the service by the event, I can read that the author of the thread has commented some script lines

cat: /var/log/rh-mariadb105/nextcloud.log: No such file or directory

Restart the service please

I can’t restart the service. It fails to start, and there is no log generated. That folder exists, it’s just empty.

Reinstall

nethserver-nextcloud
nethserver-rh-mariadb105

yum reinstall nethserver-nextcloud nethserver-rh-mariadb105

No change. Everything is the same.

Did you attempt to downgrade nextcloud at some point? (just to know, don’t try it now)

No. I have not downgraded Nextcloud ever. Also the reinstall caused the Cron and Backup issue to reappear.

what is the output on your server .

[root@prometheus ~]# /opt/rh/rh-mariadb105/root/bin/mysqlshow --socket="/var/run/rh-mariadb105-mariadb/nextcloud-mysql.sock" 
+--------------------+
|     Databases      |
+--------------------+
| information_schema |
| mysql              |
| nextcloud          |
| performance_schema |
+--------------------+
[root@prometheus ~]# /opt/rh/rh-mariadb105/root/bin/mysql --socket="/var/run/rh-mariadb105-mariadb/nextcloud-mysql.sock" 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 146
Server version: 10.5.8-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]>
1 Like