Nextcloud - Internal Server Error

Arthur · July 6, 2021, 8:06am

The server has Nextcloud installed.
Now it shows Internal Server Error

The server encountered an internal error and was unable to complete your request.
Please contact the server administrator if this error reappears multiple times, please include the technical details below in your report.
More details can be found in the server log.

All the services are active and running. I have noticed the following warning near Mariadb service.

Jul 05 18:02:24 srv.domain.local systemd[1]: Starting MariaDB 10.5 database server...
Jul 05 18:02:24 srv.domain.local scl[1111]: Database MariaDB is probably initialized in /var/opt/rh/rh-mariadb105/lib/mysql-nextcloud already, nothing is done.
Jul 05 18:02:24 srv.domain.local scl[1111]: If this is not the case, make sure the /var/opt/rh/rh-mariadb105/lib/mysql-nextcloud is empty before running mysql-prepare-db-dir.
Jul 05 18:02:24 srv.domain.local mysqld-scl-helper[1160]: 2021-07-05 18:02:24 0 [Note] /opt/rh/rh-mariadb105/root/usr/libexec/mysqld (mysqld 10.5.8-MariaDB) starting as process 1160 ..

dnutan · July 6, 2021, 8:10am

Can you list the installed nextcloud apps? or do you remember having installed documentserver or alike (OnlyOffice or Collabora Office, or any other app that requires java or tomcat)?

Arthur · July 6, 2021, 8:37am

The application list is following

Crontab manager Nethserver integration of crontab manager 1.0.4
Fail2ban Nethserver integration of Fail2ban 1.7.1
File server Share files with your working group 4.6.0
Guacamole Provides a clientless remote desktop gateway 0.0.1
mrmarkuz repo is online Provides mrmarkuz repo, open link to get repoview 0.0.1
Nextcloud Provides Nextcloud for NethServer 1.17.0
NFS server Share files with NFS 1.0.1
phpMyAdmin Provide the administration of MySQL over the Web 1.2.5
Stephdl is online You can support the stephane de Labrusse (stephdl) software repository by clicking on the button 1.1.7
Web server Web server configuration with named virtual hosts and reverse proxy 3.12.0

Also I’m planning to install one of these Offices (OnlyOffice or Collabora Office).

Just found the following logs on /var/log/rh-mariadb105/nextcloud.log

2021-07-06 11:20:02 442 [Warning] Access denied for user 'nextcloud'@'localhost' (using password: YES)

Don’t know why, but maybe the cause is Nextcloud update 2 days ago.

dnutan · July 6, 2021, 8:48am

Whats the outcome of these commands?

mysqlshow

/opt/rh/rh-mariadb105/root/bin/mysqlshow --socket="/var/run/rh-mariadb105-mariadb/nextcloud-mysql.sock"

grep \'db /usr/share/nextcloud/config/config.php |grep -v dbpass

Arthur · July 6, 2021, 9:04am

mysqlshow

+--------------------+
|     Databases      |
+--------------------+
| information_schema |
| guacamole          |
| mysql              |
| performance_schema |
| phpmyadmin         |

DB missing. I have reinstalled nextcloud. So I need to resotore it from backup or there are other choices?
grep 'db /usr/share/nextcloud/config/config.php |grep -v dbpass

  'dbtype' => 'mysql',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost:/run/rh-mariadb105-mariadb/nextcloud-mysql.sock',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'nextcloud',

dnutan · July 6, 2021, 9:06am

It shall be on mariadb105 (use the second command above)

Arthur · July 6, 2021, 10:29am

It shall be on mariadb105 (use the second command above)
What do you mean with second command?
the files are on /var/opt/rh/rh-mariadb105/lib/mysql-nextcloud/

dnutan · July 6, 2021, 10:36am

I mean:

Arthur · July 6, 2021, 10:44am

yes, the db is seen

+--------------------+
|     Databases      |
+--------------------+
| information_schema |
| mysql              |
| nextcloud          |
| performance_schema |

ok, but why the error that access denied?

dnutan · July 6, 2021, 10:53am

Check if the dbuser nextcloud exists:

/opt/rh/rh-mariadb105/root/bin/mysql --socket="/var/run/rh-mariadb105-mariadb/nextcloud-mysql.sock" -e 'SELECT user FROM mysql.user'

If it exists, then you might need to check for a password mismatch (dbpassword in nextcloud config.php file and password on /var/lib/nethserver/secrets/nextcloud).

Arthur · July 6, 2021, 11:12am

Exists

±------------+
| User |
±------------+
| mariadb.sys |
| mysql |
| nextcloud |
| root |
±------------+

‘dbpassword’ value is the same as on /var/lib/nethserver/secrets/nextcloud

dnutan · July 6, 2021, 1:20pm

If passwords are the same what is left is to check them against the mysql nextcloud user password or the mysql nextcloud user privileges:

mysql --socket="/var/run/rh-mariadb105-mariadb/nextcloud-mysql.sock" -e "SHOW GRANTS FOR 'nextcloud'@'localhost'"
+------------------------------------------------------------------------------------------------------------------+
| Grants for nextcloud@localhost                                                                                   |
+------------------------------------------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO `nextcloud`@`localhost` IDENTIFIED BY PASSWORD '*xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' |
| GRANT ALL PRIVILEGES ON `nextcloud`.* TO `nextcloud`@`localhost`                                                 |
+------------------------------------------------------------------------------------------------------------------+

Or to run signal-event nethserver-nextcloud-update hoping it completes what was left.

Arthur · July 6, 2021, 1:45pm

ok, a little bit closer! I had to reset the password and now I got the warning
Downgrading is not supported and is likely to cause unpredictable issues (from 21.0.3.1 to 21.0.2.1)

dnutan · July 6, 2021, 1:55pm

nethserver-nextcloud-1.17.1-1 (for Nextcloud 21.0.3.1) was released today.
Looks like you managed to install 21.0.3.1 (maybe from testing repo?) but now you are offered 21.0.2.1

rpm -q nethserver-nextcloud
yum check-update
yum list nethserver-nextcloud --show-duplicates

Arthur · July 6, 2021, 2:08pm

here’s output
rpm -q nethserver-nextcloud

nethserver-nextcloud-1.17.0-1.ns7.noarch

yum check-update

nethserver-cockpit.noarch                                                                        1.10.3-1.ns7                                                                   nethserver-updates
nethserver-cockpit-lib.noarch                                                                    1.10.3-1.ns7                                                                   nethserver-updates
nethserver-firewall-base.noarch                                                                  3.15.7-1.ns7                                                                   nethserver-updates

yum list nethserver-nextcloud --show-duplicates

Installed Packages
nethserver-nextcloud.noarch                                                                    1.17.0-1.ns7                                                                    @nethserver-updates
Available Packages
nethserver-nextcloud.noarch                                                                    1.15.0-1.ns7                                                                    nethserver-base    
nethserver-nextcloud.noarch                                                                    1.16.0-1.ns7                                                                    nethserver-updates 
nethserver-nextcloud.noarch                                                                    1.16.1-1.ns7                                                                    nethserver-updates 
nethserver-nextcloud.noarch                                                                    1.16.2-1.ns7                                                                    nethserver-updates 
nethserver-nextcloud.noarch                                                                    1.16.3-1.ns7                                                                    nethserver-updates 
nethserver-nextcloud.noarch                                                                    1.16.4-1.ns7                                                                    nethserver-updates 
nethserver-nextcloud.noarch                                                                    1.16.5-1.ns7                                                                    nethserver-updates 
nethserver-nextcloud.noarch                                                                    1.16.6-1.ns7                                                                    nethserver-updates 
nethserver-nextcloud.noarch                                                                    1.16.7-1.ns7                                                                    nethserver-updates 
nethserver-nextcloud.noarch                                                                    1.17.0-1.ns7                                                                    nethserver-updates

dnutan · July 6, 2021, 2:10pm

From where came the message? maybe was a partial manual update using nextcloud integrated update assistant (from the update button on Nextcloud interface)

Arthur · July 6, 2021, 2:12pm

The message came form browser.

dnutan · July 6, 2021, 2:16pm

21.0.3 should soon propagate to mirrors. In the meantime, you can choose to download manually download it…
Or install it directly from the link (installing it directly from yum via direct URL is sometimes considered insecure)

yum install http://packages.nethserver.org/nethserver/7/updates/x86_64/Packages/nethserver-nextcloud-1.17.1-1.ns7.noarch.rpm

Arthur · July 6, 2021, 2:30pm

After installation the same internal error has been generated. I looked at nextcloud log and seen the same mariadb access denied error. Have changed the password for user and nextcloud works!
One error which have been fixed after update stayed without changes.
There are some errors regarding your setup.

* Last background job execution ran 4 days ago. Something seems wrong. [Check the background job settings](https://biz.itkonsultanti.lv/index.php/settings/admin#backgroundjobs)

Thank you very much!

dnutan · July 6, 2021, 2:48pm

The message shall go away the next time the nextcloud cron job runs.