Nextcloud migration not working - data directory not writable

Environment: NS8 on Debian 11.7.0 on 2 nodes installed from ISO in Proxmox

After finishing the Nextcloud migration the ns8-action is stuck at 33%

grafik

NS8 /var/log/messages:

May 14 10:59:45 node1 nextcloud1[5627]: New nextcloud instance
May 14 10:59:45 node1 nextcloud1[5627]: Installing with MySQL database
May 14 10:59:45 node1 nextcloud1[5627]: Starting nextcloud installation
May 14 10:59:46 node1 podman[6141]: 2023-05-14 10:59:46.433832585 +0200 CEST m=+0.799223614 container exec 2f9a80bdcd0ca96f4ffc6ecedb2d2a46068f17aaae0c2e7d19713d83cc0cc9fc (image=ghcr.io/nethserver/nextcloud-app:1.0.0, name=nextcloud-app, PODMAN_SYSTEMD_UNIT=nextcloud-app.service, io.buildah.version=1.23.1)
May 14 10:59:46 node1 traefik1[818]: 192.168.3.159 - - [14/May/2023:08:59:45 +0000] "GET /cluster-admin/api/module/nextcloud1/task/fa3b3bd6-350b-4728-bde8-f4e560a8ffbf/context HTTP/2.0" 200 348 "-" "-" 1008 "ApiServer-https@redis" "http://127.0.0.1:9311" 540ms
May 14 10:59:46 node1 nextcloud1[5627]: Your data directory is not writable.
May 14 10:59:46 node1 nextcloud1[5627]: Permissions can usually be fixed by giving the web server write access to the root directory. See https://docs.nextcloud.com/server/26/go.php?to=admin-dir_permissions.
May 14 10:59:46 node1 nextcloud1[5627]: 
May 14 10:59:46 node1 nextcloud1[5627]: An unhandled exception has been thrown:
May 14 10:59:46 node1 nextcloud1[5627]: Exception: Environment not properly prepared. in /var/www/html/lib/private/Console/Application.php:167
May 14 10:59:46 node1 nextcloud1[5627]: Stack trace:
May 14 10:59:46 node1 nextcloud1[5627]: #0 /var/www/html/console.php(99): OC\Console\Application->loadCommands(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
May 14 10:59:46 node1 nextcloud1[5627]: #1 /var/www/html/occ(11): require_once('/var/www/html/c...')
May 14 10:59:46 node1 nextcloud1[5627]: #2 {main}Retrying install...
May 14 10:59:47 node1 podman[6141]: 2023-05-14 10:59:47.025479881 +0200 CEST m=+1.390871123 container exec_died 2f9a80bdcd0ca96f4ffc6ecedb2d2a46068f17aaae0c2e7d19713d83cc0cc9fc (image=ghcr.io/nethserver/nextcloud-app:1.0.0, name=nextcloud-app, execID=8dd5b68a68fe49abd76cb26c22d94e38e7ea14d0e4d2fa22f032e02d163b55a2)
May 14 10:59:50 node1 traefik1[818]: 192.168.3.159 - - [14/May/2023:08:59:50 +0000] "GET /cluster-admin/api/module/nextcloud1/task/fa3b3bd6-350b-4728-bde8-f4e560a8ffbf/context HTTP/2.0" 200 348 "-" "-" 1009 "ApiServer-https@redis" "http://127.0.0.1:9311" 34ms
May 14 10:59:52 node1 podman[6213]: 2023-05-14 10:59:52.350181755 +0200 CEST m=+0.158254766 container exec 2f9a80bdcd0ca96f4ffc6ecedb2d2a46068f17aaae0c2e7d19713d83cc0cc9fc (image=ghcr.io/nethserver/nextcloud-app:1.0.0, name=nextcloud-app, PODMAN_SYSTEMD_UNIT=nextcloud-app.service, io.buildah.version=1.23.1)
May 14 10:59:52 node1 podman[6213]: 2023-05-14 10:59:52.76967327 +0200 CEST m=+0.577746329 container exec_died 2f9a80bdcd0ca96f4ffc6ecedb2d2a46068f17aaae0c2e7d19713d83cc0cc9fc (image=ghcr.io/nethserver/nextcloud-app:1.0.0, name=nextcloud-app, execID=ad06f44664c0f5970f8eeef777fd27aeb5e5f63662546a12cd7baca4c186309a)
May 14 10:59:55 node1 traefik1[818]: 192.168.3.159 - - [14/May/2023:08:59:55 +0000] "GET /cluster-admin/api/module/nextcloud1/task/fa3b3bd6-350b-4728-bde8-f4e560a8ffbf/context HTTP/2.0" 200 348 "-" "-" 1010 "ApiServer-https@redis" "http://127.0.0.1:9311" 41ms
May 14 10:59:57 node1 nextcloud1[5627]: Your data directory is not writable.
May 14 10:59:57 node1 nextcloud1[5627]: Permissions can usually be fixed by giving the web server write access to the root directory. See https://docs.nextcloud.com/server/26/go.php?to=admin-dir_permissions.
May 14 10:59:57 node1 nextcloud1[5627]: 
May 14 10:59:57 node1 nextcloud1[5627]: An unhandled exception has been thrown:
May 14 10:59:57 node1 nextcloud1[5627]: Exception: Environment not properly prepared. in /var/www/html/lib/private/Console/Application.php:167
May 14 10:59:57 node1 nextcloud1[5627]: Stack trace:
May 14 10:59:57 node1 nextcloud1[5627]: #0 /var/www/html/console.php(99): OC\Console\Application->loadCommands(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
May 14 10:59:57 node1 nextcloud1[5627]: #1 /var/www/html/occ(11): require_once('/var/www/html/c...')
May 14 10:59:57 node1 nextcloud1[5627]: #2 {main}Retrying install...
May 14 10:59:58 node1 podman[6288]: 2023-05-14 10:59:58.036521913 +0200 CEST m=+0.140257827 container exec 2f9a80bdcd0ca96f4ffc6ecedb2d2a46068f17aaae0c2e7d19713d83cc0cc9fc (image=ghcr.io/nethserver/nextcloud-app:1.0.0, name=nextcloud-app, PODMAN_SYSTEMD_UNIT=nextcloud-app.service, io.buildah.version=1.23.1)
May 14 10:59:58 node1 podman[6288]: 2023-05-14 10:59:58.485609322 +0200 CEST m=+0.589345498 container exec_died 2f9a80bdcd0ca96f4ffc6ecedb2d2a46068f17aaae0c2e7d19713d83cc0cc9fc (image=ghcr.io/nethserver/nextcloud-app:1.0.0, name=nextcloud-app, execID=47cf8aae802184614bd57ce1a4a34f7578065ab3caa130a63bc4c271f9883909)
May 14 10:59:59 node1 traefik1[818]: 192.168.3.159 - - [14/May/2023:08:59:59 +0000] "POST /api/v4/users/status/ids HTTP/2.0" 200 123 "-" "-" 1011 "mattermost1-https@redis" "http://127.0.0.1:20009" 1ms
May 14 11:00:00 node1 traefik1[818]: 192.168.3.159 - - [14/May/2023:09:00:00 +0000] "GET /cluster-admin/api/module/nextcloud1/task/fa3b3bd6-350b-4728-bde8-f4e560a8ffbf/context HTTP/2.0" 200 348 "-" "-" 1012 "ApiServer-https@redis" "http://127.0.0.1:9311" 16ms
May 14 11:00:03 node1 podman[6357]: 2023-05-14 11:00:03.75769704 +0200 CEST m=+0.149970212 container exec 2f9a80bdcd0ca96f4ffc6ecedb2d2a46068f17aaae0c2e7d19713d83cc0cc9fc (image=ghcr.io/nethserver/nextcloud-app:1.0.0, name=nextcloud-app, PODMAN_SYSTEMD_UNIT=nextcloud-app.service, io.buildah.version=1.23.1)
May 14 11:00:04 node1 podman[6357]: 2023-05-14 11:00:04.213279391 +0200 CEST m=+0.605552601 container exec_died 2f9a80bdcd0ca96f4ffc6ecedb2d2a46068f17aaae0c2e7d19713d83cc0cc9fc (image=ghcr.io/nethserver/nextcloud-app:1.0.0, name=nextcloud-app, execID=95ac36fe8503864e7b8569713dbc80a5077181a17e2ea5f0bc5c81b8c4563c28)
May 14 11:00:05 node1 traefik1[818]: 192.168.3.159 - - [14/May/2023:09:00:05 +0000] "GET /cluster-admin/api/module/nextcloud1/task/fa3b3bd6-350b-4728-bde8-f4e560a8ffbf/context HTTP/2.0" 200 348 "-" "-" 1013 "ApiServer-https@redis" "http://127.0.0.1:9311" 22ms
May 14 11:00:07 node1 nextcloud1[5627]: Your data directory is not writable.
May 14 11:00:07 node1 nextcloud1[5627]: Permissions can usually be fixed by giving the web server write access to the root directory. See https://docs.nextcloud.com/server/26/go.php?to=admin-dir_permissions.
May 14 11:00:07 node1 nextcloud1[5627]: 
May 14 11:00:07 node1 nextcloud1[5627]: An unhandled exception has been thrown:
May 14 11:00:07 node1 nextcloud1[5627]: Exception: Environment not properly prepared. in /var/www/html/lib/private/Console/Application.php:167
May 14 11:00:07 node1 nextcloud1[5627]: Stack trace:
May 14 11:00:07 node1 nextcloud1[5627]: #0 /var/www/html/console.php(99): OC\Console\Application->loadCommands(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
May 14 11:00:07 node1 nextcloud1[5627]: #1 /var/www/html/occ(11): require_once('/var/www/html/c...')
May 14 11:00:07 node1 nextcloud1[5627]: #2 {main}Retrying install...

NS7 side:

Copied command from cockpit:

[root@testserver ~]#  echo '{"app":"nethserver-nextcloud","action":"finish","migrationConfig":{"virtualHost":"nextcloud.ns8.domain.tld"}}' | /usr/bin/setsid /usr/bin/sudo /usr/libexec/nethserver/api/nethserver-ns8-migration/migration/update | jq
{
  "progress": "0.00",
  "time": "0.0",
  "exit": 0,
  "event": "migration-sync",
  "state": "running",
  "step": 0,
  "pid": 0,
  "action": ""
}
rsync: failed to connect to 10.5.4.1 (10.5.4.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]

EDIT:

In Alma Linux 9.2 I got the same repeating error after finishing the Nextcloud migration:

May 14 23:06:05 alma1 nextcloud1[3546]: Nextcloud or one of the apps require upgrade - only a limited number of commands are available
May 14 23:06:05 alma1 nextcloud1[3546]: You may use your browser or the occ upgrade command to do the upgrade
May 14 23:06:05 alma1 nextcloud1[3546]: {"reqId":"DKabknPACivnokn2x2ty","level":3,"time":"2023-05-14T21:06:05+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fopen(/var/www/html/data/data_dir_writability_test_64614d3d2b366.tmp): Failed to open stream: Permission denied at /var/www/html/lib/private/legacy/OC_Util.php#581","userAgent":"--","version":"25.0.6.1","data":{"app":"PHP"}}
May 14 23:06:05 alma1 nextcloud1[3546]: parse error: Invalid numeric literal at line 1, column 5
May 14 23:06:05 alma1 nextcloud1[3546]: {"reqId":"DKabknPACivnokn2x2ty","level":3,"time":"2023-05-14T21:06:05+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fopen(/var/www/html/data/nextcloud.log): Failed to open stream: Permission denied at /var/www/html/lib/private/Log/File.php#84","userAgent":"--","version":"25.0.6.1","data":{"app":"PHP"}}
May 14 23:06:06 alma1 nextcloud1[4821]: Nextcloud or one of the apps require upgrade - only a limited number of commands are available
May 14 23:06:06 alma1 nextcloud1[4821]: You may use your browser or the occ upgrade command to do the upgrade
May 14 23:06:06 alma1 nextcloud1[4821]: {"reqId":"3S9yi0NI72nJjlATLzMM","level":3,"time":"2023-05-14T21:06:06+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fopen(/var/www/html/data/data_dir_writability_test_64614d3e2d1b9.tmp): Failed to open stream: Permission denied at /var/www/html/lib/private/legacy/OC_Util.php#581","userAgent":"--","version":"25.0.6.1","data":{"app":"PHP"}}
May 14 23:06:06 alma1 nextcloud1[4821]: Your data directory is not writable.
May 14 23:06:06 alma1 nextcloud1[4821]: Permissions can usually be fixed by giving the web server write access to the root directory. See https://docs.nextcloud.com/server/26/go.php?to=admin-dir_permissions.
May 14 23:06:06 alma1 nextcloud1[4821]: 
May 14 23:06:06 alma1 nextcloud1[4821]: An unhandled exception has been thrown:
May 14 23:06:06 alma1 nextcloud1[4821]: Exception: Environment not properly prepared. in /var/www/html/lib/private/Console/Application.php:167
May 14 23:06:06 alma1 nextcloud1[4821]: Stack trace:
May 14 23:06:06 alma1 nextcloud1[4821]: #0 /var/www/html/console.php(99): OC\Console\Application->loadCommands(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
May 14 23:06:06 alma1 nextcloud1[4821]: #1 /var/www/html/occ(11): require_once('/var/www/html/c...')
May 14 23:06:06 alma1 nextcloud1[4821]: {"reqId":"3S9yi0NI72nJjlATLzMM","level":3,"time":"2023-05-14T21:06:06+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fopen(/var/www/html/data/nextcloud.log): Failed to open stream: Permission denied at /var/www/html/lib/private/Log/File.php#84","userAgent":"--","version":"25.0.6.1","data":{"app":"PHP"}}
May 14 23:06:06 alma1 nextcloud1[4821]: #2 {main}Installing of nextcloud failed!```
5 Likes

Thank you for testing and reporting!
I’ve added a card for beta 2.

Regarding Debian, you could try to fix by entering the container and changing data owner:

ssh nextcloud1@localhost
podman exec -ti nextcloud-app /bin/bash

Try to inspect the permission of /var/www/html inside the container.

In Alma Linux 9.2 I got the same repeating error after finishing the Nextcloud migration:

Maybe something about selinux. Is there anything inside the audit.log of the host?

1 Like

Yes, that seemed to be the issue, from inside the nextcloud-app container (/bin/bash wasn’t available) I saw the owner of the data dir wasn’t www-data:

nextcloud1@node1:~$ podman exec -ti nextcloud-app /bin/sh
/var/www/html # ls -ld data
drwxr-x---    5 48       48            4096 May 15 15:21 data

After executing

/var/www/html # chown -R www-data:www-data data

and saving Nextcloud settings in the web UI I’m one step further:

After clicking the update button there was an undefined ldap error so I’m stuck in maintenance mode. So I tried the following:

/var/www/html # ./occ maintenance:mode --off
Console has to be executed with the user that owns the file config/config.php
Current user id: 0
Owner id of config.php: 82
Try adding 'sudo -u #82' to the beginning of the command (without the single quotes)
If running with 'docker exec' try adding the option '-u 82' to the docker command (without the single quotes)

/var/www/html # sudo -u #82 occ maintenance:mode --off
/bin/sh: sudo: not found

/var/www/html # su #82 -c "occ maintenance:mode --off"

but without success.
I think the main issue is the wrong owner of the data dir.

EDIT:

I tried the migration again, this time with Alma Linux 9.2 instead of Debian and I could get to the Nextcloud login page but I can’t login, the external LDAP (Neth7 AD) seems not reachable for Nextcloud.

I entered the container and set owner of data dir to www-data, this time I entered the container as user 82:

podman exec -ti -u 82 nextcloud-app /bin/sh

I ran the ./occ update manually and got the LDAP errors again but it finished and disabled maintenance mode:

Migrate old user accessibility config
 Starting ...
    0/0 [------->--------------------]   0%Repair error: Lost connection to LDAP server.
...
Repair error: Lost connection to LDAP server.

Here is the part of the audit.log, starting from the first occurence of “nextcloud”:

type=ADD_GROUP msg=audit(1684187092.533:142): pid=2113 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:unconfined_service_t:s0 msg='op=add-group acct="nextcloud1" exe="/usr/sbin/useradd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset"
type=ADD_USER msg=audit(1684187092.549:143): pid=2113 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:unconfined_service_t:s0 msg='op=add-user acct="nextcloud1" exe="/usr/sbin/useradd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset"
type=USER_MGMT msg=audit(1684187092.720:144): pid=2113 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:unconfined_service_t:s0 msg='op=add-home-dir id=1003 exe="/usr/sbin/useradd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset" ID="nextcloud1"
type=SERVICE_START msg=audit(1684187093.464:145): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=user-runtime-dir@1003 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset"
type=USER_ACCT msg=audit(1684187093.538:146): pid=2176 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='op=PAM:accounting grantors=pam_unix acct="nextcloud1" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset"
type=CRED_ACQ msg=audit(1684187093.539:147): pid=2176 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='op=PAM:setcred grantors=? acct="nextcloud1" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'^]UID="root" AUID="unset"
type=USER_ROLE_CHANGE msg=audit(1684187093.540:148): pid=2176 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='pam: default-context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 selected-context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset"
type=LOGIN msg=audit(1684187093.541:149): pid=2176 uid=0 subj=system_u:system_r:init_t:s0 old-auid=4294967295 auid=1003 tty=(none) old-ses=4294967295 ses=7 res=1^]UID="root" OLD-AUID="unset" AUID="nextcloud1"
type=SYSCALL msg=audit(1684187093.541:149): arch=c000003e syscall=1 success=yes exit=4 a0=7 a1=7ffc9a7f8270 a2=4 a3=3eb items=0 ppid=1 pid=2176 auid=1003 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=7 comm="(systemd)" exe="/usr/lib/systemd/systemd" subj=system_u:system_r:init_t:s0 key=(null)^]ARCH=x86_64 SYSCALL=write AUID="nextcloud1" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"
type=PROCTITLE msg=audit(1684187093.541:149): proctitle="(systemd)"
type=USER_START msg=audit(1684187093.546:150): pid=2176 uid=0 auid=1003 ses=7 subj=system_u:system_r:init_t:s0 msg='op=PAM:session_open grantors=pam_selinux,pam_selinux,pam_loginuid,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="nextcloud1" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="nextcloud1"
type=SERVICE_START msg=audit(1684187093.730:151): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=user@1003 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset"
type=CRYPTO_KEY_USER msg=audit(1684187204.514:152): pid=2844 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:2b:30:66:25:7c:c0:44:79:16:8f:2c:70:bb:dd:1f:cd:95:25:c3:8f:20:6a:da:d8:b8:53:75:0f:63:46:3c:18 direction=? spid=2844 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset" SUID="root"
type=CRYPTO_SESSION msg=audit(1684187204.519:153): pid=2843 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=start direction=from-server cipher=aes256-gcm@openssh.com ksize=256 mac=<implicit> pfs=curve25519-sha256 spid=2844 suid=74 rport=50194 laddr=::1 lport=22  exe="/usr/sbin/sshd" hostname=? addr=::1 terminal=? res=success'^]UID="root" AUID="unset" SUID="sshd"
type=CRYPTO_SESSION msg=audit(1684187204.522:154): pid=2843 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=start direction=from-client cipher=aes256-gcm@openssh.com ksize=256 mac=<implicit> pfs=curve25519-sha256 spid=2844 suid=74 rport=50194 laddr=::1 lport=22  exe="/usr/sbin/sshd" hostname=? addr=::1 terminal=? res=success'^]UID="root" AUID="unset" SUID="sshd"
type=USER_AUTH msg=audit(1684187204.637:155): pid=2843 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=pubkey_auth grantors=auth-key acct="nextcloud1" exe="/usr/sbin/sshd" hostname=? addr=::1 terminal=? res=success'^]UID="root" AUID="unset"
type=CRYPTO_KEY_USER msg=audit(1684187204.637:156): pid=2843 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=negotiate kind=auth-key fp=SHA256:64:f1:d0:cd:68:fe:95:1e:83:ab:38:4a:16:8c:0e:1a:01:c5:77:b2:a6:a5:b2:e3:94:b1:ee:6d:28:1a:ac:3f exe="/usr/sbin/sshd" hostname=? addr=::1 terminal=? res=success'^]UID="root" AUID="unset"
type=USER_ACCT msg=audit(1684187204.666:157): pid=2843 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=PAM:accounting grantors=pam_unix acct="nextcloud1" exe="/usr/sbin/sshd" hostname=::1 addr=::1 terminal=ssh res=success'^]UID="root" AUID="unset"
type=CRYPTO_KEY_USER msg=audit(1684187204.667:158): pid=2843 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=session fp=? direction=both spid=2844 suid=74 rport=50194 laddr=::1 lport=22  exe="/usr/sbin/sshd" hostname=? addr=::1 terminal=? res=success'^]UID="root" AUID="unset" SUID="sshd"
type=CRED_ACQ msg=audit(1684187204.669:159): pid=2843 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="nextcloud1" exe="/usr/sbin/sshd" hostname=::1 addr=::1 terminal=ssh res=success'^]UID="root" AUID="unset"
type=LOGIN msg=audit(1684187204.669:160): pid=2843 uid=0 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 old-auid=4294967295 auid=1003 tty=(none) old-ses=4294967295 ses=8 res=1^]UID="root" OLD-AUID="unset" AUID="nextcloud1"
type=SYSCALL msg=audit(1684187204.669:160): arch=c000003e syscall=1 success=yes exit=4 a0=3 a1=7ffcf0a826b0 a2=4 a3=3eb items=0 ppid=751 pid=2843 auid=1003 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=8 comm="sshd" exe="/usr/sbin/sshd" subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 key=(null)^]ARCH=x86_64 SYSCALL=write AUID="nextcloud1" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"
type=PROCTITLE msg=audit(1684187204.669:160): proctitle=737368643A206E657874636C6F756431205B707269765D
type=USER_ROLE_CHANGE msg=audit(1684187204.671:161): pid=2843 uid=0 auid=1003 ses=8 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='pam: default-context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 selected-context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 exe="/usr/sbin/sshd" hostname=::1 addr=::1 terminal=ssh res=success'^]UID="root" AUID="nextcloud1"
type=USER_START msg=audit(1684187204.702:162): pid=2843 uid=0 auid=1003 ses=8 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_selinux,pam_loginuid,pam_selinux,pam_namespace,pam_keyinit,pam_keyinit,pam_limits,pam_systemd,pam_unix,pam_umask,pam_lastlog acct="nextcloud1" exe="/usr/sbin/sshd" hostname=::1 addr=::1 terminal=ssh res=success'^]UID="root" AUID="nextcloud1"
type=CRYPTO_KEY_USER msg=audit(1684187204.704:163): pid=2846 uid=0 auid=1003 ses=8 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:2b:30:66:25:7c:c0:44:79:16:8f:2c:70:bb:dd:1f:cd:95:25:c3:8f:20:6a:da:d8:b8:53:75:0f:63:46:3c:18 direction=? spid=2846 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="nextcloud1" SUID="root"
type=CRED_ACQ msg=audit(1684187204.706:164): pid=2846 uid=0 auid=1003 ses=8 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="nextcloud1" exe="/usr/sbin/sshd" hostname=::1 addr=::1 terminal=ssh res=success'^]UID="root" AUID="nextcloud1"
type=USER_LOGIN msg=audit(1684187204.752:165): pid=2843 uid=0 auid=1003 ses=8 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=1003 exe="/usr/sbin/sshd" hostname=? addr=::1 terminal=/dev/pts/2 res=success'^]UID="root" AUID="nextcloud1" ID="nextcloud1"
type=USER_START msg=audit(1684187204.753:166): pid=2843 uid=0 auid=1003 ses=8 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=1003 exe="/usr/sbin/sshd" hostname=? addr=::1 terminal=/dev/pts/2 res=success'^]UID="root" AUID="nextcloud1" ID="nextcloud1"
type=CRYPTO_KEY_USER msg=audit(1684187204.755:167): pid=2843 uid=0 auid=1003 ses=8 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:2b:30:66:25:7c:c0:44:79:16:8f:2c:70:bb:dd:1f:cd:95:25:c3:8f:20:6a:da:d8:b8:53:75:0f:63:46:3c:18 direction=? spid=2847 suid=1003  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="nextcloud1" SUID="nextcloud1"
type=BPF msg=audit(1684187204.779:168): prog-id=34 op=LOAD
type=BPF msg=audit(1684187204.779:169): prog-id=35 op=LOAD
type=SERVICE_START msg=audit(1684187204.886:170): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-hostnamed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset"
type=SERVICE_STOP msg=audit(1684187234.892:171): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-hostnamed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset"
type=BPF msg=audit(1684187234.920:172): prog-id=0 op=UNLOAD
type=BPF msg=audit(1684187234.920:173): prog-id=0 op=UNLOAD
type=SERVICE_STOP msg=audit(1684187510.325:174): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=sssd-kcm comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset"
type=SERVICE_START msg=audit(1684187578.829:175): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-tmpfiles-clean comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset"
type=SERVICE_STOP msg=audit(1684187578.829:176): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-tmpfiles-clean comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'^]UID="root" AUID="unset"
1 Like

I do not see any error on the audit log: I do not think the problem is related to selinux.

If you access the NS8 UI and try to configure the user domain from there, does nextcloud can access the ldap server?

In the meanwhile, I’ve added a card also for nextcloud migration

1 Like

I tried…

  • Reconfiguring the external AD domain
  • Migrate the Active Directory to make it local
  • Create another local AD

…all without success.

After Nextcloud login I always get internal server error:

grep nextcloud /var/log/messages
May 16 11:10:00 alma1 nextcloud1[1870]: crond: wakeup dt=60
May 16 11:10:00 alma1 nextcloud1[1870]: crond: file www-data:
May 16 11:10:00 alma1 nextcloud1[1870]: crond:  line php -f /var/www/html/cron.php
May 16 11:10:00 alma1 nextcloud1[1870]: crond:  job: 0 php -f /var/www/html/cron.php
May 16 11:10:00 alma1 nextcloud1[1870]: crond: child running /bin/ash
May 16 11:10:00 alma1 nextcloud1[1870]: crond: USER www-data pid  63 cmd php -f /var/www/html/cron.php
May 16 11:10:10 alma1 nextcloud1[1870]: crond: wakeup dt=10
May 16 11:11:00 alma1 nextcloud1[1870]: crond: wakeup dt=50
May 16 11:11:00 alma1 nextcloud1[1870]: crond: file www-data:
May 16 11:11:00 alma1 nextcloud1[1870]: crond:  line php -f /var/www/html/cron.php
May 16 11:12:00 alma1 nextcloud1[1870]: crond: wakeup dt=60
May 16 11:12:00 alma1 nextcloud1[1870]: crond: file www-data:
May 16 11:12:00 alma1 nextcloud1[1870]: crond:  line php -f /var/www/html/cron.php
May 16 11:13:00 alma1 nextcloud1[1870]: crond: wakeup dt=60
May 16 11:13:00 alma1 nextcloud1[1870]: crond: file www-data:
May 16 11:13:00 alma1 nextcloud1[1870]: crond:  line php -f /var/www/html/cron.php
May 16 11:13:18 alma1 nextcloud1[873]: task/module/nextcloud1/33981fa8-38b1-453d-bf0a-c62672ce2c32: get-name/50get_name is starting
May 16 11:13:18 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:18 +0000] "POST /cluster-admin/api/module/nextcloud1/tasks HTTP/2.0" 201 240 "-" "-" 834 "ApiServer-https@redis" "http://127.0.0.1:9311" 26ms
May 16 11:13:18 alma1 nextcloud1[873]: task/module/nextcloud1/1dbeb4c3-de19-4883-aab4-0f191afd5cb2: get-status/20read is starting
May 16 11:13:19 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:18 +0000] "POST /cluster-admin/api/module/nextcloud1/tasks HTTP/2.0" 201 241 "-" "-" 835 "ApiServer-https@redis" "http://127.0.0.1:9311" 68ms
May 16 11:13:19 alma1 nextcloud1[873]: task/module/nextcloud1/03083c6b-4de2-43ad-8686-63254bda8ce3: get-configuration/20read is starting
May 16 11:13:19 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:18 +0000] "POST /cluster-admin/api/module/nextcloud1/tasks HTTP/2.0" 201 246 "-" "-" 836 "ApiServer-https@redis" "http://127.0.0.1:9311" 83ms
May 16 11:13:19 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:19 +0000] "GET /cluster-admin/api/module/nextcloud1/task/33981fa8-38b1-453d-bf0a-c62672ce2c32/context HTTP/2.0" 200 251 "-" "-" 838 "ApiServer-https@redis" "http://127.0.0.1:9311" 21ms
May 16 11:13:19 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:19 +0000] "GET /cluster-admin/api/module/nextcloud1/task/33981fa8-38b1-453d-bf0a-c62672ce2c32/context HTTP/2.0" 200 251 "-" "-" 839 "ApiServer-https@redis" "http://127.0.0.1:9311" 27ms
May 16 11:13:19 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:19 +0000] "GET /cluster-admin/api/module/nextcloud1/task/1dbeb4c3-de19-4883-aab4-0f191afd5cb2/context HTTP/2.0" 200 251 "-" "-" 841 "ApiServer-https@redis" "http://127.0.0.1:9311" 39ms
May 16 11:13:19 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:19 +0000] "GET /cluster-admin/api/module/nextcloud1/task/03083c6b-4de2-43ad-8686-63254bda8ce3/context HTTP/2.0" 200 257 "-" "-" 845 "ApiServer-https@redis" "http://127.0.0.1:9311" 19ms
May 16 11:13:19 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:19 +0000] "GET /cluster-admin/api/module/nextcloud1/task/03083c6b-4de2-43ad-8686-63254bda8ce3/context HTTP/2.0" 200 257 "-" "-" 844 "ApiServer-https@redis" "http://127.0.0.1:9311" 33ms
May 16 11:13:19 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:19 +0000] "GET /cluster-admin/api/module/nextcloud1/task/1dbeb4c3-de19-4883-aab4-0f191afd5cb2/context HTTP/2.0" 200 251 "-" "-" 840 "ApiServer-https@redis" "http://127.0.0.1:9311" 121ms
May 16 11:13:20 alma1 nextcloud1[873]: task/module/nextcloud1/669a9966-4b0e-4dbe-99eb-ea75fd3e9d0d: get-configuration/20read is starting
May 16 11:13:20 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:19 +0000] "POST /cluster-admin/api/module/nextcloud1/tasks HTTP/2.0" 201 245 "-" "-" 846 "ApiServer-https@redis" "http://127.0.0.1:9311" 65ms
May 16 11:13:20 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:20 +0000] "GET /cluster-admin/api/module/nextcloud1/task/669a9966-4b0e-4dbe-99eb-ea75fd3e9d0d/context HTTP/2.0" 200 256 "-" "-" 848 "ApiServer-https@redis" "http://127.0.0.1:9311" 14ms
May 16 11:13:20 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:20 +0000] "GET /cluster-admin/api/module/nextcloud1/task/669a9966-4b0e-4dbe-99eb-ea75fd3e9d0d/context HTTP/2.0" 200 256 "-" "-" 851 "ApiServer-https@redis" "http://127.0.0.1:9311" 11ms
May 16 11:13:22 alma1 nextcloud1[873]: task/module/nextcloud1/33981fa8-38b1-453d-bf0a-c62672ce2c32: action "get-name" status is "completed" (0) at step 50get_name
May 16 11:13:22 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:22 +0000] "GET /cluster-admin/api/module/nextcloud1/task/33981fa8-38b1-453d-bf0a-c62672ce2c32/context HTTP/2.0" 200 251 "-" "-" 858 "ApiServer-https@redis" "http://127.0.0.1:9311" 21ms
May 16 11:13:22 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:22 +0000] "GET /cluster-admin/api/module/nextcloud1/task/33981fa8-38b1-453d-bf0a-c62672ce2c32/context HTTP/2.0" 200 251 "-" "-" 859 "ApiServer-https@redis" "http://127.0.0.1:9311" 24ms
May 16 11:13:22 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:22 +0000] "GET /cluster-admin/api/module/nextcloud1/task/33981fa8-38b1-453d-bf0a-c62672ce2c32/status HTTP/2.0" 200 149 "-" "-" 862 "ApiServer-https@redis" "http://127.0.0.1:9311" 10ms
May 16 11:13:23 alma1 nextcloud1[873]: task/module/nextcloud1/1dbeb4c3-de19-4883-aab4-0f191afd5cb2: action "get-status" status is "completed" (0) at step validate-output.json
May 16 11:13:23 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:23 +0000] "GET /cluster-admin/api/module/nextcloud1/task/1dbeb4c3-de19-4883-aab4-0f191afd5cb2/context HTTP/2.0" 200 251 "-" "-" 867 "ApiServer-https@redis" "http://127.0.0.1:9311" 15ms
May 16 11:13:23 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:23 +0000] "GET /cluster-admin/api/module/nextcloud1/task/1dbeb4c3-de19-4883-aab4-0f191afd5cb2/context HTTP/2.0" 200 251 "-" "-" 868 "ApiServer-https@redis" "http://127.0.0.1:9311" 12ms
May 16 11:13:23 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:23 +0000] "GET /cluster-admin/api/module/nextcloud1/task/1dbeb4c3-de19-4883-aab4-0f191afd5cb2/status HTTP/2.0" 200 598 "-" "-" 869 "ApiServer-https@redis" "http://127.0.0.1:9311" 12ms
May 16 11:13:24 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:24 +0000] "GET /cluster-admin/api/module/nextcloud1/task/03083c6b-4de2-43ad-8686-63254bda8ce3/context HTTP/2.0" 200 257 "-" "-" 870 "ApiServer-https@redis" "http://127.0.0.1:9311" 11ms
May 16 11:13:24 alma1 nextcloud1[873]: task/module/nextcloud1/03083c6b-4de2-43ad-8686-63254bda8ce3: action "get-configuration" status is "completed" (0) at step 20read
May 16 11:13:24 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:24 +0000] "GET /cluster-admin/api/module/nextcloud1/task/03083c6b-4de2-43ad-8686-63254bda8ce3/context HTTP/2.0" 200 257 "-" "-" 871 "ApiServer-https@redis" "http://127.0.0.1:9311" 16ms
May 16 11:13:24 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:24 +0000] "GET /cluster-admin/api/module/nextcloud1/task/03083c6b-4de2-43ad-8686-63254bda8ce3/context HTTP/2.0" 200 257 "-" "-" 872 "ApiServer-https@redis" "http://127.0.0.1:9311" 11ms
May 16 11:13:24 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:24 +0000] "GET /cluster-admin/api/module/nextcloud1/task/03083c6b-4de2-43ad-8686-63254bda8ce3/status HTTP/2.0" 200 266 "-" "-" 873 "ApiServer-https@redis" "http://127.0.0.1:9311" 26ms
May 16 11:13:24 alma1 nextcloud1[873]: task/module/nextcloud1/669a9966-4b0e-4dbe-99eb-ea75fd3e9d0d: action "get-configuration" status is "completed" (0) at step 20read
May 16 11:13:24 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:24 +0000] "GET /cluster-admin/api/module/nextcloud1/task/669a9966-4b0e-4dbe-99eb-ea75fd3e9d0d/context HTTP/2.0" 200 256 "-" "-" 874 "ApiServer-https@redis" "http://127.0.0.1:9311" 18ms
May 16 11:13:24 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:24 +0000] "GET /cluster-admin/api/module/nextcloud1/task/669a9966-4b0e-4dbe-99eb-ea75fd3e9d0d/context HTTP/2.0" 200 256 "-" "-" 875 "ApiServer-https@redis" "http://127.0.0.1:9311" 16ms
May 16 11:13:24 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:24 +0000] "GET /cluster-admin/api/module/nextcloud1/task/669a9966-4b0e-4dbe-99eb-ea75fd3e9d0d/status HTTP/2.0" 200 266 "-" "-" 876 "ApiServer-https@redis" "http://127.0.0.1:9311" 9ms
May 16 11:13:29 alma1 nextcloud1[873]: task/module/nextcloud1/7983e5c2-ae1a-480b-a816-cb910ddf48c3: configure-module/20configure is starting
May 16 11:13:29 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:29 +0000] "POST /cluster-admin/api/module/nextcloud1/tasks HTTP/2.0" 201 322 "-" "-" 877 "ApiServer-https@redis" "http://127.0.0.1:9311" 22ms
May 16 11:13:29 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:29 +0000] "GET /cluster-admin/api/module/nextcloud1/task/7983e5c2-ae1a-480b-a816-cb910ddf48c3/context HTTP/2.0" 200 331 "-" "-" 878 "ApiServer-https@redis" "http://127.0.0.1:9311" 12ms
May 16 11:13:29 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:29 +0000] "GET /cluster-admin/api/module/nextcloud1/task/7983e5c2-ae1a-480b-a816-cb910ddf48c3/context HTTP/2.0" 200 331 "-" "-" 879 "ApiServer-https@redis" "http://127.0.0.1:9311" 11ms
May 16 11:13:30 alma1 nextcloud1[873]: task/module/nextcloud1/7983e5c2-ae1a-480b-a816-cb910ddf48c3: configure-module/30traefik is starting
May 16 11:13:30 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:30 +0000] "GET /cluster-admin/api/module/nextcloud1/task/7983e5c2-ae1a-480b-a816-cb910ddf48c3/context HTTP/2.0" 200 331 "-" "-" 880 "ApiServer-https@redis" "http://127.0.0.1:9311" 15ms
May 16 11:13:30 alma1 nextcloud1[873]: Traceback (most recent call last):
May 16 11:13:30 alma1 nextcloud1[873]:  File "/home/nextcloud1/.config/actions/configure-module/30traefik", line 49, in <module>
May 16 11:13:30 alma1 nextcloud1[873]:    response = agent.tasks.run(
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/local/agent/pypkg/agent/tasks/run.py", line 39, in run
May 16 11:13:30 alma1 nextcloud1[873]:    results = runp([taskrq], **kwargs)
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/local/agent/pypkg/agent/tasks/run.py", line 50, in runp
May 16 11:13:30 alma1 nextcloud1[873]:    return asyncio.run(_runp(tasks, **kwargs))
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/lib64/python3.9/asyncio/runners.py", line 44, in run
May 16 11:13:30 alma1 nextcloud1[873]:    return loop.run_until_complete(main)
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/lib64/python3.9/asyncio/base_events.py", line 647, in run_until_complete
May 16 11:13:30 alma1 nextcloud1[873]:    return future.result()
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/local/agent/pypkg/agent/tasks/run.py", line 120, in _runp
May 16 11:13:30 alma1 nextcloud1[873]:    return await asyncio.gather(*runners, return_exceptions=(len(tasks) > 1))
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/local/agent/pypkg/agent/tasks/run.py", line 129, in _run_with_protocol
May 16 11:13:30 alma1 nextcloud1[873]:    return await run_apiclient(taskrq, **pconn)
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/local/agent/pypkg/agent/tasks/apiclient.py", line 47, in run_apiclient
May 16 11:13:30 alma1 nextcloud1[873]:    taskctx['status_path'] = await _retry_request(_apost_task, taskrq, client=client, theaders=theaders, **kwargs)
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/local/agent/pypkg/agent/tasks/apiclient.py", line 191, in _retry_request
May 16 11:13:30 alma1 nextcloud1[873]:    raise exhttp
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/local/agent/pypkg/agent/tasks/apiclient.py", line 166, in _retry_request
May 16 11:13:30 alma1 nextcloud1[873]:    retval = await request_procedure(*args, **kwargs)
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/local/agent/pypkg/agent/tasks/apiclient.py", line 246, in _apost_task
May 16 11:13:30 alma1 nextcloud1[873]:    async with client.post(
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/local/agent/pyenv/lib64/python3.9/site-packages/aiohttp/client.py", line 1117, in __aenter__
May 16 11:13:30 alma1 nextcloud1[873]:    self._resp = await self._coro
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/local/agent/pyenv/lib64/python3.9/site-packages/aiohttp/client.py", line 625, in _request
May 16 11:13:30 alma1 nextcloud1[873]:    resp.raise_for_status()
May 16 11:13:30 alma1 nextcloud1[873]:  File "/usr/local/agent/pyenv/lib64/python3.9/site-packages/aiohttp/client_reqrep.py", line 1000, in raise_for_status
May 16 11:13:30 alma1 nextcloud1[873]:    raise ClientResponseError(
May 16 11:13:30 alma1 nextcloud1[873]: aiohttp.client_exceptions.ClientResponseError: 403, message='Forbidden', url=URL('http://cluster-leader:9311/api/module/traefik1/tasks')
May 16 11:13:30 alma1 nextcloud1[873]: task/module/nextcloud1/7983e5c2-ae1a-480b-a816-cb910ddf48c3: action "configure-module" status is "aborted" (1) at step 30traefik
May 16 11:13:30 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:30 +0000] "GET /cluster-admin/api/module/nextcloud1/task/7983e5c2-ae1a-480b-a816-cb910ddf48c3/context HTTP/2.0" 200 331 "-" "-" 881 "ApiServer-https@redis" "http://127.0.0.1:9311" 14ms
May 16 11:13:30 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:30 +0000] "GET /cluster-admin/api/module/nextcloud1/task/7983e5c2-ae1a-480b-a816-cb910ddf48c3/status HTTP/2.0" 200 803 "-" "-" 882 "ApiServer-https@redis" "http://127.0.0.1:9311" 11ms
May 16 11:13:57 alma1 nextcloud1[1870]: 127.0.0.1 -  16/May/2023:09:13:57 +0000 "GET /index.php" 200
May 16 11:13:57 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:57 +0000] "GET /login HTTP/2.0" 200 5932 "-" "-" 883 "nextcloud1-https@redis" "http://127.0.0.1:20009" 103ms
May 16 11:13:58 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:58 +0000] "GET /apps/theming/js/theming.js?v=2a1c1a0f-0 HTTP/2.0" 200 60 "-" "-" 884 "nextcloud1-https@redis" "http://127.0.0.1:20009" 0ms
May 16 11:13:58 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:13:58 +0000] "GET /dist/core-login.js?v=2a1c1a0f-0 HTTP/2.0" 200 30949 "-" "-" 885 "nextcloud1-https@redis" "http://127.0.0.1:20009" 4ms
May 16 11:14:00 alma1 nextcloud1[1870]: crond: wakeup dt=60
May 16 11:14:00 alma1 nextcloud1[1870]: crond: file www-data:
May 16 11:14:00 alma1 nextcloud1[1870]: crond:  line php -f /var/www/html/cron.php
May 16 11:14:01 alma1 nextcloud1[1870]: 127.0.0.1 -  16/May/2023:09:14:00 +0000 "POST /index.php" 500
May 16 11:14:01 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:14:00 +0000] "POST /login HTTP/2.0" 500 3619 "-" "-" 886 "nextcloud1-https@redis" "http://127.0.0.1:20009" 279ms
May 16 11:14:03 alma1 nextcloud1[1870]: 127.0.0.1 -  16/May/2023:09:14:03 +0000 "GET /index.php" 200
May 16 11:14:03 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:14:03 +0000] "GET /login HTTP/2.0" 200 5935 "-" "-" 887 "nextcloud1-https@redis" "http://127.0.0.1:20009" 77ms
May 16 11:14:13 alma1 nextcloud1[1870]: 127.0.0.1 -  16/May/2023:09:14:12 +0000 "POST /index.php" 500
May 16 11:14:13 alma1 traefik1[987]: 192.168.3.159 - - [16/May/2023:09:14:12 +0000] "POST /login HTTP/2.0" 500 3619 "-" "-" 888 "nextcloud1-https@redis" "http://127.0.0.1:20009" 280ms
May 16 11:15:00 alma1 nextcloud1[1870]: crond: wakeup dt=60
May 16 11:15:00 alma1 nextcloud1[1870]: crond: file www-data:
May 16 11:15:00 alma1 nextcloud1[1870]: crond:  line php -f /var/www/html/cron.php
May 16 11:15:00 alma1 nextcloud1[1870]: crond:  job: 0 php -f /var/www/html/cron.php
May 16 11:15:00 alma1 nextcloud1[1870]: crond: child running /bin/ash
May 16 11:15:00 alma1 nextcloud1[1870]: crond: USER www-data pid  78 cmd php -f /var/www/html/cron.php
May 16 11:15:10 alma1 nextcloud1[1870]: crond: wakeup dt=10
May 16 11:16:00 alma1 nextcloud1[1870]: crond: wakeup dt=50
May 16 11:16:00 alma1 nextcloud1[1870]: crond: file www-data:
May 16 11:16:00 alma1 nextcloud1[1870]: crond:  line php -f /var/www/html/cron.php
May 16 11:17:00 alma1 nextcloud1[1870]: crond: wakeup dt=60
May 16 11:17:00 alma1 nextcloud1[1870]: crond: file www-data:
May 16 11:17:00 alma1 nextcloud1[1870]: crond:  line php -f /var/www/html/cron.php
May 16 11:18:00 alma1 nextcloud1[1870]: crond: wakeup dt=60
May 16 11:18:00 alma1 nextcloud1[1870]: crond: file www-data:
May 16 11:18:00 alma1 nextcloud1[1870]: crond:  line php -f /var/www/html/cron.php
May 16 11:19:00 alma1 nextcloud1[1870]: crond: wakeup dt=60
May 16 11:19:00 alma1 nextcloud1[1870]: crond: file www-data:
May 16 11:19:00 alma1 nextcloud1[1870]: crond:  line php -f /var/www/html/cron.php
1 Like

This could be a bug due to some ACL refactor.
I think we should investigate this:

May 16 11:13:30 alma1 nextcloud1[873]: aiohttp.client_exceptions.ClientResponseError: 403, message='Forbidden', url=URL('http://cluster-leader:9311/api/module/traefik1/tasks')

Thank you Mark!

(We will need a bit of time to try fixing it!)

1 Like

well…I bet that our migration path are broken, we have the same effect with mattermost

[mattermost1@R1-pve ~]$ podman exec -ti mattermost-app /bin/bash
mattermost@mattermost:~$ ls -la
total 248
drwxr-xr-x. 1 mattermost mattermost     41 May 30 06:52 .
dr-xr-xr-x. 1 root       root           68 May 30 06:52 ..
-rw-r--r--. 1 mattermost mattermost   1239 Apr 13 16:36 MIT-COMPILED-LICENSE.md
-rw-r--r--. 1 mattermost mattermost 218333 Apr 13 16:36 NOTICE.txt
-rw-r--r--. 1 mattermost mattermost   7145 Apr 13 16:36 README.md
drwxr-xr-x. 2 mattermost mattermost     37 Apr 13 16:37 bin
drwxr-xr-x. 2 mattermost mattermost      6 May 30 06:52 bleve-indexes
drwxr-xr-x. 1 mattermost mattermost   4096 Apr 13 16:36 client
drwxr-xr-x. 2 mattermost mattermost     42 May 30 06:52 config
drwxr-xr-x. 4        992        988     35 May 30  2023 data
drwxr-xr-x. 2 mattermost mattermost     44 Apr 13 16:36 fonts
drwxr-xr-x. 2 mattermost mattermost   4096 Apr 13 16:36 i18n
drwxr-xr-x. 2 mattermost mattermost      6 Apr 13 16:37 logs
-rw-r--r--. 1 mattermost mattermost    632 Apr 13 16:36 manifest.txt
drwxr-xr-x. 8 mattermost mattermost    166 May 30 06:52 plugins
drwxr-xr-x. 2 mattermost mattermost   4096 Apr 13 16:37 prepackaged_plugins
drwxr-xr-x. 2 mattermost mattermost   4096 Apr 13 16:36 templates
[root@NS1 ~]# id mattermost
uid=992(mattermost) gid=988(mattermost) groups=988(mattermost)

this is not wokable anymore (it was before)
nethserver-ns8-migration/migrate at master · NethServer/nethserver-ns8-migration · GitHub

same for nethserver-ns8-migration/migrate at master · NethServer/nethserver-ns8-migration · GitHub

it seems that our rsync container does not honor anymore the ownership we are reclaiming. cc @giacomo

3 Likes
May 30 10:19:34 R1-pve.rocky9-pve.org nextcloud1[12922]: task/module/nextcloud1/79c70949-0dd7-4ba2-b12e-5c0dfb7cd89a: import-module/05create_volumes is starting
May 30 10:19:34 R1-pve.rocky9-pve.org nextcloud1[12922]: podman volume create nextcloud-app-data
May 30 10:19:34 R1-pve.rocky9-pve.org nextcloud1[12922]: nextcloud-app-data
May 30 10:19:34 R1-pve.rocky9-pve.org nextcloud1[12922]: task/module/nextcloud1/79c70949-0dd7-4ba2-b12e-5c0dfb7cd89a: import-module/10recvstate is starting
May 30 10:19:35 R1-pve.rocky9-pve.org nextcloud1[12922]: podman run --rm --privileged --network=host --workdir=/srv --env=RSYNCD_NETWORK=10.5.4.0/24 --env=RSYNCD_ADDRESS=cluster-localnode --env=RSYNCD_PORT=20013 --env=RSYNCD_USER=nextcloud1 --env=RSYNCD_PASSWORD=1274534b680a3-2bc6-41cc-a2c8-0155c01344cd --env=RSYNCD_SYSLOG_TAG=nextcloud1 --volume=/dev/log:/dev/log --name=rsync-nextcloud1 --volume=/home/nextcloud1/.config/state:/srv/state --volume=nextcloud-app-data:/srv/volumes/nextcloud-app-data ghcr.io/nethserver/rsync:1.0.1
May 30 10:19:35 R1-pve.rocky9-pve.org podman[14134]: 
May 30 10:19:35 R1-pve.rocky9-pve.org systemd[12906]: Started libcrun container.
May 30 10:19:35 R1-pve.rocky9-pve.org nextcloud1[14148]: rsyncd version 3.2.7 starting, listening on port 20013
May 30 10:19:36 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:19:36.100 * 1 changes in 5 seconds. Saving...
May 30 10:19:36 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:19:36.101 * Background saving started by pid 72
May 30 10:19:36 R1-pve.rocky9-pve.org redis[7650]: 72:C 30 May 2023 08:19:36.125 * DB saved on disk
May 30 10:19:36 R1-pve.rocky9-pve.org redis[7650]: 72:C 30 May 2023 08:19:36.125 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
May 30 10:19:36 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:19:36.203 * Background saving terminated with success
May 30 10:19:37 R1-pve.rocky9-pve.org nextcloud1[14153]: connect from UNDETERMINED (10.5.4.2)
May 30 10:19:37 R1-pve.rocky9-pve.org nextcloud1[14153]: module-list request from UNDETERMINED (10.5.4.2)
May 30 10:19:37 R1-pve.rocky9-pve.org cluster[7808]: task/cluster/675c2fce-1fce-402f-932b-edb5d09dbe9c: list-user-domains/50read is starting
May 30 10:19:38 R1-pve.rocky9-pve.org cluster[7808]: task/cluster/675c2fce-1fce-402f-932b-edb5d09dbe9c: action "list-user-domains" status is "completed" (0) at step validate-output.json
May 30 10:19:42 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:19:42.097 * 1 changes in 5 seconds. Saving...
May 30 10:19:42 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:19:42.097 * Background saving started by pid 73
May 30 10:19:42 R1-pve.rocky9-pve.org redis[7650]: 73:C 30 May 2023 08:19:42.120 * DB saved on disk
May 30 10:19:42 R1-pve.rocky9-pve.org redis[7650]: 73:C 30 May 2023 08:19:42.120 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
May 30 10:19:42 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:19:42.198 * Background saving terminated with success
May 30 10:20:36 R1-pve.rocky9-pve.org nextcloud1[14157]: connect from UNDETERMINED (10.5.4.2)
May 30 10:20:36 R1-pve.rocky9-pve.org nextcloud1[14157]: rsync allowed access on module data from UNDETERMINED (10.5.4.2)
May 30 10:20:36 R1-pve.rocky9-pve.org nextcloud1[14157]: rsync to data/ from nextcloud1@UNDETERMINED (10.5.4.2)
May 30 10:20:36 R1-pve.rocky9-pve.org nextcloud1[14157]: receiving file list
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14159]: connect from UNDETERMINED (10.5.4.2)
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14159]: rsync allowed access on module data from UNDETERMINED (10.5.4.2)
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14157]: sent 24 bytes  received 57 bytes  total size 230
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14159]: rsync to data/volumes/nextcloud-app-data/data/ from nextcloud1@UNDETERMINED (10.5.4.2)
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14159]: receiving file list
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14159]: sent 760 bytes  received 25970437 bytes  total size 25961309
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14161]: connect from UNDETERMINED (10.5.4.2)
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14161]: rsync allowed access on module data from UNDETERMINED (10.5.4.2)
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14161]: rsync to data/state/restore/ from nextcloud1@UNDETERMINED (10.5.4.2)
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14161]: receiving file list
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14163]: connect from UNDETERMINED (10.5.4.2)
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14163]: rsync allowed access on module data from UNDETERMINED (10.5.4.2)
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14161]: sent 40 bytes  received 169460 bytes  total size 169324
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14163]: rsync to data/state/ from nextcloud1@UNDETERMINED (10.5.4.2)
May 30 10:20:37 R1-pve.rocky9-pve.org nextcloud1[14163]: receiving file list
May 30 10:20:38 R1-pve.rocky9-pve.org nextcloud1[14163]: sent 40 bytes  received 1317 bytes  total size 1216
May 30 10:20:38 R1-pve.rocky9-pve.org cluster[7808]: task/cluster/70258ccd-f033-4ec8-b340-c5373af45a54: list-user-domains/50read is starting
May 30 10:20:38 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:20:38.768 * 1 changes in 5 seconds. Saving...
May 30 10:20:38 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:20:38.769 * Background saving started by pid 74
May 30 10:20:38 R1-pve.rocky9-pve.org redis[7650]: 74:C 30 May 2023 08:20:38.788 * DB saved on disk
May 30 10:20:38 R1-pve.rocky9-pve.org redis[7650]: 74:C 30 May 2023 08:20:38.789 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
May 30 10:20:38 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:20:38.869 * Background saving terminated with success
May 30 10:20:39 R1-pve.rocky9-pve.org cluster[7808]: task/cluster/70258ccd-f033-4ec8-b340-c5373af45a54: action "list-user-domains" status is "completed" (0) at step validate-output.json
May 30 10:20:44 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:20:44.040 * 1 changes in 5 seconds. Saving...
May 30 10:20:44 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:20:44.040 * Background saving started by pid 75
May 30 10:20:44 R1-pve.rocky9-pve.org redis[7650]: 75:C 30 May 2023 08:20:44.076 * DB saved on disk
May 30 10:20:44 R1-pve.rocky9-pve.org redis[7650]: 75:C 30 May 2023 08:20:44.077 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
May 30 10:20:44 R1-pve.rocky9-pve.org redis[7650]: 1:M 30 May 2023 08:20:44.141 * Background saving terminated with success
May 30 10:21:26 R1-pve.rocky9-pve.org systemd[12906]: Starting Mark boot as successful...
May 30 10:21:26 R1-pve.rocky9-pve.org systemd[12906]: Finished Mark boot as successful.

UNDETERMINED why ?

2 Likes

need to check, it seems we need to use

rsync -tr --owner --group --usermap=48:82 --groupmap=48:82 --numeric-ids --delete --exclude=appdata\* --exclude=nextcloud.log /var/lib/nethserver/nextcloud/ "${RSYNC_ENDPOINT}"/data/volumes/nextcloud-app-data/data/

–chown seems dead even if it is documented

I need to do more tests

3 Likes

https://download.samba.org/pub/rsync/rsync.1#opt--chown

Or we need to add -s because we have different version of rsync, it enables wildcard

need to do furthers tests too

2 Likes

well we have two pb here

  • one bad ownership of file synchronization
  • one specific to nextcloud relevant to the upgrade to nc26 (not inspected)
3 Likes

with the nethserver-ns8-migration-1.0.2 the migration should be workable for now

we recommend before to migrate nextcloud to upgrade nextcloud to nc26
yum install http://packages.nethserver.org/nethserver/7.9.2009/testing/x86_64/Packages/nethserver-nextcloud-1.21.4-1.2.g0409507.ns7.noarch.rpm

4 Likes

The migration with updated NS8 core, ns8-migration-tool and Nextcloud from testing worked like a charm. :+1:

3 Likes