Slapd does not start

pike · November 29, 2022, 6:49pm

NethServer Version: 7.9
Module: Users and Groups (OpenLDAP)
After several weeks of downtime Let’sencrypt cryed about soon to expire certificate. So I boot up my test installation and the certificate was updated. Yay.
However.

slapd decided to take some nap. Lazy boy/daemon. However:

nudged a start. Mope.
verified the firewall access. Still not willing to cooperate
throwed a signal-event nethserver-directory-update. Nothing new (or good)
rebooted the whole guy. Still nothing useful.

I scavenged into logs, i found this detail.


 systemd Tue Nov 29 2022 19:30:39 GMT+0100 
Failed to start OpenLDAP Server Daemon.
CODE_FILE	src/core/job.c
CODE_FUNCTION	job_log_status_message
CODE_LINE	774
MESSAGE_ID	be02cf6855d2428ba40df7e9d022f03d
PRIORITY	3
RESULT	failed
SYSLOG_FACILITY	3
SYSLOG_IDENTIFIER	systemd
UNIT	slapd.service
_BOOT_ID	bf1b5ac42fbc479280364f163a5f0239
_CAP_EFFECTIVE	1fffffffff
_CMDLINE	/usr/lib/systemd/systemd --switched-root --system --deserialize 22
_COMM	systemd
_EXE	/usr/lib/systemd/systemd
_GID	0
_HOSTNAME	public.hostname.tld
_MACHINE_ID	0cf84a4c9dea459c952212641958b596
_PID	1
_SOURCE_REALTIME_TIMESTAMP	1669746639869891
_SYSTEMD_CGROUP	/
_TRANSPORT	journal
_UID	0
__CURSOR	s=0752fac9da8f467d919639a310070d72;i=199d;b=bf1b5ac42fbc479280364f163a5f0239;m=8928584af;t=5eea02f563cbd;x=ca308046d98ef6c2
__MONOTONIC_TIMESTAMP	36817962159
__REALTIME_TIMESTAMP	1669746639912125

As usual, the scapegoat can be sacrified to greater goods, but starting from scratch is quite a pain in the hardware

dnutan · November 29, 2022, 9:59pm

anything else from:

systemctl status -l slapd
journalctl -xe

pike · November 29, 2022, 10:16pm

Here there are fresh outputs.

systemctl status -l slapd

slapd.service - OpenLDAP Server Daemon
   Loaded: loaded (/usr/lib/systemd/system/slapd.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/slapd.service.d
           └─nethserver.conf
   Active: failed (Result: exit-code) since Tue 2022-11-29 19:35:53 CET; 3h 38min ago
     Docs: man:slapd
           man:slapd-config
           man:slapd-hdb
           man:slapd-mdb
           file:///usr/share/doc/openldap-servers/guide.html
  Process: 1376 ExecStart=/usr/sbin/slapd -u ldap -h ${SLAPD_URLS} $SLAPD_OPTIONS (code=exited, status=1/FAILURE)
  Process: 1119 ExecStartPre=/usr/libexec/openldap/check-config.sh (code=exited, status=0/SUCCESS)

Nov 29 19:35:52 public.hostname.tld runuser[1287]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 19:35:52 public.hostname.tld runuser[1287]: pam_unix(runuser:session): session closed for user ldap
Nov 29 19:35:52 public.hostname.tld runuser[1302]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 19:35:52 public.hostname.tld runuser[1308]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 19:35:52 public.hostname.tld runuser[1317]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 19:35:52 public.hostname.tld slapd[1376]: @(#) $OpenLDAP: slapd 2.4.44 (Feb 23 2022 17:11:27) $
                                                          mockbuild@x86-01.bsys.centos.org:/builddir/build/BUILD/openldap-2.4.44/openldap-2.4.44/servers/slapd
Nov 29 19:35:53 public.hostname.tld systemd[1]: slapd.service: control process exited, code=exited status=1
Nov 29 19:35:53 public.hostname.tld systemd[1]: Failed to start OpenLDAP Server Daemon.
Nov 29 19:35:53 public.hostname.tld systemd[1]: Unit slapd.service entered failed state.
Nov 29 19:35:53 public.hostname.tld systemd[1]: slapd.service failed.

journalctl -xe

Nov 29 23:06:54 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry cycle in 1 steps: 0 items checked, 0 significant (0 made persist
Nov 29 23:06:54 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in ham: {nil}
Nov 29 23:06:54 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in spam: {nil}
Nov 29 23:06:54 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, total: {nil}
Nov 29 23:08:32 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry step 1: 0 items checked, 0 significant (0 made persistent), 0 in
Nov 29 23:08:32 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry cycle in 1 steps: 0 items checked, 0 significant (0 made persist
Nov 29 23:08:32 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in ham: {nil}
Nov 29 23:08:32 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in spam: {nil}
Nov 29 23:08:32 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, total: {nil}
Nov 29 23:10:25 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry step 1: 0 items checked, 0 significant (0 made persistent), 0 in
Nov 29 23:10:25 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry cycle in 1 steps: 0 items checked, 0 significant (0 made persist
Nov 29 23:10:25 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in ham: {nil}
Nov 29 23:10:25 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in spam: {nil}
Nov 29 23:10:25 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, total: {nil}
Nov 29 23:12:21 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry step 1: 0 items checked, 0 significant (0 made persistent), 0 in
Nov 29 23:12:21 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry cycle in 1 steps: 0 items checked, 0 significant (0 made persist
Nov 29 23:12:21 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in ham: {nil}
Nov 29 23:12:21 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in spam: {nil}
Nov 29 23:12:21 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, total: {nil}
Nov 29 23:13:42 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry step 1: 0 items checked, 0 significant (0 made persistent), 0 in
Nov 29 23:13:42 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry cycle in 1 steps: 0 items checked, 0 significant (0 made persist
Nov 29 23:13:42 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in ham: {nil}
Nov 29 23:13:42 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in spam: {nil}
Nov 29 23:13:42 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, total: {nil}
Nov 29 23:14:01 public.hostname.tld systemd[1]: Created slice User Slice of root.
-- Subject: Unit user-0.slice has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit user-0.slice has finished starting up.
--
-- The start-up result is done.
Nov 29 23:14:01 public.hostname.tld systemd[1]: Started Session 12 of user root.
-- Subject: Unit session-12.scope has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit session-12.scope has finished starting up.
--
-- The start-up result is done.
Nov 29 23:14:01 public.hostname.tld CROND[27349]: (root) CMD ([ -x /usr/sbin/clamav-unofficial-sigs.sh ] && /bin/bash /usr/sbin/clamav-unofficial-sigs.sh)
Nov 29 23:14:01 public.hostname.tld clamd[1671]: SelfCheck: Database status OK.
Nov 29 23:14:14 public.hostname.tld sshd[28123]: pam_listfile(sshd:auth): Refused user root for service sshd
Nov 29 23:14:14 public.hostname.tld sshd[28119]: Accepted keyboard-interactive/pam for root from 172.31.111.56 port 63232 ssh2
...skipping...
Nov 29 23:06:54 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry cycle in 1 steps: 0 items checked, 0 significant (0 made persist
Nov 29 23:06:54 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in ham: {nil}
Nov 29 23:06:54 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in spam: {nil}
Nov 29 23:06:54 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, total: {nil}
Nov 29 23:08:32 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry step 1: 0 items checked, 0 significant (0 made persistent), 0 in
Nov 29 23:08:32 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry cycle in 1 steps: 0 items checked, 0 significant (0 made persist
Nov 29 23:08:32 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in ham: {nil}
Nov 29 23:08:32 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in spam: {nil}
Nov 29 23:08:32 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, total: {nil}
Nov 29 23:10:25 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry step 1: 0 items checked, 0 significant (0 made persistent), 0 in
Nov 29 23:10:25 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry cycle in 1 steps: 0 items checked, 0 significant (0 made persist
Nov 29 23:10:25 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in ham: {nil}
Nov 29 23:10:25 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in spam: {nil}
Nov 29 23:10:25 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, total: {nil}
Nov 29 23:12:21 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry step 1: 0 items checked, 0 significant (0 made persistent), 0 in
Nov 29 23:12:21 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry cycle in 1 steps: 0 items checked, 0 significant (0 made persist
Nov 29 23:12:21 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in ham: {nil}
Nov 29 23:12:21 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in spam: {nil}
Nov 29 23:12:21 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, total: {nil}
Nov 29 23:13:42 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry step 1: 0 items checked, 0 significant (0 made persistent), 0 in
Nov 29 23:13:42 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry cycle in 1 steps: 0 items checked, 0 significant (0 made persist
Nov 29 23:13:42 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in ham: {nil}
Nov 29 23:13:42 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in spam: {nil}
Nov 29 23:13:42 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, total: {nil}
Nov 29 23:14:01 public.hostname.tld systemd[1]: Created slice User Slice of root.
-- Subject: Unit user-0.slice has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit user-0.slice has finished starting up.
--
-- The start-up result is done.
Nov 29 23:14:01 public.hostname.tld systemd[1]: Started Session 12 of user root.
-- Subject: Unit session-12.scope has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit session-12.scope has finished starting up.
--
-- The start-up result is done.
Nov 29 23:14:01 public.hostname.tld CROND[27349]: (root) CMD ([ -x /usr/sbin/clamav-unofficial-sigs.sh ] && /bin/bash /usr/sbin/clamav-unofficial-sigs.sh)
Nov 29 23:14:01 public.hostname.tld clamd[1671]: SelfCheck: Database status OK.
Nov 29 23:14:14 public.hostname.tld sshd[28123]: pam_listfile(sshd:auth): Refused user root for service sshd
Nov 29 23:14:14 public.hostname.tld sshd[28119]: Accepted keyboard-interactive/pam for root from 172.31.111.56 port 63232 ssh2
Nov 29 23:14:15 public.hostname.tld systemd-logind[1027]: New session 13 of user root.
-- Subject: A new session 13 has been created for user root
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- Documentation: http://www.freedesktop.org/wiki/Software/systemd/multiseat
--
-- A new session with the ID 13 has been created for the user root.
--
-- The leading process of the session is 28119.
Nov 29 23:14:15 public.hostname.tld systemd[1]: Started Session 13 of user root.
-- Subject: Unit session-13.scope has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit session-13.scope has finished starting up.
--
-- The start-up result is done.
Nov 29 23:14:15 public.hostname.tld sshd[28119]: pam_unix(sshd:session): session opened for user root by (uid=0)
Nov 29 23:15:10 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry step 1: 0 items checked, 0 significant (0 made persistent), 0 in
Nov 29 23:15:10 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:440: finished expiry cycle in 1 steps: 0 items checked, 0 significant (0 made persist
Nov 29 23:15:10 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in ham: {nil}
Nov 29 23:15:10 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, in spam: {nil}
Nov 29 23:15:10 public.hostname.tld rspamd[4005]: <7y4xnn>; lua; bayes_expiry.lua:447: tokens occurrences, total: {nil}

After a systemctl start slapd

--
-- Unit slapd-create-dhparam.service has finished starting up.
--
-- The start-up result is done.
Nov 29 23:19:11 public.hostname.tld runuser[28516]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28516]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28540]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28540]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28542]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28542]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28544]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28544]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28546]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28546]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28548]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28548]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28550]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28550]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28552]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28552]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28554]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28554]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28556]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28556]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28558]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28558]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28560]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28560]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld runuser[28562]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Nov 29 23:19:13 public.hostname.tld runuser[28562]: pam_unix(runuser:session): session closed for user ldap
Nov 29 23:19:13 public.hostname.tld slapd[28565]: @(#) $OpenLDAP: slapd 2.4.44 (Feb 23 2022 17:11:27) $
                                                           mockbuild@x86-01.bsys.centos.org:/builddir/build/BUILD/openldap-2.4.44/openldap-2.4.44/servers/slapd
Nov 29 23:19:14 public.hostname.tld systemd[1]: slapd.service: control process exited, code=exited status=1
Nov 29 23:19:14 public.hostname.tld systemd[1]: Failed to start OpenLDAP Server Daemon.
-- Subject: Unit slapd.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit slapd.service has failed.
--
-- The result is failed.
Nov 29 23:19:14 public.hostname.tld systemd[1]: Unit slapd.service entered failed state.
Nov 29 23:19:14 public.hostname.tld systemd[1]: slapd.service failed.
Nov 29 23:19:14 public.hostname.tld polkitd[801]: Unregistered Authentication Agent for unix-process:28503:1342829 (system bus name :1.99, object path /org/freedesktop

davidep · November 30, 2022, 7:32am

Just to check if the issue comes from the certificate… Switch temporarily to the self signed certificate and try to start slapd again.

pike · November 30, 2022, 10:33am

Thanks for anyone is giving support. I’ll answer tomorrow after the test.

pike · December 1, 2022, 9:08am

Long story short: things are quite the same.
Trying to change the default certificate i had this error (sorry for italian)

Text for the command echo '{"props":{"KeyFile":"/etc/pki/tls/private/NSRV.key","CrtFile":"/etc/pki/tls/certs/NSRV.crt","ChainFile":""},"action":"set-default"}' | /usr/bin/setsid /usr/bin/sudo /usr/libexec/nethserver/api/system-certificate/update | jq

But the setting succeeded.

I anyway I poked for the start of slapd with this result.

 systemctl start slapd
Job for slapd.service failed because the control process exited with error code. See "systemctl status slapd.service" and "journalctl -xe" for details.

Dec 01 09:55:44 public.hostname.tldpolkitd[801]: Registered Authentication Agent for unix-process:11631:13802172 (system bus name :1.435 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedes
Dec 01 09:55:44 public.hostname.tldsystemd[1]: Starting OpenLDAP Server Daemon...
-- Subject: Unit slapd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit slapd.service has begun starting up.
Dec 01 09:55:44 public.hostname.tldsystemd[1]: Started Create the dhparam group for slapd.
-- Subject: Unit slapd-create-dhparam.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit slapd-create-dhparam.service has finished starting up.
--
-- The start-up result is done.
Dec 01 09:55:44 public.hostname.tldrunuser[11642]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:44 public.hostname.tldrunuser[11642]: pam_unix(runuser:session): session closed for user ldap
Dec 01 09:55:44 public.hostname.tldrunuser[11653]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:44 public.hostname.tldrunuser[11653]: pam_unix(runuser:session): session closed for user ldap
Dec 01 09:55:44 public.hostname.tldrunuser[11655]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:44 public.hostname.tldrunuser[11655]: pam_unix(runuser:session): session closed for user ldap
Dec 01 09:55:44 public.hostname.tldrunuser[11657]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:44 public.hostname.tldrunuser[11657]: pam_unix(runuser:session): session closed for user ldap
Dec 01 09:55:44 public.hostname.tldrunuser[11659]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:44 public.hostname.tldrunuser[11659]: pam_unix(runuser:session): session closed for user ldap
Dec 01 09:55:44 public.hostname.tldrunuser[11661]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:45 public.hostname.tldrunuser[11661]: pam_unix(runuser:session): session closed for user ldap
Dec 01 09:55:45 public.hostname.tldrunuser[11663]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:45 public.hostname.tldrunuser[11663]: pam_unix(runuser:session): session closed for user ldap
Dec 01 09:55:45 public.hostname.tldrunuser[11665]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:45 public.hostname.tldrunuser[11665]: pam_unix(runuser:session): session closed for user ldap
Dec 01 09:55:45 public.hostname.tldrunuser[11667]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:45 public.hostname.tldrunuser[11667]: pam_unix(runuser:session): session closed for user ldap
Dec 01 09:55:45 public.hostname.tldrunuser[11669]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:45 public.hostname.tldrunuser[11669]: pam_unix(runuser:session): session closed for user ldap
Dec 01 09:55:45 public.hostname.tldrunuser[11671]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:45 public.hostname.tldrunuser[11671]: pam_unix(runuser:session): session closed for user ldap
Dec 01 09:55:45 public.hostname.tldsystemd[1]: slapd.service: control process exited, code=exited status=1
Dec 01 09:55:45 public.hostname.tldrunuser[11673]: pam_unix(runuser:session): session opened for user ldap by (uid=0)
Dec 01 09:55:45 public.hostname.tldsystemd[1]: Failed to start OpenLDAP Server Daemon.
-- Subject: Unit slapd.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit slapd.service has failed.
--

Server has been rebooted, slapd keept sleeping, maybe it knows that winter is coming…

davidep · December 1, 2022, 3:45pm

This seems good news: the DB should be still intact…

…that does not help

Can you see anything interesting in /var/log/messages?

pike · December 1, 2022, 4:10pm

I don’t think that i can discriminate among “interesting”, “boring” and “useless” from /var/log/messages.
Anyway saturday i’m gonna have a run.

pike · December 3, 2022, 1:11pm

Not an update: a nice SSD decided to panic (SATAFIRM S11 anyone?). I hate crappy hardware.
Unfortunately the dude was so precise with backup (310 days old) that’s still shocked and panicking.

I’ll try again soon.

pike · December 27, 2022, 2:52pm

Sorry for the delay… System fully updated.
And, now this is the answer

starting slapd.

DNSMasq is running. The installation is a “simple” green adapter configured, and told me “missing network adapter”. I removed it in the interface (maybe the network card greet me for good…), and after removal… still the same

The detail from the log

Failed to start OpenLDAP Server Daemon.
CODE_FILE	src/core/job.c
CODE_FUNCTION	job_log_status_message
CODE_LINE	774
MESSAGE_ID	be02cf6855d2428ba40df7e9d022f03d
PRIORITY	3
RESULT	failed
SYSLOG_FACILITY	3
SYSLOG_IDENTIFIER	systemd
UNIT	slapd.service
_BOOT_ID	9e02524324044dedb5e4b60681cf1fa9
_CAP_EFFECTIVE	1fffffffff
_CMDLINE	/usr/lib/systemd/systemd --switched-root --system --deserialize 22
_COMM	systemd
_EXE	/usr/lib/systemd/systemd
_GID	0
_HOSTNAME	that.not.concern
_MACHINE_ID	0cf84a4c9dea459c952212641958b596
_PID	1
_SOURCE_REALTIME_TIMESTAMP	1672152741773683
_SYSTEMD_CGROUP	/
_TRANSPORT	journal
_UID	0
__CURSOR	s=379bedc1c236415084ac90cc3355bd5b;i=13be;b=9e02524324044dedb5e4b60681cf1fa9;m=34bd59f0;t=5f0d0662f0632;x=354171e74e5c4959
__MONOTONIC_TIMESTAMP	884824560
__REALTIME_TIMESTAMP	1672152741774898

Available for any test.

Ok. I decided to remove some of the unused packages for gateway. Let’s see if i can turn out some garbage.

pike · December 27, 2022, 3:24pm

System is gone slower than a injured turtle. Maybe slapd is a victim, not a reaper…

pike · December 30, 2022, 2:37pm

So.
I changed the hard drive with a same age/less beaten-up.
Speed did not changed.
Slapd still not wishing to work.

Open to suggestions…

Andy_Wismer · December 31, 2022, 7:10am

@pike

Hi Michael

I assume you have a working backup?
What kind of hardware is underneath?

I understand / assume this is native install, not a VM.

It can help, eg if the disk / system had issues (including power outages), that you boot up the system with SystemRescueCD ( https://www.system-rescue.org/ - New Name! ) and run xfs_repair from there.
xfs_repair needs to be started with an unmounted file system, so SystemRescueCD fits the job well. It’s already part of the toolbox, and doesn’t mount the LVM from NethServer while booting. so you can run xfs_repair on both partitions (Boot & LVM / root).

SystemRescueCD works well for both native installs and VMs.

xfs_repair /dev/sda1

and

xfs_repair /dev/VolGroup/lv_root

→ run both!

Can the backup be restored eg into a VM? (Only possible if such is available…), but a nice test…

Sometimes, with NethServer, it has helped to try a “Restore Config” after erasing the LDAP component (eg Account Provider).

My 2 cents
Andy

pike · December 31, 2022, 8:34am

I do-ish

I think “old crap” should define it fast and good enough

It is

Yes-is. I can convert it into a VMWare Player host (i can access to ESX but seem not nice to borrow them because they’re not mine.

However, do things I already know might be a path maybe might solve the issue (slapd will start again?) but won’t find the root cause.

And because it’s my scapegoat (aka test installation) I could totally wipe the floor/hard drive down and start from scratch retaining only the hostname. I’m not currently into any kind of design/test new installations, so no point in doing that, complaining for the lack of support or… try to push someone around to have more help.

However, cockpit bothered me several times, on package management, with this “nice” error.

Andy_Wismer · December 31, 2022, 8:36am

Verify it’s not a disk issue - can’t lose anything, but maybe gain a working system again…

davidep · December 31, 2022, 9:53am

Is LDAP DB data still there? The DB dump command is:

slapcat -n 2

Its output should already be in

/var/lib/nethserver/backup/ldap-data.ldif

Similar check for DB config:

slapcat -n 0

If one of them is missing a restore-config from the last valid backup is needed…

dnutan · December 31, 2022, 10:12am

above logs doesn’t tell much about the problem, maybe the troubleshooting section on the openldap manual can give some hints on what can be checked.
https://www.openldap.org/doc/admin24/guide.html#Troubleshooting

pike · December 31, 2022, 1:27pm

@davidep @dnutan @Andy_Wismer thanks anyway for all your suggestions.

slapcat -n 2

63b0385d hdb_db_open: warning - no DB_CONFIG file found in directory /var/lib/ldap: (2).
Expect poor performance for suffix "dc=directory,dc=nh".
63b0385d bdb(dc=directory,dc=nh): BDB0060 PANIC: fatal region error detected; run recovery
63b0385d hdb_db_open: database "dc=directory,dc=nh" cannot be opened, err -30973. Restore from backup!
63b0385d backend_startup_one (type=hdb, suffix="dc=directory,dc=nh"): bi_db_open failed! (-30973)
slap_startup failed

Seems correct: slapd don’t want to start because… it has no good db to use.

davidep · December 31, 2022, 2:03pm

IIRC in some cases db_recover works

db_recover(1) [osx man page]

Run it as user ldap, or fix the ownership of files after the command runs

pike · January 2, 2023, 2:15pm

Indeed… it worked.

And userlist is populated. Now:

Backup
Shutdown
deep diagnose about other disk status (I think it’s waiting to be trashed)
maybe a reinstall? IDK. Some network adapters were unplugged, and… IDK.