Nethserver Most recent update causes ejabberd to fail

NethServer Version: NethServer 7.9.2009

Hello @support_team,

I’m having serious trouble with updating my Nethserver with the latest updates. When I try to update today I receive the following error:

Error

System not updated.

The following command has failed:
system-packages/update

Unfortunately we couldn’t catch the exact error. If you want to help, please click on the button below to copy the failed command to the clipboard, paste it into the Terminal and submit command output to the developers.

When I copy the failed command (pressing the button) the following is the result.

[root@nethserver ~]# echo ‘{“action”:“update”,“packages”:}’ | /usr/bin/setsid /usr/bin/sudo /usr/libexec/nethserver/api/system-packages/update | jq
{
“state”: “running”,
“steps”: -1,
“event”: “Initialization”
}
{
“state”: “running”,
“steps”: -1,
“event”: “Resolving RPM dependencies”
}
{
“state”: “running”,
“steps”: -1,
“event”: “Downloading Packages”
}
{
“state”: “running”,
“steps”: -1,
“event”: “Running Test Transaction”
}
{
“state”: “running”,
“steps”: -1,
“event”: “Running Transaction”
}
{
“status”: “success”,
“steps”: -1,
“event”: “Transaction end”
}
{
“state”: “success”

I then check my services and see that my ejabberd wasn’t running.

ejabberd : The service is either not running or not enabled
So I checked the status of the ejabberd service. Here is what I found in status:

  • ejabberd.service - ejabberd XMPP Server
    Loaded: loaded (/usr/lib/systemd/system/ejabberd.service; enabled; vendor preset: disabled)
    Active: activating (start) since Thu 2021-01-28 19:28:27 EST; 19s ago
    Control: 6091 (sh)
    CGroup: /system.slice/ejabberd.service
    |-6091 /bin/sh -c /opt/ejabberd-20.03/bin/ejabberdctl start && /opt/ejabberd-20.03/bin/ejabberdctl started
    |-6119 /opt/ejabberd-20.03/bin/epmd -daemon
    |-6122 /bin/sh /opt/ejabberd-20.03/bin/ejabberdctl started
    `-6583 sleep 2

Jan 28 19:28:27 nethserver.local systemd[1]: Starting ejabberd XMPP Server…

My Nethserver then showed another error at the top right of the window:

Error

dns.service_action_error

The following command has failed:
system-services/update

Unfortunately we couldn’t catch the exact error. If you want to help, please click on the button below to copy the failed command to the clipboard, paste it into the Terminal and submit command output to the developers.

I copied the command into my terminal and this is the result:

echo ‘{“action”:“start”,“name”:“ejabberd”}’ | /usr/bin/setsid /usr/bin/sudo /usr/libexec/nethserver/api/system-services/update | jq
{
“id”: “1611880383”,
“type”: “EventFailed”,
“message”: “Action failed”

Has anyone found errors with the most recent update? What can I do to get past this.

Thank you.

ejabberd seems loaded and enabled. You may try to restart and disable/enable the service in UI.

Did you check /var/log/messages for ejabberd entries?

1 Like

Hello @mrmarkuz,

Thank you for your reply. Here is what I see when I tail /var/log/messages:

I updated my Nethserver the first time and the updates failed with what I outlined above. I restored from backup and tried the updates a second time and once again the updates failed with the same errors outlined above. In both cases it’s my ejabberd that is in trouble and will not start. I’m not sure if there are any other issues. The failed update message wasn’t specific to what the failure was.

I’ve since restored from backup a third time and I’m running my office without the updates installed. But I’ve kept a copy of my nethserver that did fail but the NIC is disabled so I can read any errors but I do not have GUI access. If there are other logs I can provide to help with this investigation please let me know.

Thank you.

You may check /var/log/ejabberd/ejabberd.log and maybe other logfiles in that directory to get more information about what’s not working.

2 Likes

Hello @mrmarkuz,

Looking at /var/log/ejabberd/ejabberd.log I can see the following errrors:

[root@nethserver ejabberd]# tail ejabberd.log
2021-01-31 10:43:08.310 [notice] <0.120.0>@lager_file_backend:152 Changed loghwm of /var/log/ejabberd/error.log to 100
2021-01-31 10:43:08.310 [notice] <0.120.0>@lager_file_backend:152 Changed loghwm of /var/log/ejabberd/ejabberd.log to 100
2021-01-31 10:43:08.417 [info] <0.106.0>@ejabberd_config:load:82 Loading configuration from /etc/ejabberd/ejabberd.yml
2021-01-31 10:43:08.556 [warning] <0.106.0>@gen_mod:warn_soft_dep_fail:582 Module mod_mam is recommended for module mod_muc but is not found in the config
2021-01-31 10:43:08.573 [info] <0.106.0>@ejabberd_config:load:89 Configuration loaded successfully
2021-01-31 10:43:08.837 [info] <0.345.0>@gen_mod:start_modules:130 Loading modules for swaslab.local
2021-01-31 10:43:09.076 [info] <0.106.0>@ejabberd_cluster_mnesia:wait_for_sync:123 Waiting for Mnesia synchronization to complete
2021-01-31 10:43:09.116 [error] <0.369.0>@ejabberd_pkix:log_errors:401 Failed to read PEM file /etc/ejabberd/ejabberd.pem: at line 61: no matching private key found for this certificate
2021-01-31 10:43:09.116 [warning] <0.369.0>@ejabberd_pkix:log_warnings:393 Invalid certificate in /etc/ejabberd/ejabberd.pem: at line 28: certificate is signed by unknown CA
2021-01-31 10:43:09.116 [critical] <0.369.0>@ejabberd_pkix:stop_ejabberd:352 ejabberd initialization was aborted due to invalid certificates configuration

I’m not sure if these errors were there prior to the update. Any idea why the recent updates would cause these above errors?

Thank you.

Did you upload a certificate? It seems the cert is not ok.

You may try to reset the used certificate as default.

grafik

1 Like

Hello @mrmarkuz,

That’s what’s strange. I didn’t make any changes to my certificate. Are you aware of any recent updates that would have touched the certificate for ejabberd?

I’ll reset my certificate after hours tonight on this broken server and see if this fixes the problem.

Thank you.

1 Like

signal-event nethserver-ejabberd-update

it should handle the certificate creation and the restart of the service

the certificate path is : /etc/ejabberd/ejabberd.pem

2 Likes

Hello @stephdl,

Thank you for your reply. I’ve tried this command to update ejabberd and I’m still receiving the same errors as before. I’ve not changed my certificate so I can’t understand why this problem is occurring.

Perhaps I should remove ejabberd all together and re-install it? What is the command to remove ejabberd from my Nethserver. Then I will try and re-install it fresh.

Thank you.

rpm -e --nodeps nethserver-ejabberd ejabberd
rm -f /etc/ejabberd/*

1 Like

I finally figured out my problem. I had a certificate issue whereby the certificate used for ejabberd was not correct.

I first removed ejabberd using the command @stephdl provided me. I then upgraded my Nethserver and rebooted Nethserver. I then re-installed chat/ejabberd and updated the .pem cert file. Once I did all this my ejabberd/chat app now works properly once again.

Thanks @mrmarkuz and @stephdl for providing support to me for these questions.

4 Likes