Cockpit: SSL/TLS certificate update is not complete (no problem with Nethgui)

Neustradamus · November 26, 2019, 8:30pm

There is a problem with SSL/TLS certificate generation.

The process is not complete with Cockpit, no problem with Nethgui:

https://ip:9090/nethserver#/certificates

The connection is cut off before the end of process.

Can you solve this problem?

davidep · November 26, 2019, 8:58pm

Could you attach an excerpt of /var/log/messages with the bug evidence?

Sometimes a link to gist.github.com helps with log files attachment.

danb35 · November 25, 2019, 3:44pm

Yes, that’s going to happen when you restart the web server to reload a new cert. Which is one of the reasons it shouldn’t work the way you’re asking for it to work.

Neustradamus · November 25, 2019, 3:51pm

No problem in Nethgui.

giacomo · November 25, 2019, 4:47pm

You’re right, but we can’t workaround it: this is the upstream desired behavior.

Neustradamus · November 26, 2019, 2:14am

The solution is to used nethgui?
It is not planned to remove it?
It will be always developped if all can be in Cockpit?
Same for missing Alias part (root) in Email / Cockpit?

Neustradamus · November 26, 2019, 8:27pm

The solution is to used nethgui for SSL/TLS certificate change?
Any news about the cut off connection which kill certificate system in the server?

It’s unthinkable to have not an important functional thing

dnutan · November 26, 2019, 10:02pm

You can change the certificate from Cockpit. It is working.

If you are referring to the “Reconnect” button and logout on certificate change, I think saying it kills the certificate system in the server is exaggerated. The cert is modified as expected, the ulterior actions derived from it are forced mostly by the browser. You’ve to approve your trust in the new certificate on the browser as a security measure. Moreover it is a self-signed cert, as has already been told (untrusted by browser by default). Same happens on nethgui but without immediately forcing a logout.
On the “Reconnect” button behavior for other scenarios, upstream is working on more relaxed rules.

Neustradamus · November 27, 2019, 12:52pm

Please move to bug again, thanks.
Logs for the difference and to see the problem:

Cockpit:

XXX XX XX:XX:XX server cockpit-bridge: No entry for terminal type "unknown";
XXX XX XX:XX:XX server cockpit-bridge: using dumb terminal settings.
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: OLD pki=configuration|CertificateDuration|3650|ChainFile||CommonName||CountryCode||CrtFile||EmailAddress||KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality||Organization||OrganizationalUnitName||State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: NEW pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode||CrtFile||EmailAddress||KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality||Organization||OrganizationalUnitName||State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: OLD pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode||CrtFile||EmailAddress||KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality||Organization||OrganizationalUnitName||State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: NEW pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress||KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality||Organization||OrganizationalUnitName||State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: OLD pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress||KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality||Organization||OrganizationalUnitName||State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: NEW pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality||Organization||OrganizationalUnitName||State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: OLD pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality||Organization||OrganizationalUnitName||State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: NEW pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality|Hometown|Organization||OrganizationalUnitName||State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: OLD pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality|Hometown|Organization||OrganizationalUnitName||State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: NEW pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality|Hometown|Organization|Example1|OrganizationalUnitName||State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: OLD pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality|Hometown|Organization|Example1|OrganizationalUnitName||State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: NEW pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality|Hometown|Organization|Example1|OrganizationalUnitName|Main|State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: OLD pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality|Hometown|Organization|Example1|OrganizationalUnitName|Main|State||SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: NEW pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality|Hometown|Organization|Example1|OrganizationalUnitName|Main|State|SomeState|SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: OLD pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality|Hometown|Organization|Example1|OrganizationalUnitName|Main|State|SomeState|SubjectAltName|
XXX XX XX:XX:XX server /sbin/e-smith/db[10214]: /var/lib/nethserver/db/configuration: NEW pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality|Hometown|Organization|Example1|OrganizationalUnitName|Main|State|SomeState|SubjectAltName|domain.tld
XXX XX XX:XX:XX server esmith::event[10217]: Event: certificate-update
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/backup-config.d/nethserver-certificates.include
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/httpd/conf.d/nethserver.conf
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/httpd/admin-conf/httpd.conf
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/pki/tls/certs/localhost.crt
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/pki/tls/certs/httpd-admin.crt
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/pki/tls/private/localhost.key
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/pki/tls/private/httpd-admin.key
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/pki/dovecot/certs/dovecot.pem
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/pki/dovecot/private/dovecot.pem
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/cockpit/ws-certs.d/99-nethserver.cert
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/postfix/postfix.crt
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/postfix/postfix.key
XXX XX XX:XX:XX server esmith::event[10217]: expanding /etc/ejabberd/ejabberd.pem
XXX XX XX:XX:XX server esmith::event[10217]: expanding /var/lib/nethserver/certs/ca.cnf
XXX XX XX:XX:XX server esmith::event[10217]: Action: /etc/e-smith/events/actions/generic_template_expand SUCCESS [0.842604]
XXX XX XX:XX:XX server systemd: Reloading.
XXX XX XX:XX:XX server systemd: Starting Cockpit motd updater service...
XXX XX XX:XX:XX server systemd: Stopping Cockpit Web Service...
XXX XX XX:XX:XX server update-motd: /usr/share/cockpit/motd/update-motd: line 24: /run/cockpit/active.motd: No such file or directory
XXX XX XX:XX:XX server systemd: Started Cockpit motd updater service.
XXX XX XX:XX:XX server systemd: Stopped Cockpit Web Service.
XXX XX XX:XX:XX server systemd: Starting Cockpit Web Service...
XXX XX XX:XX:XX server systemd: Started Cockpit Web Service.
XXX XX XX:XX:XX server esmith::event[10217]: Action: /etc/e-smith/events/certificate-update/S20nethserver-cockpit-conf SUCCESS [1.650682]
XXX XX XX:XX:XX server cockpit-ws: Using certificate: /etc/cockpit/ws-certs.d/99-nethserver.cert
XXX XX XX:XX:XX server systemd-logind: Removed session 5.

Nethgui:

XXX XX XX:XX:XX server /sbin/e-smith/db[14953]: /var/lib/nethserver/db/configuration: OLD pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality|Hometown|Organization|Example1|OrganizationalUnitName|Main|State|SomeState|SubjectAltName|domain.tld
XXX XX XX:XX:XX server /sbin/e-smith/db[14953]: /var/lib/nethserver/db/configuration: NEW pki=configuration|CertificateDuration|3650|ChainFile||CommonName|NethServer|CountryCode|--|CrtFile||EmailAddress|admin1@domain.tld|KeyFile||LetsEncrypt|disabled|LetsEncryptDomains||LetsEncryptMail||LetsEncryptRenewDays|30|Locality|Hometown|Organization|Example2|OrganizationalUnitName|Main|State|SomeState|SubjectAltName|domain.tld
XXX XX XX:XX:XX server systemd: Started Session c76 of user root.
XXX XX XX:XX:XX server systemd: Started Session c77 of user root.
XXX XX XX:XX:XX server systemd: Started Session c78 of user root.
XXX XX XX:XX:XX server esmith::event[14973]: Event: certificate-update
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/backup-config.d/nethserver-certificates.include
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/httpd/conf.d/nethserver.conf
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/httpd/admin-conf/httpd.conf
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/pki/tls/certs/localhost.crt
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/pki/tls/certs/httpd-admin.crt
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/pki/tls/private/localhost.key
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/pki/tls/private/httpd-admin.key
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/pki/dovecot/certs/dovecot.pem
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/pki/dovecot/private/dovecot.pem
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/cockpit/ws-certs.d/99-nethserver.cert
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/postfix/postfix.crt
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/postfix/postfix.key
XXX XX XX:XX:XX server esmith::event[14973]: expanding /etc/ejabberd/ejabberd.pem
XXX XX XX:XX:XX server esmith::event[14973]: expanding /var/lib/nethserver/certs/ca.cnf
XXX XX XX:XX:XX server esmith::event[14973]: Action: /etc/e-smith/events/actions/generic_template_expand SUCCESS [0.525573]
XXX XX XX:XX:XX server systemd: Reloading.
XXX XX XX:XX:XX server systemd: Starting Cockpit motd updater service...
XXX XX XX:XX:XX server systemd: Stopping Cockpit Web Service...
XXX XX XX:XX:XX server systemd: Stopped Cockpit Web Service.
XXX XX XX:XX:XX server systemd: Starting Cockpit Web Service...
XXX XX XX:XX:XX server update-motd: /usr/share/cockpit/motd/update-motd: line 24: /run/cockpit/active.motd: No such file or directory
XXX XX XX:XX:XX server systemd: Started Cockpit motd updater service.
XXX XX XX:XX:XX server systemd-logind: Removed session 585.
XXX XX XX:XX:XX server esmith::event[14973]: Action: /etc/e-smith/events/certificate-update/S20nethserver-cockpit-conf SUCCESS [0.865906]
XXX XX XX:XX:XX server systemd: Started Cockpit Web Service.
XXX XX XX:XX:XX server cockpit-ws: Using certificate: /etc/cockpit/ws-certs.d/99-nethserver.cert
XXX XX XX:XX:XX server esmith::event[14973]: Using configuration from /var/lib/nethserver/certs/ca.cnf
XXX XX XX:XX:XX server esmith::event[14973]: Action: /etc/e-smith/events/certificate-update/S30nethserver-openvpn-crl SUCCESS [0.129367]
XXX XX XX:XX:XX server systemd: Reloading.
XXX XX XX:XX:XX server esmith::event[14973]: [INFO] service httpd reload
XXX XX XX:XX:XX server systemd: Reloading The Apache HTTP Server.
XXX XX XX:XX:XX server systemd: Reloaded The Apache HTTP Server.
XXX XX XX:XX:XX server systemd: Reloading.
XXX XX XX:XX:XX server esmith::event[14973]: [INFO] service postfix restart
XXX XX XX:XX:XX server systemd: Stopping Postfix Mail Transport Agent...
XXX XX XX:XX:XX server systemd: Stopped Postfix Mail Transport Agent.
XXX XX XX:XX:XX server systemd: Starting Postfix Mail Transport Agent...
XXX XX XX:XX:XX server systemd: Started Postfix Mail Transport Agent.
XXX XX XX:XX:XX server systemd: Reloading.
XXX XX XX:XX:XX server esmith::event[14973]: [INFO] service dovecot restart
XXX XX XX:XX:XX server systemd: Stopping Dovecot IMAP/POP3 email server...
XXX XX XX:XX:XX server systemd: Stopped Dovecot IMAP/POP3 email server.
XXX XX XX:XX:XX server systemd: Starting Dovecot IMAP/POP3 email server...
XXX XX XX:XX:XX server systemd: Can't open PID file /var/run/dovecot/master.pid (yet?) after start: No such file or directory
XXX XX XX:XX:XX server systemd: Started Dovecot IMAP/POP3 email server.
XXX XX XX:XX:XX server systemd: Reloading.
XXX XX XX:XX:XX server esmith::event[14973]: [INFO] service ejabberd restart
XXX XX XX:XX:XX server systemd: Stopping ejabberd XMPP Server...
XXX XX XX:XX:XX server epmd: epmd: got KILL_REQ - terminates normal
XXX XX XX:XX:XX server systemd: Stopped ejabberd XMPP Server.
XXX XX XX:XX:XX server systemd: Starting ejabberd XMPP Server...
XXX XX XX:XX:XX server systemd: Started ejabberd XMPP Server.
XXX XX XX:XX:XX server esmith::event[14973]: Action: /etc/e-smith/events/actions/adjust-services SUCCESS [15.847613]
XXX XX XX:XX:XX server systemd: Started Delayed graceful restart of httpd-admin.
XXX XX XX:XX:XX server esmith::event[14973]: Action: /etc/e-smith/events/certificate-update/S99nethserver-httpd-admin-asyncreload SUCCESS [0.035917]
XXX XX XX:XX:XX server esmith::event[14973]: Event: certificate-update SUCCESS
XXX XX XX:XX:XX server systemd: Started Session c79 of user root.
XXX XX XX:XX:XX server systemd: Stopped Delayed graceful restart of httpd-admin.
XXX XX XX:XX:XX server systemd: Starting Graceful restart of httpd-admin...
XXX XX XX:XX:XX server systemd: Started Graceful restart of httpd-admin.

danb35 · November 27, 2019, 1:11pm

It’s working as designed; it isn’t a bug. “It doesn’t do what I want it to do” isn’t necessarily a bug.

Neustradamus · November 27, 2019, 1:12pm

It is normal that the Cockpit does not work completely?

Neustradamus · November 27, 2019, 8:59pm

@davidep, @giacomo: In a previous message, I have sent logs with the problem in Cockpit and the log for Nethgui which works.

Can you now move "again) this ticket in bugs section because there is a bug.

davidep · November 27, 2019, 9:21pm

I think Marc gave you a good answer. I’d like to set it as solution.

I have a Firefox 70. I’ll possibly check how it works with it.

Neustradamus · November 27, 2019, 10:11pm

No, the problem is when we edit “update” the SSL/TLS certificate:

https://IP:9090/nethserver#/certificates (does not work, there is a problem, it is not complete, see the log, it is not finished)
https://IP:980/en-US/Pki (it works without problem, it is complete)

dnutan · November 27, 2019, 10:58pm

Do you mean reloading/restarting additional services?

Neustradamus · November 27, 2019, 11:04pm

Yes, the process is not finished (not complete), you can see log (difference between Nethgui and Cockpit).
-> S20nethserver-cockpit-conf must launch after S99nethserver-httpd-admin-asyncreload.

dnutan · November 27, 2019, 11:12pm

OK, if your’e referring to the reload/restart of services I think most probably it is a bug. The log helped. Thanks.
Please, next time try to be more specific about the problem from the beginning, otherwise unclear sentences or incomplete description (cut-off, not complete) can be interpreted in different ways and others have to guess their meaning, and in the meantime time is lost.

Neustradamus · November 27, 2019, 11:15pm

For me, it was clear: The connection between computer manager and the server has been cut off because the service has been reloaded before the end of process.

dnutan · November 27, 2019, 11:20pm

It seems for others it was not so clear. At least it wasn’t for me.

davidep · November 28, 2019, 1:50pm

Yes it is a reproducible bug, as reported by @Neustradamus.

To fix it I propose to remove the cockpit service restarts from this action:

[root@vm5 ~]# find  /etc/e-smith/events/ | grep cockpit-conf
/etc/e-smith/events/actions/nethserver-cockpit-conf
/etc/e-smith/events/certificate-update/S20nethserver-cockpit-conf
/etc/e-smith/events/nethserver-cockpit-update/S20nethserver-cockpit-conf

The cockpit service uses systemd socket activation. Its server process terminates when the last user session ends, and is spawned from scratch when a new connection is established on port 9090.

I think there’s no hurry to restart the server process during a certificate generation.

Possibly we don’t even need to restart it during the update event.

What do you think @giacomo @edoardo_spadoni?