Certificate Revocation Problem

Kernel Release 3.10.0-1062.18.1.el7.x86_64

Operating System NethServer release 7.7.1908 (final)

I’m using LetsEncrypt certificates. I have not had any notification of certificate revokes, and up to today, 13 April, everything was fine. Today, I can access the three sites I’m running, as usual, on https.
However, if I try to browse to <IP_Address>:980, in Firefox or Chrome, I get notification of revoked certificates, and both browsers refuse to go further. Then it gets weird…I can ssh into the console no problem, and I can go to <IP_Address>:9090 and get a warning but I can proceed to the console by accepting the warning.

I thought the certs could be renewed by issuing “/usr/local/bin/certbot-auto --apache”. That completed fine, but didn’t make a change. I thought about a reboot at this point, but I’ve still got service and I don’t want to reboot and maybe loose what I have got…

Anybody got any suggestions?

Thanks

Jim

Almost looks like your system isn’t serving the same certificate for the old server manager as it is for your web sites and for Cockpit, which would be very strange. You can start by comparing the output (from a *nix or Mac client machine) of openssl s_client -connect yourdomain.com:443 with openssl s_client -connect yourdomain.com:980. Those commands will give you the actual certificates your system is presenting on each port. You can visually look and see if they look the same, but it’d probably be easier to paste them into an online certificate decoder (e.g., Certificate Decoder - Decode certificates to view their contents). Or you can decide it yourself by running openssl x509 -in certificate.crt -text -noout, where certificate.crt is the contents of the cert saved to a text file.

Keep in mind that you’ll always get certificate errors if you’re using a trusted cert and browsing to an IP address, as the cert only covers FQDNs, not IP addresses.

Thanks for the quick response. I checked the certificates, they are as you suggest, different. They start with the same strings, but the last 340 characters differ. I am using the FQDN, not the IP addresses. The server and vHosts are behind NAT, but the same occurs either side of the NAT barrier.

I have a single certificate, valid for all three sites, set up as default

Thanks

Jim

That isn’t what you said in your OP:

…though the certificate error that would give you is not that the cert is revoked.

My speculation at this point is that your cert is in fact revoked, but that fact isn’t obvious on your main site because revocation is broken. You can test this using the ssllabs.com tester (SSL Server Test (Powered by Qualys SSL Labs)), which does check revocation status. If I’m correct, the reason for the difference is that the server manager uses OCSP stapling, which attaches a certificate of the current revocation status to server’s certificate. However, the main Apache instance doesn’t do this, and apparently Cockpit doesn’t either.

If I’m correct that your cert is revoked, the fix is simple: certbot renew --force-renewal.

1 Like

I tried the suggested fix:
[root@bastion ~]# certbot renew --force-renewal
Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/<FQDN#1 obscured>.conf


Could not choose appropriate plugin: The requested apache plugin does not appear to be installed
Attempting to renew cert (<FQDN#1 obscured>) from /etc/letsencrypt/renewal/<FQDN#1 obscured>.conf produced an unexpected error: The requested apache plugin does not appear to be installed. Skipping.


Processing /etc/letsencrypt/renewal/<FQDN#1 obscured>.conf


Could not choose appropriate plugin: The requested apache plugin does not appear to be installed
Attempting to renew cert (<FQDN#2 obscured>) from /etc/letsencrypt/renewal/<FQDN#2 obscured>.conf produced an unexpected error: The requested apache plugin does not appear to be installed. Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/<FQDN#1 obscured>/fullchain.pem (failure)
/etc/letsencrypt/live/<FQDN#2 obscured>/fullchain.pem (failure)


All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/<FQDN#1 obscured>/fullchain.pem (failure)
/etc/letsencrypt/live/<FQDN#2 obscured>/fullchain.pem (failure)


2 renew failure(s), 0 parse failure(s)

So it would appear that apache plugin does not appear to be installed…but what apache plugin?

The file /etc/letsencrypt/renewal/<FQDN#1 obscured>.conf contains:
# renew_before_expiry = 30 days
version = 1.3.0
archive_dir = /etc/letsencrypt/archive/FQDN#1 obscured
cert = /etc/letsencrypt/live/FQDN#1 obscured/cert.pem
privkey = /etc/letsencrypt/live/FQDN#1 obscured/privkey.pem
chain = /etc/letsencrypt/live/FQDN#1 obscured/chain.pem
fullchain = /etc/letsencrypt/live/FQDN#1 obscured/fullchain.pem

Options used in the renewal process

[renewalparams]
authenticator = apache
account = cobscured***6
server = https://acme-v02.api.letsencrypt.org/directory
installer = apache

I’ll Google this, but I’m 'way out of my comfort zone, so all help is much appreciated!

Jim

Did you first confirm that your cert was actually revoked using the ssllabs test?

That shouldn’t be happening, unless your previous attempt at running certbot apache confused it. Try re-requesting the cert through the (Cockpit) server manager–that should reset the options.

1 Like

You suggested:

“Try re-requesting the cert through the (Cockpit) server manager–that should reset the options.”

Do you mean by selecting Certificates> Action>Request LetsEncrypt Certificates, and adding all three FQDNs to the list before clicking " Request"?

Sorry to be so pedantic, but I want to be sure I do this right.

Thanks

Jim

Yes, that’s it exactly.

Dan, you just saved my sanity and my websites! Thanks, man!

It’s all back where it should be, and I have some reading to do (LetsEncrypt docs and logs!).

Really appreciate your speedy and complete response, I’ll mark this as solved. Much Kudos.

Thanks
Jim