LE-Cert renewal failed

Hello, today my certs are outdated, the automatic renewal is failed.
Is this normal, do I always have to renew manually?

best regards, Marko

No, never have to do this…Looking at the log over here it checks once a day:
ls -ltr /var/log/letsencrypt

@capote

Another confirmation:

NethServers with several aliases on the ssl cert normally update the cert with NO issues.
I’ve had problems with LE in the past, but mostly “leftovers” from an earlier problem, like files not removed.
Normally it just works.

My 2 cents
Andy

You’ve really answered your own question here–there is an automatic renewal task for whatever cert you obtained using the GUI (legacy or cockpit), which you already know, if you know that it’s failed as you said above–and you should know that, because it emails you when it fails, so you should have been getting emails daily for the last month saying that, as well as a couple of emails from Let’s Encrypt warning you of the impending expiration. IOW, this shouldn’t have been anything like a surprise.

The question, of course, is why it failed, which you can find in the certbot logs in /var/log/letsencrypt. If you look at the latest file or two, they should show what’s going on.

Now, if this is a cert you didn’t obtain through the GUI, whatever method you used to obtain it should have its own renewal mechanism–but it should still be on a schedule.

The Certs where requested by the Cockpit GUI and I got daily mail with expiration warnings.
I thought this is only for information w/o tasks for me.
Insode of cockpit I don’t find the access option for /var/log/letsencrypt

In NethGUI I found the last entry:

2021-01-19 05:57:37,560:DEBUG:certbot.ocsp:Querying OCSP for /etc/letsencrypt/archive/dargels.de/cert5.pem 2021-01-19 05:57:37,561:DEBUG:certbot.ocsp:openssl ocsp -no_nonce -issuer /etc/letsencrypt/archive/dargels.de/chain5.pem -cert /etc/letsencrypt/archive/dargels.de/cert5.pem -CAfile /etc/letsencrypt/archive/dargels.de/chain5.pem -verify_other /etc/letsencrypt/archive/dargels.de/chain5.pem -trust_other -timeout 10 -header Host r3.o.lencr.org -url http://r3.o.lencr.org 2021-01-19 05:57:37,728:INFO:certbot._internal.renewal:Cert not yet due for renewal 2021-01-19 05:57:37,728:INFO:certbot._internal.main:Keeping the existing certificate 2021-01-19 05:57:37,729:DEBUG:certbot.display.util:Notifying user: Certificate not yet due for renewal; no action taken.

affected Cert:


image

hmmm,
Your domains of the archived and live certificates to not match. cerbot check’s da***ls.de (archived) you use my****.de (live)

If things are working properly, you should never get expiration warnings.

It’s still on your server–suggest looking there. Also, the output of certbot certificates would be helpful.

[root@ns-srv01 ~]# certbot certificates

Saving debug log to /var/log/letsencrypt/letsencrypt.log


Found the following certs:
Certificate Name: alles-<…>.de
Serial Number: 346c5309d4a958479898651b081979bf98e
Key Type: RSA
Domains: alles-<…>.de imap.alles-<…>.de mail.alles-<…>.de webtop.alles-<…>.de www.alles-<…>.de
Expiry Date: 2021-01-26 09:16:49+00:00 (VALID: 6 days)
Certificate Path: /etc/letsencrypt/live/alles-<…>.de/fullchain.pem
Private Key Path: /etc/letsencrypt/live/alles-<…>.de/privkey.pem
Certificate Name: dargels.de
Serial Number: 45ffccdacdc341f39c611da05ab646b5f30
Key Type: RSA
Domains: dargels.de collabora.dargels.de imap.dargels.de mail.dargels.de nextcloud.dargels.de ns-srv01.dargels.de smtp.dargels.de status.dargels.de webtop.dargels.de wp.dargels.de www.dargels.de
Expiry Date: 2021-04-13 03:34:48+00:00 (VALID: 83 days)
Certificate Path: /etc/letsencrypt/live/dargels.de/fullchain.pem
Private Key Path: /etc/letsencrypt/live/dargels.de/privkey.pem
Certificate Name: myancestry.de
Serial Number: 4da88e81fd9e01d282dfdf18e63a7068003
Key Type: RSA
Domains: myancestry.de imap.myancestry.de mail.myancestry.de smtp.myancestry.de www.myancestry.de
Expiry Date: 2021-01-19 07:33:22+00:00 (INVALID: EXPIRED)
Certificate Path: /etc/letsencrypt/live/myancestry.de/fullchain.pem
Private Key Path: /etc/letsencrypt/live/myancestry.de/privkey.pem


Your domains of the archived and live certificates to not match. cerbot check’s da***ls.de (archived) you use my****.de (live)

Digging deeper:

2021-01-19 05:57:36,866:DEBUG:certbot._internal.main:certbot version: 1.10.1 2021-01-19 05:57:36,866:DEBUG:certbot._internal.main:Arguments: ['--text', '--non-interactive', '--agree-tos', '--email', 'mail@mail.com', '--preferred-challenges', 'http', '--webroot', '--webroot-path', '/var/www/html/', '-d', 'dargels.de', '-d', 'collabora.dargels.de', '-d', 'imap.dargels.de', '-d', 'mail.dargels.de', '-d', 'nextcloud.dargels.de', '-d', 'smtp.dargels.de', '-d', 'wp.dargels.de', '-d', 'www.dargels.de', '-d', 'webtop.dargels.de', '-d', 'ns-srv01.dargels.de', '-d', 'status.dargels.de', '--quiet'] 2021-01-19 05:57:36,866:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot) 2021-01-19 05:57:36,883:DEBUG:certbot._internal.log:Root logging level set at 30 2021-01-19 05:57:36,884:INFO:certbot._internal.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log 2021-01-19 05:57:36,884:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer None 2021-01-19 05:57:36,886:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * webroot Description: Place files in webroot directory Interfaces: IAuthenticator, IPlugin Entry point: webroot = certbot._internal.plugins.webroot:Authenticator Initialized: Prep: True 2021-01-19 05:57:36,886:DEBUG:certbot._internal.plugins.selection:Selected authenticator and installer None 2021-01-19 05:57:36,886:INFO:certbot._internal.plugins.selection:Plugins selected: Authenticator webroot, Installer None 2021-01-19 05:57:36,913:DEBUG:certbot._internal.main:Picked account: <Account(RegistrationResource(body=Registration(status=None, terms_of_service_agreed=None, agreement=None, only_return_existing=None, contact=(), key=None, external_account_binding=None), uri=u'https://acme-v02.api.letsencrypt.org/acme/acct/98774805', new_authzr_uri=None, terms_of_service=None), 4507e00e979072793c396c2a3ee407aa, Meta(creation_host=u'ns-srv01.dargels.de', register_to_eff=None, creation_dt=datetime.datetime(2020, 10, 8, 22, 54, 4, tzinfo=)))> 2021-01-19 05:57:36,918:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory. 2021-01-19 05:57:36,926:INFO:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org 2021-01-19 05:57:37,536:DEBUG:urllib3.connectionpool:"GET /directory HTTP/1.1" 200 658 2021-01-19 05:57:37,537:DEBUG:acme.client:Received response: HTTP 200 content-length: 658 strict-transport-security: max-age=604800 server: nginx connection: keep-alive cache-control: public, max-age=0, no-cache date: Tue, 19 Jan 2021 04:57:37 GMT x-frame-options: DENY content-type: application/json

{
“9JjJeItAKvI”: “https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417”,
“keyChange”: “https://acme-v02.api.letsencrypt.org/acme/key-change”,
“meta”: {
“caaIdentities”: [
“letsencrypt.org”
],
“termsOfService”: “https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf”,
“website”: “https://letsencrypt.org”
},
“newAccount”: “https://acme-v02.api.letsencrypt.org/acme/new-acct”,
“newNonce”: “https://acme-v02.api.letsencrypt.org/acme/new-nonce”,
“newOrder”: “https://acme-v02.api.letsencrypt.org/acme/new-order”,
“revokeCert”: “https://acme-v02.api.letsencrypt.org/acme/revoke-cert”
}
2021-01-19 05:57:37,550:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer <certbot._internal.cli.cli_utils._Default object at 0x7f19e4f9b5d0>
2021-01-19 05:57:37,560:DEBUG:certbot.ocsp:Querying OCSP for /etc/letsencrypt/archive/dargels.de/cert5.pem
2021-01-19 05:57:37,561:DEBUG:certbot.ocsp:openssl ocsp -no_nonce -issuer /etc/letsencrypt/archive/dargels.de/chain5.pem -cert /etc/letsencrypt/archive/dargels.de/cert5.pem -CAfile /etc/letsencrypt/archive/dargels.de/chain5.pem -verify_other /etc/letsencrypt/archive/dargels.de/chain5.pem -trust_other -timeout 10 -header Host r3.o.lencr.org -url http://r3.o.lencr.org
2021-01-19 05:57:37,728:INFO:certbot._internal.renewal:Cert not yet due for renewal
2021-01-19 05:57:37,728:INFO:certbot._internal.main:Keeping the existing certificate
2021-01-19 05:57:37,729:DEBUG:certbot.display.util:Notifying user: Certificate not yet due for renewal; no action taken.

2021-01-19 16:07:53,790:DEBUG:certbot._internal.main:certbot version: 1.10.1 2021-01-19 16:07:53,790:DEBUG:certbot._internal.main:Arguments: [] 2021-01-19 16:07:53,790:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot) 2021-01-19 16:07:53,807:DEBUG:certbot._internal.log:Root logging level set at 20 2021-01-19 16:07:53,808:INFO:certbot._internal.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log 2021-01-19 16:07:53,875:DEBUG:certbot.ocsp:Querying OCSP for /etc/letsencrypt/live/alles-<..>.de/cert.pem 2021-01-19 16:07:53,875:DEBUG:certbot.ocsp:openssl ocsp -no_nonce -issuer /etc/letsencrypt/live/alles-<..>.de/chain.pem -cert /etc/letsencrypt/live/alles-e<..>.de/cert.pem -CAfile /etc/letsencrypt/live/alles-<..>.de/chain.pem -verify_other /etc/letsencrypt/live/alles-<..>.dechain.pem -trust_other -timeout 10 -header Host ocsp.int-x3.letsencrypt.org -url http://ocsp.int-x3.letsencrypt.org 2021-01-19 16:07:54,051:DEBUG:certbot.ocsp:Querying OCSP for /etc/letsencrypt/live/dargels.de/cert.pem 2021-01-19 16:07:54,051:DEBUG:certbot.ocsp:openssl ocsp -no_nonce -issuer /etc/letsencrypt/live/dargels.de/chain.pem -cert /etc/letsencrypt/live/dargels.de/cert.pem -CAfile /etc/letsencrypt/live/dargels.de/chain.pem -verify_other /etc/letsencrypt/live/dargels.de/chain.pem -trust_other -timeout 10 -header Host r3.o.lencr.org -url http://r3.o.lencr.org 2021-01-19 16:07:54,253:DEBUG:certbot.display.util:Notifying user: Found the following certs: Certificate Name: alles-e****.de Serial Number: 346c5309d4a958479898651b081979bf98e Key Type: RSA Domains: alles-<..>.de imap.alles-<..>.de mail.alles-<..>.de webtop.alles-<..>.de www.alles-<..>.de Expiry Date: 2021-01-26 09:16:49+00:00 (VALID: 6 days) Certificate Path: /etc/letsencrypt/live/alles-<..>.de/fullchain.pem Private Key Path: /etc/letsencrypt/live/alles-<..>.de/privkey.pem Certificate Name: dargels.de Serial Number: 45ffccdacdc341f39c611da05ab646b5f30 Key Type: RSA Domains: dargels.de collabora.dargels.de imap.dargels.de mail.dargels.de nextcloud.dargels.de ns-srv01.dargels.de smtp.dargels.de status.dargels.de webtop.dargels.de wp.dargels.de www.dargels.de Expiry Date: 2021-04-13 03:34:48+00:00 (VALID: 83 days) Certificate Path: /etc/letsencrypt/live/dargels.de/fullchain.pem Private Key Path: /etc/letsencrypt/live/dargels.de/privkey.pem Certificate Name: myancestry.de Serial Number: 4da88e81fd9e01d282dfdf18e63a7068003 Key Type: RSA Domains: myancestry.de imap.myancestry.de mail.myancestry.de smtp.myancestry.de www.myancestry.de Expiry Date: 2021-01-19 07:33:22+00:00 (INVALID: EXPIRED) Certificate Path: /etc/letsencrypt/live/myancestry.de/fullchain.pem Private Key Path: /etc/letsencrypt/live/myancestry.de/privkey.pem

I think I need to manually refresh because my site is unreachable.

So the myancestry.de cert has expired, as of earlier this morning. The alles-…de cert is also about to expire. You should be able to renew both of them by running certbot renew.

But now to why that happened. The Neth GUI doesn’t support more than one cert very well; my guess is that you most recently issued the dargels.de cert through the GUI, so that’s the only one it keeps track of–the output of config show pki would confirm this.

Thank you for the hint:

[root@ns-srv01 ~]# certbot renew

Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/alles-ernst.de.conf


Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator webroot, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Renewing an existing certificate for alles-ernst.de and 4 more domains
Performing the following challenges:
http-01 challenge for alles-<…>.de
http-01 challenge for imap.alles-<…>.de
http-01 challenge for mail.alles-<…>.de
http-01 challenge for webtop.<…>.de
http-01 challenge for www.alles-<…>.de
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification…
Cleaning up challenges


new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/alles-ernst.de/fullchain.pem



Processing /etc/letsencrypt/renewal/dargels.de.conf


Cert not yet due for renewal


Processing /etc/letsencrypt/renewal/myancestry.de.conf


Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator webroot, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Renewing an existing certificate for myancestry.de and 4 more domains
Performing the following challenges:
http-01 challenge for imap.myancestry.de
http-01 challenge for mail.myancestry.de
http-01 challenge for myancestry.de
http-01 challenge for smtp.myancestry.de
http-01 challenge for www.myancestry.de
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification…
Cleaning up challenges


new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/myancestry.de/fullchain.pem



The following certs are not due for renewal yet:
/etc/letsencrypt/live/dargels.de/fullchain.pem expires on 2021-04-13 (skipped)
Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/alles-<…>.de/fullchain.pem (success)
/etc/letsencrypt/live/myancestry.de/fullchain.pem (success)


Would it be a good idea to configure a cron job for certbot renew?

Best regrads, Marko

It would. For reasons known only to the devs, they only try to renew the one cert they’re tracking (the one named under the pki key in the config database)–anything else is just left to expire. Here’s how I do it in the wiki:

1 Like

the new certificate has a CN issuer declaration like “R3”


image

not like before:

I hope, it is only a cosmetic problem.

It isn’t a problem at all–that’s the new intermediate CA cert they stood up a while back.

1 Like

@danb35 (sorry @capote for a small hijack of the post)

for acme-dns port 53 should probably forwarded to the instance acme-dns runs on right?

second: puzzeld by (@ end of Issuing the certificate) :confused:

You’ll need to once again log into your DNS host and add the record specified. You’ll only need to do this once for each hostname. Wait a few minutes, then press Enter.

isn’t this obsolete now?

If your Neth system is behind a firewall, yes, port 53 (UDP and TCP) need to be forwarded to wherever acme-dns is running.

Why do you think this?

If you’ve installed a previous version (before version 0.8) of acme-dns, and activated HTTPS for the API, you must remove the DNS CNAME record you created for _acme-challenge.acme.example.com. Acme-dns now handles its own certificate using DNS validation, but this record will conflict with that process.

Especially the last sentence.
Do I understand correctly now you set acme-dns up and add the CNAME records per (sub)domain as instructed by acme-dns-auth.py ?

Yes. You won’t need (and shouldn’t have) a CNAME for _acme-challenge.acme.example.com, now that acme-dns now handles its own certs internally, but you’ll still need to create CNAMEs for any other (sub)domain you want to issue. You’ll only need to do that once per (sub)domain, though.

3 Likes