LetsEncrypt Certs not renewing

I Have a Nethserver, System version - NethServer release 7.8.2003 (final) - Kernel release 3.10.0-1127.19.1.el7.x86_64

I browsed to one of the vhosts I support, www.netunity.co.uk, and got a certificate warning, which said the cert expired on 9 November 2020.

I ran “certbot renew” and got the following output

"[root@bastion netunity]# certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/www.blake-online.net.conf


Cert not yet due for renewal


Processing /etc/letsencrypt/renewal/www.bluestarline.org.conf


Cert not yet due for renewal


The following certs are not due for renewal yet:
/etc/letsencrypt/live/www.blake-online.net/fullchain.pem expires on 2021-01-08 (skipped)
/etc/letsencrypt/live/www.bluestarline.org/fullchain.pem expires on 2021-01-10 (skipped)
No renewals were attempted.


[root@bastion netunity]#"

But when I inspect the config, I see:

"# renew_before_expiry = 30 days
version = 1.9.0
archive_dir = /etc/letsencrypt/archive/www.blake-online.net
cert = /etc/letsencrypt/live/www.blake-online.net/cert.pem
privkey = /etc/letsencrypt/live/www.blake-online.net/privkey.pem
chain = /etc/letsencrypt/live/www.blake-online.net/chain.pem
fullchain = /etc/letsencrypt/live/www.blake-online.net/fullchain.pem

Options used in the renewal process

[renewalparams]
authenticator = webroot
account = cc380dc4c392efda55242f9e557f2846
server = https://acme-v02.api.letsencrypt.org/directory
webroot_path = /var/www/html,
[[webroot_map]]
www.blake-online.net = /var/www/html
www.bluestarline.org = /var/www/html
www.netunity.co.uk = /var/www/html"

So I’m confused. LetsEncrypt knows about the Netunity domain, but isn’t trying to renew it, despite the certs being out of date, and although I did all the certificate configs at the same time, and (I thought) in the same way, it appears that my other two vhosts are not due for renewal untilk next January (which is reasonable, really, that would be 3 months from 9/11/2020 which is when the Netunity cert expired) but my Netunity domain is being ignored.

Any advice/help would be much appreciated!

Jim

What’s the output of certbot certificates?

[root@bastion netunity]# certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log


Found the following certs:
Certificate Name: www.blake-online.net
Serial Number: 4e608d0350e5684a1d0339634d7aff6a7d3
Domains: www.bluestarline.org www.blake-online.net www.netunity.co.uk
Expiry Date: 2021-01-08 22:31:02+00:00 (VALID: 42 days)
Certificate Path: /etc/letsencrypt/live/www.blake-online.net/fullchain.pem
Private Key Path: /etc/letsencrypt/live/www.blake-online.net/privkey.pem
Certificate Name: www.bluestarline.org
Serial Number: 4a28c732d2a19faf5182e02cfed4cbc0e6a
Domains: www1.blake-online.net www.bluestarline.org www.littleclose.co.uk www.netunity.co.uk
Expiry Date: 2021-01-10 10:16:51+00:00 (VALID: 43 days)
Certificate Path: /etc/letsencrypt/live/www.bluestarline.org/fullchain.pem
Private Key Path: /etc/letsencrypt/live/www.bluestarline.org/privkey.pem


[root@bastion netunity]#

Hmmmm, I don’t understand why NetUnity’s certificate is not noted, but that looks like a good avenue to persue. Can you advise my next step?

Thanks for the quick response!
Jim

No, next January would be two months from this November, which is as expected–a cert ordinarily attempts to renew 30 days prior to renewal.

NetUnity doesn’t have its own certificate; it’s an added name on both of your other certs–which is perfectly normal as far as it goes. That virtual host is using the bluestarline.org cert, but for some reason it’s serving an expired cert on that virtual host, which isn’t at all normal.

Is NetUnity a virtual host on your Neth system, or is it hosted on a different system? If it is hosted on your Neth system, can you post the contents of /etc/httpd/conf.d/virtualhosts.conf and the output of db vhosts show <whatever this vhost is named>?

NetUnity is hosted on the Neth server:

/etc/httpd/conf.d/virtualhosts.conf looks like this:

[root@bastion jim]# more /etc/httpd/conf.d/virtualhosts.conf

# ================= DO NOT MODIFY THIS FILE =================
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at NethServer official site: https://www.nethserver.org
#
#
#
# 10default_vhost
#
<VirtualHost _default_:80>
    IncludeOptional conf.d/default-virtualhost.inc
</VirtualHost>
#
# 20virtual_hosts -- expand extra virtual hosts
#


#
# Virtual Host blake-online - www1.blake-online.net:443
#

<VirtualHost *:443>
    DocumentRoot "/var/lib/nethserver/vhost/blake-online"
    ServerName www1.blake-online.net


    # 20forcessl_redirect disabled

    #
    #  20ssl_engine
    #
    SSLEngine on


    # 30directory -- basic directory setup
    <Directory /var/lib/nethserver/vhost/blake-online>
      AllowOverride All
      Options FollowSymLinks
      Options +Indexes
      Options +Includes
      php_admin_value memory_limit 512M

      <RequireAll>
        # access public
        Require all granted

        # authentication disabled

      </RequireAll>
    </Directory>

</VirtualHost>


#
# Virtual Host blake-online - www1.blake-online.net:80
#

<VirtualHost *:80>
    DocumentRoot "/var/lib/nethserver/vhost/blake-online"
    ServerName www1.blake-online.net


    # 20forcessl_redirect disabled



    # 30directory -- basic directory setup
    <Directory /var/lib/nethserver/vhost/blake-online>
      AllowOverride All
      Options FollowSymLinks
      Options +Indexes
      Options +Includes
      php_admin_value memory_limit 512M

      <RequireAll>
        # access public
        Require all granted

        # authentication disabled

      </RequireAll>
    </Directory>

</VirtualHost>


#
# Virtual Host blue_star_line - www.bluestarline.org:443
#

<VirtualHost *:443>
    DocumentRoot "/var/lib/nethserver/vhost/blue_star_line"
    ServerName www.bluestarline.org


    # 20forcessl_redirect disabled

    #
    #  20ssl_engine
    #
    SSLEngine on
    SSLCertificateFile "/etc/letsencrypt/live/www.bluestarline.org/cert.pem"
    SSLCertificateKeyFile "/etc/letsencrypt/live/www.bluestarline.org/privkey.pe
m"
    SSLCertificateChainFile "/etc/letsencrypt/live/www.bluestarline.org/chain.pe
m"


    # 30directory -- basic directory setup
    <Directory /var/lib/nethserver/vhost/blue_star_line>
      AllowOverride All
      Options FollowSymLinks
      Options +Indexes
      Options +Includes
      php_admin_value memory_limit 512M

      <RequireAll>
        # access public
        Require all granted

        # authentication disabled

      </RequireAll>
    </Directory>

</VirtualHost>


#
# Virtual Host blue_star_line - www.bluestarline.org:80
#

<VirtualHost *:80>
    DocumentRoot "/var/lib/nethserver/vhost/blue_star_line"
    ServerName www.bluestarline.org


    # 20forcessl_redirect disabled



    # 30directory -- basic directory setup
    <Directory /var/lib/nethserver/vhost/blue_star_line>
      AllowOverride All
      Options FollowSymLinks
      Options +Indexes
      Options +Includes
      php_admin_value memory_limit 512M

      <RequireAll>
        # access public
        Require all granted

        # authentication disabled

      </RequireAll>
    </Directory>

</VirtualHost>


#
# Virtual Host netunity - www.netunity.co.uk:443
#

<VirtualHost *:443>
    DocumentRoot "/var/lib/nethserver/vhost/netunity"
    ServerName www.netunity.co.uk


    # 20forcessl_redirect disabled

    #
    #  20ssl_engine
    #
    SSLEngine on


    # 30directory -- basic directory setup
    <Directory /var/lib/nethserver/vhost/netunity>
      AllowOverride All
      Options FollowSymLinks
      Options +Indexes
      Options +Includes
      php_admin_value memory_limit 512M

      <RequireAll>
        # access public
        Require all granted

        # authentication disabled

      </RequireAll>
    </Directory>

</VirtualHost>


#
# Virtual Host netunity - www.netunity.co.uk:80
#

<VirtualHost *:80>
    DocumentRoot "/var/lib/nethserver/vhost/netunity"
    ServerName www.netunity.co.uk


    # 20forcessl_redirect disabled



    # 30directory -- basic directory setup
    <Directory /var/lib/nethserver/vhost/netunity>
      AllowOverride All
      Options FollowSymLinks
      Options +Indexes
      Options +Includes
      php_admin_value memory_limit 512M

      <RequireAll>
        # access public
        Require all granted

        # authentication disabled

      </RequireAll>
    </Directory>

</VirtualHost>

Unfortunately, db vhosts show

[root@bastion jim]# db vhosts show
gives me an error “bash: db: command not found”

Thanks

Jim

If you’d used su - to become root rather than just su, you’d have root’s environment, including the $PATH. But no matter, /sbin/e-smith/db vhosts show netunity.

Hi Dan,
Apologies for the error…here’s the output:

[root@bastion jim]# /sbin/e-smith/db vhosts show netunity
netunity=vhost
Access=public
Description=Company Site
ForceSslStatus=disabled
FtpPassword=
FtpStatus=enabled
Indexes=disabled
PasswordStatus=disabled
ServerNames=www.netunity.co.uk
SslCertificate=
status=enabled
[root@bastion jim]#

What about /sbin/e-smith/config show pki?

[root@bastion jim]# /sbin/e-smith/config show pki
pki=configuration
CertificateDuration=3650
ChainFile=/etc/letsencrypt/live/www.blake-online.net/chain.pem
CommonName=
CountryCode=
CrtFile=/etc/letsencrypt/live/www.blake-online.net/cert.pem
EmailAddress=
KeyFile=/etc/letsencrypt/live/www.blake-online.net/privkey.pem
LetsEncrypt=disabled
LetsEncryptDomains=www.bluestarline.org,www.blake-online.net,www.netunity.co.uk
LetsEncryptMail=jim@blake-online.net
LetsEncryptRenewDays=30
Locality=
Organization=
OrganizationalUnitName=
State=
SubjectAltName=
[root@bastion jim]#

That’s curious. This indicates that the blake-online.net cert is the default, but it isn’t what the netunity virtual host is actually serving:

dan@Dan-Hack-Mini  ~  openssl s_client -connect www.netunity.co.uk:443
CONNECTED(00000005)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = www.bluestarline.org
verify error:num=10:certificate has expired
notAfter=Nov  9 22:38:59 2020 GMT
verify return:1
depth=0 CN = www.bluestarline.org
notAfter=Nov  9 22:38:59 2020 GMT
verify return:1
---
Certificate chain
 0 s:/CN=www.bluestarline.org
   i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
 1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---

What happens if you just reload the Apache configuration? systemctl reload httpd

I did that, but it made no difference: If you go to https://www.netunity.co.uk, you still get an “expired certificate” error. I tried to copy and paste the output, but it got mangled. You can see it on the site, though, its publicly available

Let’s see what cert files are specified throughout the Apache configuration. In /etc/httpd/conf.d/, run grep ".pem" *.

Its not looking good:

[root@bastion jim]# cd /etc/httpd/conf.d/
[root@bastion conf.d]# grep “.pem” *.
grep: *.: No such file or directory
[root@bastion conf.d]#

Just looking around a bit: is it likely that having the wrong certificate set as default has caused this:


Though I haven’t changed the default, so I’m not sure this ius relevant

Jim

Hi Jim.

You have 3 domains: blake-online.net, bluestarline.org and netunity.co.uk.

Their root directory are:
blake-online.net/var/lib/nethserver/vhost/blake-online
bluestarline.org/var/lib/nethserver/vhost/blue_star_line
netunity.co.uk/var/lib/nethserver/vhost/netunity

In /etc/httpd/conf.d/virtualhosts.conf, blake-online.net is described as www1.blake-online.net which I think denotes an error entered when you configure the DNS page for this domain.
On the DNS configuration page, use the domain name blake-online.net, make sure to click Wildcard DNS record.

Request one Let’s Encrypt Certificate:
I you want to use only 1 certificate, the best choice is to request it for netunity.co.uk (put this domain as the first line) and then add all the others after, including all their CNAMEs.
† Make sure that all domains have all their CNAMEs (www, mail, etc…) in the DNS records at your domain registrar.

netunity.co.uk
www.netunity.co.uk
mail.netunity.co.uk

bluestarline.org
www.bluestarline.org
mail.bluestarline.org

netunity.co.uk

■ → Set this certificate as the default one.

Request 3 Let’s Encrypt Certificates:
If you want to have different certificates, one for each domain, you have to request 3 certificates.
One for:
netunity.co.uk
www.netunity.co.uk
mail.netunity.co.uk

One for:
bluestarline.org
www.bluestarline.org
mail.bluestarline.org

And one for:
netunity.co.uk
www.netunity.co.uk
mail.netunity.co.uk

blake-online.net
www.blake-online.net
www.blake-online.net

Assign the proper certificate to the proper domain:
In Cockpit:
Applications → Web server → Settings → Virtual hosts → netunity.co.uk → Edit → Advanced settings → SSL/TLS certificate → chose its proper Let’s Encrypt certificate → finish by cliking Edit.
● Repeat the procedure for your other domains.

All should work fine,

Michel-André

CORRECTION:
The 3rd domain for Request 3 Let’s Encrypt Certificates: is blake-online.net and not netunity.co.uk

Argh, that’s what I get for worrying about proper punctuation. The trailing period isn’t part of the command to run.

It’s certainly possible, though I’m not clear why it would be–the Netunity name is on both certs, and both are still valid.

Why would you recommend this? Why do you think it matters which name appears in which order on the cert?

Hi Dan
I shoud have realised the trailing . was punctuation! Sorry! Here is the output of the command in its correct form, and having go no output, I listed the contents of the directory as well:

[root@bastion ~]#
[root@bastion ~]# cd /etc/httpd/conf.d/
[root@bastion conf.d]# grep “.pem” *
[root@bastion conf.d]# ls -al
total 100
drwxr-xr-x. 2 root root 4096 Nov 4 11:33 .
drwxr-xr-x. 7 root root 130 Oct 12 12:16 …
-rw-r–r-- 1 root root 127 Oct 9 08:04 apacheStatus.conf
-rw-r–r-- 1 root root 463 Aug 3 2017 authnz_pam.conf
-rw-r–r-- 1 root root 2926 Apr 2 2020 autoindex.conf
-rw-r–r-- 1 root root 951 Apr 13 2020 default-virtualhost.inc
-rw-r–r-- 1 root root 521 Sep 3 10:52 ejabberd.conf
-rw-r–r--. 1 root root 231 Apr 17 2018 letsencrypt.conf
-rw-r–r-- 1 root root 578 Mar 1 2019 lightsquid.conf
-rw-r–r-- 1 root root 1619 Jun 4 21:14 nethserver.conf
-rw-r–r-- 1 root root 330 Feb 27 2020 nextcloud.conf
-rw-r–r-- 1 root root 691 Apr 1 2020 php.conf
-rw-r–r-- 1 root root 1799 Mar 2 2019 phpMyAdmin.conf
-rw-r–r--. 1 root root 261 Mar 1 2019 proxypass.conf
-rw-r–r-- 1 root root 366 Apr 2 2020 README
-rw-r–r-- 1 root root 899 Mar 1 2019 roundcubemail.conf
-rw-r–r-- 1 root root 298 May 27 2020 squid.conf
-rw-r–r-- 1 root root 9443 Nov 27 2019 ssl.conf
-rw-r–r-- 1 root root 1252 Nov 27 2019 userdir.conf
-rw-r–r-- 1 root root 4481 May 19 2020 virtualhosts.conf
-rw-r–r-- 1 root root 1296 Sep 4 2019 welcome.conf
-rw-r–r-- 1 root root 625 Nov 18 2019 wpad.conf
-rw-r–r-- 1 root root 1000 May 19 2020 zz_nextcloud.conf
[root@bastion conf.d]#

Cheers

Jim

Further info: I thought it was just the netunity.co.uk domain that was experiencing problems, but on digging a bit deeper, I found I had set the browser up to accept bad certificates. What is actually happening is that www.bluestarline.org works properly, but both the other domains, netunity.co.uk and blake-online.net are using bad certificates…sorry I din’t check this earlier, I may have misled you

Cheers
Jim

Hi Jim,

Can you explain why you use www1 ?

Michel-André

I wish I could! It was done a while back and I don’t know why. Should I change it?
Indeed, would it be sensible to do a full re-installation of the certificate mechanism?

Cheers

Jim