sektor
May 25, 2024, 3:38pm
21
This is what I am seeing in the http access log on the server. After running /usr/libexec/nethserver/letsencrypt-certs -t -v
66.133.109.36 - - [25/May/2024:11:40:04 -0400] “GET /.well-known/acme-challenge/RJbnP_vrHTOOk02-sw9hsuNFrM34cFRXoA_6lOrxCFk HTTP/1.1” 200 87 “-” “Mozilla/5.0 (compatible; Let’s Encrypt validation server; +https://www.letsencrypt.org)”
mrmarkuz
May 26, 2024, 9:15am
22
sektor:
timeout during connect
I can also reproduce the Timeout during connect (likely firewall problem) error, when a domain name doesn’t point to the correct ip in DNS. Did you check if DNS resolves the domain names in the cert correctly?
1 Like
sektor
May 26, 2024, 11:05am
23
Yes, dns points to the correct IP.
Hi @sektor ,
Try to reach it with TOR (Tor Project | Download ) as it doesn’t use the DNS server of your station but its own one.
Also, at https://mxtoolbox.com , test the DNS Lookup.
Michel-André
1 Like
sektor
May 26, 2024, 12:01pm
25
I have validated dns is pointing correctly, because I can see it hitting the firewall and passing through to the server when I try to test the cert renewal, thus the 200 request in the http access log.
1 Like
mrmarkuz
May 26, 2024, 7:27pm
26
Could you please share the part of /var/log/letsencrypt/letsencrypt.log when you were running /usr/libexec/nethserver/letsencrypt-certs -t -v
Maybe there’s an httpd issue regarding vhosts, please run following command to get the apache vhosts:
httpd -S
2 Likes
sektor
May 27, 2024, 1:31am
27
Below is the httpd -S output, which has been the same since day 1 and it won’t let me save the whole lets encrypt log from when I ran the command, because it exceeds the character limit of the forum. Is there a particular section you are looking for?
Although I will say this, because I don’t know if I mentioned it in the beginning I noticed that httpd was only listening to port 80 on IPv6, but I don’t use IPv6 so I made sure all that was turned off and confirmed it was listening on IPv4 again.
Forgive me for redacting my domain name and ip address, also being I managed to acquire the lets encrypt ip addresses I adjusted my port 80 rule to only allow those ip addresses and I can verify if something is not getting through or not.
httpd -S output
VirtualHost configuration:
httpd access log
Salut @sektor
If I remember well, been there done that, until I found that those IP addresses change.
Correct me if I’m wrong,
Michel-André
sektor
May 27, 2024, 2:19am
29
Salut @sektor
If I remember well, been there done that, until I found that those IP addresses change.
Correct me if I’m wrong,
Michel-André
[quote=“michelandre, post:28, topic:23683, full:true”]
I am aware that the ip addresses change, because of load balancing etc but that’s an easy problem to fix
Now if I can figure out why my certs won’t renew, although I may have found something interesting.
A noticeable number of Let's Encrypt users who previously had many successful certificate renewals have been having renewal difficulties since April 2024. Many of these have been attributable to a recent change on the Let's Encrypt side. If your...
Reading time: 1 mins 🕑
Likes: 17 ❤
1 Like
Salut @sektor
The easiest way to test if you can answer the challenges or not.
Make sure you have those DNS records at Cloudflare.com , or at least 1 of them:
toto.org
server-name.toto.org
mail.toto.org
smtp.toto.org
imap.toto.org
www.toto.org
Get your Cloudflare API token as I mentionned in an above post.
Install .acme.sh script
# yum install -y socat
# curl https://get.acme.sh | sh -s email=toto@toto.org
# /root/.acme.sh/acme.sh -v
# cat /root/.acme.sh/account.conf | grep ACCOUNT_EMAIL
# /root/.acme.sh/acme.sh --set-default-ca --server letsencrypt
# config setprop pki CrtFile /etc/pki/tls/certs/cert.crt
# config setprop pki ChainFile /etc/pki/tls/certs/cert-chain.crt
# config setprop pki KeyFile /etc/pki/tls/private/cert.key
# config setprop pki EmailAddress toto@toto.org
# config show pki
# export CF_Key="Cloudflare-API-token"
# export CF_Email="Your-email-address"
# env | grep -i CF_
TEST CERTIFICATE
# /root/.acme.sh/acme.sh \
--issue \
--dns dns_cf \
-d toto.org \
-d server-name.toto.org \
-d mail.toto.org \
-d smtp.toto.org \
-d imap.toto.org \
-d www.toto.org \
--cert-file /etc/pki/tls/certs/cert.crt \
--ca-file /etc/pki/tls/certs/cert-chain.crt \
--key-file /etc/pki/tls/private/cert.key \
--reloadcmd "/sbin/e-smith/signal-event certificate-update" \
--test \
--force
PRODUCTION CERTIFICATE
# /root/.acme.sh/acme.sh \
--issue \
--dns dns_cf \
-d toto.org \
-d server-name.toto.org \
-d mail.toto.org \
-d smtp.toto.org \
-d imap.toto.org \
-d www.toto.org \
--cert-file /etc/pki/tls/certs/cert.crt \
--ca-file /etc/pki/tls/certs/cert-chain.crt \
--key-file /etc/pki/tls/private/cert.key \
--reloadcmd "/sbin/e-smith/signal-event certificate-update" \
--force
Below, the [ / ] are [ | ] ./ ]. env | grep -i CF_
# crontab -l | grep acme
# systemclt restart httpd
# systemctl status httpd | grep Active
Michel-André
1 Like
sektor
May 27, 2024, 4:45am
31
I think I found the problem and turns out my firewall is blocking, based on that forum post I shared in a reply they are validating from multiple countries that people could be blocking for various reasons, so I was able to do a successful test once I figured that out.
So I readjusted my firewall rules to allow the traffic properly.
I was able to renew my certificate once I adjusted my firewall rules accordingly, but it turns out the issue was because they are doing verifications from countries I was inherently blocking do to malicious activity and the timing fits, because when they implemented that change was around the time my certificate would have tried to auto renew.
But I do know they use a lot of aws ip addresses as well as several of their own lets encrypt ip addresses.
1 Like
Salut @sektor ,
Can you share that rule ?
For sure, it will help other people,
Michel-André
sektor
May 27, 2024, 3:53pm
33
I use pfsense as my firewall with pf blocker NG to do GEO IP blocking, so what I did was I moved inbound port 80 rule for my server above by Europe and Asia block rule and in regards to the ACL list I created it was fairly easy, I just ran the update script on the server and looked at the logs in regards to what was being blocked ip wise and added those addresses to the list.
The reason why I had to move my port 80 rule above the Europe and Asia block rules is because pfsense rule list works top down, all block rules are at the top and your allow rules are at the bottom that way all trash “essentially” gets filtered before it gets to your allow rule, so with that being the case the Europe and Asia block rules were still blocking despite adding the ip addresses to the allow list.
If you don’t do any type of GEO IP blocking then you “should” be fine.
Below are all the ip addresses that I am allowing and I confirmed those by cross referencing what I was see in the http access log on neth server, by the way the ip addresses that are used with -t are different then it just runs the renew regularly and all those ip addresses are included as well.
66.133.109.36
1 Like
