Accessing TLS certificates

NethServer Version: NS8
Module: HTTP Routes and TLS Certificates
Hi
I am trying to operate MeshCentral on a vm (proxmox) and need a certificate to get it going.
Because I am using Nethsever I created an HTTP Route for MeshCentral and requested a certificate.
The certificate has been obtained but now I need to add it to MeshCentral’s config.json however I cannot locate where Nethserver has stored the file. Could anybody tell me how to access it?

1 Like

config.json is located in /opt/meshcentral/meshcentral-data

Hi @mrmarkuz

I think Bob is looking for the LE certificates in NethServer, as MeshCentral is installed on a separate VM, as I understannd it…

These were in NS7 in /etc/letsencrypt/archive/SERVER-FQDN/ and are in PEM format, AFAIK…

My 2 cents
Andy

4 Likes

Hi Bob,

I’m facing the same issue. I have Mesh Central installed and would like to use the certificates stored in the new NS8 cluster-admin certificate store.

The reason I need access to this is My pfSense has a bug with the ACME script, and doesn’t start the stand alone web server. I’ve got a certificate installed for an http route in NS8 and I thought I could access this and use it to direct users to my mesh Central instance.

However I am getting an internal server error when using the NS8 http route which I believe is due to mismatched certificates… One authentic (NS8) and one self hosted (mesh Central)

If anyone could point either of us in the right direction it’d be very much appreciated.

Sorry I couldn’t be any help Bob, but hopefully one of the Guru’s will spot this and tell us both the way to the elusive certificate treasure chest.

not very convenient but the cert are stored inside the acme.json in a podman volumes

[root@R4-pve ~]# locate acme.json
/home/traefik1/.local/share/containers/storage/volumes/traefik-acme/_data/acme.json
2 Likes

Due to changes done in march, the above path no longer works. I have included a couple of scripts that extract the required certificate and sends to the local PC/instance requiring the keys. This is for mesh central but will also work with frappe and other server on a split DNS network.

#!/bin/bash

# --- Configuration ---
# Path to Traefik's acme.json file
# Path to Traefik's acme.json file
ACME_JSON_PATH="/home/traefik1/.config/state/acme/acme.json" # Please verify this path as it depends on your instance


# Domain for the Mesh Central certificate
DOMAIN="mc.mydomain.co.nz" 

# Mesh Central server details
MESH_SERVER_USER="your_ssh_user" # Replace with your SSH username for the Mesh server
MESH_SERVER_HOST="mesh_server_ip_or_hostname" # Replace with your Mesh server's IP or hostname

# Remote directory on Mesh Central server to store certs temporarily
REMOTE_TMP_DIR="/tmp/new_certs"

# --- Script ---
set -e # Exit immediately if a command exits with a non-zero status.

echo "Starting certificate sync for $DOMAIN..."

# Create a temporary directory for local cert files
TMP_DIR=$(mktemp -d)
# Ensure the temporary directory is cleaned up on exit
trap 'rm -rf -- "$TMP_DIR"' EXIT

# Extract certificate and key using jq, then decode from Base64
echo "Extracting and decoding certificate and key..."
jq -r --arg domain "$DOMAIN" '.acmeServer.Certificates[] | select(.domain.main == $domain) | .certificate' "$ACME_JSON_PATH" | base64 -d > "$TMP_DIR/cert.pem"
jq -r --arg domain "$DOMAIN" '.acmeServer.Certificates[] | select(.domain.main == $domain) | .key' "$ACME_JSON_PATH" | base64 -d > "$TMP_DIR/key.pem"

# Check if the files were created and are not empty
if [ ! -s "$TMP_DIR/cert.pem" ] || [ ! -s "$TMP_DIR/key.pem" ]; then
    echo "Error: Failed to extract certificate or key for $DOMAIN. Please check the domain name and acme.json path."
    exit 1
fi

echo "Certificate and key extracted successfully."

# Transfer the files to the Mesh Central server using scp
echo "Connecting to $MESH_SERVER_HOST to transfer files..."
ssh "$MESH_SERVER_USER@$MESH_SERVER_HOST" "mkdir -p $REMOTE_TMP_DIR"
scp "$TMP_DIR/cert.pem" "$TMP_DIR/key.pem" "$MESH_SERVER_USER@$MESH_SERVER_HOST:$REMOTE_TMP_DIR/"

echo "Files transferred successfully to $REMOTE_TMP_DIR on $MESH_SERVER_HOST."
echo "Certificate sync complete."

After saving the script, I make it executable. ie

chmod +x /usr/local/bin/sync_mesh_certs.sh

I then set up a cron job on the nethserver to run this each day, but it could be run weekly, or even monthly.

0 2 * * * /usr/local/bin/sync_mesh_certs.sh >> /var/log/sync_mesh_certs.log 2>

And for the script to run, setup ssh keys.

ssh-keygen -t rsa -b 4096 

Then, copy the public key to your Mesh Central server:

bash ssh-copy-id your_ssh_user@mesh_server_ip_or_host 

Part 2: Script for the Mesh Central Server This script will run on your Mesh Central server. It checks for new certificate files, moves them to the correct location, and restarts the Mesh Central service. I’ve called it update_mesh_cert.sh

#!/bin/bash

# --- Configuration ---
# Directory where new certs are copied by the sync script
SOURCE_DIR="/tmp/new_certs"

# Mesh Central's certificate and key file paths
# These paths are typically configured in Mesh Central's config.json
# Default location is often inside the meshcentral-data directory
MESH_CERT_PATH="/opt/meshcentral/meshcentral-data/webserver-cert-public.crt"
MESH_KEY_PATH="/opt/meshcentral/meshcentral-data/webserver-cert-private.key"

# --- Script ---
set -e

echo "Checking for new certificates in $SOURCE_DIR..."

# Check if source files exist and are not empty
if [ ! -s "$SOURCE_DIR/cert.pem" ] || [ ! -s "$SOURCE_DIR/key.pem" ]; then
    echo "No new certificate files found. Exiting."
    exit 0
fi

echo "New certificate files found. Updating..."

# Move the new certificate and key into place
mv "$SOURCE_DIR/cert.pem" "$MESH_CERT_PATH"
mv "$SOURCE_DIR/key.pem" "$MESH_KEY_PATH"

# Set correct permissions (optional, but good practice)
# The user running Mesh Central needs to be able to read these files.
# Adjust 'meshcentral_user' if you run it under a different account.
# chown meshcentral_user:meshcentral_user "$MESH_CERT_PATH" "$MESH_KEY_PATH"
chmod 600 "$MESH_KEY_PATH" # Private key should be readable only by the owner

echo "Certificates updated. Restarting Mesh Central service..."

# Restart Mesh Central service to apply the new certificates
# This command may vary based on your setup (e.g., pm2, systemd)
# Example for systemd:
systemctl restart meshcentral

# Example for PM2 (if you use it to manage Mesh Central):
# pm2 restart meshcentral

echo "Mesh Central service restarted. Update complete."

# Clean up the source directory
rm -rf "$SOURCE_DIR"

Save the script: Save the code above to /usr/local/bin/update_mesh_certs.sh on your Mesh Central server and make it executable.

chmod +x /usr/local/bin/update_mesh_cert.sh

And last setup the cron job.

5 2 * * * /usr/local/bin/update_mesh_certs.sh >> /var/log/update_mesh_cert.log 2>&1

Hope this helps someone. I’ts not 100% error proof, but the scripts above are a good starting point :grinning_face:


2 Likes

Yes, there have been some changes to certificate management in Traefik, including the acme.json path.

At that time, we added a get-certificate command that applications like Mail, NethVoice, and Ejabberd use to get a copy of the local node’s certificate. With the right environment variables, it can also be used in a shell script like yours to fetch any certificate from any node in the cluster.

First, create credentials for the script. For simplicity, let’s create a cluster administrator (owner role):

add-user --role owner --password S3CR3T syncert

Then create a directory where the certificate and private key will be stored:

mkdir mycerts
cd mycerts

Obtain the certificate for a given FQDN (e.g. www.example.org) with this command. Note that the FQDN must belong to the local node:

runagent -c env -u AGENT_STATE_DIR \
    REDIS_USER=syncert  \
    REDIS_PASSWORD=S3CR3T \
    get-certificate --cert-file cert.pem --key-file key.pem www.example.com

If you want to collect certificates from other nodes, set NODE_ID in the environment:

runagent -c env -u AGENT_STATE_DIR \
    REDIS_USER=syncert  \
    REDIS_PASSWORD=S3CR3T \
    NODE_ID=3 \
    get-certificate --cert-file cert.pem --key-file key.pem db.example.org

The get-certificate command exit codes have the following meanings:

  • 0 – a new certificate was obtained and saved. Use this exit code to trigger certificate propagation.

  • 3 – the certificate was obtained, but it matches the existing cert.pem. No action required.

  • Other – an error occurred.


An alternative solution could be built with a lower-level call, like:

api-cli run module/traefik1/get-certificate --data '{"fqdn":"www.example.com"}'

This command does not require additional credentials, but works only on the leader node. The command output is in JSON format and can be parsed with jq.


Anyone want to write a Howto?

3 Likes

oh, I like the

api-cli run module/traefik1/get-certificate --data ‘{“fqdn”:“mydomain.co.nz”}’

I can look at the phasing the output:
Warning: using user “cluster” credentials from the environment
{“fqdn”: "mydomain.co.nz”, “type”: “internal”, “obtained”: true, “certificates”: [{“cert”: …etc}

If the path changes it’ll still work, I’ll modify my script to use and re-post.

Question: When you say it only works on the leader node, if I have client nodes I need to use the first option you gave to get the certificate information from these client nodes, or does the leader node access these as well?

1 Like

The api-cli run command needs to be executed on the leader but it accesses other nodes traefik modules as well.

2 Likes

With additional credentials you can run api-cli on every node.

api-cli login -h

Edit: pass credentials also with

 api-cli run -h
2 Likes

Ok, which explains why you posted the first option. this is far better on multi-node setups. :smile:

I think the first option is also useful for the exit codes 0 and 3, but it’s a matter of taste.

BTW I updated the previous comment. The “run” subcommand should understand also --username and --password params.

2 Likes

Well I was just being lazy. I’ll re-write the script taking in the new method, test then post it. :rofl:

2 Likes