I am trying to add a new node to my cluster. I’ve taken a look at the post linked to below, and as far as I can tell, I have met requirements. I’ve installed, added the node to the cluster, and it shows up in the Nodes
I can ssh from one to another, the worker node can resolve the address of the leader. But the leader node says the cluster node is offline.
How do I make sure the ports at 55820 are open? I do notice that if I do
The 0.0.0.0 vs * you saw is just ss formatting — both mean “all interfaces,” nothing’s wrong there.
A few things to check to narrow down where the VPN join is failing:
Check the WireGuard handshake status on both nodes:
wg
Look at the latest handshake line for the peer. If it’s recent (updated roughly every minute), the tunnel is up. If stale or missing, traffic isn’t flowing over UDP/55820 — a firewall/NAT issue, not NS8’s node logic.
Ping the VPN IPs directly, not just the LAN hostnames:
ping 10.5.4.1
ping 10.5.4.2
(adjust to your actual cluster VPN addresses). If SSH works over the LAN but ping fails over the VPN overlay, the tunnel isn’t passing traffic even if the socket is listening.
Check Redis on the worker node:
systemctl status redis
Cluster state/node status reflects Redis replica status, so if it’s not running on the worker or it can’t reach the leader, the leader on its side will keep showing it offline even if VPN and SSH are fine.
Let us know what you get from these and we can dig further.
Hi - thanks for the reply. Tying to find my netstat and nslookup.
On the Cluster Status page of the web management app it says “All nodes are online” Going to the nodes page, the worker node has an alert. In See details, I get ‘Cluster node 4 is offline’.
Anyway -
wg on the Leader
[root@eastman ~]# wg
interface: wg0
public key: <deleted>
private key: (hidden)
listening port: 55820
peer: <deleted>
endpoint: <external IP>:55820
allowed ips: 10.5.4.4/32
latest handshake: 53 seconds ago
transfer: 46.46 MiB received, 58.03 MiB sent
persistent keepalive: every 25 seconds
On the Worker
[root@te-kuru ~]# wg
interface: wg0
public key: <deleted>
private key: (hidden)
listening port: 55820
peer: <deleted>
endpoint: <external IP>:55820
allowed ips: 10.5.4.1/32
latest handshake: 42 seconds ago
transfer: 54.54 MiB received, 46.01 MiB sent
persistent keepalive: every 25 seconds
[root@te-kuru ~]#
The keys look like they match.
Ping
I can ping from one to the other, and vice versa, using allowed ips.
The VPN tunnel itself looks fine (handshakes recent on both sides, keys match, ping works). So the issue is likely higher up, in how cluster status is being tracked/reported.
A few things to help pin it down:
On the Nodes page, is the alert icon red or yellow? Red means the Redis replica’s idle time is above 8 seconds; yellow is a different alert, so if you see yellow, open the node details for the specific message.
Does the offline status flap if you reload the page a few times in a row, or is it consistently showing offline every time?
Redis replication status — please paste the output of this on both nodes:
redis-cli info replication
This will show whether the worker’s Redis is actually replicating from the leader (or vice versa), which is separate from just “is the redis service running.”
Thanks for the detailed output so far, it’s helping rule things out.
A disabled node_exporter service is a symptom of errors during the cluster join procedure. You should either reinstall te-kuru worker and join it again as a new node (node 5?), or at least make sure core services are enabled on it at boot, running:
