After migration dnsmasq gets not started

Bug
, , ,
1

NethServer Version: NS8 on Rocky Linux 9.4
Module: dnsmasq
Repositories: default and nethforge, no testing enabled

Problem nr. 3: I installed dnsmasq for the local dns management. Unfortunately it will not start up after 5 tries. It seems to complain about CNAME loops?

Abstract of log (journalctl):

Jun 24 11:35:07 server2 systemd[1]: Starting DNSmasq podman container...
Jun 24 11:35:08 server2 podman[2989444]: 2024-06-24 11:35:08.105995624 +0200 CEST m=+0.060257260 container create 3c2f9077d85a2aa9a0676bebe098babc1f14bf71e1ae8f1bc53d06dac381feaf (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsm>
Jun 24 11:35:08 server2 podman[2989444]: 2024-06-24 11:35:08.089349054 +0200 CEST m=+0.043610704 image pull 48bd9fe84aa10cd1690dc8ac1b695060f74f38ca2948342a8ac05a9edea3f498 ghcr.io/nethserver/dnsmasq-server:latest
Jun 24 11:35:08 server2 podman[2989444]: 2024-06-24 11:35:08.189587263 +0200 CEST m=+0.143848898 container init 3c2f9077d85a2aa9a0676bebe098babc1f14bf71e1ae8f1bc53d06dac381feaf (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsmas>
Jun 24 11:35:08 server2 podman[2989444]: 2024-06-24 11:35:08.195445365 +0200 CEST m=+0.149707010 container start 3c2f9077d85a2aa9a0676bebe098babc1f14bf71e1ae8f1bc53d06dac381feaf (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsma>
Jun 24 11:35:08 server2 dnsmasq[2989472]: 
Jun 24 11:35:08 server2 dnsmasq[2989472]: dnsmasq: CNAME loop involving log.pa3hfj.nl
Jun 24 11:35:08 server2 systemd[1]: Started DNSmasq podman container.
Jun 24 11:35:08 server2 podman[2989444]: 3c2f9077d85a2aa9a0676bebe098babc1f14bf71e1ae8f1bc53d06dac381feaf
Jun 24 11:35:08 server2 podman[2989479]: 2024-06-24 11:35:08.256706483 +0200 CEST m=+0.044991749 container died 3c2f9077d85a2aa9a0676bebe098babc1f14bf71e1ae8f1bc53d06dac381feaf (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsmas>
Jun 24 11:35:08 server2 podman[2989479]: 2024-06-24 11:35:08.306154247 +0200 CEST m=+0.094439509 container remove 3c2f9077d85a2aa9a0676bebe098babc1f14bf71e1ae8f1bc53d06dac381feaf (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsm>
Jun 24 11:35:08 server2 systemd[1]: dnsmasq3.service: Main process exited, code=exited, status=1/FAILURE
Jun 24 11:35:08 server2 systemd[1]: dnsmasq3.service: Failed with result 'exit-code'.
Jun 24 11:35:08 server2 systemd[1]: dnsmasq3.service: Scheduled restart job, restart counter is at 1.
Jun 24 11:35:08 server2 systemd[1]: Stopped DNSmasq podman container.
Jun 24 11:35:08 server2 systemd[1]: Starting DNSmasq podman container...
Jun 24 11:35:09 server2 podman[2989637]: 2024-06-24 11:35:09.183920055 +0200 CEST m=+0.081250177 container create 1f814de0df48b84387c7f0ce7e81b8431dd77fc05c1c939d6df61ecce12b15be (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsm>
Jun 24 11:35:09 server2 podman[2989637]: 2024-06-24 11:35:09.2396076 +0200 CEST m=+0.136937727 container init 1f814de0df48b84387c7f0ce7e81b8431dd77fc05c1c939d6df61ecce12b15be (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsmasq,>
Jun 24 11:35:09 server2 podman[2989637]: 2024-06-24 11:35:09.24569738 +0200 CEST m=+0.143027505 container start 1f814de0df48b84387c7f0ce7e81b8431dd77fc05c1c939d6df61ecce12b15be (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsmas>
Jun 24 11:35:09 server2 systemd[1]: Started DNSmasq podman container.
Jun 24 11:35:09 server2 dnsmasq[2989662]: 
Jun 24 11:35:09 server2 dnsmasq[2989662]: dnsmasq: CNAME loop involving ha.pa3hfj.nl
Jun 24 11:35:09 server2 podman[2989637]: 1f814de0df48b84387c7f0ce7e81b8431dd77fc05c1c939d6df61ecce12b15be
Jun 24 11:35:09 server2 podman[2989637]: 2024-06-24 11:35:09.1588555 +0200 CEST m=+0.056185627 image pull 48bd9fe84aa10cd1690dc8ac1b695060f74f38ca2948342a8ac05a9edea3f498 ghcr.io/nethserver/dnsmasq-server:latest
Jun 24 11:35:09 server2 podman[2989666]: 2024-06-24 11:35:09.30363724 +0200 CEST m=+0.041578520 container died 1f814de0df48b84387c7f0ce7e81b8431dd77fc05c1c939d6df61ecce12b15be (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsmasq>
Jun 24 11:35:09 server2 podman[2989666]: 2024-06-24 11:35:09.336047632 +0200 CEST m=+0.073988909 container remove 1f814de0df48b84387c7f0ce7e81b8431dd77fc05c1c939d6df61ecce12b15be (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsm>
Jun 24 11:35:09 server2 systemd[1]: dnsmasq3.service: Main process exited, code=exited, status=1/FAILURE
Jun 24 11:35:09 server2 systemd[1]: dnsmasq3.service: Failed with result 'exit-code'.
Jun 24 11:35:09 server2 systemd[1]: dnsmasq3.service: Scheduled restart job, restart counter is at 2.
Jun 24 11:35:09 server2 systemd[1]: Stopped DNSmasq podman container.
Jun 24 11:35:09 server2 systemd[1]: Starting DNSmasq podman container...
Jun 24 11:35:10 server2 podman[2989689]: 2024-06-24 11:35:10.07184671 +0200 CEST m=+0.047414087 container create d9dd1fc800b5d1cba1d21845a3a57b2ee7310c915f857a58891410d8df266d2a (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsma>
Jun 24 11:35:10 server2 podman[2989689]: 2024-06-24 11:35:10.127561968 +0200 CEST m=+0.103129350 container init d9dd1fc800b5d1cba1d21845a3a57b2ee7310c915f857a58891410d8df266d2a (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsmas>
Jun 24 11:35:10 server2 dnsmasq[2989700]: 
Jun 24 11:35:10 server2 dnsmasq[2989700]: dnsmasq: CNAME loop involving www.taalapart.nl
Jun 24 11:35:10 server2 podman[2989689]: 2024-06-24 11:35:10.141336727 +0200 CEST m=+0.116904111 container start d9dd1fc800b5d1cba1d21845a3a57b2ee7310c915f857a58891410d8df266d2a (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsma>
Jun 24 11:35:10 server2 systemd[1]: Started DNSmasq podman container.
Jun 24 11:35:10 server2 podman[2989689]: d9dd1fc800b5d1cba1d21845a3a57b2ee7310c915f857a58891410d8df266d2a
Jun 24 11:35:10 server2 podman[2989689]: 2024-06-24 11:35:10.056453638 +0200 CEST m=+0.032021034 image pull 48bd9fe84aa10cd1690dc8ac1b695060f74f38ca2948342a8ac05a9edea3f498 ghcr.io/nethserver/dnsmasq-server:latest
Jun 24 11:35:10 server2 podman[2989704]: 2024-06-24 11:35:10.19662939 +0200 CEST m=+0.039846970 container died d9dd1fc800b5d1cba1d21845a3a57b2ee7310c915f857a58891410d8df266d2a (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsmasq>
Jun 24 11:35:10 server2 podman[2989704]: 2024-06-24 11:35:10.232286713 +0200 CEST m=+0.075504296 container remove d9dd1fc800b5d1cba1d21845a3a57b2ee7310c915f857a58891410d8df266d2a (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsm>
Jun 24 11:35:10 server2 systemd[1]: dnsmasq3.service: Main process exited, code=exited, status=1/FAILURE
Jun 24 11:35:10 server2 systemd[1]: dnsmasq3.service: Failed with result 'exit-code'.
Jun 24 11:35:10 server2 systemd[1]: dnsmasq3.service: Scheduled restart job, restart counter is at 3.
Jun 24 11:35:10 server2 systemd[1]: Stopped DNSmasq podman container.
Jun 24 11:35:10 server2 systemd[1]: Starting DNSmasq podman container...
Jun 24 11:35:11 server2 podman[2989769]: 2024-06-24 11:35:11.124543589 +0200 CEST m=+0.047026186 container create d86fee8b48680076ef6171c7f5c7bf5715fa992447d3e7ca2ba23452fa3d0db4 (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsm>
Jun 24 11:35:11 server2 podman[2989769]: 2024-06-24 11:35:11.191937441 +0200 CEST m=+0.114420032 container init d86fee8b48680076ef6171c7f5c7bf5715fa992447d3e7ca2ba23452fa3d0db4 (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsmas>
Jun 24 11:35:11 server2 dnsmasq[2989788]: 
Jun 24 11:35:11 server2 dnsmasq[2989788]: dnsmasq: CNAME loop involving mail.pa3hfj.nl
Jun 24 11:35:11 server2 podman[2989769]: 2024-06-24 11:35:11.200502274 +0200 CEST m=+0.122984864 container start d86fee8b48680076ef6171c7f5c7bf5715fa992447d3e7ca2ba23452fa3d0db4 (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsma>
Jun 24 11:35:11 server2 conmon[2989788]: conmon d86fee8b48680076ef61 <nwarn>: Failed to open cgroups file: /sys/fs/cgroup/machine.slice/libpod-d86fee8b48680076ef6171c7f5c7bf5715fa992447d3e7ca2ba23452fa3d0db4.scope/container/memory.events
Jun 24 11:35:11 server2 systemd[1]: Started DNSmasq podman container.
Jun 24 11:35:11 server2 podman[2989769]: 2024-06-24 11:35:11.105907163 +0200 CEST m=+0.028389753 image pull 48bd9fe84aa10cd1690dc8ac1b695060f74f38ca2948342a8ac05a9edea3f498 ghcr.io/nethserver/dnsmasq-server:latest
Jun 24 11:35:11 server2 podman[2989769]: d86fee8b48680076ef6171c7f5c7bf5715fa992447d3e7ca2ba23452fa3d0db4
Jun 24 11:35:11 server2 podman[2989799]: 2024-06-24 11:35:11.251474038 +0200 CEST m=+0.040427204 container died d86fee8b48680076ef6171c7f5c7bf5715fa992447d3e7ca2ba23452fa3d0db4 (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsmas>
Jun 24 11:35:11 server2 podman[2989799]: 2024-06-24 11:35:11.287767526 +0200 CEST m=+0.076720691 container remove d86fee8b48680076ef6171c7f5c7bf5715fa992447d3e7ca2ba23452fa3d0db4 (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsm>
Jun 24 11:35:11 server2 systemd[1]: dnsmasq3.service: Main process exited, code=exited, status=1/FAILURE
Jun 24 11:35:11 server2 systemd[1]: dnsmasq3.service: Failed with result 'exit-code'.
Jun 24 11:35:11 server2 systemd[1]: dnsmasq3.service: Scheduled restart job, restart counter is at 4.
Jun 24 11:35:11 server2 systemd[1]: Stopped DNSmasq podman container.
Jun 24 11:35:11 server2 systemd[1]: Starting DNSmasq podman container...
Jun 24 11:35:12 server2 podman[2989936]: 2024-06-24 11:35:12.068480792 +0200 CEST m=+0.061358736 container create 8508d247f0625a8830b695f264697ad5bb02bf30a7d30363ede918dbe99ffc2d (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsm>
Jun 24 11:35:12 server2 podman[2989936]: 2024-06-24 11:35:12.111355823 +0200 CEST m=+0.104233770 container init 8508d247f0625a8830b695f264697ad5bb02bf30a7d30363ede918dbe99ffc2d (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsmas>
Jun 24 11:35:12 server2 dnsmasq[2989954]: 
Jun 24 11:35:12 server2 dnsmasq[2989954]: dnsmasq: CNAME loop involving log.pa3hfj.nl
Jun 24 11:35:12 server2 podman[2989936]: 2024-06-24 11:35:12.124104007 +0200 CEST m=+0.116981952 container start 8508d247f0625a8830b695f264697ad5bb02bf30a7d30363ede918dbe99ffc2d (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsma>
Jun 24 11:35:12 server2 systemd[1]: Started DNSmasq podman container.
Jun 24 11:35:12 server2 podman[2989936]: 8508d247f0625a8830b695f264697ad5bb02bf30a7d30363ede918dbe99ffc2d
Jun 24 11:35:12 server2 podman[2989936]: 2024-06-24 11:35:12.048597444 +0200 CEST m=+0.041475382 image pull 48bd9fe84aa10cd1690dc8ac1b695060f74f38ca2948342a8ac05a9edea3f498 ghcr.io/nethserver/dnsmasq-server:latest
Jun 24 11:35:12 server2 podman[2989962]: 2024-06-24 11:35:12.178421626 +0200 CEST m=+0.035518032 container died 8508d247f0625a8830b695f264697ad5bb02bf30a7d30363ede918dbe99ffc2d (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsmas>
Jun 24 11:35:12 server2 podman[2989962]: 2024-06-24 11:35:12.222966917 +0200 CEST m=+0.080063311 container remove 8508d247f0625a8830b695f264697ad5bb02bf30a7d30363ede918dbe99ffc2d (image=ghcr.io/nethserver/dnsmasq-server:latest, name=dnsm>
Jun 24 11:35:12 server2 systemd[1]: dnsmasq3.service: Main process exited, code=exited, status=1/FAILURE
Jun 24 11:35:12 server2 systemd[1]: dnsmasq3.service: Failed with result 'exit-code'.
Jun 24 11:35:12 server2 systemd[1]: dnsmasq3.service: Scheduled restart job, restart counter is at 5.
Jun 24 11:35:12 server2 systemd[1]: Stopped DNSmasq podman container.
Jun 24 11:35:12 server2 systemd[1]: dnsmasq3.service: Start request repeated too quickly.
Jun 24 11:35:12 server2 systemd[1]: dnsmasq3.service: Failed with result 'exit-code'.
Jun 24 11:35:12 server2 systemd[1]: Failed to start DNSmasq podman container.
2

@Tbaile any idea?

3

Hi!

Can you please print the output of the following files?

  • dnsmasq_hosts.d/00_node_hosts
  • dnsmasq.d/01cnames.conf
  • config.json

Never occurred this issue, probably a configuration issue, you can find the files when using runagent -m dnsmasq3

4

Here is the asked info:

[root@server2 state]# cat dnsmasq_hosts.d/00_node_hosts
# This file is automatically generated by NethServer, manual changes will be lost.
192.168.0.2 server2.int.pa3hfj.nl

[root@server2 state]# cat dnsmasq.d/01cnames.conf
# This file is automatically generated by NethServer, manual changes will be lost.
cname=www.taalapart.nl,webmail.pa3hfj.nl,www.pi4zwn.nl,nvr.pa3hfj.nl,openwebrx.pa3hfj.nl,server2.int.pa3hfj.nl,mail.pa3hfj.nl,log.pi4zwn.nl,log.pa3hfj.nl,nextcloud.pa3hfj.nl,ha.pa3hfj.nl,server2.int.pa3hfj.nl

[root@server2 state]# cat config.json
{"dhcp-server": {"enabled": false, "end": "192.168.0.254", "lease": 12, "start": "192.168.0.1"}, "dns-server": {"enabled": true, "primary-server": "1.0.0.1", "secondary-server": "1.1.1.1"}, "interface": "enp6s18"}[root@server2 state]#
5

It appears that a bug has been discovered! Seems that there’s a service on server2.int.pa3hfj.nl, which conflicts to the cname of itself…

Filled a bug here: DNSMasq won't start if some routes has the same FQDN as the node · Issue #6959 · NethServer/dev · GitHub

3 Likes
6

The bug fix is in testing. @pa3hfj would you mind testing it?

This is a procedure:

  1. Enable Testing switch of the default software repository
  2. Update the Dnsmasq app, check if it solves your issue
  3. Disable the Testing switch
7

You might also want to:

systemctl restart dnsmasq3

After the update (or just save the settings through the UI)

8

I see there is in testing also an update for traefik, should I do that as well?

9

Result without updating traefik:

dnsmasq: failed to create listening socket for port 53: Address already in use
10

Tommaso wrote dnsmasq3. I think it is on purpose, because Dnsmasq is a rootful module and installs progressive service unit names.

Your command revived a stale Dnsmasq instance?

11

No, I moved dnsmasq to a worker node to see if that would get things going, but it didn’t. So I moved it back to the leader node, hence the “5”.

afbeelding

12

That’s weird, what is you installation based on? (Rocky, Ubuntu…)
Also, can you find what process holds port 53?
By the command you pasted the number of the module is omitted, are you sure you added the 5 at the end? (just double checking that we’re not accidentally trying to restart the dnsmasq installed by the OS, rather than the one from NS8)

13

Hello Tommaso, thanks for your help. I’m sorry to reply so late, but my TP-Link internet-router/firewall went crazy this morning so I had an (for now) unintended “emergency” implementation of NethSecurity to do… :grinning:

I took the opportunity to make NethSecurity also the internal dhcp and dns provider, which seems to work well.

I checked the existence of a OS version (Rocky 9.4) of dnsmasq but that is not active. The strange thing that the dnsmasq container now IS running, even before I deactivated the dns provider.

Sorry to say that this problem seems to be left a bit “hanging” due to my network problems. But you can imagine that it took quite some time and resetting of switches and servers before the problem with the router became clear to me. So I think it will be hard to investigate the dnsmasq problem any further.

I have a few annoying problems left: loki doesn’t work and LE certificates are not obtained. They are addressed in earlier posts with no solution yet.

[root@server2 ~]# systemctl status dnsmasq
Unit dnsmasq.service could not be found.

[root@server2 ~]# systemctl status dnsmasq5
â—Ź dnsmasq5.service - DNSmasq podman container
     Loaded: loaded (/etc/systemd/system/dnsmasq5.service; enabled; preset: disabled)
     Active: active (running) since Wed 2024-06-26 14:03:05 CEST; 1h 16min ago
       Docs: man:podman-generate-systemd(1)
    Process: 5697 ExecStartPre=/bin/rm -f /run/dnsmasq5.service.ctr-id (code=exited, status=0/SUCCESS)
    Process: 5699 ExecStartPre=runagent -m dnsmasq5 reload_hosts (code=exited, status=0/SUCCESS)
   Main PID: 6110 (conmon)
      Tasks: 1 (limit: 74657)
     Memory: 700.0K
        CPU: 694ms
     CGroup: /system.slice/dnsmasq5.service
             └─6110 /usr/bin/conmon --api-version 1 -c c5c1a062bd2b16059eb64978de19724600e456fea222b3a976524a07cfd31e4c -u c5c1a062bd2b16059eb64978de19724600e456fea222b3a976524a07cfd3>

Jun 26 14:03:03 server2 systemd[1]: Starting DNSmasq podman container...
Jun 26 14:03:05 server2 podman[6045]: 2024-06-26 14:03:05.514332157 +0200 CEST m=+0.142002142 container create c5c1a062bd2b16059eb64978de19724600e456fea222b3a976524a07cfd31e4c (image=>
Jun 26 14:03:05 server2 podman[6045]: 2024-06-26 14:03:05.441343048 +0200 CEST m=+0.069013038 image pull fc7ecb6484c534ea8d2c8bc544e6bc0c32495eb4e84eb451a23d5c8ffe0bbdfe ghcr.io/neths>
Jun 26 14:03:05 server2 podman[6045]: 2024-06-26 14:03:05.617297044 +0200 CEST m=+0.244967031 container init c5c1a062bd2b16059eb64978de19724600e456fea222b3a976524a07cfd31e4c (image=gh>
Jun 26 14:03:05 server2 dnsmasq[6110]: Jun 26 12:03:05 dnsmasq[1]: started, version 2.89 DNS disabled
Jun 26 14:03:05 server2 dnsmasq[6110]: Jun 26 12:03:05 dnsmasq[1]: compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset no-n>
Jun 26 14:03:05 server2 dnsmasq[6110]: Jun 26 12:03:05 dnsmasq[1]: warning: failed to change owner of /dev/stdout: Permission denied
Jun 26 14:03:05 server2 podman[6045]: 2024-06-26 14:03:05.631450295 +0200 CEST m=+0.259120282 container start c5c1a062bd2b16059eb64978de19724600e456fea222b3a976524a07cfd31e4c (image=g>
Jun 26 14:03:05 server2 systemd[1]: Started DNSmasq podman container.
Jun 26 14:03:05 server2 podman[6045]: c5c1a062bd2b16059eb64978de19724600e456fea222b3a976524a07cfd31e4c
lines 1-23/23 (END)