Getting lots of martian packets

[bobtskutter (bob)]

NethServer Version: 7.4

Hello, I’m getting lots of martian packets logged in /var/log/messages. The following lines are repeated hundreds of times for different LAN IP addresses.

May 23 21:44:03 nethserver kernel: IPv4: martian source 192.168.1.43 from 192.168.1.1, on dev eth1
May 23 21:44:03 nethserver kernel: ll header: 00000000: ff ff ff ff ff ff 02 08 20 3f 4c dd 08 06        ........ ?L...
May 23 21:44:04 nethserver kernel: IPv4: martian source 192.168.1.43 from 192.168.1.1, on dev eth1
May 23 21:44:04 nethserver kernel: ll header: 00000000: ff ff ff ff ff ff 02 08 20 3f 4c dd 08 06        ........ ?L...
May 23 21:44:05 nethserver kernel: IPv4: martian source 192.168.1.43 from 192.168.1.1, on dev eth1
May 23 21:44:05 nethserver kernel: ll header: 00000000: ff ff ff ff ff ff 02 08 20 3f 4c dd 08 06        ........ ?L...

My NethServer LAN IP is 192.168.1.1
I’ve run arping and nmap to look for duplicate IP addresses, and there arn’t any.

I’ve captured TCP data via NethServer and from another machine on the LAN.

[root@nethserver ~]# tcpdump -nei eth1 host 192.168.1.43
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
21:15:20.677958 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
21:15:20.678331 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
21:15:21.680812 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
21:15:21.681202 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
21:15:22.682765 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
21:15:22.682917 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
21:15:25.697967 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
21:15:25.698275 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
21:15:26.700788 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
21:15:26.701187 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
21:15:27.702872 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
21:15:27.703180 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28
^C
12 packets captured
12 packets received by filter
0 packets dropped by kernel

HPLaptop dave # tcpdump -nei wlo1 host 192.168.1.43
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wlo1, link-type EN10MB (Ethernet), capture size 262144 bytes
21:30:04.233433 02:08:20:3f:4c:dd > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.1.43 tell 192.168.1.1, length 46
21:30:22.244469 02:08:20:3f:4c:dd > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.1.43 tell 192.168.1.1, length 46
21:30:24.189171 02:08:20:3f:4c:dd > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.1.43 tell 192.168.1.1, length 46
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel

The arp broadcast requests are coming from the MAC address of my NethServer LAN port.

[root@nethserver ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 30.1.1.3  netmask 255.255.255.0  broadcast 30.1.1.255
        inet6 fe80::8:20ff:feee:59e1  prefixlen 64  scopeid 0x20<link>
        ether 02:08:20:ee:59:e1  txqueuelen 1000  (Ethernet)
        RX packets 1404690  bytes 1764097753 (1.6 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 785052  bytes 100487824 (95.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.1  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::8:20ff:fe3f:4cdd  prefixlen 64  scopeid 0x20<link>
        ether 02:08:20:3f:4c:dd  txqueuelen 1000  (Ethernet)
        RX packets 542738  bytes 83541122 (79.6 MiB)
        RX errors 0  dropped 1  overruns 0  frame 0
        TX packets 1360867  bytes 1719922827 (1.6 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

The IP address 192.168.1.43 was assigned by DHCP and the arp broadcasts have started when that machine was shutdown.

It really seems as though my NethServer is sending arp broadcasts to work out where a machine has gone?

Any ideas?

Thanks Bob

[m.traeumner (Michael Träumner)]

Hi @bobtskutter,
I’ve no own idea, but perhaps this link could help you to get the problem:

https://www.linuxquestions.org/questions/linux-security-4/getting-martian-source-messages-185672/

[bobtskutter (bob)]

Thank you @m.traeumner I had seen that post.

The martian packets appear to be coming from the LAN address of my NethServer.

[root@nethserver ~]# tcpdump -nei eth1 host 192.168.1.43
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
21:15:20.677958 02:08:20:3f:4c:dd > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 192.168.1.43 tell 192.168.1.1, length 28

02:08:20:3f:4c:dd
is the MAC address of the LAN port on my NethServer.

In the quote above, 192.168.1.43 was assigned to a Windows Laptop that had just been shutdown

My home network is configured with one router. Hopefully this helps:

Internet -> VDSL router -> Red IP = 30.1.1.3 **NethServer** Green IP = 192.168.1.1 -> internal network

The NethServer is virtualised ontop of KVM within an Omnios (Solaris) server. The NethServer NICs are virtio devices.
The NethServer hands out LAN IP address using DHCP.
There is only one DHCP server on the network.

Regards
Bob

[bobtskutter (bob)]

Every single time a network device that was given an IP address by DHCP is shutdown I get a flood of martian packets.

My issue seems to be related to the DHCP server.

[m.traeumner (Michael Träumner)]

@support_team
Has someone else this problem?
I don’t use DHCP of NethServer.

[mrmarkuz (Markus Neuberger)]

It may be similar error as in this topic by @vmakol related to DHCP and martian source:

Multiwan Down -Only one Connection- Works Sporadically

Setup: 3 inbound Internet connections setup in ‘Balance’ mode(50, 30, 20) with One static and two DHCP. Everything has been working fine for over a month now in this mode until this morning when two of the links were showing as ‘DOWN’. I was curious and decided to check the connections by connecting them directly to a computer, and the connection was up. Same with the other one. I tried doing some troubleshooting by ‘Releasing Role’ on the two that were not working. I get the following err…

I could not reproduce it.

[bobtskutter (bob)]

Hummm…that does make me think. My system has a single WAN connection. It is also configured with OpenVPN in “road warrior” mode, the VPN server is configured to route all traffic through the VPN tunnel. So there is almost a dual WAN connection.
I’m getting so many martian packets when a mchine goes offline that I’ve disabled martian packet logging in the shorewall.conf file.

(@mrmarkuz thanks for investigating).

[bobtskutter (bob)]

I’m still getting martian packets when ever a machine that had a DHCP lease goes offline.

I’ve rebooted all the servers on my network, to clear their ARP cache.

The TCP packets I’ve grabbed say the martian data is coming from NethServer. Does anyone have any ideas?

In notice there is a message
shorewall: Setting up Proxy ARP...
in the messages log when ShoreWall restarts, please can someone explain what it means?

thanks
Bob

[dnutan (Marc)]

About Proxy ARP, Shorewall says:
http://www.shorewall.org/ProxyARP.htm
http://www.shorewall.net/manpages/shorewall-proxyarp.html

A more in-deep explanation of Proxy ARP:

Proxy ARP – Definition and Use Cases – Practical Networking .net

Regarding martian packets cannot help much… doesn’t seem to be your case but Shorewall manual points to some common causes:
http://www.shorewall.net/MultiISP.html#Martians

Maybe you can get some more clues from a shorewall dump or iptables.

[bobtskutter (bob)]

@dnutan thank you for the links.
I’m getting confused by the whole problem!
I’ve not customised my NethServer firewall or routing table, the server is set-up through the web interface and I’ve not made any adjustments through the command line.
I’ve been doing some more “simple” troubleshooting. The martian packets only seem to refer to systems that are on my WiFi network. I’ve a bunch of windows and android clients that seem to be generating martian packets when they are shutdown (i.e. after they disconnect from the WiFi network). I’ve one Linux WiFi device and that does not seem to create martian packets when it’s shutdown. I’m beginning to suspect it’s my UniFi access point that’s causing the problems.
Bob

[bobtskutter (bob)]

Investigations are continuing.
I’ve noticed the martian packet storm lasts for a maximum time of 15mins, or 900seconds.
The martians are not just from clients on the WiFi.
Linux clients on WiFi or cable don’t result in martians.

Just saw this post:

Network Scanner Integration with Web Interface

I guess we can close this and move our tests on

using the scan feature under diagnostics generates martian packets that seem very similar to what I’m getting?

Does NethServer run arp-scan to validate dhcp leases?

bob

[Jclendineng (Joel Clendineng)]

Check the source and make sure nothing weird is running in the background. Generally martian packets are ok, they are used by iana or whatever its called. Easy enough to block at the firewall level. I have personally never had this happen, I’d say you have a rogue program running.

[bobtskutter (bob)]

Hello @Jclendineng thanks for the information.
The source is my NethServer.
I’m not really sure what program would be classed as rogue, there are lots and lots of programs that run on the machine and I don’t know what they’re all supposed to do.
I’ve been observing network traffic with WireShark for the last few days. Hopefully something will reveal itself!
regards
Bob