many hosts respond to single arp packet

[DarakDuVal]

As described in the wiki on duplicate arp replies, I have observed many repeated replies on my network due to ARP scans. This is not limited to running the scan from a specific host, I have seen the exact same happening when using the Fing app on my phone, arp-scan on a RPi, and on any other linux client accordingly.

Here are some examples of scans with very limited number of hosts to scan for, but the time it takes is ridiculously high.

Scan of 3 hosts with interval of 20ms, which should only take 1,16s consumes more than a minute.

sudo arp-scan --interval=20000 -vvv 172.16.1.1-172.16.1.3
Interface: eth0, type: EN10MB, MAC: b8:27:eb:a7:a2:8a, IPv4: 172.16.1.254
DEBUG: pcap filter string: "ether dst b8:27:eb:a7:a2:8a and (arp or (ether[14:4]=0xaaaa0300 and ether[20:2]=0x0806) or (ether[12:2]=0x8100 and ether[16:2]=0x0806) or (ether[12:2]=0x8100 and ether[18:4]=0xaaaa0300 and ether[24:2]=0x0806))"
DEBUG: Loaded 34890 IEEE OUI/Vendor entries from /usr/share/arp-scan/ieee-oui.txt.
DEBUG: Loaded 4575 IEEE IAB/Vendor entries from /usr/share/arp-scan/ieee-iab.txt.
DEBUG: Loaded 6 MAC/Vendor entries from /usr/share/arp-scan/mac-vendor.txt.
Starting arp-scan 1.9.7 with 3 hosts (https://github.com/royhills/arp-scan)
Host List:

Entry   IP Address
1       172.16.1.1
2       172.16.1.2
3       172.16.1.3

Total of 3 host entries.

---     Sending packet #1 to host 172.16.1.1 tmo 500000
---     Ignoring 60 bytes from unknown host 172.16.1.81
---     Received packet #1 from 172.16.1.1
172.16.1.1      54:60:09:c0:a9:2a       Google, Inc.    Padding=00000000000000000000000000002e1b86bf
---     Removing host 172.16.1.1 - Received 60 bytes
---     Ignoring 60 bytes from unknown host 172.16.1.37
---     Ignoring 60 bytes from unknown host 172.16.0.2
---     Ignoring 60 bytes from unknown host 172.16.1.39
---     Ignoring 60 bytes from unknown host 172.16.1.45
---     Ignoring 60 bytes from unknown host 172.16.1.48
---     Sending packet #1 to host 172.16.1.2 tmo 500000
---     Ignoring 60 bytes from unknown host 172.16.1.54
---     Ignoring 60 bytes from unknown host 172.16.0.2
---     Ignoring 60 bytes from unknown host 172.16.1.54
---     Ignoring 60 bytes from unknown host 172.16.1.42
---     Ignoring 60 bytes from unknown host 172.16.1.42
---     Ignoring 60 bytes from unknown host 172.16.0.2
---     Ignoring 60 bytes from unknown host 172.16.1.47
---     Ignoring 60 bytes from unknown host 172.16.1.26
---     Received packet #1 from 172.16.1.3
172.16.1.3      1c:53:f9:26:64:2b       Google, Inc.    Padding=0000000000000000000000000000064e2d18
---     Removing host 172.16.1.3 - Received 60 bytes
---     Ignoring 60 bytes from unknown host 172.16.1.32
---     Ignoring 60 bytes from unknown host 172.16.1.26
---     Received packet #2 from 172.16.1.3
172.16.1.3      1c:53:f9:26:64:2b       Google, Inc.    Padding=00000000000000000000000000009d4cf7e4 (DUP: 2)
---     Removing host 172.16.1.3 - Received 60 bytes
***     remove_host called on non-live host: SHOULDN'T HAPPEN
---     Ignoring 60 bytes from unknown host 172.16.1.32
---     Ignoring 60 bytes from unknown host 172.16.1.37
---     Ignoring 60 bytes from unknown host 172.16.1.81
---     Pass 1 complete
---     Sending packet #2 to host 172.16.1.2 tmo 750000
---     Ignoring 60 bytes from unknown host 172.16.0.2
---     Pass 2 complete
---     Removing host 172.16.1.2 - Timeout
---     Ignoring 60 bytes from unknown host 172.16.1.26

30 packets received by filter, 0 packets dropped by kernel
Ending arp-scan 1.9.7: 3 hosts scanned in 67.630 seconds (0.04 hosts/sec). 3 responded

Returning to defaults still consumes approx 6s

sudo arp-scan 172.16.1.1-172.16.1.3
Interface: eth0, type: EN10MB, MAC: b8:27:eb:a7:a2:8a, IPv4: 172.16.1.254
Starting arp-scan 1.9.7 with 3 hosts (https://github.com/royhills/arp-scan)
172.16.1.3      1c:53:f9:26:64:2b       Google, Inc.
172.16.1.1      54:60:09:c0:a9:2a       Google, Inc.

6 packets received by filter, 0 packets dropped by kernel
Ending arp-scan 1.9.7: 3 hosts scanned in 5.391 seconds (0.56 hosts/sec). 2 responded

Furthermore, if the complete localnet is to be scanned, a broadcast storm will be detected causing shutdown of network traffic to the host running arp-scan, while also leaving critical network components non-responsive, e.g., access points.

What could be the cause of this? We can try and investigate on my network maybe a bit more, this is 100% reproducible.

[royhills]

Ending arp-scan 1.9.7: 3 hosts scanned in 67.630 seconds (0.04 hosts/sec). 3 responded

Version 1.9.7 is very old now and you should upgrade to the latest version. I'm guessing you're running an old debian stable or ubuntu LTS? The latest Debian/Ubuntu LTS versions should have packages for version 1.10.0.

I think the problem you are seeing is the same as in this issue: #67, which was fixed in version 1.9.8.

Furthermore, if the complete localnet is to be scanned, a broadcast storm will be detected causing shutdown of network traffic to the host running arp-scan, while also leaving critical network components non-responsive, e.g., access points.

arp-scan defaults to 2 milliseconds between outgoing ARP requests giving a rate of 500 packets per second and a bandwidth of 256Kbit/sec (the ARP request packets are 64 bytes long). You can control the outgoing ARP request packet rate with the --interval option, E.g. --interval=10 to specify a 10 milliseconds interval or 100 packets per second; or the --bandwidth option, e.g. --bandwidth=64K.

arp-scan's default of 256Kb/s is only 0.0256% of a gigabit Ethernet or 0.256% of 100Mbit which I wouldn't expect to cause an issue, but if it's reproducible then it is worth investigating. Some initial thoughts:

• Are there any logs from the switch (assuming it's a managed switch) or access point?
• Is broadcast storm control or similar configured on the switch/AP?

[DarakDuVal]

Version 1.9.7 is very old now and you should upgrade to the latest version. I'm guessing you're running an old debian stable or ubuntu LTS? The latest Debian/Ubuntu LTS versions should have packages for version 1.10.0.

I think the problem you are seeing is the same as in this issue: #67, which was fixed in version 1.9.8.

I did not realize how aged that version is - it keeps surprising me, how poorly some distros maintain their packages... my host is running

Distributor ID:	Raspbian
Description:	Raspbian GNU/Linux 11 (bullseye)
Release:	11
Codename:	bullseye

as it's being installed for use with pi.alert and pi-hole.

arp-scan defaults to 2 milliseconds between outgoing ARP requests giving a rate of 500 packets per second and a bandwidth of 256Kbit/sec (the ARP request packets are 64 bytes long). You can control the outgoing ARP request packet rate with the --interval option, E.g. --interval=10 to specify a 10 milliseconds interval or 100 packets per second; or the --bandwidth option, e.g. --bandwidth=64K.

arp-scan's default of 256Kb/s is only 0.0256% of a gigabit Ethernet or 0.256% of 100Mbit which I wouldn't expect to cause an issue, but if it's reproducible then it is worth investigating. Some initial thoughts:

Yes, I saw your extensive explanation on the readme (excellent documentation, by the way!) and tried to adjust some intervals or bandwidth settings, while that did not make any difference going to negligible values though. It appears as though some "mirror" effects kick in and multiply the few packets send out by non-addressed hosts. I would call it an avalanche that can be initiated with just a few arp packets.

• Are there any logs from the switch (assuming it's a managed switch) or access point?

Yes, Zyxel hardware managed through Nebula. There are some limited points of information, a blunt message hinting on a detected broadcast storm at the switch. The AP merely become non-responsive at some point.

• Is broadcast storm control or similar configured on the switch/AP?

I did not have storm control enabled on the access port which connects to the RPi, then I turned that on and set it to a level of 100 pckts. Will turn it off again.

The AP do not show any option for storm control.

Next, I will try to get my hands on latest version arp-scan, which will probably take ages to compile on the rpi.

[royhills]

my host is running

Distributor ID:	Raspbian
Description:	Raspbian GNU/Linux 11 (bullseye)
Release:	11
Codename:	bullseye

I think the issue is that the version of raspbian is based on debian 11 "Bullseye". Debian 12 "bookworm" includes arp-scan version 1.10.0. The latest version of rasperry pi os is based on Debian bookworm so should have arp-scan version 1.10.0.

Yes, I saw your extensive explanation on the readme (excellent documentation, by the way!) and tried to adjust some intervals or bandwidth settings, while that did not make any difference going to negligible values though. It appears as though some "mirror" effects kick in and multiply the few packets send out by non-addressed hosts. I would call it an avalanche that can be initiated with just a few arp packets.

Capturing packets with e.g. tcpdump -n -w filename.pcap -i interface-name arp and analysing the pcap file with wireshark might give some insight into what is happening.

Yes, Zyxel hardware managed through Nebula. There are some limited points of information, a blunt message hinting on a detected broadcast storm at the switch. The AP merely become non-responsive at some point.

Only the ARP requests that are sent by arp-scan should be broadcast packets. ARP responses should be unicast directed back to the requesting address, and arp-scan should not be sending ARP requests more frequently than specified with --interval, so it seems that there is something unusual going on somewhere.

Next, I will try to get my hands on latest version arp-scan, which will probably take ages to compile on the rpi.

It shouldn't take very long - it's just a small C program.

[royhills]

Issue #178 is another duplicate ARP response issue. This may be a seperate issue, but I am mentioning it in case there is a connection.