2022-08-23
Previously in
my post about monitoring my home wifi, I described using IPvlan to create an IP address for a docker image that
was accessible throughout my network. One downside I noticed was that when I
was ssh-ing into my Debian box, typing and inputting commands
would become significantly laggier.
I initially just dealt with the slowlness, but I finally got around to
investigating it. After a number of missteps, I eventually came to the
conclusion I needed to run the
arp -a command on my main Windows machine. I found out that
Debian box's Ethernet interface's IP (let's say its 192.168.1.59) had
the Debian box's Wifi interface's MAC address listed in the arp cache. So, I
did the sensible thing and cleared my cache and forced another arping. Running
the command again resulted in the Ethernet interface (192.168.1.59)
having the Wifi interface's MAC address.
By sending out manual ARP-Pings requesting the MAC address of 192.168.1.59, I noticed that I got 3 responses for each ping I sent out. Two of them having the Wifi interface's MAC address and one with the Ethernet interface's MAC address. Googling around, I found out that this is called ARP Flux.
Linux by default responds to the ARP ping to a known interface from all
possible interfaces with the MAC address of the responding interface. Since my
Debian box was connected via 3 interfaces (Ethernet, Wifi, Docker), it would
only respond 3 times with only one response having the actual Ethernet MAC
address. This would cause other machines to choose to connect via the slower
Wifi interface and thus my ssh had much more latency.
There are at least
two
solutions
to this. I chose to put
(EDIT [09/06/2022]: It turns out that arp_filter made my Wifi interface's
assigned IP always respond with the ethernet's MAC, which also seems
undesirable, so I switched to using arp_ignore and
arp_announce)
net.ipv4.conf.default.arp_filter=1
net.ipv4.conf.all.arp_filter=1
in my /etc/sysctl.conf file. Others preferred to put this instead
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2
I suppose I trusted a book with the name of
Guide to IP Layer Network Administration with Linux than other
websites, but it's likely both work. Also the
documentation
for
arp_filter seemed to cleanly express what I wanted:
1 - Allows you to have multiple network interfaces on the same subnet, and have the ARPs for each interface be answered based on whether or not the kernel would route a packet from the ARP’d IP out that interface (therefore you must use source based routing for this to work). In other words it allows control of which cards (usually 1) will respond to an arp request.
arp_ignore/arp_announce
arp_ignore
0 - (default): reply for any local target IP address, configured
on any interface
1 - reply only if the target IP address is local address
configured on the incoming interface
arp_announce
0 - (default) Use any local address, configured on any interface
2 - Always use the best local address for this target.
In this mode we ignore the source address in the IP packet
and try to select local address that we prefer for talks with
the target host. Such local address is selected by looking
for primary IP addresses on all our subnets on the outgoing
interface that include the target IP address. If no suitable
local address is found we select the first local address
we have on the outgoing interface or on all other interfaces,
with the hope we will receive reply for our request and
even sometimes no matter the source IP address we announce.
The setting seemed to fix my lagginess with ssh. I confirmed by
manually sending ARP Pings that I would only get a single reply to the correct
Ethernet interface's MAC Address. Problem solved!
Any error corrections or comments can be made by sending me a pull request.