Filter DHCP offers from other nodes on LAN

[ilario]

This issue was first reported by @nordurljosahvida on the lime-users list here and on a similar issue: #172.
In the original report, a commercial router was connected to the LAN port, and to fix this was enough to move it on the WAN port.
Instead, in this case I observed it on a isolated two LibreMesh nodes network while configuring them one by one using FirstBootWizard.

The setup:

• two LibreMesh nodes are connected via ethernet cable at their LAN ports
• my laptop is connected on the AP of one of the nodes
• the two LibreMesh nodes have different ap_ssid AP ESSID (to me it happened simply because one was already configured with a custom ESSID while the other was not yet configured (had the default "LibreMesh.org" ESSID))

The problem:

• the DHCP offers from both nodes reach my client, my client considers both valid and takes a random one
• the two DHCP offers suggest a different network domain and a different gateway
• when the DHCP offer from the wrong node (the distant one) gets accepted, my client cannot reach the gateway (e.g. I am connected to the AP of a node with 10.13.0.1/16 but my laptop accepts an IPv4 in the 10.249.0.0/16 network, and cannot reach the announced 10.249.0.1 gateway)

What I would expect:

some firewall rule should kill the DHCP requests or the DHCP offers going through br-lan

I thought that this ebtable rule (added 6 years ago in 920ba99) was in charge of filtering these offers, but seems that is not working:

lime-packages/packages/lime-proto-anygw/files/usr/lib/lua/lime/proto/anygw.lua

Line 66 in f33051f

"ebtables -A FORWARD -j DROP -d " .. anygw_mac .. "/" .. anygw_mac_mask .. "\n" ..

[ilario]

Running various times the DHCP client on my laptop, the IPv4 can change from one network to another randomly with some stickiness:

at this point I was connected to a "libremesh.org" AP with IPv4 10.249.x.x but the DHCP offer that gets accepted by my laptop if the one of the other "LibreMesh.org" node connected to the first node by cable on the LAN port, likely just because it was received few milliseconds before

# dhcpcd
dev: loaded udev
wlp3s0: connected to Access Point `libremesh.org'
enp0s25: waiting for carrier
DUID 00:01:00:01:22:29:54:8f:54:ee:75:7a:c2:1f
wlp3s0: IAID ea:06:f3:6f
wlp3s0: soliciting a DHCP lease
wlp3s0: soliciting an IPv6 router
wlp3s0: Router Advertisement from fe80::c24a:ff:fedd:6800
wlp3s0: advertised MTU 1500 is greater than link MTU 1350
wlp3s0: adding address 2a00:1508:af9:c000:1fcc:2637:9217:35bc/64
wlp3s0: adding route to 2a00:1508:af9:c000::/64
wlp3s0: adding default route via fe80::c24a:ff:fedd:6800
wlp3s0: Router Advertisement from fe80::de9f:dbff:fe37:28a9
wlp3s0: advertised MTU 1500 is greater than link MTU 1350
wlp3s0: adding address fd0d:fe46:8ce8:0:ddf8:77ea:8ed9:4ab0/64
wlp3s0: adding route to fd0d:fe46:8ce8::/64
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fef9:c0aa
wlp3s0: adding address 2a00:1508:af9:c000:1fcc:2637:9217:35bc/64
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fe0d:feaa
wlp3s0: adding address fd0d:fe46:8ce8:0:ddf8:77ea:8ed9:4ab0/64
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fe0d:feaa
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fef9:c0aa
wlp3s0: fe80::a8aa:aaff:fe0d:feaa is reachable again
wlp3s0: pid -944010337 deleted route to 2a00:1508:af9:c000::/64
wlp3s0: pid -1665407864 deleted route to fd0d:fe46:8ce8::/64
wlp3s0: fe80::a8aa:aaff:fef9:c0aa is reachable again
wlp3s0: adding route to 2a00:1508:af9:c000::/64
wlp3s0: adding route to fd0d:fe46:8ce8::/64
wlp3s0: fe80::a8aa:aaff:fef9:c0aa is reachable again
wlp3s0: offered 10.249.119.136 from 10.249.0.1
wlp3s0: ignoring offer of 10.13.119.137 from 10.13.0.1
wlp3s0: NAK: wrong server-ID from 10.13.0.1
wlp3s0: message: wrong server-ID
wlp3s0: soliciting a DHCP lease
wlp3s0: offered 10.13.119.137 from 10.13.0.1
wlp3s0: probing address 10.13.119.137/16
wlp3s0: leased 10.13.119.137 for 3600 seconds
wlp3s0: adding route to 10.13.0.0/16
wlp3s0: adding default route via 10.13.0.1
forked to background, child pid 158181

# ip r
default via 10.13.0.1 dev wlp3s0 proto dhcp src 10.13.119.137 metric 303 mtu 1350 
10.13.0.0/16 dev wlp3s0 proto dhcp scope link src 10.13.119.137 metric 303 mtu 1350 
# ping 10.13.0.1
PING 10.13.0.1 (10.13.0.1) 56(84) bytes of data.
^C
--- 10.13.0.1 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1004ms

repeating the request tend to stick to the previous configuration

# dhcpcd -k
sending signal ALRM to pid 158181
waiting for pid 158181 to exit

# dhcpcd
dev: loaded udev
wlp3s0: connected to Access Point `libremesh.org'
enp0s25: waiting for carrier
DUID 00:01:00:01:22:29:54:8f:54:ee:75:7a:c2:1f
wlp3s0: IAID ea:06:f3:6f
wlp3s0: soliciting a DHCP lease
wlp3s0: offered 10.13.119.137 from 10.13.0.1
wlp3s0: probing address 10.13.119.137/16
wlp3s0: soliciting an IPv6 router
wlp3s0: Router Advertisement from fe80::c24a:ff:fedd:6800
wlp3s0: advertised MTU 1500 is greater than link MTU 1350
wlp3s0: adding address 2a00:1508:af9:c000:1fcc:2637:9217:35bc/64
wlp3s0: adding route to 2a00:1508:af9:c000::/64
wlp3s0: adding default route via fe80::c24a:ff:fedd:6800
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fef9:c0aa
wlp3s0: adding address 2a00:1508:af9:c000:1fcc:2637:9217:35bc/64
wlp3s0: Router Advertisement from fe80::de9f:dbff:fe37:28a9
wlp3s0: advertised MTU 1500 is greater than link MTU 1350
wlp3s0: adding address fd0d:fe46:8ce8:0:ddf8:77ea:8ed9:4ab0/64
wlp3s0: adding route to fd0d:fe46:8ce8::/64
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fe0d:feaa
wlp3s0: adding address fd0d:fe46:8ce8:0:ddf8:77ea:8ed9:4ab0/64
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fef9:c0aa
wlp3s0: pid -221457259 deleted route to fd0d:fe46:8ce8::/64
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fe0d:feaa
wlp3s0: adding route to fd0d:fe46:8ce8::/64
wlp3s0: leased 10.13.119.137 for 3600 seconds
wlp3s0: adding route to 10.13.0.0/16
wlp3s0: adding default route via 10.13.0.1
forked to background, child pid 159568

and repeating a lot more, with a lot of luck, the other DHCP offer can be accepted and I can again reach the gateway

# dhcpcd -k
sending signal ALRM to pid 159568
waiting for pid 159568 to exit

# dhcpcd
dev: loaded udev
wlp3s0: connected to Access Point `libremesh.org'
enp0s25: waiting for carrier
DUID 00:01:00:01:22:29:54:8f:54:ee:75:7a:c2:1f
wlp3s0: IAID ea:06:f3:6f
wlp3s0: soliciting an IPv6 router
wlp3s0: Router Advertisement from fe80::c24a:ff:fedd:6800
wlp3s0: advertised MTU 1500 is greater than link MTU 1350
wlp3s0: adding address 2a00:1508:af9:c000:1fcc:2637:9217:35bc/64
wlp3s0: adding route to 2a00:1508:af9:c000::/64
wlp3s0: adding default route via fe80::c24a:ff:fedd:6800
wlp3s0: Router Advertisement from fe80::de9f:dbff:fe37:28a9
wlp3s0: advertised MTU 1500 is greater than link MTU 1350
wlp3s0: adding address fd0d:fe46:8ce8:0:ddf8:77ea:8ed9:4ab0/64
wlp3s0: adding route to fd0d:fe46:8ce8::/64
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fef9:c0aa
wlp3s0: adding address 2a00:1508:af9:c000:1fcc:2637:9217:35bc/64
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fe0d:feaa
wlp3s0: adding address fd0d:fe46:8ce8:0:ddf8:77ea:8ed9:4ab0/64
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fef9:c0aa
wlp3s0: Router Advertisement from fe80::a8aa:aaff:fe0d:feaa
wlp3s0: pid -965355403 deleted route to 2a00:1508:af9:c000::/64
wlp3s0: pid -1837816073 deleted route to fd0d:fe46:8ce8::/64
wlp3s0: soliciting a DHCP lease
wlp3s0: offered 10.249.119.136 from 10.249.0.1
wlp3s0: ignoring offer of 10.13.119.137 from 10.13.0.1
wlp3s0: probing address 10.249.119.136/16
wlp3s0: leased 10.249.119.136 for 3600 seconds
wlp3s0: adding route to 10.249.0.0/16
wlp3s0: adding default route via 10.249.0.1
forked to background, child pid 159941

# ip r
default via 10.249.0.1 dev wlp3s0 proto dhcp src 10.249.119.136 metric 303 mtu 1350 
10.249.0.0/16 dev wlp3s0 proto dhcp scope link src 10.249.119.136 metric 303 mtu 1350

[spiccinini]

Great debugging @ilario ! I think (but may be wrong) that anygw was implemented using a macvlan primarly for this purpose!

[ilario]

anygw was implemented using a macvlan primarly for this purpose

Interressttinngg, I have no idea of which advantages macvlan give, but I think that in no way this can fix the issue:
the issue is originated by the fact that the clients are sending their DHCP requests to a bridge (clients are connected on wlan*-ap* which is included in br-lan) so that all the other devices connected to the bridge (e.g. another router connected via ethernet on a LAN port) will receive the request and will answer.

[ilario]

I just confirmed that this is still happening with LibreMesh 2024.1-rc1 built over OpenWrt23.

This lack of filtering is generating a problem for the whole network if the LAN port of a non-LibreMesh router is connected to the LAN port of a LibreMesh router.
This has been described in @G10h4ck's email cited here.
Seems that is more common than expected. For example, when one uses a commercial router as a dumb cheap AP, disabling its DHCP server and connecting it LAN-LAN to a LibreMesh router on the roof making the mesh connection, the problematic situation can arise if the commercial router gets reset and re-activate the DHCP server. Its nonsensical DHCP offers would reach the whole network, causing a race condition, where the client takes the first received DHCP offer.

I tested the following setups:

(laptop)ethernet -wire- LANport(LibreMesh router)LANport -wire- commercial router

(laptop)wifi -wireless- AP(LibreMesh router)LANport -wire- commercial router

and in both cases the laptop received both DHCP offers.

A firewall rule blocking this should be included in lime-proto-anygw (the package that activates LibreMesh's DHCP server).

[ilario]

When the laptop connects via wifi, as it is a matter of speed, the DHCP offer which usually arrives first is the one from the LibreMesh node.

When the laptop connects via ethernet, it is very likely that the "wrong" answer arrives first.

[henmohr]

Hey. DHCP rogue is really annoying and hard to debug. These cheap devices are terribles!
I think adding something like this rule would do the work:

table bridge filter {
    chain prerouting {
        type filter hook prerouting priority 0;
        policy accept;
    }

    chain drop_ra_dhcp {
        icmpv6 type nd-router-advert drop
        ip6 version 6 udp sport 547 drop
        ip  version 4 udp sport 67 drop
    }
}

[ilario]

@henmohr can you prepare a pull request with this code, so that we can easily test it?
I suppose those rules should be included in this file:
https://github.com/libremesh/lime-packages/blob/master/packages/lime-proto-anygw/files/usr/share/lime/lime-proto-anygw_anycast-rules.nft