Closed
Description
Description
Routes that are programmed in the kernel using a route-map source address are not re-installed after the source address is removed and readded to a loopback interface.
Version
Reproduced on a very recent master branch:
topotests mmallin$ git log -n 1
commit 6c3e1e4fd636e6cdc9a14c4d19de26784a6b890e (HEAD -> master, origin/master, origin/HEAD)
Merge: 76e0c8459 b94e72048
Author: Donald Sharp <donaldsharp72@gmail.com>
Date: Tue Dec 17 14:44:18 2024 -0500
Merge pull request #17654 from mjstapp/remove_bgp_pRN
bgpd: remove unneeded printfrr reg for pRN
torm11# show version
FRRouting 10.3-dev-my-manual-build (torm11) on Linux(6.8.0-45-generic).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
'--prefix=/usr' '--localstatedir=/var/run/frr' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc/frr' '--enable-vtysh' '--enable-pimd' '--enable-sharpd' '--enable-multipath=64' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-snmp=agentx' '--enable-scripting' '--with-pkg-extra-version=-my-manual-build'
This originally seen on SONiC 202305:
leaf0# show ver
FRRouting 8.5.1 (leaf0) on Linux(5.10.0-23-2-amd64).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--localstatedir=/var/run/frr' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc/frr' '--with-vtysh-pager=/usr/bin/pager' '--libdir=/usr/lib/x86_64-linux-gnu/frr' '--with-moduledir=/usr/lib/x86_64-linux-gnu/frr/modules' '--disable-dependency-tracking' '--disable-rpki' '--disable-scripting' '--enable-pim6d' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-protobuf' '--disable-zeromq' '--enable-ospfapi' '--enable-bgp-vnc' '--enable-multipath=256' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' 'build_alias=x86_64-linux-gnu' 'PYTHON=python3'
How to reproduce
- Checkout the following branch for a topotest that has all the appropriate configuration:
git clone https://github.com/mikemallin/frr.git -b dev/bgp-evpn-mh-overlay-peering
NOTE: This is incomplete and should only be used for launching a topology with configuration - Launch the topotest:
sudo -E pytest -s --pause --vtysh=torm11 --shell=torm11 bgp_evpn_mh_overlay_peering - Observe the following routes installed in the kernel on torm11:
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip route
2.2.2.2 nhid 33 proto bgp src 1.1.1.1 metric 20
nexthop via 192.168.5.1 dev torm11-eth1 weight 1
nexthop via 192.168.1.1 dev torm11-eth0 weight 1
3.3.3.3 nhid 33 proto bgp src 1.1.1.1 metric 20
nexthop via 192.168.5.1 dev torm11-eth1 weight 1
nexthop via 192.168.1.1 dev torm11-eth0 weight 1
4.4.4.4 nhid 33 proto bgp src 1.1.1.1 metric 20
nexthop via 192.168.5.1 dev torm11-eth1 weight 1
nexthop via 192.168.1.1 dev torm11-eth0 weight 1
10.10.10.10 nhid 34 via 192.168.1.1 dev torm11-eth0 proto bgp src 1.1.1.1 metric 20
20.20.20.20 nhid 32 via 192.168.5.1 dev torm11-eth1 proto bgp src 1.1.1.1 metric 20
<snip>
- Observe the same routes in zebra:
torm11# show ip route
Codes: K - kernel route, C - connected, L - local, S - static,
R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric, t - Table-Direct,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
IPv4 unicast VRF default:
L * 1.1.1.1/32 is directly connected, lo, weight 1, 00:14:15
C>* 1.1.1.1/32 is directly connected, lo, weight 1, 00:14:15
B>* 2.2.2.2/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:14:02
* via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:14:02
B>* 3.3.3.3/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:14:02
* via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:14:02
B>* 4.4.4.4/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:14:02
* via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:14:02
B>* 10.10.10.10/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:14:03
B>* 20.20.20.20/32 [20/0] via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:14:02
- Remove the source address
1.1.1.1/32from interfaceloontorm11
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip addr del 1.1.1.1/32 dev lo
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip addr show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.100.15/32 brd 192.168.100.15 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
- Observe the routes from above in the kernel are no longer there:
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip route
45.0.0.0/24 dev vlan1000 proto kernel scope link src 45.0.0.2
45.0.0.0/24 dev vlan1000-v0 proto kernel scope link src 45.0.0.1
192.168.1.0/24 dev torm11-eth0 proto kernel scope link src 192.168.1.2
192.168.5.0/24 dev torm11-eth1 proto kernel scope link src 192.168.5.2
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11#
- The routes are still in zebra
torm11# show ip route
Codes: K - kernel route, C - connected, L - local, S - static,
R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric, t - Table-Direct,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
IPv4 unicast VRF default:
B>r 2.2.2.2/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:16:18
r via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:16:18
B>r 3.3.3.3/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:16:18
r via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:16:18
B>r 4.4.4.4/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:16:18
r via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:16:18
B>r 10.10.10.10/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:16:19
B>r 20.20.20.20/32 [20/0] via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:16:18
<snip>
- Re-add the address
1.1.1.1/32to interfacelo:
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip addr add 1.1.1.1/32 dev lo
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip addr show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.100.15/32 brd 192.168.100.15 scope global lo
valid_lft forever preferred_lft forever
inet 1.1.1.1/32 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
- The routes are not re-installed in the kernel (after waiting 5m)
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip route
45.0.0.0/24 dev vlan1000 proto kernel scope link src 45.0.0.2
45.0.0.0/24 dev vlan1000-v0 proto kernel scope link src 45.0.0.1
192.168.1.0/24 dev torm11-eth0 proto kernel scope link src 192.168.1.2
192.168.5.0/24 dev torm11-eth1 proto kernel scope link src 192.168.5.2
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11#
torm11# show bgp neigh 10.10.10.10
BGP neighbor is 10.10.10.10, remote AS 65101, local AS 65002, external link
Local Role: undefined
Remote Role: undefined
Hostname: leaf1
Member of peer-group OVERLAY for session parameters
BGP version 4, remote router ID 0.0.0.0, local router ID 192.168.100.15
BGP state = Active
Last read 00:05:38, Last write 00:05:29
Hold time is 10 seconds, keepalive interval is 3 seconds
Configured hold time is 10 seconds, keepalive interval is 3 seconds
Configured tcp-mss is 0, synced tcp-mss is 0
Configured conditional advertisements interval is 60 seconds
Graceful restart information:
Local GR Mode: Helper*
Remote GR Mode: NotApplicable
R bit: False
N bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 120
Configured LLGR Stale Path Time(sec): 0
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 2 2
Notifications: 2 0
Updates: 69 62
Keepalives: 354 348
Route Refresh: 0 0
Capability: 0 0
Total: 427 412
Minimum time between advertisement runs is 0 seconds
Update source is lo
For address family: L2VPN EVPN
OVERLAY peer-group member
Not part of any update group
NEXT_HOP is propagated unchanged to this neighbor
Community attribute sent to this neighbor(all)
advertise-all-vni
0 accepted prefixes
Connections established 2; dropped 2
Last reset 00:05:28, No path to specified Neighbor (FRRouting/10.3-dev-my-manual-build)
External BGP neighbor may be up to 255 hops away.
Local host: 1.1.1.1, Local port: 38851
Foreign host: 10.10.10.10, Foreign port: 179
Nexthop: 1.1.1.1
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
BGP Connect Retry Timer in Seconds: 10
Next connect timer due in 2 seconds
Read thread: off Write thread: off FD used: -1
- Zebra still sees the routes as rejected and doesn't re-install them
torm11# show ip route
Codes: K - kernel route, C - connected, L - local, S - static,
R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric, t - Table-Direct,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
IPv4 unicast VRF default:
L * 1.1.1.1/32 is directly connected, lo, weight 1, 00:04:11
C>* 1.1.1.1/32 is directly connected, lo, weight 1, 00:04:11
B>r 2.2.2.2/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:21:39
r via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:21:39
B>r 3.3.3.3/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:21:39
r via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:21:39
B>r 4.4.4.4/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:21:39
r via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:21:39
B>r 10.10.10.10/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:21:40
B>r 20.20.20.20/32 [20/0] via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:21:39
- The system can be recovered by issuing
clear bgp *
Expected behavior
After step 8, I was expecting the routes to be re-installed in the kernel like in step 3.
Actual behavior
See above. The routes that have the source address set are not re-installed in the kernel.
Additional context
No response
Checklist
- I have searched the open issues for this bug.
- I have not included sensitive information in this report.