What BGP PIC is
When a failure hits a router holding hundreds of thousands of BGP routes, BGP re-runs best-path selection for the affected routes one by one and writes each result into the routing table and the forwarding table (FIB) one by one. The time to recovery is proportional to the number of routes, and on a large table it runs to tens of seconds or minutes.
PIC (Prefix Independent Convergence) is a forwarding-plane mechanism that makes this time independent of the number of routes. Instead of waiting for BGP to recompute, the FIB switches the forwarding first, using only what it already holds. The mechanism is described in the IETF draft draft-ietf-rtgwg-bgp-pic (Informational).
In a network comprising thousands of BGP peers exchanging millions of routes, it is desirable to restore traffic after failure in a time period that does not depend on the number of BGP prefixes.
PIC does not change the BGP protocol. The messages exchanged with peers and the result of best-path selection are the same. What changes is how the FIB inside the router is organized, and how it is rewritten on a failure.
The hierarchical FIB: the foundation of PIC
A BGP route is a recursive route that is reached through a next hop (the BGP NEXT_HOP). To forward, the router must resolve it in stages: destination → BGP next hop → the IGP route to that next hop → the outgoing interface.
The hierarchical FIB that supports PIC turns these stages into the structure of the FIB itself. If tens of thousands of BGP routes point at the same next hop, there is a single entry for that next hop, and the BGP routes refer to it.
| Level | Entry | Count | Changed on a failure |
|---|---|---|---|
| BGP route | Destination prefix → reference to a next hop | One per route | Not changed |
| BGP next hop | Next hop → reference to an IGP route (primary / backup) | One per next hop (tens to hundreds) | This is what gets rewritten |
| IGP route / adjacency | Outgoing interface and the next router | One per link | Changed by IGP convergence |
Since only the shared next-hop entry is rewritten on a failure, the rewrite happens once no matter how many routes refer to it. That is what “independent of the number of prefixes” really means.
A forwarding plane with shared, hierarchical forwarding chains with maximal object reuse can reroute a large number of destinations by modifying only a small set of shared objects.
PIC Core and PIC Edge
Which information the switch relies on depends on where the failure is.
| Kind | Failure | BGP next hop | What the FIB does |
|---|---|---|---|
| PIC Core | A link or router inside the AS (the core). The BGP next hop itself is still reachable by another path | Unchanged | When the IGP reconverges, the IGP route the next-hop entry points at is replaced. BGP routes are not touched |
| PIC Edge | The BGP next hop (the border router) itself, or the link beyond the border router | Becomes unreachable | Switches to the backup path (another next hop) that was installed in the FIB in advance |
PIC Core needs nothing beyond the hierarchical FIB; there is no configuration. PIC Edge needs the backup path installed in the FIB in advance. The backup path is the route that came second in best-path selection (the same prefix received from another next hop).
There are two triggers for a PIC Edge switch.
| Trigger | Example | Detection |
|---|---|---|
| The router’s own link fails | The eBGP link of border router R1 goes down | Interface down (immediate) |
| The next hop disappears | Seen from internal router R3, border router R1 stops | 10.0.0.1/32 disappears from the IGP (next-hop tracking; it takes the IGP’s detection time) |
In both cases BGP afterwards withdraws the route and re-selects the best path in the normal way. PIC only repairs forwarding in the meantime; the final state of the routes is decided by BGP convergence.
Configuration and verification on IOS XR
| Item | Detail |
|---|---|
route-policy <name> / set path-selection backup 1 install | Installs one backup path in the FIB (install). Adding advertise also advertises it to peers with ADD-PATH (explained in ADD-PATH) |
additional-paths selection route-policy <name> | Applies the policy above, per address-family |
show bgp <prefix> | The path chosen as backup is marked backup |
show route <prefix> | The second next hop is listed as BGP backup path |
show cef <prefix> | The backup via is marked backup. The proof that it is in the FIB |
The PIC Core side has no configuration. In show cef <prefix>, the line via <next hop>/32, … recursive is the hierarchical FIB’s reference, and looking up that next hop with show cef 10.0.0.1/32 shows the IGP route and the outgoing interface.
Verification on real devices
Verified with five XRd 26.1.1 routers. XRd is the container form of IOS XR (XRd Control Plane), and forwarding is done in software rather than in the intended hardware. This verification also serves to confirm whether XRd’s FIB implements PIC.
- In AS 65001, R1 and R2 are the border routers and R3 is internal (iBGP full mesh,
next-hop-self, OSPF area 0). R1 and R2 are directly connected - R4 in AS 65002 connects to both R1 and R2 over eBGP and advertises 2,002 routes (Loopback1
192.168.4.0/24and Loopback2100.63.255.0/24, plus 2,000 static routes to Null0 viaredistribute static). The static routes are filler to show that recovery does not depend on the number of routes;192.168.4.0/24is the last prefix in the table and100.63.255.0/24the first - R5 in AS 65003 is the traffic source. Traffic from R5 to R4 goes R3 → R1 → R4 (R3 breaks the tie between via R1 and via R2 by Router ID). The return direction (R4 → R5) is pinned via R2 (
LOCAL_PREF 200on the routes from R2, inbound at R4), so that the return traffic never uses the R1-R4 link that the failures hit and only the forward switch is measured - Outbound and inbound policies have a different name per neighbor (all just
pass); every neighbor hassoft-reconfiguration inbound always
There are two observation points.
| Observation point | Best | Backup | Kind verified |
|---|---|---|---|
| R1 | Directly connected eBGP (R4, 10.1.4.4) | iBGP from R2 (10.0.0.2, over the direct R1-R2 link) | PIC Edge (own link failure) |
| R3 | iBGP from R1 (10.0.0.1) | iBGP from R2 (10.0.0.2) | PIC Edge (next hop lost) and PIC Core |
Loss during a failure is measured with pings from R5 to 192.168.4.1 (the last prefix in the table) and 100.63.255.1 (the first) at 100 ms intervals, and from the capture on the R3-R5 link as the difference between “the time of the last reply” and “the time of the next reply”.
Overview of the verification
| STEP | Change | What to check |
|---|---|---|
| 0 | Initial state (no PIC) | R1 holds two routes but only one in the FIB. The hierarchy in show cef |
| 1 | R1-R4 link down (no PIC) | Loss until R1 finishes processing 2,000 withdrawals (the baseline) |
| 2 | Restore | |
| 3 | additional-paths selection (backup 1 install) on R1 and R3 | The backup marks and the backup path in the FIB. The advertisement to peers is unchanged |
| 4 | R1-R4 link down (with PIC) | The same failure with a shorter loss |
| 5 | Restore | |
| 6 | Stop R1 (PIC Edge, next hop lost) | The moment the IGP on R3 loses 10.0.0.1/32, every route moves to R2 |
| 7 | Start R1 | |
| 8 | R3-R1 link down (PIC Core) | BGP unchanged (no UPDATE); recovery takes only the IGP convergence |
| 9 | Restore (final state) |
All times in the show output and the captures below are UTC (matching the routers’ syslog).
STEP 0: Initial state (no PIC)
R1 holds two routes to R4, but only one is in the FIB. In show bgp the eBGP route from R4 is best, and the iBGP route from R2 is only listed as a candidate with no mark.
RP/0/RP0/CPU0:R1#show bgp 192.168.4.0/24
Tue Sep 15 10:52:02.516 UTC
BGP routing table entry for 192.168.4.0/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 20032 20032
Last Modified: Sep 15 10:50:35.023 for 00:01:27
Paths: (2 available, best #2)
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.0.0.3 10.0.0.2
Path #1: Received by speaker 0
Not advertised to any peer
65002, (received & used)
10.0.0.2 (metric 2) from 10.0.0.2 (10.0.0.2)
Origin IGP, metric 0, localpref 100, valid, internal
Received Path ID 0, Local Path ID 0, version 0
Path #2: Received by speaker 0
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.0.0.3 10.0.0.2
65002, (received & used)
10.1.4.4 from 10.1.4.4 (10.0.0.4)
Origin IGP, metric 0, localpref 100, valid, external, best, group-best
Received Path ID 0, Local Path ID 1, version 18029
Origin-AS validity: (disabled)Only the eBGP route is in the routing table and the FIB.
RP/0/RP0/CPU0:R1#show route 192.168.4.0/24
Tue Sep 15 10:52:03.832 UTC
Routing entry for 192.168.4.0/24
Known via "bgp 65001", distance 20, metric 0
Tag 65002, type external
Installed Sep 15 10:50:35.697 for 00:01:28
Routing Descriptor Blocks
10.1.4.4, from 10.1.4.4, BGP external
Route metric is 0, Wt is 1
No advertising protos. RP/0/RP0/CPU0:R1#show cef 192.168.4.0/24
Tue Sep 15 10:52:04.970 UTC
192.168.4.0/24, version 42088, internal 0x5000001 0x40 (ptr 0x8851a970) [1], 0x0 (0x0), 0x0 (0x0)
Updated Sep 15 10:50:35.803
local adjacency to GigabitEthernet0/0/0/2
Prefix Len 24, traffic index 0, precedence n/a, priority 4
gateway array (0x89726530) reference count 2002, flags 0x2010, source rib (7), 0 backups
[1 type 3 flags 0x48501 (0x897c9118) ext 0x0 (0x0) (collapsed)]
LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]
gateway array update type-time 1 Sep 15 10:50:35.600
LDI Update time Sep 15 10:50:35.601
via 10.1.4.4/32, 2 dependencies, recursive, bgp-ext [flags 0x6020]
path-idx 0 NHID 0x0 [0x8851acf0 0x0]
next hop 10.1.4.4/32 via 10.1.4.4/32
Load distribution: 0 (refcount 1)
Hash OK Interface Address
0 Y GigabitEthernet0/0/0/2 10.1.4.4 On R3 the shape of the hierarchical FIB is visible. show cef for 192.168.4.0/24 only refers to the next hop as via 10.0.0.1/32 … recursive; the OSPF route and the outgoing interface (GigabitEthernet0/0/0/0, toward R1) appear only when you look up that next hop, show cef 10.0.0.1/32. All 2,002 routes refer to this single entry.
RP/0/RP0/CPU0:R3#show cef 192.168.4.0/24
Tue Sep 15 10:54:09.733 UTC
192.168.4.0/24, version 42072, internal 0x5000001 0x40 (ptr 0x8852ac10) [1], 0x0 (0x0), 0x0 (0x0)
Updated Sep 15 10:50:44.004
local adjacency to GigabitEthernet0/0/0/0
Prefix Len 24, traffic index 0, precedence n/a, priority 4
gateway array (0x89618ae0) reference count 2003, flags 0x2010, source rib (7), 0 backups
[1 type 3 flags 0x48501 (0x896bbd50) ext 0x0 (0x0) (collapsed)]
LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]
gateway array update type-time 1 Sep 15 09:46:25.395
LDI Update time Sep 15 09:46:25.395
via 10.0.0.1/32, 2 dependencies, recursive [flags 0x6000]
path-idx 0 NHID 0x0 [0x8852b070 0x0]
next hop 10.0.0.1/32 via 10.0.0.1/32
Load distribution: 0 (refcount 1)
Hash OK Interface Address
0 Y GigabitEthernet0/0/0/0 10.1.3.1 RP/0/RP0/CPU0:R3#show cef 10.0.0.1/32
Tue Sep 15 10:54:10.220 UTC
10.0.0.1/32, version 12, internal 0x1000001 0x10 (ptr 0x8852b070) [2], 0x400 (0x8977fbd8), 0x0 (0x0)
Updated Sep 15 09:45:20.971
local adjacency to GigabitEthernet0/0/0/0
Prefix Len 32, traffic index 0, precedence n/a, priority 1
gateway array (0x89618df8) reference count 1, flags 0x0, source rib (7), 0 backups
[2 type 3 flags 0x8401 (0x896bbeb0) ext 0x0 (0x0)]
LW-LDI[type=3, refc=1, ptr=0x8977fbd8, sh-ldi=0x896bbeb0]
gateway array update type-time 1 Sep 15 09:45:20.971
LDI Update time Sep 15 09:45:20.971
LW-LDI-TS Sep 15 09:45:20.971
via 10.1.3.1/32, GigabitEthernet0/0/0/0, 8 dependencies, weight 0, class 0 [flags 0x0]
path-idx 0 NHID 0x3 [0x8a781460 0x0]
next hop 10.1.3.1/32
local adjacency
Load distribution: 0 (refcount 2)
Hash OK Interface Address
0 Y GigabitEthernet0/0/0/0 10.1.3.1 R1 receives 2,002 routes from R4. A traceroute from R5 goes R3 → R1 → R4.
RP/0/RP0/CPU0:R1#show bgp summary
Tue Sep 15 10:51:59.520 UTC
BGP router identifier 10.0.0.1, local AS number 65001
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000000 RD version: 20032
BGP main routing table version 20032
BGP NSR Initial initsync version 5 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs
BGP is operating in STANDALONE mode.
Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 20032 20032 20032 20032 20032 0
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
10.0.0.2 0 65001 76 114 20032 0 0 01:06:46 2003
10.0.0.3 0 65001 71 114 20032 0 0 01:06:34 2
10.1.4.4 0 65002 67 77 20032 0 0 00:03:10 2002RP/0/RP0/CPU0:R5#traceroute 192.168.4.1 source 192.168.5.1 probe 3 timeout 1 maxttl 6
Tue Sep 15 10:56:29.726 UTC
Type escape sequence to abort.
Tracing the route to 192.168.4.1
1 10.3.5.3 6 msec 5 msec 5 msec
2 10.1.3.1 10 msec 8 msec 8 msec
3 10.1.4.4 12 msec * 15 msec STEP 1: R1-R4 link down (no PIC)
Without PIC, forwarding stops until R1 withdraws the routes in BGP and R3 switches over. Shutting down R1’s Gi0/0/0/2 (the R4 side is shut as well) makes R1 detect that the eBGP session is down.
RP/0/RP0/CPU0:R1#show configuration commit changes last 1
Tue Sep 15 10:57:10.630 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
interface GigabitEthernet0/0/0/2
shutdown
!
endRP/0/RP0/CPU0:R1#show logging start Sep 15 10:52:36
Tue Sep 15 11:00:53.087 UTC
Time Zone UTC, DST disabled
Syslog logging: enabled (0 messages dropped, 0 flushes, 0 overruns)
Console logging: Disabled
Monitor logging: level debugging, 0 messages logged
Trap logging: level informational, 0 messages logged
Buffer logging: level debugging, 294 messages logged
Log Buffer (2097152 bytes):
RP/0/RP0/CPU0:Sep 15 10:52:36.327 UTC: logger[68357]: %OS-SYSLOG-6-LOG_INFO : informational STEP1-BEGIN
RP/0/RP0/CPU0:Sep 15 10:52:36.715 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[15051]: Received disconnect from 10.100.3.1 port 64078:11: disconnected by user
RP/0/RP0/CPU0:Sep 15 10:52:36.715 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[15051]: Disconnected from user cisco 10.100.3.1 port 64078
RP/0/RP0/CPU0:Sep 15 10:57:05.629 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[15455]: Accepted authentication/pam for cisco from 10.100.3.1 port 64331 ssh2
RP/0/RP0/CPU0:Sep 15 10:57:09.698 UTC: ifmgr[155]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/2, changed state to Administratively Down
RP/0/RP0/CPU0:Sep 15 10:57:09.700 UTC: bgp[1084]: %ROUTING-BGP-5-ADJCHANGE : neighbor 10.1.4.4 Down - Admin. shutdown (CEASE notification sent - administrative shutdown) (VRF: default) (AS: 65002)
RP/0/RP0/CPU0:Sep 15 10:57:10.115 UTC: config[68757]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user 'cisco'. Use 'show configuration commit changes 1000000009' to view the changes.
RP/0/RP0/CPU0:Sep 15 10:57:10.394 UTC: config[68757]: %MGBL-SYS-5-CONFIG_I : Configured from console by cisco on vty0 (10.100.3.1)
RP/0/RP0/CPU0:Sep 15 10:57:12.214 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[15460]: Received disconnect from 10.100.3.1 port 64331:11: disconnected by user
RP/0/RP0/CPU0:Sep 15 10:57:12.214 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[15460]: Disconnected from user cisco 10.100.3.1 port 64331
RP/0/RP0/CPU0:Sep 15 11:00:32.241 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[15802]: Accepted authentication/pam for cisco from 10.100.3.1 port 64441 ssh2
RP/0/RP0/CPU0:Sep 15 11:00:49.893 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[15807]: Received disconnect from 10.100.3.1 port 64441:11: disconnected by user
RP/0/RP0/CPU0:Sep 15 11:00:49.893 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[15807]: Disconnected from user cisco 10.100.3.1 port 64441
RP/0/RP0/CPU0:Sep 15 11:00:50.697 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[16030]: Accepted authentication/pam for cisco from 10.100.3.1 port 64453 ssh2 R1 has no backup, so it drops the packets arriving from R3. The capture on the R1-R2 link contains not a single rerouted packet (zero matches for the filter icmp.type==8).
$ tshark -r bgp-pic-step1-r1r2.pcap -t ud -Y 'icmp.type==8'About 0.5 s after the link went down (10:57:10.189), R1 starts sending the 2,000 withdrawals (WITHDRAW) to R3, packed into 7 UPDATEs and finished within 15 ms. Replies come back at 10:57:10.804, when R3 has processed them and switched to the path via R2. The gap is 1.19 s (to 192.168.4.1) and 1.11 s (to 100.63.255.1); the pings at 100 ms intervals lost 1 of 900.
$ tshark -r bgp-pic-step1-r1r3.pcap -t ud -Y 'bgp.type==2'
294 2026-09-15 10:57:10.189326Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message
295 2026-09-15 10:57:10.189742Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message
297 2026-09-15 10:57:10.197593Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message
298 2026-09-15 10:57:10.197978Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message
299 2026-09-15 10:57:10.198319Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message, UPDATE Message
300 2026-09-15 10:57:10.198540Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message
303 2026-09-15 10:57:10.204688Z 10.0.0.1 → 10.0.0.3 BGP 829 UPDATE Message, UPDATE MessageRP/0/RP0/CPU0:R5#ping 192.168.4.1 source 192.168.5.1 count 900 interval 100 timeout 1
Tue Sep 15 10:56:51.885 UTC
Type escape sequence to abort.
Sending 900, 100-byte ICMP Echos to 192.168.4.1 timeout is 1 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 99 percent (899/900), round-trip min/avg/max = 9/13/93 msRP/0/RP0/CPU0:R5#ping 100.63.255.1 source 192.168.5.1 count 900 interval 100 timeout 1
Tue Sep 15 10:56:56.483 UTC
Type escape sequence to abort.
Sending 900, 100-byte ICMP Echos to 100.63.255.1 timeout is 1 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 99 percent (899/900), round-trip min/avg/max = 8/14/85 msR3 is left with the single route via R2.
RP/0/RP0/CPU0:R3#show bgp 192.168.4.0/24
Tue Sep 15 11:03:27.534 UTC
BGP routing table entry for 192.168.4.0/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 22033 22033
Last Modified: Sep 15 10:57:10.023 for 00:06:17
Paths: (1 available, best #1)
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.3.5.5
Path #1: Received by speaker 0
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.3.5.5
65002, (received & used)
10.0.0.2 (metric 2) from 10.0.0.2 (10.0.0.2)
Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
Received Path ID 0, Local Path ID 1, version 22033STEP 2: Restore
Both ends are set to no shutdown, back to the state of STEP 0.
STEP 3: additional-paths selection (backup 1 install) on R1 and R3
The backup path is installed in the FIB. The configuration is a route policy and one line under address-family.
RP/0/RP0/CPU0:R1#show configuration commit changes last 1
Tue Sep 15 11:13:06.000 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
route-policy PIC
set path-selection backup 1 install
end-policy
!
router bgp 65001
address-family ipv4 unicast
additional-paths selection route-policy PIC
!
!
endRP/0/RP0/CPU0:R3#show configuration commit changes last 1
Tue Sep 15 11:13:13.635 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
route-policy PIC
set path-selection backup 1 install
end-policy
!
router bgp 65001
address-family ipv4 unicast
additional-paths selection route-policy PIC
!
!
endOn R1 the iBGP route from R2 is now marked backup (and add-path). The best path is unchanged.
RP/0/RP0/CPU0:R1#show bgp 192.168.4.0/24
Tue Sep 15 11:15:24.103 UTC
BGP routing table entry for 192.168.4.0/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 26038 26038
Last Modified: Sep 15 11:13:07.023 for 00:02:17
Paths: (2 available, best #2)
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.0.0.3 10.0.0.2
Path #1: Received by speaker 0
Not advertised to any peer
65002, (received & used)
10.0.0.2 (metric 2) from 10.0.0.2 (10.0.0.2)
Origin IGP, metric 0, localpref 100, valid, internal, backup, add-path
Received Path ID 0, Local Path ID 4, version 26038
Path #2: Received by speaker 0
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.0.0.3 10.0.0.2
65002, (received & used)
10.1.4.4 from 10.1.4.4 (10.0.0.4)
Origin IGP, metric 0, localpref 100, valid, external, best, group-best
Received Path ID 0, Local Path ID 1, version 24035
Origin-AS validity: (disabled)The routing table gains Number of pic paths 1 and a BGP backup path, and the FIB gains a via marked backup [flags 0x6100]. Load distribution is still the single 0: forwarding still uses only the best path.
RP/0/RP0/CPU0:R1#show route 192.168.4.0/24
Tue Sep 15 11:15:25.392 UTC
Routing entry for 192.168.4.0/24
Known via "bgp 65001", distance 20, metric 0
Tag 65002
Number of pic paths 1 , type internal and external
Installed Sep 15 11:13:07.487 for 00:02:17
Routing Descriptor Blocks
10.0.0.2, from 10.0.0.2, BGP backup path
Route metric is 0, Wt is 1
10.1.4.4, from 10.1.4.4, BGP external
Route metric is 0, Wt is 1
No advertising protos. RP/0/RP0/CPU0:R1#show cef 192.168.4.0/24
Tue Sep 15 11:15:26.445 UTC
192.168.4.0/24, version 56108, internal 0x5000001 0x40 (ptr 0x8851a970) [1], 0x0 (0x0), 0x0 (0x0)
Updated Sep 15 11:13:07.585
local adjacency to GigabitEthernet0/0/0/2
Prefix Len 24, traffic index 0, precedence n/a, priority 4
gateway array (0x897252a0) reference count 2002, flags 0x102010, source rib (7), 0 backups
[1 type 3 flags 0x48501 (0x897c8ae8) ext 0x0 (0x0) (collapsed)]
LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]
gateway array update type-time 1 Sep 15 11:13:07.294
LDI Update time Sep 15 11:13:07.391
via 10.0.0.2/32, 3 dependencies, recursive, backup [flags 0x6100]
path-idx 0 NHID 0x0 [0x8851b150 0x0]
next hop 10.0.0.2/32 via 10.0.0.2/32
via 10.1.4.4/32, 2 dependencies, recursive, bgp-ext [flags 0x6020]
path-idx 1 NHID 0x0 [0x8851acf0 0x0], Internal 0x8abcb0a0
next hop 10.1.4.4/32 via 10.1.4.4/32
Weight distribution:
slot 0, weight 1, normalized_weight 1, class 0
Load distribution: 0 (refcount 1)
Hash OK Interface Address
0 Y GigabitEthernet0/0/0/2 10.1.4.4 R3 is the same: the iBGP route via R2 becomes the backup.
RP/0/RP0/CPU0:R3#show bgp 192.168.4.0/24
Tue Sep 15 11:17:31.451 UTC
BGP routing table entry for 192.168.4.0/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 26038 26038
Last Modified: Sep 15 11:13:15.023 for 00:04:16
Paths: (2 available, best #1)
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.3.5.5
Path #1: Received by speaker 0
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.3.5.5
65002, (received & used)
10.0.0.1 (metric 2) from 10.0.0.1 (10.0.0.1)
Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
Received Path ID 0, Local Path ID 1, version 24036
Path #2: Received by speaker 0
Not advertised to any peer
65002, (received & used)
10.0.0.2 (metric 2) from 10.0.0.2 (10.0.0.2)
Origin IGP, metric 0, localpref 100, valid, internal, backup, add-path
Received Path ID 0, Local Path ID 4, version 26038RP/0/RP0/CPU0:R3#show route 192.168.4.0/24
Tue Sep 15 11:17:32.687 UTC
Routing entry for 192.168.4.0/24
Known via "bgp 65001", distance 200, metric 0
Tag 65002
Number of pic paths 1 , type internal
Installed Sep 15 11:13:15.190 for 00:04:17
Routing Descriptor Blocks
10.0.0.1, from 10.0.0.1
Route metric is 0, Wt is 1
10.0.0.2, from 10.0.0.2, BGP backup path
Route metric is 0, Wt is 1
No advertising protos. RP/0/RP0/CPU0:R3#show cef 192.168.4.0/24
Tue Sep 15 11:17:33.979 UTC
192.168.4.0/24, version 56086, internal 0x5000001 0x40 (ptr 0x8852ac10) [1], 0x0 (0x0), 0x0 (0x0)
Updated Sep 15 11:13:15.296
local adjacency to GigabitEthernet0/0/0/0
Prefix Len 24, traffic index 0, precedence n/a, priority 4
gateway array (0x89619740) reference count 2002, flags 0x102010, source rib (7), 0 backups
[1 type 3 flags 0x48501 (0x896bc1c8) ext 0x0 (0x0) (collapsed)]
LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]
gateway array update type-time 1 Sep 15 11:13:14.983
LDI Update time Sep 15 11:13:15.000
via 10.0.0.1/32, 3 dependencies, recursive [flags 0x6000]
path-idx 0 NHID 0x0 [0x8852b070 0x0], Internal 0x8abcb0a0
next hop 10.0.0.1/32 via 10.0.0.1/32
via 10.0.0.2/32, 3 dependencies, recursive, backup [flags 0x6100]
path-idx 1 NHID 0x0 [0x8852af90 0x0]
next hop 10.0.0.2/32 via 10.0.0.2/32
Weight distribution:
slot 0, weight 1, normalized_weight 1, class 0
Load distribution: 0 (refcount 1)
Hash OK Interface Address
0 Y GigabitEthernet0/0/0/0 10.1.3.1 Nothing is sent to peers. The capture on the R1-R3 link holds only KEEPALIVEs.
$ tshark -r bgp-pic-step3-r1r3.pcap -t ud -Y 'bgp'
2 2026-09-15 11:13:05.308445Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
8 2026-09-15 11:13:25.503555Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
19 2026-09-15 11:14:05.311275Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
25 2026-09-15 11:14:25.503854Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
35 2026-09-15 11:15:05.309177Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
41 2026-09-15 11:15:25.503793Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
51 2026-09-15 11:16:05.309881Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
59 2026-09-15 11:16:25.504323Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
69 2026-09-15 11:17:05.310111Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
75 2026-09-15 11:17:25.504881Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
85 2026-09-15 11:18:05.310320Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
91 2026-09-15 11:18:25.505322Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
102 2026-09-15 11:19:05.310912Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
108 2026-09-15 11:19:25.505121Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
237 2026-09-15 11:20:05.310737Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE MessageSTEP 4: R1-R4 link down (with PIC)
The same failure, and this time no loss. All 900 pings to both destinations were answered, and the largest gap between replies in the capture is 0.17 s (the normal jitter, unrelated to the failure).
RP/0/RP0/CPU0:R5#ping 192.168.4.1 source 192.168.5.1 count 900 interval 100 timeout 1
Tue Sep 15 11:20:15.498 UTC
Type escape sequence to abort.
Sending 900, 100-byte ICMP Echos to 192.168.4.1 timeout is 1 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (900/900), round-trip min/avg/max = 9/17/81 msRP/0/RP0/CPU0:R5#ping 100.63.255.1 source 192.168.5.1 count 900 interval 100 timeout 1
Tue Sep 15 11:20:20.013 UTC
Type escape sequence to abort.
Sending 900, 100-byte ICMP Echos to 100.63.255.1 timeout is 1 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (900/900), round-trip min/avg/max = 9/16/76 msThe capture timestamps show what happened. R1’s interface goes down at 11:20:33.613, and 25 ms later (11:20:33.638) the packets from R5 appear on the R1-R2 link: R1’s FIB has switched to the backup path (R2).
RP/0/RP0/CPU0:R1#show logging start Sep 15 11:15:58
Tue Sep 15 11:24:14.025 UTC
Time Zone UTC, DST disabled
Syslog logging: enabled (0 messages dropped, 0 flushes, 0 overruns)
Console logging: Disabled
Monitor logging: level debugging, 0 messages logged
Trap logging: level informational, 0 messages logged
Buffer logging: level debugging, 364 messages logged
Log Buffer (2097152 bytes):
RP/0/RP0/CPU0:Sep 15 11:15:58.724 UTC: logger[67665]: %OS-SYSLOG-6-LOG_INFO : informational STEP4-BEGIN
RP/0/RP0/CPU0:Sep 15 11:15:59.192 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[18466]: Received disconnect from 10.100.3.1 port 65190:11: disconnected by user
RP/0/RP0/CPU0:Sep 15 11:15:59.192 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[18466]: Disconnected from user cisco 10.100.3.1 port 65190
RP/0/RP0/CPU0:Sep 15 11:20:29.443 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[18867]: Accepted authentication/pam for cisco from 10.100.3.1 port 65372 ssh2
RP/0/RP0/CPU0:Sep 15 11:20:33.613 UTC: ifmgr[155]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/2, changed state to Administratively Down
RP/0/RP0/CPU0:Sep 15 11:20:33.614 UTC: bgp[1084]: %ROUTING-BGP-5-ADJCHANGE : neighbor 10.1.4.4 Down - Admin. shutdown (CEASE notification sent - administrative shutdown) (VRF: default) (AS: 65002)
RP/0/RP0/CPU0:Sep 15 11:20:34.095 UTC: config[68074]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user 'cisco'. Use 'show configuration commit changes 1000000012' to view the changes.
RP/0/RP0/CPU0:Sep 15 11:20:34.305 UTC: config[68074]: %MGBL-SYS-5-CONFIG_I : Configured from console by cisco on vty0 (10.100.3.1)
RP/0/RP0/CPU0:Sep 15 11:20:36.138 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[18872]: Received disconnect from 10.100.3.1 port 65372:11: disconnected by user
RP/0/RP0/CPU0:Sep 15 11:20:36.138 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[18872]: Disconnected from user cisco 10.100.3.1 port 65372
RP/0/RP0/CPU0:Sep 15 11:23:54.506 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[19230]: Accepted authentication/pam for cisco from 10.100.3.1 port 65473 ssh2
RP/0/RP0/CPU0:Sep 15 11:24:10.995 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[19235]: Received disconnect from 10.100.3.1 port 65473:11: disconnected by user
RP/0/RP0/CPU0:Sep 15 11:24:10.995 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[19235]: Disconnected from user cisco 10.100.3.1 port 65473
RP/0/RP0/CPU0:Sep 15 11:24:11.907 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[19418]: Accepted authentication/pam for cisco from 10.100.3.1 port 65482 ssh2 $ tshark -r bgp-pic-step4-r1r2.pcap -t ud -Y 'icmp.type==8'
5 2026-09-15 11:20:33.638052Z 192.168.5.1 → 100.63.255.1 ICMP 114 Echo (ping) request id=0x483b, seq=124/31744, ttl=253
6 2026-09-15 11:20:33.686031Z 192.168.5.1 → 192.168.4.1 ICMP 114 Echo (ping) request id=0x47f7, seq=171/43776, ttl=253
7 2026-09-15 11:20:33.738905Z 192.168.5.1 → 100.63.255.1 ICMP 114 Echo (ping) request id=0x483b, seq=125/32000, ttl=253
8 2026-09-15 11:20:33.786051Z 192.168.5.1 → 192.168.4.1 ICMP 114 Echo (ping) request id=0x47f7, seq=172/44032, ttl=253
9 2026-09-15 11:20:33.886000Z 192.168.5.1 → 100.63.255.1 ICMP 114 Echo (ping) request id=0x483b, seq=126/32256, ttl=253
10 2026-09-15 11:20:33.886339Z 192.168.5.1 → 192.168.4.1 ICMP 114 Echo (ping) request id=0x47f7, seq=173/44288, ttl=253
11 2026-09-15 11:20:33.940047Z 192.168.5.1 → 100.63.255.1 ICMP 114 Echo (ping) request id=0x483b, seq=127/32512, ttl=253
12 2026-09-15 11:20:33.986276Z 192.168.5.1 → 192.168.4.1 ICMP 114 Echo (ping) request id=0x47f7, seq=174/44544, ttl=253
<snip>(以下 7 行)The BGP withdrawals come afterwards. R1 starts sending WITHDRAWs to R3 at 11:20:34.194, 0.58 s after the link went down. In STEP 1 packets were dropped during this time; with PIC the FIB has already rerouted them, so forwarding is not interrupted when BGP finishes withdrawing and R3 moves to the path via R2.
$ tshark -r bgp-pic-step4-r1r3.pcap -t ud -Y 'bgp.type==2'
314 2026-09-15 11:20:34.193733Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message
315 2026-09-15 11:20:34.193943Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message
317 2026-09-15 11:20:34.201938Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message
318 2026-09-15 11:20:34.202073Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message
319 2026-09-15 11:20:34.202178Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message, UPDATE Message
320 2026-09-15 11:20:34.202658Z 10.0.0.1 → 10.0.0.3 BGP 1294 UPDATE Message
323 2026-09-15 11:20:34.210916Z 10.0.0.1 → 10.0.0.3 BGP 829 UPDATE Message, UPDATE MessageSTEP 5: Restore
Both ends are set to no shutdown, back to the state of STEP 3.
STEP 6: Stop R1 (PIC Edge, next hop lost)
When the next hop 10.0.0.1 disappears as seen from R3, the switch to R2 happens the moment the IGP detects it. After R1 is stopped in CML, OSPF on R3 drops the adjacency with R1 when the dead interval (40 s) expires (11:36:27.680).
RP/0/RP0/CPU0:R3#show logging start Sep 15 11:33:25
Tue Sep 15 11:41:08.035 UTC
Time Zone UTC, DST disabled
Syslog logging: enabled (0 messages dropped, 0 flushes, 0 overruns)
Console logging: Disabled
Monitor logging: level debugging, 0 messages logged
Trap logging: level informational, 0 messages logged
Buffer logging: level debugging, 316 messages logged
Log Buffer (2097152 bytes):
RP/0/RP0/CPU0:Sep 15 11:33:26.088 UTC: logger[69019]: %OS-SYSLOG-6-LOG_INFO : informational STEP6-BEGIN
RP/0/RP0/CPU0:Sep 15 11:33:26.488 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[19820]: Received disconnect from 10.100.3.1 port 49531:11: disconnected by user
RP/0/RP0/CPU0:Sep 15 11:33:26.488 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[19820]: Disconnected from user cisco 10.100.3.1 port 49531
RP/0/RP0/CPU0:Sep 15 11:36:27.680 UTC: ospf[1035]: %ROUTING-OSPF-5-ADJCHG : Process 1, Nbr 10.0.0.1 on GigabitEthernet0/0/0/0 in area 0 from FULL to DOWN, Neighbor Down: dead timer expired, vrf default vrfid 0x60000000
RP/0/RP0/CPU0:Sep 15 11:38:36.759 UTC: bgp[1084]: %ROUTING-BGP-5-ADJCHANGE : neighbor 10.0.0.1 Down - BGP Notification sent, hold time expired (VRF: default) (AS: 65001)
RP/0/RP0/CPU0:Sep 15 11:40:45.905 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[20428]: Accepted authentication/pam for cisco from 10.100.3.1 port 49861 ssh2
RP/0/RP0/CPU0:Sep 15 11:41:05.189 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[20433]: Received disconnect from 10.100.3.1 port 49861:11: disconnected by user
RP/0/RP0/CPU0:Sep 15 11:41:05.189 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[20433]: Disconnected from user cisco 10.100.3.1 port 49861
RP/0/RP0/CPU0:Sep 15 11:41:06.007 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[20660]: Accepted authentication/pam for cisco from 10.100.3.1 port 49867 ssh2 Replies came back at 11:36:27.914, 0.23 s after the adjacency dropped. By contrast, BGP reinstalled the route via R2 in the routing table at 11:36:32 (5 s later), and the iBGP session with R1 went down on hold-time expiry at 11:38:36 (2 minutes later). The recovery of forwarding waited for neither.
RP/0/RP0/CPU0:R3#show route 192.168.4.0/24
Tue Sep 15 11:40:53.491 UTC
Routing entry for 192.168.4.0/24
Known via "bgp 65001", distance 200, metric 0
Tag 65002, type internal
Installed Sep 15 11:36:32.691 for 00:04:20
Routing Descriptor Blocks
10.0.0.2, from 10.0.0.2
Route metric is 0, Wt is 1
No advertising protos. RP/0/RP0/CPU0:R3#show bgp summary
Tue Sep 15 11:40:50.197 UTC
BGP router identifier 10.0.0.3, local AS number 65001
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000000 RD version: 32045
BGP main routing table version 32045
BGP NSR Initial initsync version 7 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs
BGP is operating in STANDALONE mode.
Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 32045 32045 32045 32045 32045 0
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
10.0.0.1 0 65001 183 119 0 0 0 00:02:13 Idle
10.0.0.2 0 65001 125 120 32045 0 0 01:55:24 2003
10.3.5.5 0 65003 119 143 32045 0 0 01:55:26 1The gap is 35 s, but that is the time OSPF needs to notice that R1 has stopped. The pings lost 35 of 900.
RP/0/RP0/CPU0:R5#ping 192.168.4.1 source 192.168.5.1 count 900 interval 100 timeout 1
Tue Sep 15 11:35:38.037 UTC
Type escape sequence to abort.
Sending 900, 100-byte ICMP Echos to 192.168.4.1 timeout is 1 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.
..................................!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 96 percent (865/900), round-trip min/avg/max = 9/14/81 msSTEP 7: Start R1
Once R1 has booted and converged, R3’s best path is via R1 again and the path via R2 is the backup again. R1’s PIC configuration is still in place.
RP/0/RP0/CPU0:R3#show bgp 192.168.4.0/24
Tue Sep 15 11:52:12.614 UTC
BGP routing table entry for 192.168.4.0/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 34047 34047
Last Modified: Sep 15 11:46:49.023 for 00:05:23
Paths: (2 available, best #1)
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.3.5.5
Path #1: Received by speaker 0
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.3.5.5
65002, (received & used)
10.0.0.1 (metric 2) from 10.0.0.1 (10.0.0.1)
Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
Received Path ID 0, Local Path ID 1, version 34047
Path #2: Received by speaker 0
Not advertised to any peer
65002, (received & used)
10.0.0.2 (metric 2) from 10.0.0.2 (10.0.0.2)
Origin IGP, metric 0, localpref 100, valid, internal, backup, add-path
Received Path ID 0, Local Path ID 6, version 34047STEP 8: R3-R1 link down (PIC Core)
The BGP next hop does not change; recovery takes only the IGP convergence. Shutting down R3’s Gi0/0/0/0 (toward R1; the R1 side is shut as well) makes OSPF drop the adjacency at once and replace the route to 10.0.0.1/32 with the one via R2 (10.2.3.2).
RP/0/RP0/CPU0:R3#show configuration commit changes last 1
Tue Sep 15 11:55:34.788 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
interface GigabitEthernet0/0/0/0
shutdown
!
endRP/0/RP0/CPU0:R3#show logging start Sep 15 11:52:45
Tue Sep 15 12:01:27.313 UTC
Time Zone UTC, DST disabled
Syslog logging: enabled (0 messages dropped, 0 flushes, 0 overruns)
Console logging: Disabled
Monitor logging: level debugging, 0 messages logged
Trap logging: level informational, 0 messages logged
Buffer logging: level debugging, 357 messages logged
Log Buffer (2097152 bytes):
RP/0/RP0/CPU0:Sep 15 11:52:45.283 UTC: logger[67286]: %OS-SYSLOG-6-LOG_INFO : informational STEP8-BEGIN
RP/0/RP0/CPU0:Sep 15 11:52:45.684 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[22184]: Received disconnect from 10.100.3.1 port 50302:11: disconnected by user
RP/0/RP0/CPU0:Sep 15 11:52:45.684 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[22184]: Disconnected from user cisco 10.100.3.1 port 50302
RP/0/RP0/CPU0:Sep 15 11:55:29.338 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[22468]: Accepted authentication/pam for cisco from 10.100.3.1 port 50417 ssh2
RP/0/RP0/CPU0:Sep 15 11:55:33.809 UTC: ifmgr[155]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/0, changed state to Administratively Down
RP/0/RP0/CPU0:Sep 15 11:55:33.884 UTC: ospf[1035]: %ROUTING-OSPF-5-ADJCHG : Process 1, Nbr 10.0.0.1 on GigabitEthernet0/0/0/0 in area 0 from FULL to DOWN, Neighbor Down: interface down or detached, vrf default vrfid 0x60000000
RP/0/RP0/CPU0:Sep 15 11:55:34.201 UTC: config[67579]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user 'cisco'. Use 'show configuration commit changes 1000000004' to view the changes.
RP/0/RP0/CPU0:Sep 15 11:55:34.403 UTC: config[67579]: %MGBL-SYS-5-CONFIG_I : Configured from console by cisco on vty0 (10.100.3.1)
RP/0/RP0/CPU0:Sep 15 11:55:36.521 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[22473]: Received disconnect from 10.100.3.1 port 50417:11: disconnected by user
RP/0/RP0/CPU0:Sep 15 11:55:36.521 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[22473]: Disconnected from user cisco 10.100.3.1 port 50417
RP/0/RP0/CPU0:Sep 15 12:01:02.445 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[22976]: Accepted authentication/pam for cisco from 10.100.3.1 port 50588 ssh2
RP/0/RP0/CPU0:Sep 15 12:01:24.094 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[22981]: Received disconnect from 10.100.3.1 port 50588:11: disconnected by user
RP/0/RP0/CPU0:Sep 15 12:01:24.094 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[22981]: Disconnected from user cisco 10.100.3.1 port 50588
RP/0/RP0/CPU0:Sep 15 12:01:24.920 UTC: ssh_syslog_proxy[1188]: %SECURITY-SSHD_SYSLOG_PRX-6-INFO_GENERAL : sshd[23182]: Accepted authentication/pam for cisco from 10.100.3.1 port 50609 ssh2 The only FIB entry rewritten is the one for the next hop 10.0.0.1/32 (Updated 11:55:34.001, 0.2 s after the link went down). The entry for 192.168.4.0/24 still refers to via 10.0.0.1/32; none of the 2,002 BGP routes was touched.
RP/0/RP0/CPU0:R3#show cef 10.0.0.1/32
Tue Sep 15 12:01:13.134 UTC
10.0.0.1/32, version 76120, internal 0x1000001 0x11 (ptr 0x8852b070) [3], 0x400 (0x8977fd58), 0x0 (0x0)
Updated Sep 15 11:55:34.001
local adjacency to GigabitEthernet0/0/0/1
Prefix Len 32, traffic index 0, precedence n/a, priority 1
gateway array (0x89619110) reference count 3, flags 0x0, source rib (7), 0 backups
[4 type 3 flags 0x8401 (0x896bbfb8) ext 0x0 (0x0)]
LW-LDI[type=3, refc=1, ptr=0x8977fd58, sh-ldi=0x896bbfb8]
gateway array update type-time 1 Sep 15 09:45:22.015
LDI Update time Sep 15 09:45:22.016
LW-LDI-TS Sep 15 11:55:34.001
via 10.2.3.2/32, GigabitEthernet0/0/0/1, 10 dependencies, weight 0, class 0 [flags 0x0]
path-idx 0 NHID 0x4 [0x8a781640 0x0]
next hop 10.2.3.2/32
local adjacency
Load distribution: 0 (refcount 4)
Hash OK Interface Address
0 Y GigabitEthernet0/0/0/1 10.2.3.2 RP/0/RP0/CPU0:R3#show cef 192.168.4.0/24
Tue Sep 15 12:01:12.394 UTC
192.168.4.0/24, version 82129, internal 0x5000001 0x40 (ptr 0x8852ac10) [1], 0x0 (0x0), 0x0 (0x0)
Updated Sep 15 11:55:44.203
local adjacency to GigabitEthernet0/0/0/1
Prefix Len 24, traffic index 0, precedence n/a, priority 4
gateway array (0x89619218) reference count 2002, flags 0x102010, source rib (7), 0 backups
[1 type 3 flags 0x48501 (0x896bc010) ext 0x0 (0x0) (collapsed)]
LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]
gateway array update type-time 1 Sep 15 11:55:44.047
LDI Update time Sep 15 11:55:44.047
via 10.0.0.1/32, 3 dependencies, recursive, backup [flags 0x6100]
path-idx 0 NHID 0x0 [0x8852b070 0x0]
next hop 10.0.0.1/32 via 10.0.0.1/32
via 10.0.0.2/32, 3 dependencies, recursive [flags 0x6000]
path-idx 1 NHID 0x0 [0x8852af90 0x0], Internal 0x8abcb190
next hop 10.0.0.2/32 via 10.0.0.2/32
Weight distribution:
slot 0, weight 1, normalized_weight 1, class 0
Load distribution: 0 (refcount 1)
Hash OK Interface Address
0 Y GigabitEthernet0/0/0/1 10.2.3.2 BGP is unchanged too. The best path in show bgp is still via R1, and there is no UPDATE in the R1-R2 capture (the R1-R3 iBGP session runs between loopbacks, so it simply continues over R2). All 900 pings were answered, with a largest gap of 0.17 s.
RP/0/RP0/CPU0:R3#show bgp 192.168.4.0/24
Tue Sep 15 12:01:09.800 UTC
BGP routing table entry for 192.168.4.0/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 36050 36050
Last Modified: Sep 15 11:55:44.023 for 00:05:25
Paths: (2 available, best #2)
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.3.5.5
Path #1: Received by speaker 0
Not advertised to any peer
65002, (received & used)
10.0.0.1 (metric 3) from 10.0.0.1 (10.0.0.1)
Origin IGP, metric 0, localpref 100, valid, internal, backup, add-path
Received Path ID 0, Local Path ID 6, version 36050
Path #2: Received by speaker 0
Advertised IPv4 Unicast paths to peers (in unique update groups):
10.3.5.5
65002, (received & used)
10.0.0.2 (metric 2) from 10.0.0.2 (10.0.0.2)
Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
Received Path ID 0, Local Path ID 1, version 36050$ tshark -r bgp-pic-step8-r1r2.pcap -t ud -Y 'bgp'
13 2026-09-15 11:55:46.411919Z 10.0.0.2 → 10.0.0.1 BGP 73 KEEPALIVE Message
15 2026-09-15 11:55:46.616703Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
17 2026-09-15 11:55:48.411131Z 10.0.0.1 → 10.0.0.2 BGP 73 KEEPALIVE Message
19 2026-09-15 11:55:48.614275Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
33 2026-09-15 11:56:46.411704Z 10.0.0.2 → 10.0.0.1 BGP 73 KEEPALIVE Message
35 2026-09-15 11:56:46.616677Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
38 2026-09-15 11:56:48.411278Z 10.0.0.1 → 10.0.0.2 BGP 73 KEEPALIVE Message
39 2026-09-15 11:56:48.614087Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
54 2026-09-15 11:57:46.412896Z 10.0.0.2 → 10.0.0.1 BGP 73 KEEPALIVE Message
56 2026-09-15 11:57:46.618245Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
58 2026-09-15 11:57:48.411705Z 10.0.0.1 → 10.0.0.2 BGP 73 KEEPALIVE Message
59 2026-09-15 11:57:48.614340Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
74 2026-09-15 11:58:46.413118Z 10.0.0.2 → 10.0.0.1 BGP 73 KEEPALIVE Message
76 2026-09-15 11:58:46.618728Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
79 2026-09-15 11:58:48.412295Z 10.0.0.1 → 10.0.0.2 BGP 73 KEEPALIVE Message
80 2026-09-15 11:58:48.614606Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
95 2026-09-15 11:59:46.413603Z 10.0.0.2 → 10.0.0.1 BGP 73 KEEPALIVE Message
97 2026-09-15 11:59:46.618137Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
99 2026-09-15 11:59:48.412508Z 10.0.0.1 → 10.0.0.2 BGP 73 KEEPALIVE Message
100 2026-09-15 11:59:48.615230Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
116 2026-09-15 12:00:46.413448Z 10.0.0.2 → 10.0.0.1 BGP 73 KEEPALIVE Message
118 2026-09-15 12:00:46.618769Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
120 2026-09-15 12:00:48.413002Z 10.0.0.1 → 10.0.0.2 BGP 73 KEEPALIVE Message
121 2026-09-15 12:00:48.615528Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
136 2026-09-15 12:01:46.413924Z 10.0.0.2 → 10.0.0.1 BGP 73 KEEPALIVE Message
138 2026-09-15 12:01:46.619187Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
141 2026-09-15 12:01:48.413550Z 10.0.0.1 → 10.0.0.2 BGP 73 KEEPALIVE Message
142 2026-09-15 12:01:48.615974Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE Message
157 2026-09-15 12:02:46.414214Z 10.0.0.2 → 10.0.0.1 BGP 73 KEEPALIVE Message
159 2026-09-15 12:02:46.619105Z 10.0.0.3 → 10.0.0.1 BGP 73 KEEPALIVE Message
162 2026-09-15 12:02:48.413863Z 10.0.0.1 → 10.0.0.2 BGP 73 KEEPALIVE Message
163 2026-09-15 12:02:48.616112Z 10.0.0.1 → 10.0.0.3 BGP 73 KEEPALIVE MessageRP/0/RP0/CPU0:R5#ping 192.168.4.1 source 192.168.5.1 count 900 interval 100 timeout 1
Tue Sep 15 11:55:15.404 UTC
Type escape sequence to abort.
Sending 900, 100-byte ICMP Echos to 192.168.4.1 timeout is 1 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (900/900), round-trip min/avg/max = 8/16/86 msSTEP 9: Restore (final state)
With both ends set to no shutdown, R1 and R3 return to the state of STEP 3 (one best path and one backup path).
RP/0/RP0/CPU0:R1#show route 192.168.4.0/24
Tue Sep 15 12:06:06.749 UTC
Routing entry for 192.168.4.0/24
Known via "bgp 65001", distance 20, metric 0
Tag 65002
Number of pic paths 1 , type internal and external
Installed Sep 15 11:46:46.961 for 00:19:19
Routing Descriptor Blocks
10.0.0.2, from 10.0.0.2, BGP backup path
Route metric is 0, Wt is 1
10.1.4.4, from 10.1.4.4, BGP external
Route metric is 0, Wt is 1
No advertising protos. RP/0/RP0/CPU0:R3#show route 192.168.4.0/24
Tue Sep 15 12:08:11.546 UTC
Routing entry for 192.168.4.0/24
Known via "bgp 65001", distance 200, metric 0
Tag 65002
Number of pic paths 1 , type internal
Installed Sep 15 12:04:06.535 for 00:04:05
Routing Descriptor Blocks
10.0.0.1, from 10.0.0.1
Route metric is 0, Wt is 1
10.0.0.2, from 10.0.0.2, BGP backup path
Route metric is 0, Wt is 1
No advertising protos. Verification config and show output
In each STEP the following files were collected from all five routers, one set per router (four in STEP 6, while R1 was stopped). The verification config is ..._run.txt (the final state is the one from STEP 9).
| File | Contents |
|---|---|
..._show.txt | show bgp summary / show bgp and advertised-routes / routes / received routes per peer (narrowed with include "192.168|100.63.255|Processed" to leave out the 2,000 filler routes) / show bgp <prefix> (192.168.4.0/24, 100.63.255.0/24, 100.64.0.0/24, 192.168.5.0/24) / show route summary / show route and show cef (the prefixes above plus 10.0.0.1/32 and 10.0.0.2/32) / show bgp neighbors / show bgp process / show rpl route-policy. R1-R3 also show ospf neighbor / show route ospf |
..._log.txt | show logging limited to that STEP. A logmsg marker is put at the start of each STEP and its time is given to show logging start |
..._run.txt | show running-config at that STEP (= the verification config of the STEP) |
..._ping.txt / ..._ping2.txt | ping and traceroute from R5 to 192.168.4.1 and 100.63.255.1. In the failure STEPs (1, 4, 6, 8), 900 pings at 100 ms intervals per destination, started before the failure (_ping.txt for 192.168.4.1, _ping2.txt for 100.63.255.1); 50 pings otherwise |
..._trace.txt | show bgp trace neighbor / show bgp trace error |
..._commit.cfg | Only the configuration actually committed in that STEP. Only for routers whose configuration changed. Stopping and starting R1 (STEP 6 and 7) are CML operations, so there is none |
STEP 0: Initial state (no PIC)
| Router | show output | syslog | running-config | ping | trace | commit |
|---|---|---|---|---|---|---|
| R1 | show | log | run | - | trace | - |
| R2 | show | log | run | - | trace | - |
| R3 | show | log | run | - | trace | - |
| R4 | show | log | run | - | trace | - |
| R5 | show | log | run | ping | trace | - |
STEP 1: R1-R4 link down (no PIC)
| Router | show output | syslog | running-config | ping | trace | commit |
|---|---|---|---|---|---|---|
| R1 | show | log | run | - | trace | cfg |
| R2 | show | log | run | - | trace | - |
| R3 | show | log | run | - | trace | - |
| R4 | show | log | run | - | trace | cfg |
| R5 | show | log | run | ping / ping2 | trace | - |
STEP 2: Restore
| Router | show output | syslog | running-config | ping | trace | commit |
|---|---|---|---|---|---|---|
| R1 | show | log | run | - | trace | cfg |
| R2 | show | log | run | - | trace | - |
| R3 | show | log | run | - | trace | - |
| R4 | show | log | run | - | trace | cfg |
| R5 | show | log | run | ping | trace | - |
STEP 3: additional-paths selection (backup 1 install) on R1 and R3
| Router | show output | syslog | running-config | ping | trace | commit |
|---|---|---|---|---|---|---|
| R1 | show | log | run | - | trace | cfg |
| R2 | show | log | run | - | trace | - |
| R3 | show | log | run | - | trace | cfg |
| R4 | show | log | run | - | trace | - |
| R5 | show | log | run | ping | trace | - |
STEP 4: R1-R4 link down (with PIC)
| Router | show output | syslog | running-config | ping | trace | commit |
|---|---|---|---|---|---|---|
| R1 | show | log | run | - | trace | cfg |
| R2 | show | log | run | - | trace | - |
| R3 | show | log | run | - | trace | - |
| R4 | show | log | run | - | trace | cfg |
| R5 | show | log | run | ping / ping2 | trace | - |
STEP 5: Restore
| Router | show output | syslog | running-config | ping | trace | commit |
|---|---|---|---|---|---|---|
| R1 | show | log | run | - | trace | cfg |
| R2 | show | log | run | - | trace | - |
| R3 | show | log | run | - | trace | - |
| R4 | show | log | run | - | trace | cfg |
| R5 | show | log | run | ping | trace | - |
STEP 6: Stop R1 (PIC Edge, next hop lost)
| Router | show output | syslog | running-config | ping | trace | commit |
|---|---|---|---|---|---|---|
| R1 | - | - | - | - | - | - |
| R2 | show | log | run | - | trace | - |
| R3 | show | log | run | - | trace | - |
| R4 | show | log | run | - | trace | - |
| R5 | show | log | run | ping / ping2 | trace | - |
STEP 7: Start R1
| Router | show output | syslog | running-config | ping | trace | commit |
|---|---|---|---|---|---|---|
| R1 | show | log | run | - | trace | - |
| R2 | show | log | run | - | trace | - |
| R3 | show | log | run | - | trace | - |
| R4 | show | log | run | - | trace | - |
| R5 | show | log | run | ping | trace | - |
STEP 8: R3-R1 link down (PIC Core)
| Router | show output | syslog | running-config | ping | trace | commit |
|---|---|---|---|---|---|---|
| R1 | show | log | run | - | trace | cfg |
| R2 | show | log | run | - | trace | - |
| R3 | show | log | run | - | trace | cfg |
| R4 | show | log | run | - | trace | - |
| R5 | show | log | run | ping / ping2 | trace | - |
STEP 9: Restore (final state)
| Router | show output | syslog | running-config | ping | trace | commit |
|---|---|---|---|---|---|---|
| R1 | show | log | run | - | trace | cfg |
| R2 | show | log | run | - | trace | - |
| R3 | show | log | run | - | trace | cfg |
| R4 | show | log | run | - | trace | - |
| R5 | show | log | run | ping | trace | - |
Packet captures were taken per STEP on the R3-R5 link (R5’s pings, used to measure the gaps between replies), the R1-R2 link and the R1-R3 link. The two links on R1’s side could not be captured in STEP 6, while R1 was stopped, nor in STEP 7, right after it was started ("—").
| STEP | R3-R5 | R1-R2 | R1-R3 |
|---|---|---|---|
| 0 | pcap | pcap | pcap |
| 1 | pcap | pcap | pcap |
| 2 | pcap | pcap | pcap |
| 3 | pcap | pcap | pcap |
| 4 | pcap | pcap | pcap |
| 5 | pcap | pcap | pcap |
| 6 | pcap | — | — |
| 7 | pcap | — | — |
| 8 | pcap | pcap | pcap |
| 9 | pcap | pcap | pcap |
References
| Source | Title | Summary |
|---|---|---|
| draft-ietf-rtgwg-bgp-pic-23 | BGP Prefix Independent Convergence | The hierarchical FIB and how PIC Core / Edge work (Informational) |
| RFC 4271 | A Border Gateway Protocol 4 (BGP-4) | The BGP procedures that PIC leaves unchanged |
Related articles
- BGP (Border Gateway Protocol)
- BGP Messages
- BGP Neighbor States
- iBGP and eBGP
- BGP next-hop-self
- BGP Route Reflector
- BGP Path Attributes and Best Path Selection
- BGP ORIGIN Attribute
- BGP AS_PATH Attribute
- BGP NEXT_HOP Attribute
- BGP LOCAL_PREF Attribute
- BGP MED Attribute
- BGP WEIGHT Attribute
- BGP COMMUNITY Attribute
- BGP Route Aggregation (aggregate-address)
- MP-BGP (Multiprotocol Extensions)
- BGP Extended Community
- BGP Multipath (maximum-paths)
- BGP Four-Octet AS Numbers (RFC 6793)
- Removing Private AS Numbers in BGP (remove-private-AS)
- BGP ADD-PATH (RFC 7911)
- BGP PIC (Prefix Independent Convergence)