MP-BGP (Multiprotocol Extensions)

What MP-BGP is

BGP was originally a protocol that carried only IPv4 unicast routes. MP-BGP (Multiprotocol Extensions for BGP-4, RFC 4760) is the extension that lets BGP carry reachability for IPv6, VPNs, L2VPN and other non-IPv4 families.

The mechanism is just two new path attributes in the UPDATE message.

Attribute	Type code	Role
`MP_REACH_NLRI`	14	“For this address family, these destinations are reachable via this next hop”
`MP_UNREACH_NLRI`	15	“For this address family, withdraw these destinations”

Both are optional non-transitive, so a router that does not support them ignores them and does not pass them on. Message formats, the state machine and best-path selection are unchanged, so one TCP session (port 179) can carry several address families at once.

Address families (AFI / SAFI)

The kind of route is identified by the pair AFI (Address Family Identifier, 2 bytes) and SAFI (Subsequent AFI, 1 byte). The AFI is the network-layer protocol and the SAFI is how it is used.

AFI / SAFI	Route type	Used for
1 / 1	IPv4 unicast	Ordinary IPv4
2 / 1	IPv6 unicast	Ordinary IPv6
1 / 2, 2 / 2	IPv4 / IPv6 multicast	Routes for multicast RPF
1 / 4, 2 / 4	IPv4 / IPv6 labeled unicast	Labeled unicast (RFC 8277)
1 / 128, 2 / 128	VPNv4 / VPNv6	MPLS L3VPN (RFC 4364)
25 / 65, 25 / 70	VPLS, EVPN	L2VPN
16388 / 71	BGP-LS	IGP link-state information

AFI values are in IANA Address Family Numbers; RFC 4760 defines SAFI 1 (unicast) and 2 (multicast), and IANA manages the rest. Each address family has its own BGP table and policies. On IOS XR you write network statements and route-policy per address-family block.

MP_REACH_NLRI and MP_UNREACH_NLRI

Format of the MP_REACH_NLRI attribute value

MP_REACH_NLRI puts the next hop and the destination prefixes, together with the AFI / SAFI, into one attribute. In the basic RFC 4271 form the next hop was the NEXT_HOP attribute (NEXT_HOP) and the prefixes were the NLRI field of the UPDATE itself. The next hop length is given by a 1-byte field, so any address type fits (4 for IPv4, 16 or 32 for IPv6).

Format of the MP_UNREACH_NLRI attribute value

MP_UNREACH_NLRI holds only the AFI / SAFI and the prefixes to withdraw. Both encode NLRI as <length in bits, prefix>, the same as RFC 4271.

Rule	RFC 4760
An UPDATE carrying `MP_REACH_NLRI` must also carry `ORIGIN` and `AS_PATH`, and `LOCAL_PREF` on iBGP	3 (MUST)
An UPDATE with no NLRI other than `MP_REACH_NLRI` should not carry `NEXT_HOP`; a receiver ignores it if present	3 (SHOULD NOT / SHOULD)
An UPDATE with only `MP_UNREACH_NLRI` need not carry other attributes	4
One attribute carries exactly one `<AFI, SAFI>`	11

IOS XR carries IPv4 unicast routes in MP_REACH_NLRI as well. On the other hand, IPv4 withdrawals use the Withdrawn Routes field of the UPDATE itself (the basic RFC 4271 form), while IPv6 withdrawals use MP_UNREACH_NLRI.

Address families are agreed with the OPEN capability

Which address families are exchanged is announced with the Multiprotocol Extensions Capability (capability code 1, the RFC 5492 mechanism) in the OPEN message.

Format of the Multiprotocol Extensions Capability value

Rule	Details
One capability per AFI / SAFI pair	Carrying IPv4 and IPv6 means two capabilities
Only AFI / SAFI pairs advertised by both sides are exchanged (RFC 4760 8)	A family advertised by one side only does not flow. Unknown capabilities are ignored (RFC 5492 3, MUST)
When no AFI / SAFI is in common	RFC 5492 5 says a speaker may send a NOTIFICATION (OPEN Message Error / Unsupported Capability) and terminate; if it does, the Data field must list the capabilities that caused it, and the peering should not be re-established automatically

What IOS XR (XRd 26.1.1) actually does is below; it differs from RFC 5492 5 in two points.

Situation	IOS XR behavior
An address family is added to a neighbor	To exchange capabilities again, the whole session, including the existing address families, is torn down and re-established. Until it comes back, all routes received from that neighbor are removed. No NOTIFICATION is sent when closing (syslog: `Address family activated`)
An address family is removed from a neighbor	Likewise, the whole session is torn down and re-established. A NOTIFICATION (Cease / Other Configuration Change) is sent when closing
Re-established with some family in common	One-sided families are not negotiated; the session comes up with the common ones only
No family in common at all	The side that receives the OPEN sends Unsupported Capability. The Data field is empty, and reconnection attempts continue every few tens of seconds

Adding or removing an address family on a live neighbor also interrupts the existing families, such as IPv4, until the session is re-established (about 20 to 50 seconds in this verification).

The IPv6 next hop (RFC 2545)

How IPv6 unicast (AFI 2 / SAFI 1) is carried is defined in RFC 2545. Because IPv6 has link-local addresses, the MP_REACH_NLRI next hop comes in two forms.

Next hop length	Content	When (RFC 2545 3)
32	Global + link-local (`fe80::/10`)	Only when the advertising speaker, the next hop and the receiving peer share a subnet (shall)
16	Global only	All other cases. When advertising to an internal (iBGP) peer, the link-local address may be removed (may)

IOS XR uses 32 bytes on same-subnet eBGP and 16 bytes on iBGP even over the link. What decides the length is eBGP versus iBGP, not whether the session runs over the link or over Loopback0.

Whether the TCP session runs over IPv4 or IPv6 is independent of what BGP carries, so IPv6 routes can be carried over an IPv4 session. RFC 2545 4 notes that the next hop then needs “additional explicit configuration”. IOS XR uses the global IPv6 address of the interface the session runs on (the update-source interface if one is configured) as the next hop. If that interface has no global IPv6 address, the session does not come up at all, IPv4 included (Idle (No global IPv6 address configured)).

In practice, IPv6 unicast is usually carried over IPv6 sessions. The next hop falls out naturally and the session itself proves IPv6 reachability. The reverse (IPv4 routes over an IPv6 session) uses the Extended Next Hop Encoding Capability (code 5, RFC 8950), which XRd advertises by default.

Configuration on IOS XR

Item	Details
Global `address-family`	Enables that address family; `network` statements go here
Neighbor `address-family`	Families exchanged with that neighbor. Only what you configure goes into the capabilities
`show bgp ipv6 unicast …`	Use `ipv6 unicast` for the IPv6 table, neighbors and summary (`show bgp` is IPv4)
`show bgp neighbors`	`Neighbor capabilities` shows `advertised` / `received` per family. A neighbor with nothing agreed shows `(NoNeg)` in `show bgp ipv6 unicast summary`

The actual configuration is shown with show configuration commit changes in each STEP.

Verification on real devices

Verified with three XRd 26.1.1 routers.

R1 in AS 65001 and R2 in AS 65002 run eBGP; R2 and R3 run iBGP (R2 uses next-hop-self). AS 65002 runs OSPF / OSPFv3
Each router advertises 192.168.N.0/24 and 2001:db8:N::/64 on Loopback1 with network statements
STEP 0 has IPv4 sessions only (link addresses for R1 - R2, Loopback0 for R2 - R3). R1’s Gi0/0/0/0 has no IPv6 address at first. The figure shows the final state
Outbound policies have a different name per neighbor (all just pass); every neighbor has soft-reconfiguration inbound always
Every STEP checks reachability between R1’s and R3’s Loopback1 with IPv4 / IPv6 ping (50 packets) and traceroute

Overview of the verification

STEP	Change	What to check
0	Initial state (IPv4 only)	One capability, IPv4. IPv4 routes also arrive in `MP_REACH_NLRI`
1	Add IPv6 unicast to the IPv4 sessions	R1 has no global IPv6 address and the session does not come up, IPv4 included. On the Loopback0 iBGP session the next hop is Loopback0
2	Add an IPv6 address to R1’s link	Established with both IPv4 and IPv6; the next hop is 32 bytes
3	Separate IPv6 sessions (R2 - R3 on link addresses)	eBGP: 32 bytes; iBGP over the link: 16 bytes
4	Re-create the R2 - R3 IPv6 session on Loopback0	iBGP on Loopback0: also 16 bytes
5	Only R2 adds IPv6 unicast to the IPv4 session	Established with the common IPv4 only; IPv6 is `(NoNeg)`
6	Only R2 removes IPv4 unicast from the IPv4 session	Nothing in common: Unsupported Capability with empty Data, and reconnections continue
7	Restore R2’s IPv4 session	Established again
8	Remove R1’s `network` statements	IPv6 withdrawn with `MP_UNREACH_NLRI`, IPv4 with Withdrawn Routes
9	Restore R1’s `network` statements (final state)	Configuration matches STEP 4 and reachability returns

Success rate of ping from R1’s Loopback1 to R3’s Loopback1:

STEP	0	1	2	3	4	5	6	7	8	9
IPv4	100%	0%	100%	100%	100%	100%	0%	100%	0%	100%
IPv6	0%	0%	100%	100%	100%	100%	100%	100%	0%	100%

STEP 0: Initial state (IPv4 only)

There is a single capability, IPv4 unicast, and IPv4 routes also arrive in MP_REACH_NLRI. There are no IPv6 routes yet, so IPv6 ping fails.

R2 show bgp neighbors (STEP 0)
RP/0/RP0/CPU0:R2#show bgp neighbors
Sun Sep 13 09:38:16.965 UTC
<snip>
BGP neighbor is 10.1.2.1
 Remote AS 65001, local AS 65002, external link
 Description: eBGP to R1 (IPv4 session)
 Remote router ID 10.0.0.1
  BGP state = Established, up for 00:02:26
<snip>
  Neighbor capabilities:
    Route refresh: advertised (old + new) and received (old + new)
    4-byte AS: advertised and received
    Address family IPv4 Unicast: advertised and received

R1’s OPEN (No.30 in the R1-R2 capture) has one Multiprotocol Extensions Capability, AFI: IPv4 / SAFI: Unicast.

No.30 OPEN (R1 → R2) tshark -V
Border Gateway Protocol - OPEN Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 75
    Type: OPEN Message (1)
<snip>
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 6
            Capability: Multiprotocol extensions capability
                Type: Multiprotocol extensions capability (1)
                Length: 4
                AFI: IPv4 (1)
                Reserved: 00
                SAFI: Unicast (1)

Download the pcap of the packet in the tshark output above (No.30 OPEN)

In the following UPDATE (No.34) the IPv4 route is inside MP_REACH_NLRI (AFI 1 / SAFI 1).

No.34 UPDATE (R1 → R2) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 60
    Type: UPDATE Message (2)
<snip>
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
            Type Code: MP_REACH_NLRI (14)
            Length: 13
            Address family identifier (AFI): IPv4 (1)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 10.1.2.1
                IPv4 Address: 10.1.2.1
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                192.168.1.0/24
                    MP Reach NLRI prefix length: 24
                    MP Reach NLRI IPv4 prefix: 192.168.1.0

Download the pcap of the packet in the tshark output above (No.34 UPDATE)

R1 ping (STEP 0, IPv6)
RP/0/RP0/CPU0:R1#ping 2001:db8:3::1 source 2001:db8:1::1 count 50 timeout 1
Sun Sep 13 09:37:41.984 UTC
Type escape sequence to abort.
Sending 50, 100-byte ICMP Echos to 2001:db8:3::1, timeout is 1 seconds:
..UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
Success rate is 0 percent (0/50)

STEP 1: Add IPv6 unicast to the IPv4 sessions

R1 - R2 goes to Idle (No global IPv6 address configured) and does not come up, IPv4 included. R2 - R3 on Loopback0 comes up, and R3’s route has Loopback0’s 2001:db8::3 as the next hop.

address-family ipv6 unicast was added to the existing IPv4 neighbors on all three routers (R2 and R3 likewise).

R1 show configuration commit changes last 1 (STEP 1)
RP/0/RP0/CPU0:R1#show configuration commit changes last 1
Sun Sep 13 09:40:34.412 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
router bgp 65001
 neighbor 10.1.2.2
  address-family ipv6 unicast
   route-policy FROM-R2-V6 in
   route-policy TO-R2-V6 out
   soft-reconfiguration inbound always
  !
 !
!
end

R1 closes the session and stays Idle because Gi0/0/0/0 has no global IPv6 address to use as the next hop.

R1 syslog (STEP 1)
RP/0/RP0/CPU0:R1#show logging start Sep 13 09:37:50
Sun Sep 13 09:43:07.919 UTC
<snip>
RP/0/RP0/CPU0:Sep 13 09:40:35.505 UTC: bgp[1084]: %ROUTING-BGP-5-ADJCHANGE : neighbor 10.1.2.2 Down - Address family activated (VRF: default) (AS: 65002) 

R1 show bgp neighbors (STEP 1)
RP/0/RP0/CPU0:R1#show bgp neighbors
Sun Sep 13 09:42:57.079 UTC
<snip>
BGP neighbor is 10.1.2.2
 Remote AS 65002, local AS 65001, external link
 Description: eBGP to R2 (IPv4 session)
 Remote router ID 0.0.0.0
  BGP state = Idle (No global IPv6 address configured)
<snip>
  Last reset 00:02:21, due to Address family activated

R2 keeps trying to connect, but when R1 receives the OPEN it answers with neither OPEN nor NOTIFICATION; it closes with a TCP RST.

R1-R2 packet capture (STEP 1, tshark summary)
549.179586     10.1.2.2 → 10.1.2.1     TCP 62 56142 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1460 WS=1
549.186616     10.1.2.2 → 10.1.2.1     BGP 137 OPEN Message
549.220950     10.1.2.1 → 10.1.2.2     TCP 54 179 → 56142 [RST, ACK] Seq=1 Ack=1 Win=16384 Len=0
589.507687     10.1.2.2 → 10.1.2.1     TCP 62 21219 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
589.515877     10.1.2.2 → 10.1.2.1     BGP 137 OPEN Message
589.550086     10.1.2.1 → 10.1.2.2     TCP 54 179 → 21219 [RST, ACK] Seq=1 Ack=1 Win=16384 Len=0

IPv4 routes can no longer cross R1 - R2 either, so IPv4 ping fails too.

R1 ping (STEP 1, IPv4)
RP/0/RP0/CPU0:R1#ping 192.168.3.1 source 192.168.1.1 count 50 timeout 1
Sun Sep 13 09:43:21.734 UTC
Type escape sequence to abort.
Sending 50, 100-byte ICMP Echos to 192.168.3.1 timeout is 1 seconds:
..................................................
Success rate is 0 percent (0/50)

On R2 - R3 (an IPv4 session between Loopback0s), the IPv6 route R3 sent (No.60 in the R2-R3 capture) has R3’s Loopback0 2001:db8::3 as the next hop (16 bytes). The address of the update-source interface is used, not the link address 2001:db8:23::3.

No.60 UPDATE (R3 → R2, IPv4 session) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 78
    Type: UPDATE Message (2)
<snip>
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
            Type Code: MP_REACH_NLRI (14)
            Length: 30
            Address family identifier (AFI): IPv6 (2)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 2001:db8::3
                IPv6 Address: 2001:db8::3
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                2001:db8:3::/64
                    MP Reach NLRI prefix length: 64
                    MP Reach NLRI IPv6 prefix: 2001:db8:3::

Download the pcap of the packet in the tshark output above (No.60 UPDATE)

R2 show bgp ipv6 unicast (STEP 1)
RP/0/RP0/CPU0:R2#show bgp ipv6 unicast
Sun Sep 13 09:44:48.309 UTC
BGP router identifier 10.0.0.2, local AS number 65002
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0800000   RD version: 6
BGP main routing table version 6
BGP NSR Initial initsync version 4 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
*> 2001:db8:2::/64    ::                       0         32768 i
*>i2001:db8:3::/64    2001:db8::3              0    100      0 i

Processed 2 prefixes, 2 paths

STEP 2: Add an IPv6 address to R1’s link

R1 - R2 comes up with both IPv4 and IPv6, and the next hop of R1’s IPv6 route is 32 bytes, global + link-local. Both IPv4 and IPv6 ping succeed.

R1 show configuration commit changes last 1 (STEP 2)
RP/0/RP0/CPU0:R1#show configuration commit changes last 1
Sun Sep 13 09:47:35.607 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
interface GigabitEthernet0/0/0/0
 ipv6 address 2001:db8:12::1/64
!
end

R2 show bgp neighbors (STEP 2)
RP/0/RP0/CPU0:R2#show bgp neighbors
Sun Sep 13 09:50:42.773 UTC
<snip>
BGP neighbor is 10.1.2.1
 Remote AS 65001, local AS 65002, external link
 Description: eBGP to R1 (IPv4 session)
 Remote router ID 10.0.0.1
  BGP state = Established, up for 00:03:01
<snip>
  Neighbor capabilities:
    Route refresh: advertised (old + new) and received (old + new)
    4-byte AS: advertised and received
    Address family IPv4 Unicast: advertised and received
    Address family IPv6 Unicast: advertised and received

The IPv6 route R1 sent over the IPv4 session (No.112 in the R1-R2 capture). R1 and R2 are same-subnet eBGP peers, so the link-local address is included as RFC 2545 3 requires.

No.112 UPDATE (R1 → R2, IPv4 session) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 93
    Type: UPDATE Message (2)
<snip>
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
            Type Code: MP_REACH_NLRI (14)
            Length: 46
            Address family identifier (AFI): IPv6 (2)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: IPv6=2001:db8:12::1 Link-local=fe80::5054:ff:febf:f3a6
                IPv6 Address: 2001:db8:12::1
                Link-local Address: fe80::5054:ff:febf:f3a6
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                2001:db8:1::/64
                    MP Reach NLRI prefix length: 64
                    MP Reach NLRI IPv6 prefix: 2001:db8:1::

Download the pcap of the packet in the tshark output above (No.112 UPDATE)

R2 show bgp ipv6 unicast (STEP 2)
RP/0/RP0/CPU0:R2#show bgp ipv6 unicast
Sun Sep 13 09:50:43.987 UTC
BGP router identifier 10.0.0.2, local AS number 65002
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0800000   RD version: 7
BGP main routing table version 7
BGP NSR Initial initsync version 4 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
*> 2001:db8:1::/64    2001:db8:12::1           0             0 65001 i
*> 2001:db8:2::/64    ::                       0         32768 i
*>i2001:db8:3::/64    2001:db8::3              0    100      0 i

Processed 3 prefixes, 3 paths

STEP 3: Separate IPv6 sessions (R2 - R3 on link addresses)

The eBGP next hop (R1 - R2) is 32 bytes; the iBGP one on link addresses (R2 - R3) is 16 bytes. Even on the same subnet, iBGP carries no link-local address.

IPv6 unicast was removed from the IPv4 sessions and separate IPv6 neighbors were created (R1 and R3 have the matching configuration).

R2 show configuration commit changes last 1 (STEP 3)
RP/0/RP0/CPU0:R2#show configuration commit changes last 1
Sun Sep 13 09:52:45.302 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
router bgp 65002
 neighbor 10.0.0.3
  no address-family ipv6 unicast
 !
 neighbor 10.1.2.1
  no address-family ipv6 unicast
 !
 neighbor 2001:db8:12::1
  remote-as 65001
  description eBGP to R1 (IPv6 session)
  address-family ipv6 unicast
   route-policy FROM-R1-V6S in
   route-policy TO-R1-V6S out
   soft-reconfiguration inbound always
  !
 !
 neighbor 2001:db8:23::3
  remote-as 65002
  description iBGP to R3 (IPv6 session, link)
  address-family ipv6 unicast
   next-hop-self
   route-policy FROM-R3-V6L in
   route-policy TO-R3-V6L out
   soft-reconfiguration inbound always
  !
 !
!
end

No.156 UPDATE (R1 → R2, IPv6 session) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 93
    Type: UPDATE Message (2)
<snip>
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
            Type Code: MP_REACH_NLRI (14)
            Length: 46
            Address family identifier (AFI): IPv6 (2)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: IPv6=2001:db8:12::1 Link-local=fe80::5054:ff:febf:f3a6
                IPv6 Address: 2001:db8:12::1
                Link-local Address: fe80::5054:ff:febf:f3a6

Download the pcap of the packet in the tshark output above (No.156 UPDATE)

The route R3 sent over iBGP on the link address (No.135 in the R2-R3 capture) has only 2001:db8:23::3 as the next hop.

No.135 UPDATE (R3 → R2, IPv6 session) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 78
    Type: UPDATE Message (2)
<snip>
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
            Type Code: MP_REACH_NLRI (14)
            Length: 30
            Address family identifier (AFI): IPv6 (2)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 2001:db8:23::3
                IPv6 Address: 2001:db8:23::3
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                2001:db8:3::/64
                    MP Reach NLRI prefix length: 64
                    MP Reach NLRI IPv6 prefix: 2001:db8:3::

Download the pcap of the packet in the tshark output above (No.135 UPDATE)

R3 show bgp ipv6 unicast (STEP 3)
RP/0/RP0/CPU0:R3#show bgp ipv6 unicast
Sun Sep 13 09:56:51.515 UTC
BGP router identifier 10.0.0.3, local AS number 65002
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0800000   RD version: 11
BGP main routing table version 11
BGP NSR Initial initsync version 4 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
*>i2001:db8:1::/64    2001:db8:23::2           0    100      0 65001 i
*>i2001:db8:2::/64    2001:db8:23::2           0    100      0 i
*> 2001:db8:3::/64    ::                       0         32768 i

Processed 3 prefixes, 3 paths

STEP 4: Re-create the R2 - R3 IPv6 session on Loopback0

iBGP on Loopback0 also has a 16-byte next hop. Together with STEP 3, what decides the length is eBGP versus iBGP.

R2 show configuration commit changes last 1 (STEP 4)
RP/0/RP0/CPU0:R2#show configuration commit changes last 1
Sun Sep 13 09:57:46.984 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
router bgp 65002
 neighbor 2001:db8::3
  remote-as 65002
  description iBGP to R3 (IPv6 session, Loopback0)
  update-source Loopback0
  address-family ipv6 unicast
   next-hop-self
   route-policy FROM-R3-V6S in
   route-policy TO-R3-V6S out
   soft-reconfiguration inbound always
  !
 !
 no neighbor 2001:db8:23::3
!
end

No.192 UPDATE (R3 → R2, IPv6 session) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 78
    Type: UPDATE Message (2)
<snip>
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
            Type Code: MP_REACH_NLRI (14)
            Length: 30
            Address family identifier (AFI): IPv6 (2)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 2001:db8::3
                IPv6 Address: 2001:db8::3
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                2001:db8:3::/64
                    MP Reach NLRI prefix length: 64
                    MP Reach NLRI IPv6 prefix: 2001:db8:3::

Download the pcap of the packet in the tshark output above (No.192 UPDATE)

R3 show bgp ipv6 unicast (STEP 4)
RP/0/RP0/CPU0:R3#show bgp ipv6 unicast
Sun Sep 13 10:01:53.636 UTC
BGP router identifier 10.0.0.3, local AS number 65002
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0800000   RD version: 15
BGP main routing table version 15
BGP NSR Initial initsync version 4 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
*>i2001:db8:1::/64    2001:db8::2              0    100      0 65001 i
*>i2001:db8:2::/64    2001:db8::2              0    100      0 i
*> 2001:db8:3::/64    ::                       0         32768 i

Processed 3 prefixes, 3 paths

STEP 5: Only R2 adds IPv6 unicast to the IPv4 session

R2 tears down the whole session, IPv4 included, and re-establishes it about 40 seconds later. In the meantime R1 and R2 remove the IPv4 routes received from each other. No NOTIFICATION is sent when closing. After the reset the session comes up with the common IPv4 only; IPv6 is advertised only and (NoNeg). IPv6 routes keep flowing over the IPv6 session.

R2 show configuration commit changes last 1 (STEP 5)
RP/0/RP0/CPU0:R2#show configuration commit changes last 1
Sun Sep 13 10:02:54.361 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
router bgp 65002
 neighbor 10.1.2.1
  address-family ipv6 unicast
   route-policy FROM-R1-V6 in
   route-policy TO-R1-V6 out
   soft-reconfiguration inbound always
  !
 !
!
end

R2 syslog (STEP 5)
RP/0/RP0/CPU0:R2#show logging start Sep 13 10:01:27
Sun Sep 13 10:06:06.994 UTC
<snip>
RP/0/RP0/CPU0:Sep 13 10:02:55.569 UTC: bgp[1084]: %ROUTING-BGP-5-ADJCHANGE : neighbor 10.1.2.1 Down - Address family activated (VRF: default) (AS: 65001) 
<snip>
RP/0/RP0/CPU0:Sep 13 10:03:37.714 UTC: bgp[1084]: %ROUTING-BGP-5-ADJCHANGE : neighbor 10.1.2.1 Up (VRF: default) (AS: 65001) 

Between the teardown and the re-establishment, both sides remove the IPv4 routes received from the other (purged).

R2 show bgp trace neighbor (STEP 5)
RP/0/RP0/CPU0:R2#show bgp trace neighbor | utility tail -n 300
Sun Sep 13 10:06:15.463 UTC
<snip>
Sep 13 10:02:55.569 default-bgp/spkr-tr2-nbr 0/RP0/CPU0 t4742  [NBR]:3969: nbr reset: vrf 0x60000000 10.1.2.1, loc 6, reason: Address family activated
<snip>
Sep 13 10:02:57.571 default-bgp/spkr-tr2-nbr 0/RP0/CPU0 t4809  [NBR]:3656: prefixes from nbr 10.1.2.1, purged 1, marked gr stale 0, afi: IPv4 Unicast
<snip>
Sep 13 10:03:35.699 default-bgp/spkr-tr2-nbr 0/RP0/CPU0 t4732  [NBR]:11331: send open nbr: 10.1.2.1, loc 1

R1 show bgp trace neighbor (STEP 5)
RP/0/RP0/CPU0:R1#show bgp trace neighbor | utility tail -n 300
Sun Sep 13 10:05:15.820 UTC
<snip>
Sep 13 10:02:55.575 default-bgp/spkr-tr2-nbr 0/RP0/CPU0 t4653  [NBR]:3969: nbr reset: vrf 0x60000000 10.1.2.2, loc 6, reason: Peer closing down the session
<snip>
Sep 13 10:02:57.577 default-bgp/spkr-tr2-nbr 0/RP0/CPU0 t4739  [NBR]:3656: prefixes from nbr 10.1.2.2, purged 2, marked gr stale 0, afi: IPv4 Unicast
<snip>
Sep 13 10:03:37.704 default-bgp/spkr-tr2-nbr 0/RP0/CPU0 t4732  [NBR]:2632: send open nbr: 10.1.2.2, loc 3

After the reset, R2’s OPEN (No.234) has two capabilities, IPv4 and IPv6, and R1’s OPEN (No.237) has one, IPv4.

No.234 OPEN (R2 → R1) tshark -V
Border Gateway Protocol - OPEN Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 83
    Type: OPEN Message (1)
<snip>
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 6
            Capability: Multiprotocol extensions capability
                Type: Multiprotocol extensions capability (1)
                Length: 4
                AFI: IPv4 (1)
                Reserved: 00
                SAFI: Unicast (1)
<snip>
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 6
            Capability: Multiprotocol extensions capability
                Type: Multiprotocol extensions capability (1)
                Length: 4
                AFI: IPv6 (2)
                Reserved: 00
                SAFI: Unicast (1)

Download the pcap of the packet in the tshark output above (No.234 OPEN)

No.237 OPEN (R1 → R2) tshark -V
Border Gateway Protocol - OPEN Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 75
    Type: OPEN Message (1)
<snip>
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 6
            Capability: Multiprotocol extensions capability
                Type: Multiprotocol extensions capability (1)
                Length: 4
                AFI: IPv4 (1)
                Reserved: 00
                SAFI: Unicast (1)

Download the pcap of the packet in the tshark output above (No.237 OPEN)

R2 show bgp neighbors (STEP 5)
RP/0/RP0/CPU0:R2#show bgp neighbors
Sun Sep 13 10:05:54.117 UTC
<snip>
BGP neighbor is 10.1.2.1
 Remote AS 65001, local AS 65002, external link
 Description: eBGP to R1 (IPv4 session)
 Remote router ID 10.0.0.1
  BGP state = Established, up for 00:02:16
<snip>
  Neighbor capabilities:
    Route refresh: advertised (old + new) and received (old + new)
    4-byte AS: advertised and received
    Address family IPv4 Unicast: advertised and received
    Address family IPv6 Unicast: advertised

R2 show bgp ipv6 unicast summary (STEP 5)
RP/0/RP0/CPU0:R2#show bgp ipv6 unicast summary
Sun Sep 13 10:05:55.218 UTC
<snip>
Neighbor        Spk    AS MsgRcvd MsgSent       TblVer  InQ OutQ  Up/Down  St/PfxRcd
10.1.2.1          0 65001      49      62            0    0    0 00:02:17    (NoNeg)
2001:db8::3       0 65002      11      12           13    0    0 00:07:41          1
2001:db8:12::1    0 65001      16      19           13    0    0 00:12:26          1

STEP 6: Only R2 removes IPv4 unicast from the IPv4 session

R2 now advertises IPv6 only and R1 IPv4 only, so the side that receives the OPEN returns Unsupported Capability. The Data field is empty and both keep reconnecting. IPv4 ping fails; IPv6 still works over the IPv6 session.

R2 show configuration commit changes last 1 (STEP 6)
RP/0/RP0/CPU0:R2#show configuration commit changes last 1
Sun Sep 13 10:07:42.655 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
router bgp 65002
 neighbor 10.1.2.1
  no address-family ipv4 unicast
 !
!
end

R2 first closes the session with Cease (Other Configuration Change).

R2 syslog (STEP 6)
RP/0/RP0/CPU0:R2#show logging start Sep 13 10:06:21
Sun Sep 13 10:11:41.367 UTC
<snip>
RP/0/RP0/CPU0:Sep 13 10:07:43.890 UTC: bgp[1084]: %ROUTING-BGP-5-ADJCHANGE : neighbor 10.1.2.1 Down - Address family removed (CEASE notification sent - configuration change) (VRF: default) (AS: 65001) 

No.275 NOTIFICATION (R2 → R1) tshark -V
Border Gateway Protocol - NOTIFICATION Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 21
    Type: NOTIFICATION Message (3)
    Major error Code: Cease (6)
    Minor error Code (Cease): Other Configuration Change (6)

Download the pcap of the packet in the tshark output above (No.275 NOTIFICATION)

After that, whether R1 or R2 opens the connection, the side that receives the OPEN answers with a NOTIFICATION and the session closes.

R1-R2 packet capture (STEP 6, tshark summary)
2128.224240     10.1.2.2 → 10.1.2.1     BGP 75 NOTIFICATION Message
2164.646868     10.1.2.1 → 10.1.2.2     TCP 62 52370 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
2164.654024     10.1.2.1 → 10.1.2.2     BGP 129 OPEN Message
2166.659440     10.1.2.2 → 10.1.2.1     BGP 75 NOTIFICATION Message
2207.340018     10.1.2.1 → 10.1.2.2     TCP 62 25772 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
2207.350087     10.1.2.1 → 10.1.2.2     BGP 129 OPEN Message
2209.355341     10.1.2.2 → 10.1.2.1     BGP 75 NOTIFICATION Message
2239.147132     10.1.2.2 → 10.1.2.1     TCP 62 26263 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
2239.154154     10.1.2.2 → 10.1.2.1     BGP 129 OPEN Message
2241.160555     10.1.2.1 → 10.1.2.2     BGP 75 NOTIFICATION Message

The message is 21 bytes, with nothing after the error codes. The capabilities RFC 5492 5 requires in the Data field (MUST) are absent.

No.289 NOTIFICATION (R2 → R1) tshark -V
Border Gateway Protocol - NOTIFICATION Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 21
    Type: NOTIFICATION Message (3)
    Major error Code: OPEN Message Error (2)
    Minor error Code (Open Message): Unsupported Capability (7)

Download the pcap of the packet in the tshark output above (No.289 NOTIFICATION)

R2 show bgp neighbors (STEP 6)
RP/0/RP0/CPU0:R2#show bgp neighbors
Sun Sep 13 10:11:30.753 UTC
<snip>
BGP neighbor is 10.1.2.1
 Remote AS 65001, local AS 65002, external link
 Description: eBGP to R1 (IPv4 session)
 Remote router ID 0.0.0.0
  BGP state = Connect
<snip>
  Last reset 00:00:43, due to BGP Notification sent: unsupported/disjoint capability
  Time since last notification sent to neighbor: 00:00:43
  Error Code: unsupported/disjoint capability
  Notification data sent:
    None
  Time since last notification received from neighbor: 00:01:54
  Error Code: unsupported/disjoint capability
  Notification data received:
    None

R1 ping (STEP 6, IPv4)
RP/0/RP0/CPU0:R1#ping 192.168.3.1 source 192.168.1.1 count 50 timeout 1
Sun Sep 13 10:10:08.859 UTC
Type escape sequence to abort.
Sending 50, 100-byte ICMP Echos to 192.168.3.1 timeout is 1 seconds:
..................................................
Success rate is 0 percent (0/50)

R1 ping (STEP 6, IPv6)
RP/0/RP0/CPU0:R1#ping 2001:db8:3::1 source 2001:db8:1::1 count 50 timeout 1
Sun Sep 13 10:10:59.928 UTC
Type escape sequence to abort.
Sending 50, 100-byte ICMP Echos to 2001:db8:3::1, timeout is 1 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (50/50), round-trip min/avg/max = 6/9/51 ms

STEP 7: Restore R2’s IPv4 session

Once IPv4 unicast is common again, the next reconnection comes up. The sessions are the same as in STEP 4.

R2 show configuration commit changes last 1 (STEP 7)
RP/0/RP0/CPU0:R2#show configuration commit changes last 1
Sun Sep 13 10:14:32.181 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
router bgp 65002
 neighbor 10.1.2.1
  address-family ipv4 unicast
   route-policy FROM-R1-V4 in
   route-policy TO-R1-V4 out
   soft-reconfiguration inbound always
  !
  no address-family ipv6 unicast
 !
!
end

R2 syslog (STEP 7)
RP/0/RP0/CPU0:R2#show logging start Sep 13 10:11:52
Sun Sep 13 10:17:40.016 UTC
<snip>
RP/0/RP0/CPU0:Sep 13 10:14:40.435 UTC: bgp[1084]: %ROUTING-BGP-5-ADJCHANGE : neighbor 10.1.2.1 Up (VRF: default) (AS: 65001) 

STEP 8: Remove R1’s `network` statements

The IPv6 withdrawal is an UPDATE with only MP_UNREACH_NLRI; the IPv4 withdrawal uses the Withdrawn Routes field of the UPDATE itself. Both routes disappear from R3 and ping fails.

R1 show configuration commit changes last 1 (STEP 8)
RP/0/RP0/CPU0:R1#show configuration commit changes last 1
Sun Sep 13 10:50:26.942 UTC
!! Building configuration...
!! IOS XR Configuration 26.1.1
router bgp 65001
 address-family ipv4 unicast
  no network 192.168.1.0/24
 !
 address-family ipv6 unicast
  no network 2001:db8:1::/64
 !
!
end

No.747 UPDATE (R1 → R2, IPv6 session) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 39
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 0
    Total Path Attribute Length: 16
    Path attributes
        Path Attribute - MP_UNREACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
            Type Code: MP_UNREACH_NLRI (15)
            Length: 12
            Address family identifier (AFI): IPv6 (2)
            Subsequent address family identifier (SAFI): Unicast (1)
            Withdrawn Routes
                2001:db8:1::/64
                    MP Unreach NLRI prefix length: 64
                    MP Unreach NLRI IPv6 prefix: 2001:db8:1::

Download the pcap of the packet in the tshark output above (No.747 UPDATE)

No.751 UPDATE (R1 → R2, IPv4 session) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 27
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 4
    Withdrawn Routes
        192.168.1.0/24
            Withdrawn route prefix length: 24
            Withdrawn prefix: 192.168.1.0
    Total Path Attribute Length: 0

Download the pcap of the packet in the tshark output above (No.751 UPDATE)

R3 show bgp ipv6 unicast (STEP 8)
RP/0/RP0/CPU0:R3#show bgp ipv6 unicast
Sun Sep 13 10:56:03.683 UTC
BGP router identifier 10.0.0.3, local AS number 65002
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0800000   RD version: 18
BGP main routing table version 18
BGP NSR Initial initsync version 4 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
*>i2001:db8:2::/64    2001:db8::2              0    100      0 i
*> 2001:db8:3::/64    ::                       0         32768 i

Processed 2 prefixes, 2 paths

STEP 9: Restore R1’s `network` statements (final state)

The running-config of all three routers matches STEP 4, and IPv4 / IPv6 ping succeed again.

R1 ping (STEP 9, IPv6)
RP/0/RP0/CPU0:R1#ping 2001:db8:3::1 source 2001:db8:1::1 count 50 timeout 1
Sun Sep 13 11:41:53.372 UTC
Type escape sequence to abort.
Sending 50, 100-byte ICMP Echos to 2001:db8:3::1, timeout is 1 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (50/50), round-trip min/avg/max = 8/10/34 ms

Verification config and show output

Each STEP captures the following files from all three routers, one file per router. The verification config is the ..._run.txt (final state: STEP 9).

File	Content
`..._show.txt`	`show version` / `show interface description` / `show route` / `show route ipv6` / `show bgp summary` / `show bgp ipv6 unicast summary` / `show bgp` / `show bgp ipv6 unicast` / `show bgp <prefix>` (six) / `show bgp neighbors` / `show bgp ipv6 unicast neighbors` / `show bgp update-group` / `show bgp ipv6 unicast update-group` / `show bgp ipv6 unicast nexthops` / `show ospf neighbor` / `show ospfv3 neighbor` / per-peer `advertised-routes`, `routes`, `received routes` (IPv4 and IPv6)
`..._log.txt`	`show logging` limited to that STEP (a `logmsg` marker at the start of each STEP, its time passed to `show logging start`)
`..._run.txt`	`show running-config` at that STEP (the verification config of that STEP)
`..._ping.txt`	`ping` (50 packets, timeout 1 s) and `traceroute` between R1’s and R3’s Loopback1 (R1 and R3 only)
`..._trace.txt`	`show bgp trace neighbor` / `show bgp trace error`
`..._commit.cfg`	Only the configuration actually committed in that STEP, for the routers that changed

STEP 0: Initial state (IPv4 only)

Router	show output	syslog	running-config	ping	trace	commit
R1	show	log	run	ping	trace	-
R2	show	log	run	-	trace	-
R3	show	log	run	ping	trace	-

STEP 1: Add IPv6 unicast to the IPv4 sessions

Router	show output	syslog	running-config	ping	trace	commit
R1	show	log	run	ping	trace	cfg
R2	show	log	run	-	trace	cfg
R3	show	log	run	ping	trace	cfg

STEP 2: Add an IPv6 address to R1’s link

Router	show output	syslog	running-config	ping	trace	commit
R1	show	log	run	ping	trace	cfg
R2	show	log	run	-	trace	-
R3	show	log	run	ping	trace	-

STEP 3: Separate IPv6 sessions (R2 - R3 on link addresses)

Router	show output	syslog	running-config	ping	trace	commit
R1	show	log	run	ping	trace	cfg
R2	show	log	run	-	trace	cfg
R3	show	log	run	ping	trace	cfg

STEP 4: Re-create the R2 - R3 IPv6 session on Loopback0

Router	show output	syslog	running-config	ping	trace	commit
R1	show	log	run	ping	trace	-
R2	show	log	run	-	trace	cfg
R3	show	log	run	ping	trace	cfg

STEP 5: Only R2 adds IPv6 unicast to the IPv4 session

Router	show output	syslog	running-config	ping	trace	commit
R1	show	log	run	ping	trace	-
R2	show	log	run	-	trace	cfg
R3	show	log	run	ping	trace	-

STEP 6: Only R2 removes IPv4 unicast from the IPv4 session

Router	show output	syslog	running-config	ping	trace	commit
R1	show	log	run	ping	trace	-
R2	show	log	run	-	trace	cfg
R3	show	log	run	ping	trace	-

STEP 7: Restore R2’s IPv4 session

Router	show output	syslog	running-config	ping	trace	commit
R1	show	log	run	ping	trace	-
R2	show	log	run	-	trace	cfg
R3	show	log	run	ping	trace	-

STEP 8: Remove R1’s network statements

Router	show output	syslog	running-config	ping	trace	commit
R1	show	log	run	ping	trace	cfg
R2	show	log	run	-	trace	-
R3	show	log	run	ping	trace	-

STEP 9: Restore R1’s network statements (final state)

Router	show output	syslog	running-config	ping	trace	commit
R1	show	log	run	ping	trace	cfg
R2	show	log	run	-	trace	-
R3	show	log	run	ping	trace	-

Packet captures were taken on the R1-R2 and R2-R3 links. The lab was stopped once after STEP 8, so STEP 0-8 and STEP 9 are in separate files.

Range	R1-R2	R2-R3
STEP 0-8	pcap	pcap
STEP 9	pcap	pcap

References

Source	Title	Summary
RFC 4760	Multiprotocol Extensions for BGP-4	`MP_REACH_NLRI` (3) and `MP_UNREACH_NLRI` (4), SAFI (6, 9), capability (8)
RFC 5492	Capabilities Advertisement with BGP-4	Exchanging capabilities in OPEN optional parameters (3) and Unsupported Capability (5)
RFC 2545	Use of BGP-4 Multiprotocol Extensions for IPv6 Inter-Domain Routing	The IPv6 next hop (global + link-local, length 16 / 32) and transport independence
RFC 8950	Advertising IPv4 Network Layer Reachability Information (NLRI) with an IPv6 Next Hop	Extended Next Hop Encoding Capability for IPv4 routes with an IPv6 next hop
IANA Address Family Numbers	Address Family Numbers	List of AFI values