BGP ADD-PATH (RFC 7911)

What ADD-PATH is

BGP can advertise only one path per prefix to each peer. The NLRI of an UPDATE consists of nothing but a prefix length and a prefix, so sending the same prefix twice means the second one replaces the first.

RFC 7911 (Proposed Standard) removes that limit by prepending a Path Identifier to the NLRI so that paths can be told apart.

NLRI encoding with ADD-PATH
+--------------------------------+
| Path Identifier (4 octets)     |
+--------------------------------+
| Length (1 octet)               |
+--------------------------------+
| Prefix (variable)              |
+--------------------------------+

Agreeing through a capability

Whether ADD-PATH is used is decided by a capability in the OPEN message.

Item	Value
Capability Code	69
Value	A sequence of `<AFI, SAFI, Send/Receive>`
Send/Receive	1 = can receive / 2 = can send / 3 = both

Both the sending and the receiving side have to line up.

For a BGP speaker to be able to send multiple paths to its peer, that BGP speaker MUST advertise the ADD-PATH Capability with the Send/Receive field set to either 2 or 3, and MUST receive from its peer the ADD-PATH Capability with the Send/Receive field set to either 1 or 3.

In other words, configuring only one side sends nothing extra. The sender must advertise 2 or 3 and must receive 1 or 3 from its peer before anything takes effect.

Because the capability is agreed in the OPEN message, changing the configuration alone does not apply it. An existing session keeps the old value, so it has to be re-established with clear bgp.

What a Path Identifier means

A Path Identifier is a 4-octet number, but the value itself carries no meaning.

The assignment of the Path Identifier for a path by a BGP speaker is purely a local matter.

The only requirement is that (prefix, Path Identifier) is unique per advertising destination. The receiver must not read anything into the number either.

A BGP speaker that receives a route should not assume that the identifier carries any particular semantics.

The same path may well carry a different number towards a different peer.

The standard does not say how many paths to advertise

RFC 7911 defines neither the number of paths nor how to choose them. The only guidance is this.

A BGP speaker SHOULD include the best route when more than one path is advertised to a neighbor, unless it is a path received from that neighbor.

If you advertise more than one, the best path should be among them, but a path received from that peer is excluded (it is not sent back). How many and which ones are left to the implementation and to policy.

How this differs from multipath

ADD-PATH is easy to confuse with BGP multipath, but they operate at different layers.

Item	ADD-PATH	Multipath
What it changes	Number of paths advertised to a peer	Number of paths installed in the forwarding table
Standardised	RFC 7911	No (a per-implementation extension)
Difference seen by the peer	Several paths arrive for the same prefix	None (only the best path is advertised)
Mechanism	Needs a capability and Path Identifiers	Local configuration only

IOS XR’s set path-selection has two separate keywords, advertise and install, and the distinction shows up directly in the command. advertise is the advertisement side (ADD-PATH), install is insertion into the RIB (multipath).

What it is for

Where this matters is a route reflector.

An RR reflects only the best path to its clients, so a client cannot learn about another exit inside the AS. If the RR reflects multiple paths with ADD-PATH, clients hold an alternative path in advance. It is also a prerequisite for fast switchover such as BGP PIC.

The same holds even with multipath configured. Give a client maximum-paths and it still installs one path if only one arrives from the RR. BGP multipath is not complete as a local setting on its own: under an RR it depends on several paths actually being delivered.

Configuration on IOS XR

It goes under address-family, and which paths get advertised is decided by a route policy.

On the route reflector
route-policy ADD-PATH-ALL
  set path-selection all advertise
end-policy
!
router bgp 65001
 address-family ipv4 unicast
  additional-paths receive
  additional-paths send
  additional-paths selection route-policy ADD-PATH-ALL

Item	Description
`additional-paths receive`	The receive side of capability 69 (sets 1 in Send/Receive)
`additional-paths send`	The send side (sets 2). Both together give 3
`additional-paths selection route-policy <name>`	Decides how many and which paths to advertise
`set path-selection all advertise`	Advertise every candidate
`set path-selection backup 1 advertise`	Advertise a single backup path
`show bgp <prefix>`	A path selected for advertisement is marked `add-path`

Put `additional-paths send` directly under `address-family`

A command of the same name also exists under a neighbor’s address-family, and both levels list it in the help.

Help directly under address-family (XRd 26.1.1)
RP/0/RP0/CPU0:R1(config-bgp-af)#  additional-paths ?

  advertise-limit  Limit the number of paths to be advertised (default 32)
  receive          Additional paths Receive capability
  selection        Additional paths selection
  send             Additional paths Send capability
RP/0/RP0/CPU0:R1(config-bgp-af)#  additional-paths 

% Incomplete command.
RP/0/RP0/CPU0:R1(config-bgp-af)#

RP/0/RP0/CPU0:R1(config-bgp-nbr-af)#   additional-paths ?

  receive  Additional paths Receive capability
  send     Additional paths Send capability
RP/0/RP0/CPU0:R1(config-bgp-nbr-af)#   additional-paths 

% Incomplete command.
RP/0/RP0/CPU0:R1(config-bgp-nbr-af)#

Writing it under a neighbor, however, does not advertise the send capability. The BGP command reference for the Cisco ASR 9000 Series lists its Command Modes as “IPv4 address family configuration” and others, and states this.

When the additional-paths send command is configured, the send capability is automatically enabled for all internal BGP neighbors for the specified address family.

It is not configured per neighbor; it applies to every internal BGP neighbor of that address family.

What makes this confusing is that these two lines in show bgp neighbors mean different things.

Line	Meaning
`Addpath send capability enabled by configuration`	`additional-paths send` is present under a `neighbor`
`Additional-paths Send: advertised`	The setting directly under `address-family` is actually advertising the capability

The first can appear without the second. If you are sure you configured it but no extra paths arrive, look at the second line, not the first.

Verification on real devices

This is checked on six XRd 26.1.1 routers. A route reflector topology is used because that is where ADD-PATH is actually needed.

In AS 65001, R1 is the route reflector and R2, R3 and R4 are clients. OSPF area 0 provides reachability between Loopback0 addresses, and iBGP runs between them
R5 (AS 65002) and R6 (AS 65003) both advertise the same 192.168.100.0/24, so R1 receives two paths and picks one as best
R2 and R3 apply next-hop-self to the routes they learn externally

Router	Role	External session
R1	Route reflector	None
R2	Client	eBGP with R5 (AS 65002)
R3	Client	eBGP with R6 (AS 65003)
R4	Client. Observation point	None
R5 / R6	External ASes advertising `192.168.100.0/24`	—

Using R4 as the observation point is the key. R4 has no external session, so every route in its show bgp output was reflected by R1. On R2 or R3 two paths would be visible to begin with (their own external route plus the reflected one), which could not be told apart from the effect of ADD-PATH.

Overview of the verification

STEP	Operation	What it confirms
0	Initial state (no ADD-PATH configuration)	The RR holds two paths but gives its client one. R4 forwards over one path
1	Reset the R1 - R4 session	The default OPEN carries no capability 69
2	`additional-paths receive` on R1 to R4	Both sides send `Receive (1)`. That alone adds nothing
3	`additional-paths send` under `neighbor` on R1, plus a selection policy	Send is still not advertised
4	`additional-paths send` directly under `address-family` on R1	Now `Both (3)`. Two paths reach R4, and two enter the FIB
5	Change the selection policy to `backup 1 advertise`	Which paths get advertised is decided by policy
6	Remove all ADD-PATH configuration	Back to one path

The only difference between STEP 3 and STEP 4 is where additional-paths send was written.

R4 carries maximum-paths ibgp 2 and bgp bestpath as-path multipath-relax from the start, and they are never changed for the whole run. That confines the reason R4’s forwarding entries change to ADD-PATH alone.

R4 running-config (as of STEP 0; never changed afterwards)
router bgp 65001
 bgp router-id 10.0.0.4
 bgp bestpath as-path multipath-relax
 address-family ipv4 unicast
  maximum-paths ibgp 2
 !

STEP 1, 2, 3 and 4 capture the packets between R1 and R4, each spanning a clear bgp that re-establishes the session.

STEP 0: a route reflector reflects only the best path

R1 holds two paths, but only one reaches R4.

STEP 0: show bgp 192.168.100.0/24 on R1
RP/0/RP0/CPU0:R1#show bgp 192.168.100.0/24
Fri Sep 11 22:30:39.555 UTC
BGP routing table entry for 192.168.100.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                  3            3
Last Modified: Sep 11 22:24:55.023 for 00:05:44
Paths: (2 available, best #1)
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.0.0.3        10.0.0.4        
  Path #1: Received by speaker 0
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.0.0.3        10.0.0.4        
  65002, (Received from a RR-client), (received & used)
    10.0.0.2 (metric 2) from 10.0.0.2 (10.0.0.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 3
  Path #2: Received by speaker 0
  Not advertised to any peer
  65003, (Received from a RR-client), (received & used)
    10.0.0.3 (metric 2) from 10.0.0.3 (10.0.0.3)
      Origin IGP, metric 0, localpref 100, valid, internal, group-best
      Received Path ID 0, Local Path ID 0, version 0

The best one is Path #1 (via AS 65002), and Path #2 says Not advertised to any peer.

STEP 0: show bgp 192.168.100.0/24 on R4
RP/0/RP0/CPU0:R4#show bgp 192.168.100.0/24
Fri Sep 11 22:33:42.460 UTC
BGP routing table entry for 192.168.100.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                  3            3
Last Modified: Sep 11 22:24:55.023 for 00:08:47
Paths: (1 available, best #1)
  Not advertised to any peer
  Path #1: Received by speaker 0
  Not advertised to any peer
  65002, (received & used)
    10.0.0.2 (metric 3) from 10.0.0.1 (10.0.0.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 3
      Originator: 10.0.0.2, Cluster list: 10.0.0.1

R4 has no external session, so this single path is the one reflected by R1. The forwarding table naturally holds one path too.

STEP 0: show route 192.168.100.0/24 on R4
RP/0/RP0/CPU0:R4#show route 192.168.100.0/24
Fri Sep 11 22:33:41.009 UTC

Routing entry for 192.168.100.0/24
  Known via "bgp 65001", distance 200, metric 0
  Tag 65002, type internal
  Installed Sep 11 22:24:55.502 for 00:08:45
  Routing Descriptor Blocks
    10.0.0.2, from 10.0.0.1
      Route metric is 0, Wt is 1
  No advertising protos. 

maximum-paths ibgp 2 is configured, yet there is a single Routing Descriptor Block. There is only one path to install.

STEP 1: capability 69 is not advertised by default

ADD-PATH appears in the OPEN message only once it is configured.

The R1 - R4 session is reset and the OPEN that R1 sent is extracted.

STEP 1: No.16 OPEN (R1 to R4), tshark -V
Border Gateway Protocol - OPEN Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 75
    Type: OPEN Message (1)
    Version: 4
    My AS: 65001
    Hold Time: 180
    BGP Identifier: 10.0.0.1
    Optional Parameters Length: 46
    Optional Parameters
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 6
            Capability: Multiprotocol extensions capability
                Type: Multiprotocol extensions capability (1)
                Length: 4
                AFI: IPv4 (1)
                Reserved: 00
                SAFI: Unicast (1)
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 2
            Capability: Route Refresh Capability (Cisco)
                Type: Route Refresh Capability (Cisco) (128)
                Length: 0
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 2
            Capability: Route refresh capability
                Type: Route refresh capability (2)
                Length: 0
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 6
            Capability: Support for 4-octet AS number capability
                Type: Support for 4-octet AS number capability (65)
                Length: 4
                AS Number: 65001
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 20
            Capability: Extended Next Hop Encoding
                Type: Extended Next Hop Encoding (5)
                Length: 18
                AFI: IPv4 (1)
                SAFI: Unicast (1)
                Next hop AFI: IPv6 (2)
                AFI: IPv4 (1)
                SAFI: Multicast (2)
                Next hop AFI: IPv6 (2)
                AFI: IPv4 (1)
                SAFI: Labeled VPN Unicast (128)
                Next hop AFI: IPv6 (2)

Download the pcap of the packet in the tshark output above (No.16 OPEN)

There are five capabilities: Multiprotocol (1), Route Refresh (128 and 2), 4-octet AS (65) and Extended Next Hop Encoding (5). 69 is not among them.

STEP 2: the receive side alone adds no paths

With Receive (1) on both sides, no router meets the sender’s condition.

Only additional-paths receive is added to address-family ipv4 unicast on R1, R2, R3 and R4.

STEP 2: running-config on R4 (extract)
router bgp 65001
 bgp router-id 10.0.0.4
 bgp bestpath as-path multipath-relax
 address-family ipv4 unicast
  additional-paths receive
  maximum-paths ibgp 2
 !

After the session is re-established, both sides advertise capability 69.

STEP 2: capability 69 in No.16 OPEN (R1 to R4)
Border Gateway Protocol - OPEN Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 83
    Type: OPEN Message (1)
<snip>
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 6
            Capability: Support for Additional Paths
                Type: Support for Additional Paths (69)
                Length: 4
                AFI: IPv4 (1)
                SAFI: Unicast (1)
                Send/Receive: Receive (1)

Download the pcap of the packet in the tshark output above (No.16 OPEN)

STEP 2: capability 69 in No.19 OPEN (R4 to R1)
Border Gateway Protocol - OPEN Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 83
    Type: OPEN Message (1)
<snip>
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 6
            Capability: Support for Additional Paths
                Type: Support for Additional Paths (69)
                Length: 4
                AFI: IPv4 (1)
                SAFI: Unicast (1)
                Send/Receive: Receive (1)

Download the pcap of the packet in the tshark output above (No.19 OPEN)

Both say Receive (1). show bgp neighbors on R4 still reports Additional-paths operation: None.

STEP 2: show bgp neighbors on R4 (the 10.0.0.1 part)
RP/0/RP0/CPU0:R4#show bgp neighbors
Fri Sep 11 22:47:57.014 UTC
<snip>
BGP neighbor is 10.0.0.1
 Remote AS 65001, local AS 65001, internal link
<snip>
  AF-dependent capabilities:
    Additional-paths Receive: advertised and received
    Extended Nexthop Encoding: advertised and received
  Route refresh request: received 0, sent 0
  Policy for incoming advertisements is FROM-R1
  Policy for outgoing advertisements is TO-R1
  1 accepted prefixes, 1 are bestpaths
  Exact no. of prefixes denied: 0
  Cumulative no. of prefixes denied: 0
  Number of routes received before ingress policy run: 1
  Accepted prefixes unmodified: 1
  Accepted prefixes (modified and unmodified): 1
  Accepted prefixes (modified and unmodified) high water mark: 1 (Sep 11 22:24:55.501)
  Accepted prefixes modified: 0
  Accepted prefixes modified high water mark: 0 (not set)
  Denied prefixes: 0
  Denied prefixes high water mark: 0 (not set)
  Prefix advertised 0, suppressed 0, withdrawn 0
  AIGP is enabled
  An EoR was received during read-only mode
  Last ack version 8, Last synced ack version 0
  Outstanding version objects: current 0, max 0, refresh 0
  Additional-paths operation: None

The number of paths is unchanged from STEP 0.

STEP 2: show bgp 192.168.100.0/24 on R4
RP/0/RP0/CPU0:R4#show bgp 192.168.100.0/24
Fri Sep 11 22:47:56.730 UTC
BGP routing table entry for 192.168.100.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                  8            8
Last Modified: Sep 11 22:44:13.023 for 00:03:43
Paths: (1 available, best #1)
  Not advertised to any peer
  Path #1: Received by speaker 0
  Not advertised to any peer
  65002, (received & used)
    10.0.0.2 (metric 3) from 10.0.0.1 (10.0.0.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 8
      Originator: 10.0.0.2, Cluster list: 10.0.0.1

STEP 3: under `neighbor`, Send is not advertised

The configuration is accepted and even gets a line of its own. The capability still does not change.

additional-paths send goes under R1’s neighbor 10.0.0.4, and the selection policy directly under address-family.

STEP 3: R1 neighbor 10.0.0.4 (extract)
 neighbor 10.0.0.4
  remote-as 65001
  description iBGP to R4
  update-source Loopback0
  address-family ipv4 unicast
   route-policy FROM-R4 in
   route-reflector-client
   route-policy TO-R4 out
   soft-reconfiguration inbound always
   additional-paths send
  !
 !

STEP 3: R1 route policy and address-family (extract)
route-policy ADD-PATH-ALL
  set path-selection all advertise
end-policy
!
router bgp 65001
 bgp router-id 10.0.0.1
 address-family ipv4 unicast
  additional-paths receive
  additional-paths selection route-policy ADD-PATH-ALL
 !

Even after the session is re-established, R1’s OPEN still says Receive (1).

STEP 3: capability 69 in No.19 OPEN (R1 to R4)
Border Gateway Protocol - OPEN Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 83
    Type: OPEN Message (1)
<snip>
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 6
            Capability: Support for Additional Paths
                Type: Support for Additional Paths (69)
                Length: 4
                AFI: IPv4 (1)
                SAFI: Unicast (1)
                Send/Receive: Receive (1)

Download the pcap of the packet in the tshark output above (No.19 OPEN)

show bgp neighbors on R1 does show Addpath send capability enabled by configuration. The configuration is recognised. But AF-dependent capabilities has no Additional-paths Send line, and Additional-paths operation stays None.

STEP 3: show bgp neighbors on R1 (the 10.0.0.4 part)
RP/0/RP0/CPU0:R1#show bgp neighbors
Fri Sep 11 23:15:48.553 UTC
<snip>
BGP neighbor is 10.0.0.4
 Remote AS 65001, local AS 65001, internal link
<snip>
  AF-dependent capabilities:
    Additional-paths Receive: advertised and received
    Extended Nexthop Encoding: advertised and received
  Route refresh request: received 0, sent 0
  Policy for incoming advertisements is FROM-R4
  Policy for outgoing advertisements is TO-R4
  0 accepted prefixes, 0 are bestpaths
  Exact no. of prefixes denied: 0
  Cumulative no. of prefixes denied: 0
  Number of routes received before ingress policy run: 0
  Accepted prefixes unmodified: 0
  Accepted prefixes (modified and unmodified): 0
  Accepted prefixes (modified and unmodified) high water mark: 0 (not set)
  Accepted prefixes modified: 0
  Accepted prefixes modified high water mark: 0 (not set)
  Denied prefixes: 0
  Denied prefixes high water mark: 0 (not set)
  Prefix advertised 2, suppressed 0, withdrawn 0
  Addpath send capability enabled by configuration
  AIGP is enabled
  An EoR was not received during read-only mode
  Last ack version 32, Last synced ack version 0
  Outstanding version objects: current 1, max 3, refresh 0
  Additional-paths operation: None

R4 still receives one path.

STEP 3: show bgp 192.168.100.0/24 on R4
RP/0/RP0/CPU0:R4#show bgp 192.168.100.0/24
Fri Sep 11 23:18:47.389 UTC
BGP routing table entry for 192.168.100.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 31           31
Last Modified: Sep 11 23:14:52.023 for 00:03:55
Paths: (1 available, best #1)
  Not advertised to any peer
  Path #1: Received by speaker 0
  Not advertised to any peer
  65002, (received & used)
    10.0.0.2 (metric 3) from 10.0.0.1 (10.0.0.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 31
      Originator: 10.0.0.2, Cluster list: 10.0.0.1

STEP 4: directly under `address-family`, two paths arrive

This one line is the only difference.

STEP 4: R1 address-family (extract)
router bgp 65001
 bgp router-id 10.0.0.1
 address-family ipv4 unicast
  additional-paths receive
  additional-paths send
  additional-paths selection route-policy ADD-PATH-ALL
 !

R1’s OPEN changes to Both (3).

STEP 4: capability 69 in No.23 OPEN (R1 to R4)
Border Gateway Protocol - OPEN Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 83
    Type: OPEN Message (1)
<snip>
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 6
            Capability: Support for Additional Paths
                Type: Support for Additional Paths (69)
                Length: 4
                AFI: IPv4 (1)
                SAFI: Unicast (1)
                Send/Receive: Both (3)

Download the pcap of the packet in the tshark output above (No.23 OPEN)

R4 only has receive configured, so it stays at Receive (1).

STEP 4: capability 69 in No.20 OPEN (R4 to R1)
Border Gateway Protocol - OPEN Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 83
    Type: OPEN Message (1)
<snip>
        Optional Parameter: Capability
            Parameter Type: Capability (2)
            Parameter Length: 6
            Capability: Support for Additional Paths
                Type: Support for Additional Paths (69)
                Length: 4
                AFI: IPv4 (1)
                SAFI: Unicast (1)
                Send/Receive: Receive (1)

Download the pcap of the packet in the tshark output above (No.20 OPEN)

3 and 1 satisfy the condition in RFC 7911, and Additional-paths Send: advertised and Additional-paths operation: Send appear.

STEP 4: show bgp neighbors on R1 (the 10.0.0.4 part)
RP/0/RP0/CPU0:R1#show bgp neighbors
Fri Sep 11 23:33:11.361 UTC
<snip>
BGP neighbor is 10.0.0.4
 Remote AS 65001, local AS 65001, internal link
<snip>
  AF-dependent capabilities:
    Additional-paths Send: advertised
    Additional-paths Receive: advertised and received
    Extended Nexthop Encoding: advertised and received
  Route refresh request: received 0, sent 0
  Policy for incoming advertisements is FROM-R4
  Policy for outgoing advertisements is TO-R4
  0 accepted prefixes, 0 are bestpaths
  Exact no. of prefixes denied: 0
  Cumulative no. of prefixes denied: 0
  Number of routes received before ingress policy run: 0
  Accepted prefixes unmodified: 0
  Accepted prefixes (modified and unmodified): 0
  Accepted prefixes (modified and unmodified) high water mark: 0 (not set)
  Accepted prefixes modified: 0
  Accepted prefixes modified high water mark: 0 (not set)
  Denied prefixes: 0
  Denied prefixes high water mark: 0 (not set)
  Prefix advertised 2, suppressed 0, withdrawn 0
  Addpath send capability enabled by configuration
  AIGP is enabled
  An EoR was not received during read-only mode
  Last ack version 48, Last synced ack version 0
  Outstanding version objects: current 1, max 3, refresh 0
  Additional-paths operation: Send

Addpath send capability enabled by configuration was present in STEP 3 as well. That line only says a neighbor carries the configuration; whether the capability is advertised is told by the Additional-paths Send line. R4 reports Receive.

STEP 4: show bgp neighbors on R4 (the 10.0.0.1 part)
RP/0/RP0/CPU0:R4#show bgp neighbors
Fri Sep 11 23:36:10.883 UTC
<snip>
BGP neighbor is 10.0.0.1
 Remote AS 65001, local AS 65001, internal link
<snip>
  AF-dependent capabilities:
    Additional-paths Send: received
    Additional-paths Receive: advertised and received
    Extended Nexthop Encoding: advertised and received
  Route refresh request: received 0, sent 0
  Policy for incoming advertisements is FROM-R1
  Policy for outgoing advertisements is TO-R1
  2 accepted prefixes, 1 are bestpaths
  Exact no. of prefixes denied: 0
  Cumulative no. of prefixes denied: 0
  Number of routes received before ingress policy run: 2
  Accepted prefixes unmodified: 2
  Accepted prefixes (modified and unmodified): 2
  Accepted prefixes (modified and unmodified) high water mark: 2 (Sep 11 22:54:01.864)
  Accepted prefixes modified: 0
  Accepted prefixes modified high water mark: 0 (not set)
  Denied prefixes: 0
  Denied prefixes high water mark: 0 (not set)
  Prefix advertised 0, suppressed 0, withdrawn 0
  AIGP is enabled
  An EoR was received during read-only mode
  Last ack version 47, Last synced ack version 0
  Outstanding version objects: current 0, max 0, refresh 0
  Additional-paths operation: Receive

In R1’s table, Path #2 is now marked add-path. Look at where each path goes.

STEP 4: show bgp 192.168.100.0/24 on R1
RP/0/RP0/CPU0:R1#show bgp 192.168.100.0/24
Fri Sep 11 23:33:11.099 UTC
BGP routing table entry for 192.168.100.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 49           49
Last Modified: Sep 11 23:32:46.023 for 00:00:25
Paths: (2 available, best #1)
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.0.0.4        10.0.0.3        
  Path #1: Received by speaker 0
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.0.0.4        10.0.0.3        
  65002, (Received from a RR-client), (received & used)
    10.0.0.2 (metric 2) from 10.0.0.2 (10.0.0.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 48
  Path #2: Received by speaker 0
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.0.0.2        10.0.0.4        
  65003, (Received from a RR-client), (received & used)
    10.0.0.3 (metric 2) from 10.0.0.3 (10.0.0.3)
      Origin IGP, metric 0, localpref 100, valid, internal, group-best, add-path
      Received Path ID 0, Local Path ID 3, version 49

Path #1 (learned from R2) goes to R3 and R4; Path #2 (learned from R3) goes to R2 and R4. Neither is sent back to the peer it came from. RFC 7911’s “unless it is a path received from that neighbor” appears verbatim in the output.

R4 receives two paths.

STEP 4: show bgp 192.168.100.0/24 on R4
RP/0/RP0/CPU0:R4#show bgp 192.168.100.0/24
Fri Sep 11 23:36:10.609 UTC
BGP routing table entry for 192.168.100.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 47           47
Last Modified: Sep 11 23:32:46.023 for 00:03:24
Paths: (2 available, best #1)
  Not advertised to any peer
  Path #1: Received by speaker 0
  Not advertised to any peer
  65002, (received & used)
    10.0.0.2 (metric 3) from 10.0.0.1 (10.0.0.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best, group-best, multipath
      Received Path ID 1, Local Path ID 1, version 47
      Originator: 10.0.0.2, Cluster list: 10.0.0.1
  Path #2: Received by speaker 0
  Not advertised to any peer
  65003, (received & used)
    10.0.0.3 (metric 3) from 10.0.0.1 (10.0.0.3)
      Origin IGP, metric 0, localpref 100, valid, internal, multipath
      Received Path ID 3, Local Path ID 0, version 0
      Originator: 10.0.0.3, Cluster list: 10.0.0.1

The two are distinguished by Received Path ID 1 and Received Path ID 3. In STEP 0 it was Received Path ID 0. The values were picked by R1 and mean nothing.

The packets show it too. The same prefix is carried by two UPDATE messages with different Path Identifiers.

tshark -r bgp-add-path-step4-r1r4.pcap -Y bgp.nlri_path_id -T fields -E header=y -e frame.number -e bgp.nlri_path_id -e bgp.mp_reach_nlri_ipv4_prefix -e bgp.update.path_attribute.as_path_segment.as4
frame.number	bgp.nlri_path_id	bgp.mp_reach_nlri_ipv4_prefix	bgp.update.path_attribute.as_path_segment.as4
28	1	192.168.100.0	65002
31	3	192.168.100.0	65003

Inside the NLRI it looks like this.

STEP 4: the UPDATE in No.31 (R1 to R4), tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 85
    Type: UPDATE Message (2)
<snip>
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
            Type Code: MP_REACH_NLRI (14)
            Length: 17
            Address family identifier (AFI): IPv4 (1)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 10.0.0.3
                IPv4 Address: 10.0.0.3
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                192.168.100.0/24 PathId 3 
                    NLRI path id: 3
                    Prefix Length: 24
                    MP Reach NLRI IPv4 prefix: 192.168.100.0

Download the pcap of the packet in the tshark output above (No.31 UPDATE)

Without ADD-PATH, whichever arrived second would have overwritten the first.

And R4’s forwarding table now holds two paths.

STEP 4: show route 192.168.100.0/24 on R4
RP/0/RP0/CPU0:R4#show route 192.168.100.0/24
Fri Sep 11 23:36:09.023 UTC

Routing entry for 192.168.100.0/24
  Known via "bgp 65001", distance 200, metric 0
  Tag 65002, type internal
  Installed Sep 11 23:32:45.670 for 00:03:23
  Routing Descriptor Blocks
    10.0.0.2, from 10.0.0.1, BGP multi path
      Route metric is 0
    10.0.0.3, from 10.0.0.1, BGP multi path
      Route metric is 0
  No advertising protos. 

Not one character of R4’s configuration changed since STEP 0. maximum-paths ibgp 2 was there all along and installed a single path in STEP 0. What changed is that ADD-PATH now delivers two.

STEP 5: policy decides what gets advertised

With backup 1 advertise, the marker on Path #2 becomes backup, add-path.

STEP 5: route policy on R1
route-policy ADD-PATH-ALL
  set path-selection backup 1 advertise
end-policy
!

STEP 5: show bgp 192.168.100.0/24 on R1
RP/0/RP0/CPU0:R1#show bgp 192.168.100.0/24
Fri Sep 11 23:41:12.090 UTC
BGP routing table entry for 192.168.100.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 50           50
Last Modified: Sep 11 23:39:11.023 for 00:02:01
Paths: (2 available, best #1)
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.0.0.4        10.0.0.3        
  Path #1: Received by speaker 0
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.0.0.4        10.0.0.3        
  65002, (Received from a RR-client), (received & used)
    10.0.0.2 (metric 2) from 10.0.0.2 (10.0.0.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 48
  Path #2: Received by speaker 0
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.0.0.2        10.0.0.4        
  65003, (Received from a RR-client), (received & used)
    10.0.0.3 (metric 2) from 10.0.0.3 (10.0.0.3)
      Origin IGP, metric 0, localpref 100, valid, internal, group-best, backup, add-path
      Received Path ID 0, Local Path ID 3, version 50

In this topology there is only one non-best path, so the path selected is the same as with all. Even when the count does not change, the marker records on what grounds a path became a candidate.

set path-selection accepts these five keywords.

Help for set path-selection (XRd 26.1.1)
RP/0/RP0/CPU0:R2(config-rpl)#  set path-selection ?

  all         BGP all paths
  backup      BGP backup path
  best-path   BGP best path
  group-best  BGP group best path
  multipath   BGP multipath
RP/0/RP0/CPU0:R2(config-rpl)#  set path-selection 

STEP 6: removing the configuration returns to one path

Capability 69 is no longer agreed, so the NLRI carries no Path Identifier.

STEP 6: running-config on R1 (extract)
router bgp 65001
 bgp router-id 10.0.0.1
 address-family ipv4 unicast
 !

STEP 6: show bgp 192.168.100.0/24 on R4
RP/0/RP0/CPU0:R4#show bgp 192.168.100.0/24
Fri Sep 11 23:51:10.102 UTC
BGP routing table entry for 192.168.100.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 51           51
Last Modified: Sep 11 23:47:26.023 for 00:03:44
Paths: (1 available, best #1)
  Not advertised to any peer
  Path #1: Received by speaker 0
  Not advertised to any peer
  65002, (received & used)
    10.0.0.2 (metric 3) from 10.0.0.1 (10.0.0.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 51
      Originator: 10.0.0.2, Cluster list: 10.0.0.1

Received Path ID is back to 0, and so is the forwarding table.

STEP 6: show route 192.168.100.0/24 on R4
RP/0/RP0/CPU0:R4#show route 192.168.100.0/24
Fri Sep 11 23:51:08.843 UTC

Routing entry for 192.168.100.0/24
  Known via "bgp 65001", distance 200, metric 0
  Tag 65002, type internal
  Installed Sep 11 23:47:25.979 for 00:03:42
  Routing Descriptor Blocks
    10.0.0.2, from 10.0.0.1
      Route metric is 0, Wt is 1
  No advertising protos. 

R4 still carries maximum-paths ibgp 2. It falls back to one path because only one path now arrives.

Verification config and show output

For every STEP, the following three kinds of output were collected from all six routers, split per router. The verification config is these ..._run.txt files (the final state being the one from the last STEP).

File	Contents
`..._show.txt`	`show version` / `show interface description` / `show route` / `show route 192.168.100.0/24` / `show ospf neighbor` / the `show bgp` set / `show bgp neighbors` / `show rpl route-policy`
`..._log.txt`	`show logging` narrowed to that STEP. A marker is written with `logmsg` at the start of each STEP and that timestamp is passed to `show logging start`
`..._run.txt`	`show running-config` at that STEP (that is, the verification config for the STEP)

The ..._run.txt files for STEP 6 match STEP 0 on all six routers (no ADD-PATH settings and no route policy definition are left behind).

STEP 0: initial state (no ADD-PATH configuration)

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 1: reset the R1 - R4 session

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 2: additional-paths receive on R1 to R4

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 3: additional-paths send under neighbor on R1, plus a selection policy

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 4: additional-paths send directly under address-family on R1

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 5: change the selection policy to backup 1 advertise

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 6: remove all ADD-PATH configuration (final state)

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

The packet captures between R1 and R4 are these four.

Capture for STEP 1 (no ADD-PATH)

Capture for STEP 2 (receive only)

Capture for STEP 3 (send under neighbor)

Capture for STEP 4 (send under address-family)

References

Document	Title	Summary
RFC 7911	Advertisement of Multiple Paths in BGP	ADD-PATH. Capability 69, the 4-octet Path Identifier, Send/Receive values 1/2/3
RFC 4271	A Border Gateway Protocol 4 (BGP-4)	The original specification in which NLRI consists of prefixes alone
RFC 4456	BGP Route Reflection	Route reflection, and the premise that only the best path is reflected