BGP Neighbor States

What Is the BGP Neighbor State Machine?

BGP defines session establishment as six states (the finite state machine in chapter 8 of RFC 4271). When a session does not come up, the state it is stuck in points at the cause. This article explains each state and its transitions, then confirms the state changes on IOS XR (XRd) (the messages themselves are covered in BGP Messages).

The Six States

State	Meaning
Idle	The initial state. All connections are refused and no resources are allocated. A Start event begins a TCP connection and moves to Connect. Any error also returns to this state.
Connect	Waiting for the TCP three-way handshake to complete. On success it sends an OPEN and moves to OpenSent; on failure it moves to Active.
Active	The locally initiated TCP connection failed; the router is retrying while also waiting for an inbound connection. Once a connection succeeds it moves to OpenSent.
OpenSent	An OPEN has been sent and the peer’s OPEN is awaited. The received OPEN is validated (version, AS number, BGP Identifier, Hold Time); if it passes, a KEEPALIVE is sent and the state moves to OpenConfirm.
OpenConfirm	Waiting for a KEEPALIVE from the peer. On receipt the state moves to Established.
Established	The session is up and route information can be exchanged with UPDATE.

Active does not mean healthy: it is the state in which the locally initiated TCP connection has failed and is being retried. A session stuck here calls for a look at TCP reachability and the peer address.

Connect, OpenSent and OpenConfirm are normally passed through in an instant. A TCP connection and an OPEN exchange take milliseconds to seconds, so the states you actually see stuck in show are mostly Idle, Active and Established. Even while SYNs go unanswered and are being retransmitted, IOS XR shows Active, not Connect.

What Sends a Session Back to Idle

From any state, any of the following returns the session to Idle.

A NOTIFICATION message was sent or received
The TCP session was torn down
Neither a KEEPALIVE nor an UPDATE was received within the Hold Time (Hold Timer Expired)
An administrator ran shutdown or reset the session

After returning to Idle, the router retries once the ConnectRetryTimer expires. RFC 4271 suggests 120 seconds (chapter 10). IOS XR does not show this value, but in this lab the peer starts reconnecting about 28 seconds after the teardown.

Values Rejected by OPEN Validation

In OpenSent, the received OPEN is validated, and a problem is answered with a NOTIFICATION (error code 2, OPEN Message Error). The subcode gives the reason.

Subcode	Name	Condition
`2`	Bad Peer AS	The AS number in the OPEN is not the one expected
`3`	Bad BGP Identifier	The BGP Identifier is `0`, or a peer in the same AS (iBGP) sent the local value (RFC 6286)
`6`	Unacceptable Hold Time	The offered Hold Time is not acceptable. One and two seconds must always be rejected (RFC 4271 6.2)

A NOTIFICATION carries a data field for diagnosis, but its contents are defined only in some cases. RFC 4271 6.2 defines it for Unsupported Version Number (a 2-octet version number) and says nothing about the data for Bad Peer AS. What goes there is up to the implementation.

A duplicated BGP Identifier is a problem only within the same AS (iBGP). RFC 6286 relaxed the requirement to “unique within an AS” and limits the Bad BGP Identifier error to an OPEN from an internal peer that carries the local identifier.

How to Check the State

On IOS XR the St/PfxRcd column of show bgp summary shows the number of received prefixes for an Established peer and the state name for any other. Per-peer details come from show bgp neighbor <address>: BGP state gives the state and the reason it is stuck (for example Idle (Neighbor is shutdown)), followed by Previous State and Last reset (why the session dropped); a drop caused by a NOTIFICATION also records Error Code and Notification data. State changes are logged to syslog as %ROUTING-BGP-5-ADJCHANGE.

Common Reasons a Session Stalls

Stuck state	Common causes (IOS XR `BGP state` text)
Idle	`shutdown` is configured (`Idle (Neighbor is shutdown)`). An eBGP peer is not directly connected and `ebgp-multihop` is missing (`Idle (eBGP neighbor not directly connected)`). The route to the peer points at Null0, so no source address can be chosen (`Idle (No best local address found)`). Just after a NOTIFICATION was sent or received because OPEN validation failed.
Active	TCP port `179` is unreachable. The peer is refusing with RST. ARP cannot be resolved, so no connection request goes out at all. An ACL or firewall is blocking it. The MD5 authentication passwords do not match (covered in BGP security).
OpenSent	The peer’s `remote-as` does not match. Within the same AS (iBGP), the BGP Identifier (Router ID) is duplicated. The offered Hold Time is unacceptable.
OpenConfirm	KEEPALIVEs are not arriving. There is a one-way communication failure.

The causes behind OpenSent and OpenConfirm are rarely visible as a state. A validation failure sends a NOTIFICATION and returns to Idle at once, so in show they appear as Idle with the reason in Last reset and Error Code. The list of NOTIFICATION error codes and subcodes is in BGP Messages.

Verification on Real Devices

Four XRd 26.1.1 routers sit in three ASes: R1 - R2 is iBGP, while R2 - R3 and R3 - R4 are eBGP. Each router advertises 192.168.N.0/24.

Starting from a healthy state, faults are introduced on purpose and the state changes are observed. The capture was taken on R2 - R3, plus R3 - R4 for the R2 - R4 multihop steps and R1 - R2 for the iBGP steps. Each eBGP neighbor has its own outbound route policy name (all contain only pass).

STEP	Action	What it shows
0	Initial state	All sessions are `Established`
1	`shutdown` R2’s neighbor toward R3	How the two sides display it differently
2	Restore with `no shutdown`	Recovery time, and which side opens the TCP connection
3	Configure a nonexistent peer (`10.2.3.99`) on R2	An address on the same segment that ARP cannot resolve
4	Configure on R2 a peer (`10.9.9.9`) behind a route pointing at Null0	Where a peer with a route but no exit gets stuck
5	`session-open-mode passive-only` on both R2 and R3	The effect on an established session
6	Peer R2 - R4 (two hops, across R3) on Loopback0	Where the session gets stuck without `ebgp-multihop`
7	`ebgp-multihop 2` on R2 and R4	The session comes up
8	Clean up the test peers and set R3’s `remote-as` on R2 to a wrong value (`65999`)	The NOTIFICATION and its data when OPEN validation fails
9	Restore `remote-as` and set R3’s BGP Identifier to the same `10.0.0.2` as R2 (eBGP)	A duplicate across different ASes
10	Restore R3 and set R1’s BGP Identifier to the same `10.0.0.2` as R2 (iBGP)	A duplicate within one AS
11	Restore R1 and try to configure `timers 30 2` on R3	How a 1- or 2-second Hold Time is handled
12	Block only TCP `179` from R2 with an ACL on R3	Hold Time expiry, and what is shown while SYNs go unanswered
13	Remove the ACL (final state)	Re-establishment

STEP 1: Administrative Shutdown

shutdown R2’s neighbor toward R3.

R2 neighbor shutdown (STEP 1)
router bgp 65001
 neighbor 10.2.3.3
  shutdown
 !
!

R2, which ran shutdown, is Idle (Admin); R3 on the other side is Active. After R3 was cut off by R2’s NOTIFICATION, it keeps retrying the TCP connection itself.

R2 show bgp summary (STEP 1, excerpt)
RP/0/RP0/CPU0:R2#show bgp summary
<snip>
Neighbor        Spk    AS MsgRcvd MsgSent       TblVer  InQ OutQ  Up/Down  St/PfxRcd
10.0.0.1          0 65001      34      37            8    0    0 00:30:40          1
10.2.3.3          0 65002      31      32            0    0    0 00:04:12  Idle (Admin)

R3 show bgp summary (STEP 1, excerpt)
RP/0/RP0/CPU0:R3#show bgp summary
<snip>
Neighbor        Spk    AS MsgRcvd MsgSent       TblVer  InQ OutQ  Up/Down  St/PfxRcd
10.2.3.2          0 65001      32      31            0    0    0 00:04:55  Active
10.3.4.4          0 65003      35      37            8    0    0 00:31:23          1

show bgp neighbor gives the reason as well. R2 is Idle (Neighbor is shutdown), and Last reset records that it sent a CEASE NOTIFICATION. R3 is Active, and the reason is the received NOTIFICATION.

R2 show bgp neighbor 10.2.3.3 (STEP 1, excerpt)
BGP neighbor is 10.2.3.3
 Remote AS 65002, local AS 65001, external link
 Administratively shut down
<snip>
  BGP state = Idle (Neighbor is shutdown)
  Previous State: Closing
<snip>
  Connections established 1; dropped 1
  Local host: 10.2.3.2, Local port: 24097, IF Handle: 0x00000010
  Foreign host: 10.2.3.3, Foreign port: 179
  Last reset 00:04:14, due to Admin. shutdown (CEASE notification sent - administrative shutdown)
  Time since last notification sent to neighbor: 00:04:14
  Error Code: administrative shutdown
  Notification data sent:
    None

R3 show bgp neighbor 10.2.3.2 (STEP 1, excerpt)
BGP neighbor is 10.2.3.2
 Remote AS 65001, local AS 65002, external link
<snip>
  BGP state = Active
  Previous State: Idle
<snip>
  Connections established 1; dropped 1
  Local host: 10.2.3.3, Local port: 179, IF Handle: 0x00000008
  Foreign host: 10.2.3.2, Foreign port: 24097
  Last reset 00:04:57, due to BGP Notification received: administrative shutdown
  Time since last notification received from neighbor: 00:04:57
  Error Code: administrative shutdown
  Notification data received:
    None

In the capture, R2 sends a NOTIFICATION and closes TCP (No.1-4); about 28 seconds later R3 starts sending SYNs and R2 refuses every one with an RST (No.6-14). That is what Active actually is. The SYNs are retransmitted from the same source port at 2-, 4- and 8-second intervals.

R2 - R3 packet capture (STEP 1, tshark listing, ACK-only TCP lines omitted)
 0.000000     10.2.3.2 → 10.2.3.3     BGP 75 NOTIFICATION Message
 0.000261     10.2.3.2 → 10.2.3.3     TCP 54 24097 → 179 [FIN, ACK] Seq=22 Ack=1 Win=32040 Len=0
 0.006086     10.2.3.3 → 10.2.3.2     TCP 54 179 → 24097 [FIN, ACK] Seq=1 Ack=23 Win=32098 Len=0
28.035243     10.2.3.3 → 10.2.3.2     TCP 62 16391 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
28.038540     10.2.3.2 → 10.2.3.3     TCP 54 179 → 16391 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
30.001146     10.2.3.2 → 10.2.3.3     TCP 54 24097 → 179 [RST, ACK] Seq=23 Ack=2 Win=32040 Len=0
30.035837     10.2.3.3 → 10.2.3.2     TCP 62 [TCP Port numbers reused] 16391 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
30.038199     10.2.3.2 → 10.2.3.3     TCP 54 179 → 16391 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
34.036853     10.2.3.3 → 10.2.3.2     TCP 62 [TCP Port numbers reused] 16391 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
34.039675     10.2.3.2 → 10.2.3.3     TCP 54 179 → 16391 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
42.037529     10.2.3.3 → 10.2.3.2     TCP 62 [TCP Port numbers reused] 16391 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
42.040952     10.2.3.2 → 10.2.3.3     TCP 54 179 → 16391 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0

The NOTIFICATION is error code 6 (Cease), subcode 2 (Administratively Shutdown), which can be told apart from subcode 4 (Administratively Reset) used by clear bgp.

STEP 1 No.1 NOTIFICATION (R2 to R3), tshark -V
Border Gateway Protocol - NOTIFICATION Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 21
    Type: NOTIFICATION Message (3)
    Major error Code: Cease (6)
    Minor error Code (Cease): Administratively Shutdown (2)

Download the pcap of the packet in the tshark output above (STEP 1 No.1 NOTIFICATION)

STEP 2: Recovering with no shutdown

After no shutdown, the session was back in about 11 seconds, from the commit (12:26:50) to Up (12:27:01).

R2 syslog (STEP 2, excerpt)
RP/0/RP0/CPU0:Sep 21 12:26:50.085 UTC: config[67347]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user 'cisco'. Use 'show configuration commit changes 1000000002' to view the changes. 
RP/0/RP0/CPU0:Sep 21 12:27:01.302 UTC: bgp[1084]: %ROUTING-BGP-5-ADJCHANGE : neighbor 10.2.3.3 Up (VRF: default) (AS: 65002) 

The connection was opened by R3, which had been retrying in Active. From the moment R2 accepts R3’s SYN (the SYN/ACK in No.8), the session proceeds to OPEN, KEEPALIVE and UPDATE.

R2 - R3 packet capture (STEP 2, tshark listing)
 0.000000     10.2.3.3 → 10.2.3.2     TCP 62 16391 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
 2.000394     10.2.3.3 → 10.2.3.2     TCP 62 [TCP Port numbers reused] 16391 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
 6.001448     10.2.3.3 → 10.2.3.2     TCP 62 [TCP Port numbers reused] 16391 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
14.002077     10.2.3.3 → 10.2.3.2     TCP 62 [TCP Port numbers reused] 16391 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
14.005638     10.2.3.2 → 10.2.3.3     TCP 62 179 → 16391 [SYN, ACK] Seq=0 Ack=1 Win=16384 Len=0 MSS=1240 WS=1
14.010900     10.2.3.3 → 10.2.3.2     BGP 129 OPEN Message
16.016593     10.2.3.2 → 10.2.3.3     BGP 129 OPEN Message
16.020389     10.2.3.3 → 10.2.3.2     BGP 73 KEEPALIVE Message
16.023160     10.2.3.2 → 10.2.3.3     BGP 73 KEEPALIVE Message
16.057415     10.2.3.3 → 10.2.3.2     BGP 194 UPDATE Message, UPDATE Message, UPDATE Message
16.060325     10.2.3.2 → 10.2.3.3     BGP 190 UPDATE Message, UPDATE Message, UPDATE Message

If both sides try at the same time and two TCP connections come up, the one opened by the speaker with the higher BGP Identifier is kept (RFC 4271 6.8).

STEP 3: An Unreachable Peer

Configure the nonexistent address 10.2.3.99 as a peer on R2. The state is Active.

R2 show bgp summary (STEP 3, excerpt)
RP/0/RP0/CPU0:R2#show bgp summary
<snip>
Neighbor        Spk    AS MsgRcvd MsgSent       TblVer  InQ OutQ  Up/Down  St/PfxRcd
10.0.0.1          0 65001      44      49           10    0    0 00:40:56          1
10.2.3.3          0 65002      43      44           10    0    0 00:07:04          2
10.2.3.99         0 65099       0       0            0    0    0 00:00:00  Active

Yet the message counts and the number of established connections are 0, and even the local port is unassigned (Local port: 0). Although the state is Active, not a single TCP connection request has gone out.

R2 show bgp neighbor 10.2.3.99 (STEP 3, excerpt)
BGP neighbor is 10.2.3.99
 Remote AS 65099, local AS 65001, external link
<snip>
  BGP state = Active
  Previous State: Idle
  Last Received Message: None
<snip>
  Received 0 messages, 0 notifications, 0 in queue
  Sent 0 messages, 0 notifications, 0 in queue
<snip>
  Connections established 0; dropped 0
  Local host: 10.2.3.2, Local port: 0, IF Handle: 0x00000000
  Foreign host: 10.2.3.99, Foreign port: 0
  Last reset 00:00:00

10.2.3.99 is on the same segment but does not exist, so ARP cannot resolve it.

R2 show arp (STEP 3)
RP/0/RP0/CPU0:R2#show arp
Mon Sep 21 12:34:11.126 UTC

-------------------------------------------------------------------------------
0/RP0/CPU0
-------------------------------------------------------------------------------
Address         Age        Hardware Addr   State      Type  Interface
10.1.2.1        00:41:13   5254.0004.87f3  Dynamic    ARPA  GigabitEthernet0/0/0/0
10.1.2.2        -          5254.00c3.c843  Interface  ARPA  GigabitEthernet0/0/0/0
10.2.3.2        -          5254.0090.1ff4  Interface  ARPA  GigabitEthernet0/0/0/1
10.2.3.3        00:41:05   5254.00ed.b852  Dynamic    ARPA  GigabitEthernet0/0/0/1

STEP 4: A Peer Behind a Route With No Exit

Add a static route on R2 that sends 10.9.9.9/32 to Null0, and configure that address as a peer. There is a route, but packets have no way out.

R2 show route 10.9.9.9/32 (STEP 4)
RP/0/RP0/CPU0:R2#show route 10.9.9.9/32
Mon Sep 21 12:38:54.238 UTC

Routing entry for 10.9.9.9/32
  Known via "static", distance 1, metric 0 (connected)
  Installed Sep 21 12:36:13.486 for 00:02:40
  Routing Descriptor Blocks
    directly connected, via Null0
      Route metric is 0
  No advertising protos. 

The state is Idle with the reason No best local address found. With no outgoing interface, R2 cannot choose a source address and cannot start TCP. Unlike STEP 3 (Active, Local host: 10.2.3.2), even Local host is 0.0.0.0.

R2 show bgp neighbor 10.9.9.9 (STEP 4, excerpt)
BGP neighbor is 10.9.9.9
 Remote AS 65099, local AS 65001, external link
<snip>
  BGP state = Idle (No best local address found)
  Previous State: Idle
<snip>
  Connections established 0; dropped 0
  Local host: 0.0.0.0, Local port: 0, IF Handle: 0x00000000
  Foreign host: 10.9.9.9, Foreign port: 0
  Last reset 00:00:00

STEP 5: session-open-mode passive-only on Both Sides

session-open-mode passive-only makes the router accept inbound TCP connections only and never open one itself. Configured on both R2 and R3, it is accepted and shown as TCP open mode: passive only.

R2 show bgp neighbor 10.2.3.3 (STEP 5, excerpt)
BGP neighbor is 10.2.3.3
 Remote AS 65002, local AS 65001, external link
<snip>
  BGP state = Established, up for 00:16:35
<snip>
  TCP open mode: passive only
<snip>
  Connections established 2; dropped 1

An established session is not torn down. The setting decides whether to open a TCP connection, and does not act on a session that is already up.

STEP 6: An eBGP Peer That Is Not Directly Connected

R2 and R4 are two hops apart, with R3 in between. After adding static routes to each other’s Loopback0, configure an eBGP peering between the Loopback0 addresses (update-source Loopback0, no ebgp-multihop).

R2 show bgp neighbor 10.0.0.4 (STEP 6, excerpt)
BGP neighbor is 10.0.0.4
 Remote AS 65003, local AS 65001, external link
<snip>
  BGP state = Idle (eBGP neighbor not directly connected)
<snip>
  Connections established 0; dropped 0
  Local host: 0.0.0.0, Local port: 0, IF Handle: 0x00000000
<snip>
  Last reset 00:00:00
  External BGP neighbor not directly connected.

The state is Idle (eBGP neighbor not directly connected), and no TCP connection has been started (Connections established 0). By default, eBGP only peers with directly connected neighbors. R4 shows the same. The mechanism, including how TTL is handled, is explained in iBGP vs. eBGP.

STEP 7: Establishing With ebgp-multihop

With ebgp-multihop 2 on both R2 and R4, the session comes up and the text changes to may be up to 2 hops away.

R2 show bgp summary (STEP 7, excerpt)
RP/0/RP0/CPU0:R2#show bgp summary
<snip>
Neighbor        Spk    AS MsgRcvd MsgSent       TblVer  InQ OutQ  Up/Down  St/PfxRcd
10.0.0.1          0 65001      64      69           11    0    0 01:00:57          1
10.0.0.4          0 65003      10      12           11    0    0 00:02:54          2
10.2.3.3          0 65002      63      65           11    0    0 00:27:05          2

R2 show bgp neighbor 10.0.0.4 (STEP 7, excerpt)
BGP neighbor is 10.0.0.4
 Remote AS 65003, local AS 65001, external link
<snip>
  BGP state = Established, up for 00:02:56
<snip>
  External BGP neighbor may be up to 2 hops away.

STEP 8: Mismatched AS Numbers

Clean up the test peers and static routes, then set R3’s remote-as on R2 to the wrong value 65999. R2 expects 65999 while R3 announces 65002, so the mismatch is detected in OpenSent. The state shows as Idle.

R2 show bgp summary (STEP 8, excerpt)
RP/0/RP0/CPU0:R2#show bgp summary
<snip>
Neighbor        Spk    AS MsgRcvd MsgSent       TblVer  InQ OutQ  Up/Down  St/PfxRcd
10.0.0.1          0 65001      72      79           14    0    0 01:08:39          1
10.2.3.3          0 65999      76      80            0    0    0 00:04:27  Idle

Error Code is peer in wrong AS, and the NOTIFICATION data (EAFD0000) is recorded too.

R2 show bgp neighbor 10.2.3.3 (STEP 8, excerpt)
BGP neighbor is 10.2.3.3
 Remote AS 65999, local AS 65001, external link
<snip>
  BGP state = Idle
<snip>
  Connections established 2; dropped 2
<snip>
  Last reset 00:00:09, due to BGP Notification sent: peer in wrong AS
<snip>
  Error Code: peer in wrong AS
  Notification data sent:
    EAFD0000

In the capture, R2 answers R3’s OPEN with a NOTIFICATION right away (No.15). This is the typical OpenSent failure: TCP comes up, the session gets as far as OPEN, and drops there, repeating every 30 to 40 seconds.

R2 - R3 packet capture (STEP 8, tshark listing, excerpt)
 0.000000     10.2.3.2 → 10.2.3.3     BGP 73 KEEPALIVE Message
43.575979     10.2.3.2 → 10.2.3.3     TCP 62 45521 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1240 WS=1
43.580362     10.2.3.3 → 10.2.3.2     TCP 62 179 → 45521 [SYN, ACK] Seq=0 Ack=1 Win=16384 Len=0 MSS=1240 WS=1
43.584497     10.2.3.2 → 10.2.3.3     BGP 129 OPEN Message
45.591283     10.2.3.3 → 10.2.3.2     BGP 148 OPEN Message, KEEPALIVE Message
45.600958     10.2.3.2 → 10.2.3.3     BGP 79 NOTIFICATION Message

The NOTIFICATION is error code 2 (OPEN Message Error), subcode 2 (Bad Peer AS).

STEP 8 No.15 NOTIFICATION (R2 to R3), tshark -V
Border Gateway Protocol - NOTIFICATION Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 25
    Type: NOTIFICATION Message (3)
    Major error Code: OPEN Message Error (2)
    Minor error Code (Open Message): Bad Peer AS (2)
    Bad Peer AS: 3942449152

Download the pcap of the packet in the tshark output above (STEP 8 No.15 NOTIFICATION)

Bad Peer AS: 3942449152 does not mean that R2 objected to that AS. The four data octets are ea fd 00 00: the two octets of 0xFDEA (= 65002, the AS that R3 announced) in reverse order, padded with zeros. Wireshark reads them as a 4-byte AS number and shows 0xEAFD0000 = 3942449152. RFC 4271 does not define the data for Bad Peer AS, so this is diagnostic information put there by the XRd implementation.

STEP 8 No.15 NOTIFICATION (R2 to R3), tshark -x
52 54 00 ed b8 52 52 54 00 90 1f f4 08 00 45 c0   RT...RRT......E.
00 41 b8 7f 00 00 01 06 e6 6f 0a 02 03 02 0a 02   .A.......o......
03 03 b1 d1 00 b3 43 d0 95 a5 cd ac 7f ae 50 18   ......C.......P.
7f a2 39 ad 00 00 ff ff ff ff ff ff ff ff ff ff   ..9.............
ff ff ff ff ff ff 00 19 03 02 02 ea fd 00 00      ...............

For comparison, the data for Bad BGP Identifier in STEP 10 is 0a 00 00 02 (= 10.0.0.2), in its natural order.

STEP 9: Duplicating the BGP Identifier (eBGP)

Restore remote-as and set R3’s BGP Identifier to the same 10.0.0.2 as R2. R2 and R3 are eBGP peers in different ASes. The session is re-established, and ends up Established; from R2, the peer’s Router ID is its own 10.0.0.2.

R2 show bgp neighbor 10.2.3.3 (STEP 9, excerpt)
BGP neighbor is 10.2.3.3
 Remote AS 65002, local AS 65001, external link
<snip>
 Remote router ID 10.0.0.2
  BGP state = Established, up for 00:02:19

In the capture too, R2 and R3 both send OPENs with BGP Identifier: 10.0.0.2 and move on to KEEPALIVE.

R2 - R3 packet capture (STEP 9, OPEN and KEEPALIVE)
6	10.2.3.2	10.2.3.3	10.0.0.2	OPEN Message
9	10.2.3.3	10.2.3.2	10.0.0.2	OPEN Message

As RFC 6286 says, a duplicate with a peer in another AS is not an error.

STEP 10: Duplicating the BGP Identifier (iBGP)

Restore R3, and this time set R1’s BGP Identifier to the same 10.0.0.2 as R2. R1 and R2 are iBGP peers in the same AS. This time the session does not come up, and the neighbor toward R1 stays Idle.

R2 show bgp summary (STEP 10, excerpt)
RP/0/RP0/CPU0:R2#show bgp summary
<snip>
Neighbor        Spk    AS MsgRcvd MsgSent       TblVer  InQ OutQ  Up/Down  St/PfxRcd
10.0.0.1          0 65001      89      97            0    0    0 00:03:24  Idle
10.2.3.3          0 65002      98     101           21    0    0 00:02:57          2

Error Code: BGP identifier wrong is recorded for both sent and received: R1 and R2 reject each other’s OPEN. The data 0A000002 is 10.0.0.2.

R2 show bgp neighbor 10.0.0.1 (STEP 10, excerpt)
BGP neighbor is 10.0.0.1
 Remote AS 65001, local AS 65001, internal link
<snip>
  BGP state = Idle
<snip>
  Last reset 00:00:11, due to BGP Notification received: BGP identifier wrong
<snip>
  Error Code: BGP identifier wrong
  Notification data sent:
    0A000002
<snip>
  Error Code: BGP identifier wrong
  Notification data received:
    0A000002

Error code 2 (OPEN Message Error), subcode 3 (Bad BGP Identifier) (No.20 in the R1 - R2 capture).

STEP 10 No.20 NOTIFICATION, tshark -V
Border Gateway Protocol - NOTIFICATION Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 25
    Type: NOTIFICATION Message (3)
    Major error Code: OPEN Message Error (2)
    Minor error Code (Open Message): Bad BGP Identifier (3)
    Data: 0a000002

Download the pcap of the packet in the tshark output above (STEP 10 No.20 NOTIFICATION)

The same duplicate gives opposite results in STEP 9 and STEP 10.

Duplicated with	AS relationship	Result
R3 (eBGP)	AS 65001 and AS 65002 (different)	Comes up. `Established` with `Remote router ID 10.0.0.2`
R1 (iBGP)	AS 65001 and AS 65001 (same)	Does not come up. `Idle` with `Bad BGP Identifier (3)`

STEP 11: A 1- or 2-Second Hold Time Cannot Be Configured

With R1 restored, try to set a 2-second Hold Time (timers 30 2) on R3’s neighbor toward R2. XRd refuses it in configuration mode with % Invalid input.

Entering timers 30 2 on R3 (STEP 11)
RP/0/RP0/CPU0:R3#configure
<snip>
RP/0/RP0/CPU0:R3(config)#router bgp 65002
RP/0/RP0/CPU0:R3(config-bgp)#neighbor 10.2.3.2
RP/0/RP0/CPU0:R3(config-bgp-nbr)#timers 30 2
                                           ^
% Invalid input detected at '^' marker.
<snip>
router bgp 65002
 neighbor 10.2.3.2
<snip>
RP/0/RP0/CPU0:R3(config-bgp-nbr)#abort
RP/0/RP0/CPU0:R3#show running-config router bgp 65002 neighbor 10.2.3.2
<snip>
router bgp 65002
 neighbor 10.2.3.2

RFC 4271 6.2 says that Hold Time values of one or two seconds must be rejected. XRd enforces this at configuration time, before it could ever lead to a NOTIFICATION (Unacceptable Hold Time) from the peer. As a result, no subcode 6 NOTIFICATION was observed in this lab.

STEP 12: Stopping Reception for the Hold Time

Configure an ACL on R3 that drops only TCP 179 from R2, creating a one-way failure (ipv4 unreachables disable keeps the drops from being reported by ICMP either).

R3 blocking only BGP from R2 (STEP 12)
ipv4 access-list BLOCK-BGP-FROM-R2
 10 deny tcp host 10.2.3.2 any eq bgp
 20 deny tcp host 10.2.3.2 eq bgp any
 30 permit ipv4 any any
!
interface GigabitEthernet0/0/0/0
 ipv4 unreachables disable
 ipv4 access-group BLOCK-BGP-FROM-R2 ingress
!

R2 sends a KEEPALIVE every 60 seconds, and the last one to arrive before the ACL was applied (13:20:49) was at 13:20:18.821 (the last KEEPALIVE in the STEP 11 capture). Exactly 180 seconds later, at 13:23:18.825, R3’s Hold Timer expired.

R3 syslog (STEP 12, excerpt)
RP/0/RP0/CPU0:Sep 21 13:20:49.828 UTC: config[67373]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user 'cisco'. Use 'show configuration commit changes 1000000006' to view the changes. 
RP/0/RP0/CPU0:Sep 21 13:23:18.825 UTC: bgp[1084]: %ROUTING-BGP-5-ADJCHANGE : neighbor 10.2.3.2 Down - BGP Notification sent, hold time expired (VRF: default) (AS: 65001) 

The capture contains R2’s KEEPALIVE dropped by the ACL (13:21:18.821) and the NOTIFICATION sent by R3.

R2 - R3 packet capture (STEP 12, BGP only, with timestamps)
2026-09-21 13:21:03.537977Z     10.2.3.3 → 10.2.3.2     BGP 73 KEEPALIVE Message
2026-09-21 13:21:18.821789Z     10.2.3.2 → 10.2.3.3     BGP 73 KEEPALIVE Message
2026-09-21 13:23:06.230332Z     10.2.3.2 → 10.2.3.3     BGP 73 [TCP ACKed unseen segment] , KEEPALIVE Message
2026-09-21 13:23:18.828735Z     10.2.3.3 → 10.2.3.2     BGP 75 NOTIFICATION Message
2026-09-21 13:23:18.831584Z     10.2.3.2 → 10.2.3.3     BGP 73 KEEPALIVE Message

The error code is 4 (Hold Timer Expired); no subcodes are defined, so it is 0.

STEP 12 No.28 NOTIFICATION (R3 to R2), tshark -V
Border Gateway Protocol - NOTIFICATION Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 21
    Type: NOTIFICATION Message (3)
    Major error Code: Hold Timer Expired (4)
    Minor error Code (Hold Timer Expired): 0

Download the pcap of the packet in the tshark output above (STEP 12 No.28 NOTIFICATION)

R3, whose timer expired, records Error Code: hold time expired as a sent NOTIFICATION; R2 records it as received.

R3 show bgp neighbor 10.2.3.2 (STEP 12, excerpt)
BGP neighbor is 10.2.3.2
 Remote AS 65001, local AS 65002, external link
<snip>
  BGP state = Active
<snip>
  Last reset 00:05:18, due to BGP Notification sent: hold time expired
<snip>
  Error Code: hold time expired
<snip>
  Error Code: peer in wrong AS

R2 show bgp neighbor 10.2.3.3 (STEP 12, excerpt)
BGP neighbor is 10.2.3.3
 Remote AS 65002, local AS 65001, external link
<snip>
  BGP state = Active
<snip>
  Last reset 00:04:37, due to BGP Notification received: hold time expired
<snip>
  Error Code: peer in wrong AS
<snip>
  Error Code: hold time expired

This step also shows what is displayed while SYNs go unanswered. After the drop, R2 tries to reconnect, but its SYNs are silently discarded by R3’s ACL and retransmitted at 2-, 4- and 8-second intervals. Meanwhile R2 shows Active, not Connect.

SYNs from R2 to R3 (STEP 12, tshark listing)
180.219906     10.2.3.2 → 10.2.3.3     TCP 62 56683 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1460 WS=1
182.220440     10.2.3.2 → 10.2.3.3     TCP 62 [TCP Retransmission] 56683 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1460 WS=1
186.220540     10.2.3.2 → 10.2.3.3     TCP 62 [TCP Retransmission] 56683 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1460 WS=1
194.221552     10.2.3.2 → 10.2.3.3     TCP 62 [TCP Retransmission] 56683 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1460 WS=1
260.170479     10.2.3.2 → 10.2.3.3     TCP 62 [TCP Port numbers reused] 56683 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1460 WS=1
262.170929     10.2.3.2 → 10.2.3.3     TCP 62 [TCP Retransmission] 56683 → 179 [SYN] Seq=0 Win=32768 Len=0 MSS=1460 WS=1

Hold Timer Expired happens not only when the peer goes down but also when communication fails in only one direction. ping may still work, so L3 reachability alone cannot isolate the problem.

STEP 13: Restoring the Configuration

When the ACL is removed, the session comes back at the next connection attempt (about 51 seconds after the commit).

R3 syslog (STEP 13, excerpt)
RP/0/RP0/CPU0:Sep 21 13:30:47.148 UTC: config[68614]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user 'cisco'. Use 'show configuration commit changes 1000000007' to view the changes. 
RP/0/RP0/CPU0:Sep 21 13:31:38.231 UTC: bgp[1084]: %ROUTING-BGP-5-ADJCHANGE : neighbor 10.2.3.2 Up (VRF: default) (AS: 65001) 

Summary of Error Codes

The NOTIFICATIONs observed in this lab. Error Code in show bgp neighbor corresponds to the code on the wire.

STEP	Action	Error code	Subcode	`Error Code` in `show bgp neighbor`
1	`shutdown`	`6` Cease	`2` Administratively Shutdown	`administrative shutdown`
8	Wrong `remote-as`	`2` OPEN Message Error	`2` Bad Peer AS	`peer in wrong AS`
10	Duplicate BGP Identifier in iBGP	`2` OPEN Message Error	`3` Bad BGP Identifier	`BGP identifier wrong`
12	Nothing received within the Hold Time	`4` Hold Timer Expired	`0` (not defined)	`hold time expired`

Verification Config and Command Outputs

For every step, the following files were collected from all four routers, one per router. The verification config is the ..._run.txt file (the final state is the one from STEP 13).

File	Contents
`..._show.txt`	`show version` / `show interface description` / `show route` / `show bgp` / `show bgp summary` / `show bgp <prefix>` / `show bgp neighbors` / per-peer `advertised-routes`, `routes` and `received routes` / `show bgp update-group` / `show rpl route-policy`. R1 and R2 also include `show ospf neighbor`. R2 in STEP 3 adds `show arp`, R2 in STEP 4 adds `show route 10.9.9.9/32`, and R3 in STEPs 12 and 13 adds `show access-lists ipv4`
`..._log.txt`	`show logging` limited to that step. A `logmsg` marker is written at the start of each step and its timestamp passed to `show logging start`
`..._run.txt`	`show running-config` at that step (that is, the verification config for the step)
`..._ping.txt`	The `ping` and `traceroute` of that step (50 packets from R1, R2 and R4 toward `192.168.4.1` or `192.168.1.1`)
`..._trace.txt`	`show bgp trace neighbor` and `show bgp trace error` (internal events of the BGP process)
`..._commit.cfg`	The configuration actually committed in that step (only for the routers that were changed)
`..._exec.txt`	The attempt to enter a configuration that was refused (R3 in STEP 11)

The final state is the same as the initial one: R1 - R2 iBGP (Loopback0 peering, OSPF area 0), R2 - R3 and R3 - R4 eBGP, and each router advertising 192.168.N.0/24.

STEP 0: Initial state

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	—
R3	show	log	run	—	trace	—
R4	show	log	run	ping	trace	—

STEP 1: shutdown R2’s neighbor toward R3

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	commit
R3	show	log	run	—	trace	—
R4	show	log	run	ping	trace	—

STEP 2: Restore with no shutdown

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	commit
R3	show	log	run	—	trace	—
R4	show	log	run	ping	trace	—

STEP 3: Configure a nonexistent peer 10.2.3.99 on R2

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	commit
R3	show	log	run	—	trace	—
R4	show	log	run	ping	trace	—

STEP 4: Configure on R2 a peer 10.9.9.9 behind a route pointing at Null0

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	commit
R3	show	log	run	—	trace	—
R4	show	log	run	ping	trace	—

STEP 5: session-open-mode passive-only on both R2 and R3

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	commit
R3	show	log	run	—	trace	commit
R4	show	log	run	ping	trace	—

STEP 6: Peer R2 - R4 on Loopback0 (no ebgp-multihop)

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	commit
R3	show	log	run	—	trace	commit
R4	show	log	run	ping	trace	commit

STEP 7: ebgp-multihop 2 on R2 and R4

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	commit
R3	show	log	run	—	trace	—
R4	show	log	run	ping	trace	commit

STEP 8: Clean up the test peers and set R3’s remote-as on R2 to 65999

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	commit
R3	show	log	run	—	trace	commit
R4	show	log	run	ping	trace	commit

STEP 9: Restore remote-as and set R3’s BGP Identifier to 10.0.0.2 (eBGP)

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	commit
R3	show	log	run	—	trace	commit
R4	show	log	run	ping	trace	—

STEP 10: Restore R3 and set R1’s BGP Identifier to 10.0.0.2 (iBGP)

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	commit
R2	show	log	run	ping	trace	—
R3	show	log	run	—	trace	commit
R4	show	log	run	ping	trace	—

STEP 11: Restore R1 and try to configure timers 30 2 on R3

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	commit
R2	show	log	run	ping	trace	—
R3	show	log	run	—	trace	exec
R4	show	log	run	ping	trace	—

STEP 12: Block TCP 179 from R2 with an ACL on R3

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	—
R3	show	log	run	—	trace	commit
R4	show	log	run	ping	trace	—

STEP 13: Remove the ACL (final state)

Router	show output	syslog	running-config	ping	trace	Committed config
R1	show	log	run	ping	trace	—
R2	show	log	run	ping	trace	—
R3	show	log	run	—	trace	commit
R4	show	log	run	ping	trace	—

Packet captures were taken per step (a dash means no capture on that link).

STEP	R2 - R3	R3 - R4	R1 - R2
0	pcap	—	—
1	pcap	—	—
2	pcap	—	—
3	pcap	—	—
4	pcap	—	—
5	pcap	—	—
6	pcap	pcap	—
7	pcap	pcap	—
8	pcap	—	—
9	pcap	—	—
10	pcap	—	pcap
11	pcap	—	pcap
12	pcap	—	—
13	pcap	—	—

References

Source	Title	Summary
RFC 4271	A Border Gateway Protocol 4 (BGP-4)	The base BGP-4 specification. Chapter 8 defines the FSM, 6.2 OPEN validation, and chapter 10 the suggested timer values.
RFC 6286	Autonomous-System-Wide Unique BGP Identifier for BGP-4	Relaxes the BGP Identifier uniqueness requirement to “unique within an AS”.
IANA	Border Gateway Protocol (BGP) Parameters	The registry of numbers used in BGP, including NOTIFICATION error codes and subcodes.