BGP COMMUNITY Attribute

What COMMUNITY Is

COMMUNITY is a path attribute that carries a “tag” along with a route. Unlike AS_PATH and LOCAL_PREF, covered in BGP Path Attributes and Best Path Selection, it has no effect on best path selection by itself. It only becomes meaningful when a router that sees the tag decides, through its own policy, what to do about it.

Item	Value
Type code	8
Attribute class	Optional Transitive
Value	32 bits (4 octets) per community; several can be carried together
Notation	`AS:value` (upper 16 bits : lower 16 bits)
Reserved ranges	`0x00000000`–`0x0000FFFF` and `0xFFFF0000`–`0xFFFFFFFF`
Best path selection	No effect. It only tags the route
Definition	RFC 1997 BGP Communities Attribute (Standards Track)

Because it is Optional Transitive, a router that does not understand the community passes it on rather than dropping it. That is the point of communities: a tag set by AS 65004 can travel through AS 65001 and reach AS 65002.

The `AS:value` Notation

The 32 bits are split into an upper and a lower 16-bit half and written as 65004:1000. By convention the upper half holds your own AS number and the lower half a value whose meaning your organisation defines. RFC 1997 does not mandate this split, but using the AS number as a namespace keeps your tags from colliding with anyone else’s.

Note that because the upper half is only 16 bits, an organisation with a 4-byte AS number cannot put its own AS number there. Large Communities, described below, solve that.

Tagging and Acting on Tags Are Two Different ASes

A community does nothing on its own. In practice two roles always pair up.

Role	What it does	Typical party
Tagger	Attaches a tag with `set community` and advertises the route	Customer, downstream AS
Actor	Matches the tag on receipt with `community matches-any`, then changes `LOCAL_PREF` or narrows where the route is advertised	Upstream ISP

If the actor does nothing, the tag simply rides along and nothing happens. Conversely, when an upstream ISP publishes “tag your route with this community and we will treat it this way,” customers gain control over routing without asking the ISP to change anything. That is the main way communities are used in production.

Note that a user-defined community carries no force of its own. Unlike the well-known communities described below, such as NO_EXPORT, its meaning is decided by the receiving AS, so it behaves as advertised only when every border router in that AS carries the same match. Miss one router and that router advertises the route as usual.

A Real Example: NTT (AS2914)

NTT’s Global IP Network (AS2914) publishes the communities its customers may use in its Routing Policies. An excerpt:

Community	Meaning
`2914:429`	do not advertise to any peers
`2914:421`–`2914:423`	prepend the AS_PATH 1–3 times toward all peers
`2914:490`	customer default (LOCAL_PREF 120)
`2914:450`	peer fallback (LOCAL_PREF 96)
`65500:nnn`	do not announce to peer AS `nnn`
`2914:666`	blackhole (discard traffic toward the prefix)

A customer only has to attach one of these and advertise the route to control the LOCAL_PREF inside AS2914 or which peers see it. No configuration change is needed on the AS2914 side. STEP 8 below reproduces this pattern in the lab with 65001:429, modelled on 2914:429.

Well-Known Communities

RFC 1997 defines three values that carry the same meaning in every AS. All of them sit in the 0xFFFF reserved range.

Name	Hex	What RFC 1997 says
`NO_EXPORT`	`0xFFFFFF01`	MUST NOT be advertised outside a BGP confederation boundary
`NO_ADVERTISE`	`0xFFFFFF02`	MUST NOT be advertised to other BGP peers
`NO_EXPORT_SUBCONFED`	`0xFFFFFF03`	MUST NOT be advertised to external BGP peers (including peers in other member ASes inside a confederation)

In an ordinary design without confederations, NO_EXPORT means “do not send it outside my AS (to eBGP peers), but do distribute it over iBGP,” while NO_ADVERTISE means “do not send it anywhere, iBGP included.” STEP 6 and STEP 7 show both on real hardware.

Two more well-known communities were added later.

Name	Hex	`AS:value`	Definition
`GRACEFUL_SHUTDOWN`	`0xFFFF0000`	`65535:0`	RFC 8326 Graceful BGP Session Shutdown (Standards Track). Reduces packet loss during planned maintenance
`BLACKHOLE`	`0xFFFF029A`	`65535:666`	RFC 7999 BLACKHOLE Community (Informational). Asks a neighbouring network to discard traffic toward the prefix

Extended and Large Communities

There are two derivatives. This article does not cover them, but here is where they sit.

Attribute	Type code	Length each	Structure	Definition
COMMUNITY	8	4 octets	16 bits + 16 bits	RFC 1997
Extended Community	16	8 octets	Type/Sub-Type + value. Used for MPLS-VPN Route Targets and similar	RFC 4360
Large Community	32	12 octets	Three 32-bit fields (Global Administrator + Local Data Part 1 and 2)	RFC 8092

The essential point about Large Communities is that a 4-byte AS number fits in the namespace field. You can write 64500:1:2 and still carry two operator-defined values behind a 32-bit AS number.

Configuring Communities on IOS XR

Communities are handled in RPL (Routing Policy Language).

Tagging

route-policy TO-UPSTREAM
  if destination in (192.168.6.0/24) then
    set community (65004:1000)
  endif
  pass
end-policy

set community replaces whatever was already there. To keep the existing communities and add to them, append additive.

  set community (65001:100) additive

STEP 3 and STEP 4 measure the difference.

Matching

Define a set of values with community-set, then match with community matches-any (any one value matches) or matches-every (all values must match).

community-set CS-FROM-CUSTOMER
  65004:1000
end-set
!
route-policy FROM-CUSTOMER
  if community matches-any CS-FROM-CUSTOMER then
    set local-preference 200
  endif
  pass
end-policy

The match is evaluated against the value as it entered the policy. Running set community later in the same policy does not change the outcome of an if community matches-any written above it. To match on the rewritten value, split the work into two policies applied in sequence.

Removing

  delete community all                  ! remove everything
  delete community in CS-FROM-CUSTOMER  ! remove only what the community-set lists

eBGP Needs `send-community-ebgp`

IOS XR sends communities to iBGP peers by default but not to eBGP peers. To carry them over eBGP, add send-community-ebgp under the neighbor’s address family.

router bgp 65004
 neighbor 10.1.6.1
  address-family ipv4 unicast
   send-community-ebgp
   route-policy TO-UPSTREAM out

show bgp neighbor reports Community attribute sent to this neighbor when it is in place. Without that line, set community produces nothing the eBGP peer can see.

Verification on Real Hardware

A lab of six XRd routers (IOS XR 26.1.1) confirms the behaviour described above in nine STEPs.

The COMMUNITY verification lab topology. R1 and R2 in AS 65001 peer eBGP with R4 in AS 65002, and R4 also peers with R5 in AS 65003. R5 peers with R6 in AS 65004, and R6 also peers with R1. R6 tags 192.168.6.0/24 with a community and advertises it to R1

AS	Router	Role
65001	R1, R2	AS border. iBGP full mesh with R3 (Loopback0, `next-hop-self`) plus OSPF area 0
65001	R3	Internal. No eBGP peers
65002	R4	eBGP with R1, R2 and R5
65003	R5	eBGP with R4 and R6. Advertises `192.168.5.0/24`
65004	R6	eBGP with R5 and R1. Advertises `192.168.6.0/24`

The prefix under observation is 192.168.6.0/24, originated by R6. It travels R6 → (eBGP) → R1 → (iBGP) → R2 and R3, and R1 → (eBGP) → R4. Because it crosses both an eBGP hop and an iBGP hop, one prefix is enough to see how far a community travels and where it stops.

STEP	Change	What to look for
0	No policy	No community attached
1	R6 adds `set community (65004:1000)`	The value arrives over eBGP and crosses iBGP to R2, R3 and R4
2	R1 raises `LOCAL_PREF 200` on `matches-any`	The acting side of the pair
3	R1 adds `set community (65001:100)` without `additive`	The original `65004:1000` disappears
4	Add `additive`	Both values are carried
5	R1 inbound back to `PASS-ALL`	Only the original value remains
6	R6 sets `NO_EXPORT`	R1 and R2 stop advertising to eBGP; iBGP still carries it
7	R6 sets `NO_ADVERTISE`	R1 stops advertising over iBGP too and withdraws from R2 and R3
8	R6 sets `65001:429`; R1 drops it in `TO-R4`	Advertisement control with a user-defined community

Every eBGP neighbor carries send-community-ebgp and a separately named outbound policy (TO-R1, TO-R4 and so on, each containing only pass) from the start. The names are kept distinct per neighbor so that update-group membership stays fixed across the STEPs; neighbors that share a policy name are merged into one update-group, which merges what they are advertised. Each STEP edits only the contents of a policy.

STEP 0: Initial State

The 192.168.6.0/24 that R1 received.

R1 show bgp 192.168.6.0/24 (STEP 0)
RP/0/RP0/CPU0:R1#show bgp 192.168.6.0/24

Wed Sep  9 05:50:59.631 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                  7            7
Last Modified: Sep  9 05:14:11.774 for 00:36:47
Paths: (1 available, best #1)
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  Path #1: Received by speaker 0
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  65004, (received & used)
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 100, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 7
      Origin-AS validity: (disabled)

There is no Community: line. R6’s outbound policy contains only pass, so no community is attached.

STEP 1: R6 Tags `192.168.6.0/24` with `65004:1000`

R6’s outbound policy toward R1, TO-R1, is rewritten.

R6 show rpl route-policy TO-R1 (STEP 1)
RP/0/RP0/CPU0:R6#show rpl route-policy TO-R1

Wed Sep  9 06:14:29.887 UTC
route-policy TO-R1
  if destination in (192.168.6.0/24) then
    set community (65004:1000)
  endif
  pass
end-policy
!

A Community: line now appears on R1.

R1 show bgp 192.168.6.0/24 (STEP 1)
RP/0/RP0/CPU0:R1#show bgp 192.168.6.0/24

Wed Sep  9 06:12:17.878 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                  9            9
Last Modified: Sep  9 06:10:23.774 for 00:01:54
Paths: (1 available, best #1)
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  Path #1: Received by speaker 0
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  65004, (received & used)
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 100, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 9
      Community: 65004:1000
      Origin-AS validity: (disabled)

The capture confirms it. No.213 in the attached eBGP capture is the UPDATE sent from R6 (10.1.6.6) to R1 (10.1.6.1).

No.213 UPDATE (R6 to R1) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 67
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 0
    Total Path Attribute Length: 44
    Path attributes
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
<snip>
            Type Code: MP_REACH_NLRI (14)
            Length: 13
            Address family identifier (AFI): IPv4 (1)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 10.1.6.6
                IPv4 Address: 10.1.6.6
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                192.168.6.0/24
                    MP Reach NLRI prefix length: 24
                    MP Reach NLRI IPv4 prefix: 192.168.6.0
        Path Attribute - ORIGIN: IGP
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: ORIGIN (1)
            Length: 1
            Origin: IGP (0)
        Path Attribute - AS_PATH: 65004
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: AS_PATH (2)
            Length: 6
            AS Path segment: 65004
                Segment type: AS_SEQUENCE (2)
                Segment length (number of ASN): 1
                AS4: 65004
        Path Attribute - MULTI_EXIT_DISC: 0
            Flags: 0x80, Optional, Non-transitive, Complete
<snip>
            Type Code: MULTI_EXIT_DISC (4)
            Length: 4
            Multiple exit discriminator: 0
        Path Attribute - COMMUNITIES: 65004:1000
            Flags: 0xc0, Optional, Transitive, Complete
<snip>
            Type Code: COMMUNITIES (8)
            Length: 4
            Communities: 65004:1000
                Community: 65004:1000
                    Community AS: 65004
                    Community value: 1000

Flags: 0xc0, Optional, Transitive, Complete shows that this attribute is Optional Transitive.

Download the pcap of the packet in the tshark output above (No.213 UPDATE)

It Crosses the iBGP Hop

R1 passes the route to R2 and R3 over iBGP. IOS XR sends communities to iBGP peers by default, so the value arrives without any extra configuration.

R2 show bgp 192.168.6.0/24 (STEP 1)
RP/0/RP0/CPU0:R2#show bgp 192.168.6.0/24

Wed Sep  9 06:12:46.531 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                  9            9
Last Modified: Sep  9 06:10:23.774 for 00:02:22
Paths: (1 available, best #1)
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.2.4.4        
  Path #1: Received by speaker 0
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.2.4.4        
  65004, (received & used)
    10.0.0.1 (metric 3) from 10.0.0.1 (10.0.0.1)
      Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 9
      Community: 65004:1000

No.427 in the R1-R3 iBGP capture is that UPDATE.

No.427 UPDATE in the iBGP capture (R1 to R3) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 74
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 0
    Total Path Attribute Length: 51
    Path attributes
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
<snip>
            Type Code: MP_REACH_NLRI (14)
            Length: 13
            Address family identifier (AFI): IPv4 (1)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 10.0.0.1
                IPv4 Address: 10.0.0.1
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                192.168.6.0/24
                    MP Reach NLRI prefix length: 24
                    MP Reach NLRI IPv4 prefix: 192.168.6.0
        Path Attribute - ORIGIN: IGP
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: ORIGIN (1)
            Length: 1
            Origin: IGP (0)
        Path Attribute - AS_PATH: 65004
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: AS_PATH (2)
            Length: 6
            AS Path segment: 65004
                Segment type: AS_SEQUENCE (2)
                Segment length (number of ASN): 1
                AS4: 65004
        Path Attribute - MULTI_EXIT_DISC: 0
            Flags: 0x80, Optional, Non-transitive, Complete
<snip>
            Type Code: MULTI_EXIT_DISC (4)
            Length: 4
            Multiple exit discriminator: 0
        Path Attribute - LOCAL_PREF: 100
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: LOCAL_PREF (5)
            Length: 4
            Local preference: 100
        Path Attribute - COMMUNITIES: 65004:1000
            Flags: 0xc0, Optional, Transitive, Complete
<snip>
            Type Code: COMMUNITIES (8)
            Length: 4
            Communities: 65004:1000
                Community: 65004:1000
                    Community AS: 65004
                    Community value: 1000

Download the pcap of the packet in the tshark output above (No.427 UPDATE)

R1 also advertises the route to R4 over eBGP, so 65004:1000 reaches R4 in AS 65002 as well. A tag set by AS 65004 passed straight through AS 65001 and arrived in AS 65002. That is what Optional Transitive buys you.

R4 show bgp community 65004:1000 (STEP 1)
RP/0/RP0/CPU0:R4#show bgp community 65004:1000

Wed Sep  9 06:13:36.383 UTC
BGP router identifier 10.0.0.4, local AS number 65002
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000000   RD version: 11
BGP main routing table version 11
BGP NSR Initial initsync version 4 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
*> 192.168.6.0/24     10.1.4.1                               0 65001 65004 i
*                     10.2.4.2                               0 65001 65004 i

Processed 1 prefixes, 2 paths

STEP 2: R1 Matches the Community and Raises `LOCAL_PREF`

So far the tag has only been attached. Now R1 gets the configuration that acts on it.

R1 show rpl community-set CS-FROM-R6 (STEP 2)
RP/0/RP0/CPU0:R1#show rpl community-set CS-FROM-R6

Wed Sep  9 06:17:48.295 UTC
community-set CS-FROM-R6
  65004:1000
end-set
!

R1 show rpl route-policy FROM-R6 (STEP 2)
RP/0/RP0/CPU0:R1#show rpl route-policy FROM-R6

Wed Sep  9 06:17:48.790 UTC
route-policy FROM-R6
  if community matches-any CS-FROM-R6 then
    set local-preference 200
  endif
  pass
end-policy
!

This is applied inbound on the neighbor toward R6. localpref becomes 200.

R1 show bgp 192.168.6.0/24 (STEP 2)
RP/0/RP0/CPU0:R1#show bgp 192.168.6.0/24

Wed Sep  9 06:17:43.938 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 10           10
Last Modified: Sep  9 06:17:31.774 for 00:00:12
Paths: (2 available, best #1)
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  Path #1: Received by speaker 0
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  65004
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 200, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 10
      Community: 65004:1000
      Origin-AS validity: (disabled)
  Path #2: Received by speaker 0
  Not advertised to any peer
  65004, (received-only)
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 100, valid, external
      Received Path ID 0, Local Path ID 0, version 0
      Community: 65004:1000
      Origin-AS validity: (disabled)

Path #2, marked (received-only), is the copy before the inbound policy ran, retained by soft-reconfiguration inbound always. Its localpref is still 100 and its community is 65004:1000. Path #1 is the post-policy result with localpref 200. The difference between the two is exactly what the policy did.

STEP 3: Without `additive`, `set community` Erases the Original

set community (65001:100) is added to FROM-R6, with no additive.

R1 show rpl route-policy FROM-R6 (STEP 3)
RP/0/RP0/CPU0:R1#show rpl route-policy FROM-R6

Wed Sep  9 06:27:05.372 UTC
route-policy FROM-R6
  if community matches-any CS-FROM-R6 then
    set local-preference 200
  endif
  set community (65001:100)
  pass
end-policy
!

R1 show bgp 192.168.6.0/24 (STEP 3)
RP/0/RP0/CPU0:R1#show bgp 192.168.6.0/24

Wed Sep  9 06:26:59.651 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 12           12
Last Modified: Sep  9 06:26:26.774 for 00:00:32
Paths: (2 available, best #1)
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  Path #1: Received by speaker 0
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  65004
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 200, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 12
      Community: 65001:100
      Origin-AS validity: (disabled)
  Path #2: Received by speaker 0
  Not advertised to any peer
  65004, (received-only)
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 100, valid, external
      Received Path ID 0, Local Path ID 0, version 0
      Community: 65004:1000
      Origin-AS validity: (disabled)

Path #1 (post-policy) now carries only 65001:100: 65004:1000 is gone. Path #2 (pre-policy) still has 65004:1000.

localpref 200 is still applied. The test ran against the value as it stood before set community executed (65004:1000), exactly as described above.

There is a second side effect. This set community sits outside the if block, so it applies to every route received from R6. 192.168.5.0/24, learned via R6, is tagged 65001:100 too (Path #3).

R1 show bgp 192.168.5.0/24 (STEP 3)
RP/0/RP0/CPU0:R1#show bgp 192.168.5.0/24

Wed Sep  9 06:26:59.411 UTC
BGP routing table entry for 192.168.5.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 11           11
Last Modified: Sep  9 06:26:26.774 for 00:00:32
Paths: (4 available, best #2)
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.6.6        
  Path #1: Received by speaker 0
  Not advertised to any peer
  65002 65003, (received & used)
    10.0.0.2 (metric 3) from 10.0.0.2 (10.0.0.2)
      Origin IGP, localpref 100, valid, internal
      Received Path ID 0, Local Path ID 0, version 0
  Path #2: Received by speaker 0
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.6.6        
  65002 65003, (received & used)
    10.1.4.4 from 10.1.4.4 (10.0.0.4)
      Origin IGP, localpref 100, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 8
      Origin-AS validity: (disabled)
  Path #3: Received by speaker 0
  Not advertised to any peer
  65004 65003
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, localpref 100, valid, external, group-best
      Received Path ID 0, Local Path ID 0, version 0
      Community: 65001:100
      Origin-AS validity: (disabled)
  Path #4: Received by speaker 0
  Not advertised to any peer
  65004 65003, (received-only)
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, localpref 100, valid, external
      Received Path ID 0, Local Path ID 0, version 0
      Origin-AS validity: (disabled)

No.559 in the iBGP capture carries the replaced value to R3.

No.559 UPDATE in the iBGP capture (R1 to R3) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 74
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 0
    Total Path Attribute Length: 51
    Path attributes
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
<snip>
            Type Code: MP_REACH_NLRI (14)
            Length: 13
            Address family identifier (AFI): IPv4 (1)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 10.0.0.1
                IPv4 Address: 10.0.0.1
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                192.168.6.0/24
                    MP Reach NLRI prefix length: 24
                    MP Reach NLRI IPv4 prefix: 192.168.6.0
        Path Attribute - ORIGIN: IGP
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: ORIGIN (1)
            Length: 1
            Origin: IGP (0)
        Path Attribute - AS_PATH: 65004
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: AS_PATH (2)
            Length: 6
            AS Path segment: 65004
                Segment type: AS_SEQUENCE (2)
                Segment length (number of ASN): 1
                AS4: 65004
        Path Attribute - MULTI_EXIT_DISC: 0
            Flags: 0x80, Optional, Non-transitive, Complete
<snip>
            Type Code: MULTI_EXIT_DISC (4)
            Length: 4
            Multiple exit discriminator: 0
        Path Attribute - LOCAL_PREF: 200
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: LOCAL_PREF (5)
            Length: 4
            Local preference: 200
        Path Attribute - COMMUNITIES: 65001:100
            Flags: 0xc0, Optional, Transitive, Complete
<snip>
            Type Code: COMMUNITIES (8)
            Length: 4
            Communities: 65001:100
                Community: 65001:100
                    Community AS: 65001
                    Community value: 100

Download the pcap of the packet in the tshark output above (No.559 UPDATE)

STEP 4: With `additive`, Both Survive

additive is appended to the same line.

R1 show rpl route-policy FROM-R6 (STEP 4)
RP/0/RP0/CPU0:R1#show rpl route-policy FROM-R6

Wed Sep  9 06:32:15.350 UTC
route-policy FROM-R6
  if community matches-any CS-FROM-R6 then
    set local-preference 200
  endif
  set community (65001:100) additive
  pass
end-policy
!

R1 show bgp 192.168.6.0/24 (STEP 4)
RP/0/RP0/CPU0:R1#show bgp 192.168.6.0/24

Wed Sep  9 06:32:09.384 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 13           13
Last Modified: Sep  9 06:31:52.774 for 00:00:16
Paths: (2 available, best #1)
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  Path #1: Received by speaker 0
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  65004
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 200, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 13
      Community: 65001:100 65004:1000
      Origin-AS validity: (disabled)
  Path #2: Received by speaker 0
  Not advertised to any peer
  65004, (received-only)
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 100, valid, external
      Received Path ID 0, Local Path ID 0, version 0
      Community: 65004:1000
      Origin-AS validity: (disabled)

Community: 65001:100 65004:1000 — both are present. An upstream that wants to add its own tag while preserving the customer’s must use additive.

No.605 UPDATE in the iBGP capture (R1 to R2) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 78
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 0
    Total Path Attribute Length: 55
    Path attributes
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
<snip>
            Type Code: MP_REACH_NLRI (14)
            Length: 13
            Address family identifier (AFI): IPv4 (1)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 10.0.0.1
                IPv4 Address: 10.0.0.1
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                192.168.6.0/24
                    MP Reach NLRI prefix length: 24
                    MP Reach NLRI IPv4 prefix: 192.168.6.0
        Path Attribute - ORIGIN: IGP
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: ORIGIN (1)
            Length: 1
            Origin: IGP (0)
        Path Attribute - AS_PATH: 65004
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: AS_PATH (2)
            Length: 6
            AS Path segment: 65004
                Segment type: AS_SEQUENCE (2)
                Segment length (number of ASN): 1
                AS4: 65004
        Path Attribute - MULTI_EXIT_DISC: 0
            Flags: 0x80, Optional, Non-transitive, Complete
<snip>
            Type Code: MULTI_EXIT_DISC (4)
            Length: 4
            Multiple exit discriminator: 0
        Path Attribute - LOCAL_PREF: 200
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: LOCAL_PREF (5)
            Length: 4
            Local preference: 200
        Path Attribute - COMMUNITIES: 65001:100 65004:1000
            Flags: 0xc0, Optional, Transitive, Complete
<snip>
            Type Code: COMMUNITIES (8)
            Length: 8
            Communities: 65001:100 65004:1000
                Community: 65001:100
                    Community AS: 65001
                    Community value: 100
                Community: 65004:1000
                    Community AS: 65004
                    Community value: 1000

Download the pcap of the packet in the tshark output above (No.605 UPDATE)

Length: 8 confirms two 4-octet communities in the attribute.

STEP 5: Removing R1’s Inbound Policy

The well-known communities come next, so R1’s inbound policy toward R6 goes back to PASS-ALL.

R1 show bgp 192.168.6.0/24 (STEP 5)
RP/0/RP0/CPU0:R1#show bgp 192.168.6.0/24

Wed Sep  9 06:34:55.670 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 15           15
Last Modified: Sep  9 06:34:43.774 for 00:00:12
Paths: (1 available, best #1)
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  Path #1: Received by speaker 0
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.1.4.4        
  65004, (received & used)
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 100, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 15
      Community: 65004:1000
      Origin-AS validity: (disabled)

localpref is back to 100 and the only community is the 65004:1000 that R6 attached.

STEP 6: `NO_EXPORT` Stops at the AS Boundary

R6’s TO-R1 becomes set community (no-export).

R6 show rpl route-policy TO-R1 (STEP 6)
RP/0/RP0/CPU0:R6#show rpl route-policy TO-R1

Wed Sep  9 06:37:59.186 UTC
route-policy TO-R1
  if destination in (192.168.6.0/24) then
    set community (no-export)
  endif
  pass
end-policy
!

R1’s BGP table:

R1 show bgp 192.168.6.0/24 (STEP 6)
RP/0/RP0/CPU0:R1#show bgp 192.168.6.0/24

Wed Sep  9 06:38:46.975 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 16           16
Last Modified: Sep  9 06:37:22.774 for 00:01:24
Paths: (2 available, best #2, not advertised to EBGP peer)
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Path #1: Received by speaker 0
  Not advertised to any peer
  65002 65003 65004, (received & used)
    10.1.4.4 from 10.1.4.4 (10.0.0.4)
      Origin IGP, localpref 100, valid, external, group-best
      Received Path ID 0, Local Path ID 0, version 0
      Origin-AS validity: (disabled)
  Path #2: Received by speaker 0
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  65004, (received & used)
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 100, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 16
      Community: no-export
      Origin-AS validity: (disabled)

The not advertised to EBGP peer in Paths: (2 available, best #2, not advertised to EBGP peer) is the mark of a received NO_EXPORT. 192.168.6.0/24 has indeed dropped out of what R1 advertises to R4.

R1 show bgp neighbors 10.1.4.4 advertised-routes (STEP 6)
RP/0/RP0/CPU0:R1#show bgp neighbors 10.1.4.4 advertised-routes

Wed Sep  9 06:38:50.455 UTC
Network            Next Hop        From            AS Path
192.168.1.0/24     10.1.4.1        Local           65001i
192.168.2.0/24     10.1.4.1        10.0.0.2        65001i
192.168.3.0/24     10.1.4.1        10.0.0.3        65001i

Processed 3 prefixes, 3 paths

It is still distributed over iBGP. R2 receives the route and ends up in the same “do not advertise to eBGP” state.

R2 show bgp 192.168.6.0/24 (STEP 6)
RP/0/RP0/CPU0:R2#show bgp 192.168.6.0/24

Wed Sep  9 06:39:16.253 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 14           14
Last Modified: Sep  9 06:37:22.774 for 00:01:53
Paths: (2 available, best #1, not advertised to EBGP peer)
  Not advertised to any peer
  Path #1: Received by speaker 0
  Not advertised to any peer
  65004, (received & used)
    10.0.0.1 (metric 3) from 10.0.0.1 (10.0.0.1)
      Origin IGP, metric 0, localpref 100, valid, internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 14
      Community: no-export
  Path #2: Received by speaker 0
  Not advertised to any peer
  65002 65003 65004, (received & used)
    10.2.4.4 from 10.2.4.4 (10.0.0.4)
      Origin IGP, localpref 100, valid, external, group-best
      Received Path ID 0, Local Path ID 0, version 0
      Origin-AS validity: (disabled)

R2 show bgp neighbors 10.2.4.4 advertised-routes (STEP 6)
RP/0/RP0/CPU0:R2#show bgp neighbors 10.2.4.4 advertised-routes

Wed Sep  9 06:39:19.485 UTC
Network            Next Hop        From            AS Path
192.168.1.0/24     10.2.4.2        10.0.0.1        65001i
192.168.2.0/24     10.2.4.2        Local           65001i
192.168.3.0/24     10.2.4.2        10.0.0.3        65001i

Processed 3 prefixes, 3 paths

Nothing was configured on R2. R1 handed the route over iBGP with NO_EXPORT still attached, so R2 stopped advertising it to eBGP on its own. The rule really does apply per AS, not per router.

With both R1 and R2 holding it back, R4 in AS 65002 can no longer learn 192.168.6.0/24 through AS 65001 and is left with the path via R5 (65003 65004).

R4 show bgp 192.168.6.0/24 (STEP 6)
RP/0/RP0/CPU0:R4#show bgp 192.168.6.0/24

Wed Sep  9 06:39:59.467 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 16           16
Last Modified: Sep  9 06:37:22.774 for 00:02:36
Paths: (1 available, best #1)
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.2.4.2        10.1.4.1        
  Path #1: Received by speaker 0
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.2.4.2        10.1.4.1        
  65003 65004, (received & used)
    10.4.5.5 from 10.4.5.5 (10.0.0.5)
      Origin IGP, localpref 100, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 16
      Origin-AS validity: (disabled)

No.321 in the eBGP capture carried the NO_EXPORT.

No.321 UPDATE (R6 to R1) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 67
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 0
    Total Path Attribute Length: 44
    Path attributes
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
<snip>
            Type Code: MP_REACH_NLRI (14)
            Length: 13
            Address family identifier (AFI): IPv4 (1)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 10.1.6.6
                IPv4 Address: 10.1.6.6
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                192.168.6.0/24
                    MP Reach NLRI prefix length: 24
                    MP Reach NLRI IPv4 prefix: 192.168.6.0
        Path Attribute - ORIGIN: IGP
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: ORIGIN (1)
            Length: 1
            Origin: IGP (0)
        Path Attribute - AS_PATH: 65004
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: AS_PATH (2)
            Length: 6
            AS Path segment: 65004
                Segment type: AS_SEQUENCE (2)
                Segment length (number of ASN): 1
                AS4: 65004
        Path Attribute - MULTI_EXIT_DISC: 0
            Flags: 0x80, Optional, Non-transitive, Complete
<snip>
            Type Code: MULTI_EXIT_DISC (4)
            Length: 4
            Multiple exit discriminator: 0
        Path Attribute - COMMUNITIES: NO_EXPORT
            Flags: 0xc0, Optional, Transitive, Complete
<snip>
            Type Code: COMMUNITIES (8)
            Length: 4
            Communities: NO_EXPORT
                Community Well-known: NO_EXPORT (0xffffff01)

Community Well-known: NO_EXPORT (0xffffff01) is the value straight out of RFC 1997.

Download the pcap of the packet in the tshark output above (No.321 UPDATE)

STEP 7: `NO_ADVERTISE` Stops at the Router

TO-R1 becomes set community (no-advertise).

R6 show rpl route-policy TO-R1 (STEP 7)
RP/0/RP0/CPU0:R6#show rpl route-policy TO-R1

Wed Sep  9 06:40:52.536 UTC
route-policy TO-R1
  if destination in (192.168.6.0/24) then
    set community (no-advertise)
  endif
  pass
end-policy
!

R1 now reports not advertised to any peer.

R1 show bgp 192.168.6.0/24 (STEP 7)
RP/0/RP0/CPU0:R1#show bgp 192.168.6.0/24

Wed Sep  9 06:41:35.790 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 17           17
Last Modified: Sep  9 06:40:35.774 for 00:01:00
Paths: (3 available, best #3, not advertised to any peer)
  Not advertised to any peer
  Path #1: Received by speaker 0
  Not advertised to any peer
  65002 65003 65004, (received & used)
    10.0.0.2 (metric 3) from 10.0.0.2 (10.0.0.2)
      Origin IGP, localpref 100, valid, internal
      Received Path ID 0, Local Path ID 0, version 0
  Path #2: Received by speaker 0
  Not advertised to any peer
  65002 65003 65004, (received & used)
    10.1.4.4 from 10.1.4.4 (10.0.0.4)
      Origin IGP, localpref 100, valid, external, group-best
      Received Path ID 0, Local Path ID 0, version 0
      Origin-AS validity: (disabled)
  Path #3: Received by speaker 0
  Not advertised to any peer
  65004, (received & used)
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 100, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 17
      Community: no-advertise
      Origin-AS validity: (disabled)

192.168.6.0/24 has also disappeared from what R1 advertises to its iBGP peers.

R1 show bgp neighbors 10.0.0.2 advertised-routes (STEP 7)
RP/0/RP0/CPU0:R1#show bgp neighbors 10.0.0.2 advertised-routes

Wed Sep  9 06:41:38.012 UTC
Network            Next Hop        From            AS Path
192.168.1.0/24     10.0.0.1        Local           i
192.168.4.0/24     10.0.0.1        10.1.4.4        65002i
192.168.5.0/24     10.0.0.1        10.1.4.4        65002 65003i

Processed 3 prefixes, 3 paths

In STEP 6 this list still contained 192.168.6.0/24. NO_EXPORT distributes over iBGP; NO_ADVERTISE does not.

R1 does not merely stop sending it from now on: it withdraws the route it had already advertised. No.681 in the iBGP capture is that withdraw.

No.681 UPDATE in the iBGP capture (R1 to R3, withdraw) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 27
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 4
    Withdrawn Routes
        192.168.6.0/24
            Withdrawn route prefix length: 24
            Withdrawn prefix: 192.168.6.0
    Total Path Attribute Length: 0

Download the pcap of the packet in the tshark output above (No.681 withdraw)

R3 does not lose 192.168.6.0/24 entirely, though: it falls back to the path through R2. R2 has the 65002 65003 65004 path from R4, and with R1’s path gone that becomes best.

R3 show bgp 192.168.6.0/24 (STEP 7)
RP/0/RP0/CPU0:R3#show bgp 192.168.6.0/24

Wed Sep  9 06:42:28.378 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 17           17
Last Modified: Sep  9 06:40:36.774 for 00:01:51
Paths: (1 available, best #1)
  Not advertised to any peer
  Path #1: Received by speaker 0
  Not advertised to any peer
  65002 65003 65004, (received & used)
    10.0.0.2 (metric 2) from 10.0.0.2 (10.0.0.2)
      Origin IGP, localpref 100, valid, internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 17

The AS_PATH grew from 65004 (via R1, one hop) to 65002 65003 65004 (R2 to R4 to R5 to R6, three hops). NO_ADVERTISE does not delete the destination; it only stops one path from being distributed, so any other path takes over.

No.335 in the eBGP capture carried the NO_ADVERTISE.

No.335 UPDATE (R6 to R1) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 67
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 0
    Total Path Attribute Length: 44
    Path attributes
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
<snip>
            Type Code: MP_REACH_NLRI (14)
            Length: 13
            Address family identifier (AFI): IPv4 (1)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 10.1.6.6
                IPv4 Address: 10.1.6.6
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                192.168.6.0/24
                    MP Reach NLRI prefix length: 24
                    MP Reach NLRI IPv4 prefix: 192.168.6.0
        Path Attribute - ORIGIN: IGP
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: ORIGIN (1)
            Length: 1
            Origin: IGP (0)
        Path Attribute - AS_PATH: 65004
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: AS_PATH (2)
            Length: 6
            AS Path segment: 65004
                Segment type: AS_SEQUENCE (2)
                Segment length (number of ASN): 1
                AS4: 65004
        Path Attribute - MULTI_EXIT_DISC: 0
            Flags: 0x80, Optional, Non-transitive, Complete
<snip>
            Type Code: MULTI_EXIT_DISC (4)
            Length: 4
            Multiple exit discriminator: 0
        Path Attribute - COMMUNITIES: NO_ADVERTISE
            Flags: 0xc0, Optional, Transitive, Complete
<snip>
            Type Code: COMMUNITIES (8)
            Length: 4
            Communities: NO_ADVERTISE
                Community Well-known: NO_ADVERTISE (0xffffff02)

Download the pcap of the packet in the tshark output above (No.335 UPDATE)

STEP 8: A User-Defined Community Narrows the Advertisement (Final State)

The last STEP reproduces NTT’s 2914:429 (do not advertise to any peers) in the lab. The premise is that AS 65001 publishes 65001:429 meaning “do not advertise to R4 (AS 65002),” and R6 attaches it.

On R6 the only change is set community (65001:429). Note that the upper half is the neighbour’s AS number, not R6’s own, because AS 65001 is the party that defines what this community means.

R6 show rpl route-policy TO-R1 (STEP 8)
RP/0/RP0/CPU0:R6#show rpl route-policy TO-R1

Wed Sep  9 06:44:06.630 UTC
route-policy TO-R1
  if destination in (192.168.6.0/24) then
    set community (65001:429)
  endif
  pass
end-policy
!

On the AS 65001 side, R1 gets the match and the drop.

R1 show rpl community-set CS-NO-ADV-R4 (STEP 8)
RP/0/RP0/CPU0:R1#show rpl community-set CS-NO-ADV-R4

Wed Sep  9 06:44:56.006 UTC
community-set CS-NO-ADV-R4
  65001:429
end-set
!

R1 show rpl route-policy TO-R4 (STEP 8)
RP/0/RP0/CPU0:R1#show rpl route-policy TO-R4

Wed Sep  9 06:44:54.180 UTC
route-policy TO-R4
  if community matches-any CS-NO-ADV-R4 then
    drop
  endif
  pass
end-policy
!

R1’s BGP table. Because this is not a well-known community, there is no special not advertised to ... annotation.

R1 show bgp 192.168.6.0/24 (STEP 8)
RP/0/RP0/CPU0:R1#show bgp 192.168.6.0/24

Wed Sep  9 06:44:49.322 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 18           18
Last Modified: Sep  9 06:43:35.774 for 00:01:13
Paths: (2 available, best #2)
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  Path #1: Received by speaker 0
  Not advertised to any peer
  65002 65003 65004, (received & used)
    10.1.4.4 from 10.1.4.4 (10.0.0.4)
      Origin IGP, localpref 100, valid, external, group-best
      Received Path ID 0, Local Path ID 0, version 0
      Origin-AS validity: (disabled)
  Path #2: Received by speaker 0
  Advertised IPv4 Unicast paths to update-groups (with more than one peer):
    0.2 
  65004, (received & used)
    10.1.6.6 from 10.1.6.6 (10.0.0.6)
      Origin IGP, metric 0, localpref 100, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 18
      Community: 65001:429
      Origin-AS validity: (disabled)

192.168.6.0/24 has dropped out of what R1 advertises to R4.

R1 show bgp neighbors 10.1.4.4 advertised-routes (STEP 8)
RP/0/RP0/CPU0:R1#show bgp neighbors 10.1.4.4 advertised-routes

Wed Sep  9 06:44:51.989 UTC
Network            Next Hop        From            AS Path
192.168.1.0/24     10.1.4.1        Local           65001i
192.168.2.0/24     10.1.4.1        10.0.0.2        65001i
192.168.3.0/24     10.1.4.1        10.0.0.3        65001i

Processed 3 prefixes, 3 paths

Over iBGP it flows as usual. Unlike NO_EXPORT, this is just R1 dropping the route in one outbound policy.

R1 show bgp neighbors 10.0.0.2 advertised-routes (STEP 8)
RP/0/RP0/CPU0:R1#show bgp neighbors 10.0.0.2 advertised-routes

Wed Sep  9 06:44:50.856 UTC
Network            Next Hop        From            AS Path
192.168.1.0/24     10.0.0.1        Local           i
192.168.4.0/24     10.0.0.1        10.1.4.4        65002i
192.168.5.0/24     10.0.0.1        10.1.4.4        65002 65003i
192.168.6.0/24     10.0.0.1        10.1.6.6        65004i

Processed 4 prefixes, 4 paths

A User-Defined Community Leaks Unless Every Border Router Enforces It

Look at R2, the other AS border router in AS 65001. R2 has no drop configured.

R2 show bgp neighbors 10.2.4.4 advertised-routes (STEP 8)
RP/0/RP0/CPU0:R2#show bgp neighbors 10.2.4.4 advertised-routes

Wed Sep  9 06:45:21.155 UTC
Network            Next Hop        From            AS Path
192.168.1.0/24     10.2.4.2        10.0.0.1        65001i
192.168.2.0/24     10.2.4.2        Local           65001i
192.168.3.0/24     10.2.4.2        10.0.0.3        65001i
192.168.6.0/24     10.2.4.2        10.0.0.1        65001 65004i

Processed 4 prefixes, 4 paths

192.168.6.0/24 is advertised to R4, and it arrives there with the community still attached.

R4 show bgp 192.168.6.0/24 (STEP 8)
RP/0/RP0/CPU0:R4#show bgp 192.168.6.0/24

Wed Sep  9 06:46:02.350 UTC
BGP routing table entry for 192.168.6.0/24
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 16           16
Last Modified: Sep  9 06:37:22.774 for 00:08:39
Paths: (2 available, best #2)
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.2.4.2        10.1.4.1        
  Path #1: Received by speaker 0
  Not advertised to any peer
  65001 65004, (received & used)
    10.2.4.2 from 10.2.4.2 (10.0.0.2)
      Origin IGP, localpref 100, valid, external, group-best
      Received Path ID 0, Local Path ID 0, version 0
      Community: 65001:429
      Origin-AS validity: (disabled)
  Path #2: Received by speaker 0
  Advertised IPv4 Unicast paths to peers (in unique update groups):
    10.2.4.2        10.1.4.1        
  65003 65004, (received & used)
    10.4.5.5 from 10.4.5.5 (10.0.0.5)
      Origin IGP, localpref 100, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 16
      Origin-AS validity: (disabled)

NO_EXPORT is specified by the protocol and behaves the same on every router, while a user-defined community only does what each router’s policy makes it do — the result described above. If you publish “tag it with this community and we will not advertise it,” every border router in that AS needs the same match.

No.347 in the eBGP capture carried 65001:429.

No.347 UPDATE (R6 to R1) tshark -V
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 67
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 0
    Total Path Attribute Length: 44
    Path attributes
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
<snip>
            Type Code: MP_REACH_NLRI (14)
            Length: 13
            Address family identifier (AFI): IPv4 (1)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 10.1.6.6
                IPv4 Address: 10.1.6.6
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                192.168.6.0/24
                    MP Reach NLRI prefix length: 24
                    MP Reach NLRI IPv4 prefix: 192.168.6.0
        Path Attribute - ORIGIN: IGP
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: ORIGIN (1)
            Length: 1
            Origin: IGP (0)
        Path Attribute - AS_PATH: 65004
            Flags: 0x40, Transitive, Well-known, Complete
<snip>
            Type Code: AS_PATH (2)
            Length: 6
            AS Path segment: 65004
                Segment type: AS_SEQUENCE (2)
                Segment length (number of ASN): 1
                AS4: 65004
        Path Attribute - MULTI_EXIT_DISC: 0
            Flags: 0x80, Optional, Non-transitive, Complete
<snip>
            Type Code: MULTI_EXIT_DISC (4)
            Length: 4
            Multiple exit discriminator: 0
        Path Attribute - COMMUNITIES: 65001:429
            Flags: 0xc0, Optional, Transitive, Complete
<snip>
            Type Code: COMMUNITIES (8)
            Length: 4
            Communities: 65001:429
                Community: 65001:429
                    Community AS: 65001
                    Community value: 429

Download the pcap of the packet in the tshark output above (No.347 UPDATE)

Summary of the Verification

STEP	Community set by R6	R1 behaviour	R2 and R3 (iBGP)	R4 (AS 65002)
1	`65004:1000`	Passes it through	Receives it	Receives it
3	(R1 overwrites with `65001:100`)	Discards the original	`65001:100` only	`65001:100` only
4	(R1 adds with `additive`)	Keeps both	Both arrive	Both arrive
6	`NO_EXPORT`	Not advertised to eBGP	Receives it; R2 also stops advertising to eBGP	Does not arrive
7	`NO_ADVERTISE`	Not advertised anywhere	Does not arrive (withdrawn)	Does not arrive
8	`65001:429`	Dropped in `TO-R4`	Receives it	Not from R1, but leaks from R2

Verification Configs and show Output

For every STEP, three files were captured from each of the six routers. The verification config is the ..._run.txt (the final state is the STEP 8 set).

File	Contents
`..._show.txt`	`show version`, `show interface description`, `show route`, `show bgp`, `show bgp community <value>`, `show bgp neighbors <peer> advertised-routes / routes / received routes`, `show rpl route-policy` and more
`..._log.txt`	`show logging` limited to that STEP. A `logmsg` marker is written at the start of each STEP and its timestamp passed to `show logging start`
`..._run.txt`	`show running-config` at that point in time (that is, the verification config for the STEP)

In the final state R6’s TO-R1 contains set community (65001:429), and R1 has community-set CS-NO-ADV-R4 plus a TO-R4 that drops routes matching it. R1’s FROM-R6 and CS-FROM-R6 stopped being applied at STEP 5 but the definitions remain.

STEP 0: Initial state

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 1: R6 tags 192.168.6.0/24 with 65004:1000

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 2: R1 matches the community and raises LOCAL_PREF

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 3: set community without additive

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 4: Adding additive

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 5: Removing R1 inbound policy

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 6: R6 sets NO_EXPORT

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 7: R6 sets NO_ADVERTISE

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

STEP 8: User-defined community 65001:429 narrows the advertisement (final state)

Router	show output	syslog	running-config
R1	show	log	run
R2	show	log	run
R3	show	log	run
R4	show	log	run
R5	show	log	run
R6	show	log	run

Two captures were taken: bgp-community.pcap on the R1-R6 (eBGP) link and bgp-community-ibgp.pcap on the R1-R3 (iBGP) link.

Download the full R1-R6 (eBGP) capture

Download the full R1-R3 (iBGP) capture

References

Source	Title	Summary
RFC 1997	BGP Communities Attribute	Defines the COMMUNITY attribute: type code 8, optional transitive, four octets, the reserved ranges, and `NO_EXPORT` / `NO_ADVERTISE` / `NO_EXPORT_SUBCONFED`.
RFC 4360	BGP Extended Communities Attribute	Defines Extended Communities (type code 16, eight octets).
RFC 8092	BGP Large Communities Attribute	Defines Large Communities (type code 32, twelve octets, three 32-bit fields), which accommodate 4-byte AS numbers.
RFC 7999	BLACKHOLE Community	Defines `65535:666` (`0xFFFF029A`). Informational.
RFC 8326	Graceful BGP Session Shutdown	Defines `GRACEFUL_SHUTDOWN` (`65535:0` / `0xFFFF0000`).
NTT	Routing - NTT-GIN	The community list AS2914 publishes for its customers, including `2914:429` and `65500:nnn`. A good look at how communities are used in production.