↓ Skip to main content
KAZULOG.INFO
  1. Network Articles/
  2. MPLS-VPN Articles/

MPLS VPN Label Allocation (per-prefix / per-CE / per-VRF)

Table of Contents

What is an MPLS VPN label?

A VPN label is the label the egress PE uses to decide which VRF, and where in it, a packet goes. The egress PE chooses the value itself and tells the ingress PE by putting it in the MP-BGP NLRI (MPLS VPN MP-BGP). The ingress PE pushes this label beneath the transport label.

The core P routers only look at the transport label; only the egress PE ever looks at the VPN label. How the VPN label values are chosen is therefore up to the egress PE alone.

The allocation unit is left to the implementation

RFC 4364 section 4.3.2 leaves it to the implementation whether each route gets its own label.

Whether or not each route has a distinct label is an implementation matter.

It then lists three possible approaches. IOS XR offers each as a mode called per-vrf, per-ce and per-prefix.

Approach listed in RFC 4364IOS XR modeNumber of labelsProcessing at the egress PE
One per routeper-prefix (default)One per routeThe label alone decides the exit
One per attachment circuitper-ce (per next-hop CE)One per CEThe label alone decides the exit
One per VRFper-vrfOne per VRFIP lookup again in the VRF

With one label per VRF, the label alone does not tell which CE to send to. RFC 4364 section 4.3.2 describes what the egress PE must then do:

when the egress PE receives a packet with that label, it must look up the packet’s IP destination address in that VRF

In IOS XR’s show mpls forwarding, a label that is removed and followed by an IP lookup in the VRF is shown as Aggregate, with <VRF name>: Per-VRF Aggr in the Prefix or ID column.

IOS XR implements per-ce as Resilient per-CE. Besides the forwarding entry towards the CE, it keeps a backup entry that does an IP lookup in the VRF, marked (!) (FRR backup) in show mpls forwarding.

Connected routes get the aggregate label in every mode

The PE-CE connected routes advertised with redistribute connected point not at a CE but at the individual systems on that LAN. The label alone does not determine the outgoing layer 2 header, so the egress PE has to look up the IP address again (resolve ARP). RFC 4364 section 4.3.2 also names this as a case where an aggregate label is useful:

or if the VRF has an associated Local Area Network (LAN) interface (where there is a different outgoing layer 2 header for each system on the LAN, but a route is not distributed for each such system)

IOS XR gives connected routes the per-VRF aggregate label (Per-VRF Aggr) regardless of the label mode setting. It is this same label that the static routes move onto under per-vrf.

Pros and cons

ModeProsCons
per-prefixThe label alone decides the exit. A route’s exit can change without redistributing its label (RFC 4364 section 4.3.2)Uses one label per route. A large VRF consumes many labels on the PE
per-ceThe number of labels drops to the number of CEs. The label alone decides the exitIf the exit CE changes, the label changes too
per-vrfA single label is enoughThe egress PE needs two lookups (the label, then the IP address)

A label value is 20 bits, so the number a PE can use is finite. The more VRFs and routes a PE carries, the more the choice of mode matters.

Configuring VPN labels on IOS XR

The mode is set per VRF as label mode under address-family ipv4 unicast of the vrf under router bgp. Without it, the mode is per-prefix.

Configuring per-VRF (IOS XR)
router bgp 65001
 vrf CUST-A
  rd 65001:1
  address-family ipv4 unicast
   label mode per-vrf
   redistribute connected
   redistribute static
  !
 !
!
SettingAllocation
(not configured)per-prefix (default)
label mode per-ceOne per next-hop CE
label mode per-vrfOne per VRF

Changing label mode does not reset the BGP session. The PE simply re-sends its UPDATEs with the new labels. When the receiver gets an UPDATE for the same prefix and the same next hop, it withdraws the previous label and replaces it with the new one (RFC 8277 section 2.5).

Lab setup

Six XRd routers, with two CEs and four routes behind the egress PE2. The difference between the modes only shows when there are several CEs and several routes. PE1 and PE2 run redistribute connected and redistribute static in the VRF’s BGP, so the PE-CE connected routes are advertised as VPNv4 routes as well.

NodeRole
PE1 / PE2vrf CUST-A (RD 65001:1, RT 65001:100). redistribute connected and redistribute static
P1Core only (OSPF area 0 + LDP)
CE-A1Customer A site 1. LAN 10.1.1.0/24
CE-A2 / CE-A3Sites 2 / 3, with two LANs each (10.1.2.0/24 and 10.1.3.0/24 / 10.1.4.0/24 and 10.1.5.0/24)

Only label mode in PE2’s vrf CUST-A is changed. At every STEP, CE-A1 sends a different number of pings to each destination (10 to 10.1.2.1, 20 to 10.1.3.1, 30 to 10.1.4.1, 40 to 10.1.5.1, and 7 and 11 to the connected-route addresses 172.16.2.2 and 172.16.3.2). Between P1 and PE2 only the VPN label remains after PHP, so the capture shows directly which label carried the pings to each destination.

STEPChange (PE2 vrf CUST-A)What it shows
0Initial state (default = per-prefix)Four labels for the four static routes; the two connected routes use the aggregate label
1label mode per-ceThe static routes collapse into two labels, one per CE. The session is not reset
2label mode per-vrfEverything moves onto the single aggregate label, and Aggregate does an IP lookup
3Remove label mode (final state)Back to STEP 0

By default one per route, connected routes on the aggregate label (STEP 0)

These are the labels PE2 assigned to its own routes (Local Label). The four static routes have four labels, 24003 to 24006, while the two connected routes share 24007.

STEP 0 PE2, show bgp vpnv4 unicast labels (excerpt)
   Network            Next Hop        Rcvd Label      Local Label
Route Distinguisher: 65001:1 (default for vrf CUST-A)
Route Distinguisher Version: 20
*>i10.1.1.0/24        1.1.1.1         24003           nolabel
*> 10.1.2.0/24        172.16.2.2      nolabel         24003
*> 10.1.3.0/24        172.16.2.2      nolabel         24004
*> 10.1.4.0/24        172.16.3.2      nolabel         24005
*> 10.1.5.0/24        172.16.3.2      nolabel         24006
*>i172.16.1.0/24      1.1.1.1         24004           nolabel
*> 172.16.2.0/24      0.0.0.0         nolabel         24007
*> 172.16.3.0/24      0.0.0.0         nolabel         24007

Processed 8 prefixes, 8 paths

24007 also appears as the Local Label in the connected route’s details.

STEP 0 PE2, show bgp vpnv4 unicast rd 65001:1 172.16.2.0/24 (excerpt)
RP/0/RP0/CPU0:PE2#show bgp vpnv4 unicast rd 65001:1 172.16.2.0/24
Tue Sep 22 15:59:04.077 UTC
BGP routing table entry for 172.16.2.0/24, Route Distinguisher: 65001:1
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 14           14
    Local Label: 24007

In the forwarding table, 24003 to 24006 are Unlabelled (remove the label and send to the CE) and 24007 is Aggregate (IP lookup in the VRF).

STEP 0 PE2, show mpls forwarding
RP/0/RP0/CPU0:PE2#show mpls forwarding
Tue Sep 22 15:59:06.567 UTC
Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes       
Label  Label       or ID              Interface                    Switched    
------ ----------- ------------------ ------------ --------------- ------------
24000  24000       1.1.1.1/32         Gi0/0/0/1    10.0.23.3       16559       
24001  Pop         3.3.3.3/32         Gi0/0/0/1    10.0.23.3       626         
24002  Pop         10.0.13.0/24       Gi0/0/0/1    10.0.23.3       0           
24003  Unlabelled  10.1.2.0/24[V]     Gi0/0/0/0    172.16.2.2      2236        
24004  Unlabelled  10.1.3.0/24[V]     Gi0/0/0/0    172.16.2.2      2000        
24005  Unlabelled  10.1.4.0/24[V]     Gi0/0/0/2    172.16.3.2      4236        
24006  Unlabelled  10.1.5.0/24[V]     Gi0/0/0/2    172.16.3.2      4000        
24007  Aggregate   CUST-A: Per-VRF Aggr[V]   \
                                      CUST-A                       1872

Counting CE-A1’s pings (ICMP echo requests) in the P1-PE2 capture by label and destination gives exactly the numbers sent.

Labels and destinations counted in the STEP 0 capture
$ tshark -r mpls-vpn-label-step0-p1pe2.pcap -Y 'mpls && icmp.type == 8 && ip.src == 10.1.1.1' -T fields -e mpls.label -e ip.dst | sort | uniq -c
     10 24003	10.1.2.1
     20 24004	10.1.3.1
     30 24005	10.1.4.1
     40 24006	10.1.5.1
      7 24007	172.16.2.2
     11 24007	172.16.3.2

per-CE collapses the labels per CE (STEP 1)

With label mode per-ce, the static routes’ labels collapsed into two, one per next-hop CE (24008 for CE-A2, 24009 for CE-A3). The connected routes stayed on 24007.

STEP 1 PE2, show bgp vpnv4 unicast labels (excerpt)
   Network            Next Hop        Rcvd Label      Local Label
Route Distinguisher: 65001:1 (default for vrf CUST-A)
Route Distinguisher Version: 28
*>i10.1.1.0/24        1.1.1.1         24003           nolabel
*> 10.1.2.0/24        172.16.2.2      nolabel         24008
*> 10.1.3.0/24        172.16.2.2      nolabel         24008
*> 10.1.4.0/24        172.16.3.2      nolabel         24009
*> 10.1.5.0/24        172.16.3.2      nolabel         24009
*>i172.16.1.0/24      1.1.1.1         24004           nolabel
*> 172.16.2.0/24      0.0.0.0         nolabel         24007
*> 172.16.3.0/24      0.0.0.0         nolabel         24007

Processed 8 prefixes, 8 paths

The route details show the Resilient per-CE next-hop set.

STEP 1 PE2, show bgp vpnv4 unicast rd 65001:1 10.1.2.0/24 (excerpt)
RP/0/RP0/CPU0:PE2#show bgp vpnv4 unicast rd 65001:1 10.1.2.0/24
Tue Sep 22 16:03:27.460 UTC
BGP routing table entry for 10.1.2.0/24, Route Distinguisher: 65001:1
Versions:
  Process           bRIB/RIB   SendTblVer
  Speaker                 25           25
    Local Label: 24008
    Gateway Array ID: 1, Resilient per-CE nexthop set ID: 1

In the forwarding table, 24008 and 24009 each have a backup entry marked (!) (Aggregate, an IP lookup in the VRF) besides the entry towards the CE (Unlabelled). Since one label no longer points at a single prefix, the Prefix or ID column reads No ID.

STEP 1 PE2, show mpls forwarding
RP/0/RP0/CPU0:PE2#show mpls forwarding
Tue Sep 22 16:03:31.271 UTC
Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes       
Label  Label       or ID              Interface                    Switched    
------ ----------- ------------------ ------------ --------------- ------------
24000  24000       1.1.1.1/32         Gi0/0/0/1    10.0.23.3       16603       
24001  Pop         3.3.3.3/32         Gi0/0/0/1    10.0.23.3       626         
24002  Pop         10.0.13.0/24       Gi0/0/0/1    10.0.23.3       0           
24007  Aggregate   CUST-A: Per-VRF Aggr[V]   \
                                      CUST-A                       1872        
24008  Unlabelled  No ID              Gi0/0/0/0    172.16.2.2      4236        
       Aggregate   No ID              CUST-A                       0            (!)
24009  Unlabelled  No ID              Gi0/0/0/2    172.16.3.2      8236        
       Aggregate   No ID              CUST-A                       0            (!)

Changing the mode did not reset the BGP session (no ADJCHANGE in the syslog of PE1 or PE2). PE2 just sent one UPDATE with the new labels: in No.8 of the attached STEP 1 capture, the four routes carry 24008 and 24009. No withdrawal (MP_UNREACH_NLRI) was sent.

STEP 1 No.8 UPDATE (PE2 to PE1), MP_REACH_NLRI (tshark -V excerpt)
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
                1... .... = Optional: Set
                .0.. .... = Transitive: Not set
                ..0. .... = Partial: Not set
                ...1 .... = Extended-Length: Set
                .... 0000 = Unused: 0x0
            Type Code: MP_REACH_NLRI (14)
            Length: 77
            Address family identifier (AFI): IPv4 (1)
            Subsequent address family identifier (SAFI): Labeled VPN Unicast (128)
            Next hop:  RD=0:0 IPv4=2.2.2.2
                Route Distinguisher: 0:0
                IPv4 Address: 2.2.2.2
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                BGP Prefix
                    Prefix Length: 112
                    Label Stack: 24009 (bottom)
                    Route Distinguisher: 65001:1
                    MP Reach NLRI IPv4 prefix: 10.1.5.0
                BGP Prefix
                    Prefix Length: 112
                    Label Stack: 24009 (bottom)
                    Route Distinguisher: 65001:1
                    MP Reach NLRI IPv4 prefix: 10.1.4.0
                BGP Prefix
                    Prefix Length: 112
                    Label Stack: 24008 (bottom)
                    Route Distinguisher: 65001:1
                    MP Reach NLRI IPv4 prefix: 10.1.3.0
                BGP Prefix
                    Prefix Length: 112
                    Label Stack: 24008 (bottom)
                    Route Distinguisher: 65001:1
                    MP Reach NLRI IPv4 prefix: 10.1.2.0
Download the pcap of the packet in the tshark output above (No.8 UPDATE)

In the data plane too, CE-A2’s two destinations ride on 24008 and CE-A3’s two on 24009.

Labels and destinations counted in the STEP 1 capture
$ tshark -r mpls-vpn-label-step1-p1pe2.pcap -Y 'mpls && icmp.type == 8 && ip.src == 10.1.1.1' -T fields -e mpls.label -e ip.dst | sort | uniq -c
      7 24007	172.16.2.2
     11 24007	172.16.3.2
     10 24008	10.1.2.1
     20 24008	10.1.3.1
     30 24009	10.1.4.1
     40 24009	10.1.5.1

per-VRF uses one label and the egress does an IP lookup (STEP 2)

With label mode per-vrf, all six routes moved to 24007, the aggregate label the connected routes had used since STEP 0.

STEP 2 PE2, show bgp vpnv4 unicast labels (excerpt)
   Network            Next Hop        Rcvd Label      Local Label
Route Distinguisher: 65001:1 (default for vrf CUST-A)
Route Distinguisher Version: 32
*>i10.1.1.0/24        1.1.1.1         24003           nolabel
*> 10.1.2.0/24        172.16.2.2      nolabel         24007
*> 10.1.3.0/24        172.16.2.2      nolabel         24007
*> 10.1.4.0/24        172.16.3.2      nolabel         24007
*> 10.1.5.0/24        172.16.3.2      nolabel         24007
*>i172.16.1.0/24      1.1.1.1         24004           nolabel
*> 172.16.2.0/24      0.0.0.0         nolabel         24007
*> 172.16.3.0/24      0.0.0.0         nolabel         24007

Processed 8 prefixes, 8 paths

The labels the ingress PE1 receives (Rcvd Label) are 24007 for all six routes as well.

STEP 2 PE1, show bgp vpnv4 unicast labels (excerpt)
   Network            Next Hop        Rcvd Label      Local Label
Route Distinguisher: 65001:1 (default for vrf CUST-A)
Route Distinguisher Version: 29
*> 10.1.1.0/24        172.16.1.2      nolabel         24003
*>i10.1.2.0/24        2.2.2.2         24007           nolabel
*>i10.1.3.0/24        2.2.2.2         24007           nolabel
*>i10.1.4.0/24        2.2.2.2         24007           nolabel
*>i10.1.5.0/24        2.2.2.2         24007           nolabel
*> 172.16.1.0/24      0.0.0.0         nolabel         24004
*>i172.16.2.0/24      2.2.2.2         24007           nolabel
*>i172.16.3.0/24      2.2.2.2         24007           nolabel

Processed 8 prefixes, 8 paths

The only VRF label in PE2’s forwarding table is 24007: Aggregate, with CUST-A: Per-VRF Aggr[V] in the Prefix or ID column. After removing this label, the egress PE2 looks up the IP address in vrf CUST-A to decide where to send the packet.

STEP 2 PE2, show mpls forwarding vrf CUST-A
RP/0/RP0/CPU0:PE2#show mpls forwarding vrf CUST-A
Tue Sep 22 16:07:54.813 UTC
Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes       
Label  Label       or ID              Interface                    Switched    
------ ----------- ------------------ ------------ --------------- ------------
24007  Aggregate   CUST-A: Per-VRF Aggr[V]   \
                                      CUST-A                       14272

The pings to all six destinations arrived on 24007.

Labels and destinations counted in the STEP 2 capture
$ tshark -r mpls-vpn-label-step2-p1pe2.pcap -Y 'mpls && icmp.type == 8 && ip.src == 10.1.1.1' -T fields -e mpls.label -e ip.dst | sort | uniq -c
     10 24007	10.1.2.1
     20 24007	10.1.3.1
     30 24007	10.1.4.1
     40 24007	10.1.5.1
      7 24007	172.16.2.2
     11 24007	172.16.3.2

The full show mpls forwarding of STEP 2 (in the attached file) still lists 24008 and 24009 from STEP 1, but their counters are 0 and no traffic flows on them.

Back to the default (STEP 3)

After removing label mode, the static routes went back to one label per route (24003 to 24006).

Labels and destinations counted in the STEP 3 capture
$ tshark -r mpls-vpn-label-step3-p1pe2.pcap -Y 'mpls && icmp.type == 8 && ip.src == 10.1.1.1' -T fields -e mpls.label -e ip.dst | sort | uniq -c
     10 24003	10.1.2.1
     20 24004	10.1.3.1
     30 24005	10.1.4.1
     40 24006	10.1.5.1
      7 24007	172.16.2.2
     11 24007	172.16.3.2

References

StandardTitleWhat this article used
RFC 4364BGP/MPLS IP Virtual Private Networks (VPNs)4.3.2 (label allocation is an implementation matter; per VRF, per attachment circuit, per route; LAN interfaces and aggregate labels)
RFC 8277Using BGP to Bind MPLS Labels to Address Prefixes2.5 (changing the label bound to a prefix)

Books: Mobeen Tahir et al., Cisco IOS XR Fundamentals (Cisco Press, 2009), Chapter 9

Test environment: Cisco IOS XRd 26.1.1 (Cisco Modeling Labs)

Configurations and show output

The following files were captured from all six routers at every STEP, one file per router. The configurations under test are these ..._run.txt files (the final state is the one from the last STEP).

FileContents
..._show.txtshow version, show bgp vpnv4 unicast labels, show bgp vpnv4 unicast rd <RD> <prefix> (six routes), the three-command set (advertised-routes / routes / received routes), show mpls forwarding, show mpls forwarding vrf CUST-A, show mpls label table and others
..._log.txtshow logging narrowed to that STEP. A marker is written with logmsg at the start of each STEP and its timestamp passed to show logging start
..._run.txtshow running-config at that STEP (the configuration under test)
..._ping.txtThe ping and traceroute results for that STEP
..._trace.txtThe PE’s show bgp trace, filtered by the peer address, last 150 lines
..._clear.txtThe counters cleared before the pings (clear mpls forwarding counters on PE2)
..._commit.cfgOnly the configuration committed in that STEP, for the routers that were changed

The final state (STEP 3) is back to the same configuration as STEP 0.

STEP 0: Initial state (default = per-prefix)

Routershowsyslogrunning-configpingtraceclearcommitted config
CE-A1showlogrunping---
PE1showlogrunpingtrace--
P1showlogrunping---
PE2showlogrunpingtraceclear-
CE-A2showlogrunping---
CE-A3showlogrunping---

STEP 1: label mode per-ce

Routershowsyslogrunning-configpingtraceclearcommitted config
CE-A1showlogrunping---
PE1showlogrunpingtrace--
P1showlogrunping---
PE2showlogrunpingtraceclearcommit
CE-A2showlogrunping---
CE-A3showlogrunping---

STEP 2: label mode per-vrf

Routershowsyslogrunning-configpingtraceclearcommitted config
CE-A1showlogrunping---
PE1showlogrunpingtrace--
P1showlogrunping---
PE2showlogrunpingtraceclearcommit
CE-A2showlogrunping---
CE-A3showlogrunping---

STEP 3: Remove label mode (final state)

Routershowsyslogrunning-configpingtraceclearcommitted config
CE-A1showlogrunping---
PE1showlogrunpingtrace--
P1showlogrunping---
PE2showlogrunpingtraceclearcommit
CE-A2showlogrunping---
CE-A3showlogrunping---
Packet captures were taken per STEP.
STEPP1-PE2
0pcap
1pcap
2pcap
3pcap

Related Articles