MPLS Label Operations (push / swap / pop) and PHP

MPLS Label Operations (push / swap / pop) and PHP

An MPLS router (LSR) performs three operations on a packet: push to add a label, swap to replace the top label, and pop to remove the top label. This article explains which table each router consults and what it does as a single packet travels from the ingress PE to the egress PE, and what PHP (Penultimate Hop Popping) is, then confirms it hop by hop in an IOS XR (XRd) lab. The label format is covered in MPLS Labels and the Label Stack.

Three operations and two tables

Which table is consulted depends on whether the received packet carries a label.

A transit LSR never looks at the IP header. It consults the top label only, and either replaces it with the value in Outgoing Label (swap) or removes it when the entry says Pop.

The Outgoing Label column of show mpls forwarding shows the following in addition to a label value.

A label is replaced at every hop

A label is only meaningful between two adjacent routers, so its value changes each time the packet passes an LSR. In the figure above, the 24003 that PE1 pushed (the value P1 assigned to 5.5.5.5/32) is replaced by P1 with 30000 (the value P2 assigned to the same prefix). How the values are chosen is covered in the LDP article. Each LSR assigns its local labels independently, so two neighbours may happen to pick the same value. The swap still takes place; only the values coincide. The assignment also changes when a router reboots, so the label value itself carries no meaning.

PHP (Penultimate Hop Popping)

An egress LSR advertises implicit-null (3) for the routes connected to itself, such as its loopback. The LSR one hop before it removes the label from packets destined for that prefix (pop) before forwarding them. That behaviour is PHP.

The purpose of PHP is to reduce the number of lookups at the egress LSR (RFC 3031 section 3.16). If the egress LSR received the packet still labelled, it would first consult the LFIB to decide that the label is its own and must be removed, then consult the FIB with the IP header, which is two lookups. Removing the label one hop earlier lets the egress LSR receive an IP packet and do a single FIB lookup.

To avoid PHP, the egress LSR is made to advertise explicit-null (0). The penultimate LSR swaps the label to 0 and forwards it, and the egress LSR removes label 0 before doing the IP lookup. This is used when the TC field (the QoS class) of the label needs to reach the egress LSR. On IOS XR it is selected with label local advertise explicit-null under mpls ldp.

PHP only removes the last label one hop early. When there are two labels, as in an MPLS VPN, the penultimate LSR removes only the top transport label, and the VPN label reaches the egress PE.

Lab verification

Lab setup

CE1 — PE1 — P1 — P2 — PE2 — CE2 are connected in a line. PE1 / P1 / P2 / PE2 in AS 65001 form the MPLS network, distributing routes with OSPF (area 0, network point-to-point on every link) and labels with LDP. CE1 and CE2 exchange routes with the PEs over eBGP, and PE1 and PE2 run iBGP between their Loopback0 addresses with next-hop-self so that external routes take their own Loopback0 as the next hop. CE3 and CE4 are customers in VRF CUST-A, used here to confirm that the VPN label is not removed by PHP.

A ping from CE1 to 192.168.6.1 is captured simultaneously on the three links PE1 - P1, P1 - P2 and P2 - PE2, and the label each link carries is matched against the FIB and LFIB of each router.

Verification steps

STEP 0: Initial state (PHP, the default)

PE1 learns 192.168.6.0/24 over iBGP with next hop 5.5.5.5 (PE2’s Loopback0), so the label it pushes is held in the entry for the recursive next hop 5.5.5.5/32.

   via 10.2.3.3/32, GigabitEthernet0/0/0/1, 6 dependencies, weight 0, class 0 [flags 0x0]
    path-idx 0 NHID 0x4 [0x8a76a5a0 0x0]
    next hop 10.2.3.3/32
    local adjacency
     local label 24004      labels imposed {24003}

labels imposed {24003} is the value PE1 pushes. Next are the LFIBs of P1 and P2.

Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes
Label  Label       or ID              Interface                    Switched
------ ----------- ------------------ ------------ --------------- ------------
24000  Pop         2.2.2.2/32         Gi0/0/0/0    10.2.3.2        10426
24001  Pop         4.4.4.4/32         Gi0/0/0/1    10.3.4.4        1230
24002  Pop         10.4.5.0/24        Gi0/0/0/1    10.3.4.4        0
24003  24003       5.5.5.5/32         Gi0/0/0/1    10.3.4.4        7547

Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes
Label  Label       or ID              Interface                    Switched
------ ----------- ------------------ ------------ --------------- ------------
24000  24000       2.2.2.2/32         Gi0/0/0/0    10.3.4.3        9028
24001  Pop         3.3.3.3/32         Gi0/0/0/0    10.3.4.3        1248
24002  Pop         10.2.3.0/24        Gi0/0/0/0    10.3.4.3        0
24003  Pop         5.5.5.5/32         Gi0/0/0/1    10.4.5.5        9148

P1 receives 24003 and swaps it to 24003, and P2 receives 24003 and pops it. In this run P1 and P2 both assigned 24003 to 5.5.5.5/32, so the values coincide. Following the CE1 - CE2 echo request through the three captures gives the following.

On P2 - PE2 the EtherType is back to 0x0800 and the label is gone, because PHP made P2 remove the last label.

For a route in VRF CUST-A, PE1 pushes two labels.

   via 5.5.5.5/32, 3 dependencies, recursive [flags 0x6000]
    path-idx 0 NHID 0x0 [0x8857f048 0x0]
    recursion-via-/32
    next hop VRF - 'default', table - 0xe0000000
    next hop 5.5.5.5/32 via 24004/0/21
     next hop 10.2.3.3/32 GigabitEthernet0/0/0/1 labels imposed {24003 24005}

The CE3 - CE4 echo request travels through the three links as follows.

P2 removed only the top label 24003; the VPN label 24005 reached PE2.

STEP 1: Change P2’s dynamic label range

Configuring mpls label range table 0 30000 1048575 on P2 and rebooting it moves P2’s local labels into the 30000 range. Only the swap target changes in P1’s LFIB.

Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes
Label  Label       or ID              Interface                    Switched
------ ----------- ------------------ ------------ --------------- ------------
24000  Pop         2.2.2.2/32         Gi0/0/0/0    10.2.3.2        19988
24001  Pop         4.4.4.4/32         Gi0/0/0/1    10.3.4.4        840
24002  Pop         10.4.5.0/24        Gi0/0/0/1    10.3.4.4        0
24003  30000       5.5.5.5/32         Gi0/0/0/1    10.3.4.4        6747

The 24003 that P1 receives is unchanged; only the value it sends becomes 30000. PE1 - P1 still carries 24003 while P1 - P2 changes to 30000. The same values appear in a traceroute from CE1.

 1  10.1.2.2 11 msec  4 msec  4 msec
 2  10.2.3.3 [MPLS: Label 24003 Exp 0] 14 msec  12 msec  13 msec
 3  10.3.4.4 [MPLS: Label 30000 Exp 0] 17 msec  17 msec  12 msec
 4  10.4.5.5 12 msec  13 msec  10 msec
 5  10.5.6.6 18 msec  *  23 msec

STEP 2: label local advertise explicit-null on PE2

When PE2 advertises explicit-null (0) instead of implicit-null (3), P2’s Outgoing Label becomes Exp-Null-v4.

Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes
Label  Label       or ID              Interface                    Switched
------ ----------- ------------------ ------------ --------------- ------------
30000  Exp-Null-v4 5.5.5.5/32         Gi0/0/0/1    10.4.5.5        6684
30001  24000       2.2.2.2/32         Gi0/0/0/0    10.3.4.3        14107
30002  Pop         3.3.3.3/32         Gi0/0/0/0    10.3.4.3        1608
30003  Pop         10.2.3.0/24        Gi0/0/0/0    10.3.4.3        0

P2 swaps to 0 instead of removing the label, so P2 - PE2 also carries an MPLS header. For a VRF packet there are two labels, 0 and the VPN label.

MultiProtocol Label Switching Header, Label: 0 (IPv4 Explicit-Null), Exp: 0, S: 0, TTL: 252
    0000 0000 0000 0000 0000 .... .... .... = MPLS Label: IPv4 Explicit-Null (0)
    .... .... .... .... .... 000. .... .... = MPLS Experimental Bits: 0
    .... .... .... .... .... ...0 .... .... = MPLS Bottom Of Label Stack: 0
    .... .... .... .... .... .... 1111 1100 = MPLS TTL: 252
MultiProtocol Label Switching Header, Label: 24005, Exp: 0, S: 1, TTL: 254
    0000 0101 1101 1100 0101 .... .... .... = MPLS Label: 24005 (0x05dc5)
    .... .... .... .... .... 000. .... .... = MPLS Experimental Bits: 0
    .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label Stack: 1
    .... .... .... .... .... .... 1111 1110 = MPLS TTL: 254
Internet Protocol Version 4, Src: 192.168.7.1, Dst: 192.168.8.1

The top label is 0 with S = 0, and the second label 24005 has S = 1. Even with explicit-null only the top label is affected, and the VPN label reaches PE2. PE2 removes label 0 before the IP lookup, which makes two lookups.

STEP 3: Remove STEP 1 and STEP 2 (final state)

Removing both settings and rebooting P2 returns P2’s label range to the 24000 range and its Outgoing Label to Pop.

Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes
Label  Label       or ID              Interface                    Switched
------ ----------- ------------------ ------------ --------------- ------------
24000  Pop         5.5.5.5/32         Gi0/0/0/1    10.4.5.5        7892
24001  24000       2.2.2.2/32         Gi0/0/0/0    10.3.4.3        7050
24002  Pop         3.3.3.3/32         Gi0/0/0/0    10.3.4.3        512
24003  Pop         10.2.3.0/24        Gi0/0/0/0    10.3.4.3        0

The reboot changed the order in which P2 assigns labels, and 5.5.5.5/32 received 24000. Unlike STEP 0 this does not match P1’s value (24003), so the replacement is visible in a traceroute as well.

 1  10.1.2.2 6 msec  5 msec  5 msec
 2  10.2.3.3 [MPLS: Label 24003 Exp 0] 15 msec  14 msec  13 msec
 3  10.3.4.4 [MPLS: Label 24000 Exp 0] 13 msec  14 msec  21 msec
 4  10.4.5.5 15 msec  15 msec  14 msec
 5  10.5.6.6 19 msec  *  19 msec

For a VRF packet, the top label is removed by PHP and only the VPN label remains.

 1  10.2.7.2 6 msec  14 msec  5 msec
 2  10.2.3.3 [MPLS: Labels 24003/24005 Exp 0] 20 msec  16 msec  16 msec
 3  10.3.4.4 [MPLS: Labels 24000/24005 Exp 0] 17 msec  16 msec  17 msec
 4  10.4.5.5 [MPLS: Label 24005 Exp 0] 17 msec  18 msec  19 msec
 5  10.5.8.8 20 msec  *  18 msec

Captures

The pcaps taken simultaneously on the three links. They were captured without a filter because they contain MPLS.

STEP 0 PE1 - P1

STEP 0 P1 - P2

STEP 0 P2 - PE2

STEP 1 PE1 - P1

STEP 1 P1 - P2

STEP 1 P2 - PE2

STEP 2 PE1 - P1

STEP 2 P1 - P2

STEP 2 P2 - PE2

STEP 3 PE1 - P1

STEP 3 P1 - P2

STEP 3 P2 - PE2

Verification config and show output

The following file types were collected from all eight routers at every STEP, split per router. The verification config is these ..._run.txt files (the final state is the one from the last STEP).

STEP 0: Initial state (PHP, the default)

STEP 1: Change P2’s dynamic label range

STEP 2: label local advertise explicit-null on PE2

STEP 3: Remove STEP 1 and STEP 2 (final state)

References

Book: Luc De Ghein, MPLS Fundamentals (Cisco Press, 2006), Chapter 3

Related articles

• What Is MPLS
• MPLS Labels and the Label Stack
• MPLS Label Operations (push / swap / pop) and PHP