LDP-IGP Synchronization and LDP Session Protection

LDP-IGP Synchronization and LDP Session Protection

The IGP that distributes routes and LDP that distributes labels run independently. When the IGP picks a link on which LDP is not usable, there is no label and only the MPLS traffic is dropped. This article explains that gap and the two mechanisms that close it, then measures both in an IOS XR (XRd) lab. The basics of LDP are covered in What Is LDP.

What happens when a link without labels is chosen

MPLS forwarding assumes that LDP has finished distributing the label for the FEC towards the next hop the IGP chose. Two situations break that assumption.

The first is LDP not running on the link at all. A missing configuration, an ACL or filter, or a mismatched password stops the session from coming up; the IGP keeps choosing the link and only the MPLS traffic is dropped, permanently.

The second is the convergence gap right after a link comes up, but which of the two finishes first depends on the timers. With the IOS XR defaults, LDP sends Hellos every 5 seconds and the session is established over TCP immediately, while OSPF sends Hellos every 10 seconds and reaches FULL through ExStart / Exchange / Loading. On a point-to-point link LDP usually comes up first (measured later in this article), and in that case restoring a link does not create a blackhole.

How things break also depends on the kind of traffic. Plain IP is sent out without a label, and if the next router has an IP route it is forwarded anyway. VPN (VPNv4) traffic, on the other hand, requires the transport label and has no way to fall back to IP forwarding. The same fault drops only the VPN traffic.

LDP-IGP synchronization

RFC 5443 defines a mechanism where the IGP advertises a link at maximum cost until LDP is “fully operational” on it. Because the cost is the maximum, any other path wins and traffic does not flow onto a link that has no labels.

Fully operational means all three conditions hold (section 2).

The maximum cost differs per IGP.

There are two caveats. It is meaningless without an alternate path — if there is no other route, the link is chosen anyway. Section 3 of RFC 5443 likewise says the mechanism is for cases where “availability of LDP-signaled LSPs matters more than always choosing the optimal path”. Also, on a broadcast link the whole link is costed out, not the individual peer (section 3).

LDP session protection

An LDP session is TCP between Lo0s, but whether it may be kept depends on whether a hello adjacency is alive. When the link fails and every hello adjacency disappears, the session goes down too and the label bindings are lost. On restoration it all starts again: Hello → TCP → Initialization → label exchange.

LDP session protection fills that gap with targeted Hellos. They are the Extended Discovery messages of RFC 5036 section 2.4.2 and, unlike link Hellos (multicast), are sent as unicast to the peer’s address. Even with the link gone, the hello adjacency survives as long as an IP path to the peer remains, and so do the session and its labels. Extended Discovery is asymmetric: one side starts sending and the receiver decides whether to answer.

What session protection changes is not the speed of recovery but whether the session and its labels are lost. It does not help if the peer cannot be reached by IP.

Lab verification

Lab setup

The core is duplicated and PE1 and P1 are joined by a single link. Stopping Discovery on that one link prevents an LDP session between PE1 and P1.

By cost, PE1 → PE2 is 3 via PE1-P1-P3-PE2, 103 via PE1-P2-P1-P3-PE2 and 201 via PE1-P2-P4-PE2.

CE1 and CE2 peer eBGP into the PEs’ vrf CUST-A (RD/RT 65001:1), and PE1 - PE2 run iBGP vpnv4 between their Lo0s. Customer traffic therefore always carries two labels and has no way to fall back to IP forwarding.

How it was measured

ping 192.168.2.1 source 192.168.1.1 count 20 interval 100 was run from CE1, with traceroute for the path and the label stack. Every STEP records the ping and traceroute of all eight routers. The outage duration was measured on the CE1 - PE1 capture, from the gap between echo replies.

The IOS XR ping honours interval, but when there is no answer it waits for timeout (2 seconds by default) before sending the next one. Loss counts are comparable, but multiplying the loss count by the interval does not give the outage duration.

XRd forwards in software on CML. Read the numbers as relative comparisons within this lab.

Verification steps

The sections below are grouped by mechanism rather than by STEP order.

A link where LDP cannot come up keeps being chosen (STEP 1)

An ACL that drops only LDP Discovery was applied to both ends. TCP 646 is not blocked, because without Discovery no session is established anyway.

ipv4 access-list BLOCK-LDP-DISCOVERY
 10 deny udp any any eq ldp
 20 permit ipv4 any any
!
interface GigabitEthernet0/0/0/1
 ipv4 access-group BLOCK-LDP-DISCOVERY ingress

It takes the 15-second hold time for the session to go down after the ACL is applied. Only the session with P1 disappears; the one with P2 remains.

Peer               GR  NSR  Up Time     Discovery   Addresses     Labels    
                                        ipv4  ipv6  ipv4  ipv6  ipv4   ipv6 
-----------------  --  ---  ----------  ----------  ----------  ------------
4.4.4.4:0          N   N    00:38:11    1     0     4     0     14     0    

All customer traffic is lost. The traceroute does not get a single hop past PE1.

Success rate is 0 percent (0/20)
 1  10.1.2.2 !N  *  !N 

!N is network unreachable, and PE1 itself returns it. The reason shows in the forwarding entry.

   via 7.7.7.7/32, 0 dependencies, recursive [flags 0x6000]
    path-idx 0 NHID 0x0 [0x88521758 0x0]
    recursion-via-/32
    next hop VRF - 'default', table - 0xe0000000
    unresolved
     labels imposed {24011}

    Load distribution: 0 (refcount 1)

    Hash  OK  Interface                 Address
    0     Y   recursive                 drop           

The VPN label (24011) is known, but the transport label cannot be obtained, so the entry is “drop”.

Meanwhile, plain IP still gets through in the same state. Every ping from PE1 to PE2’s Lo0 succeeded.

Success rate is 100 percent (20/20), round-trip min/avg/max = 8/14/60 ms
 1  10.2.3.3 5 msec  5 msec  5 msec 
 2  10.3.5.5 [MPLS: Label 24009 Exp 0] 11 msec  10 msec  9 msec 
 3  10.5.7.7 10 msec  *  13 msec 

The first hop, 10.2.3.3 (P1), carries no label. PE1 sends it out unlabelled, P1 has an IP route and forwards it, and from the second hop the label P1 applied is present. Whether a packet can fall back to IP forwarding is what separates VPN traffic from plain IP.

Advertising the maximum cost (STEP 2 and 3)

mpls ldp sync is added to that interface in OSPF on PE1 and P1.

router ospf 1
 area 0
  interface GigabitEthernet0/0/0/1
   mpls ldp sync

The sync state is shown per interface, including which condition is unmet.

GigabitEthernet0/0/0/1:
  VRF: 'default' (0x60000000)
  Sync delay: Disabled
  Sync status: Not ready (No hello adjacency)
GigabitEthernet0/0/0/2:
  VRF: 'default' (0x60000000)
  Sync delay: Disabled
  Sync status: Ready
    Peers:
      4.4.4.4:0

No hello adjacency means condition 1 of fully operational is not met. Gi0/0/0/2, which has no ACL, is Ready — the two sit side by side in the same output. The configured interface cost is still 1.

GigabitEthernet0/0/0/1 is up, line protocol is up 
  Internet Address 10.2.3.2/24, Area 0, SID 0, Strict-SPF SID 0
  Label stack Primary label 1 Backup label 3 SRTE label 10
  auto path capability supported
  Process ID 1, Router ID 2.2.2.2, Network Type POINT_TO_POINT, Cost: 1

The advertised value appears in the Router-LSA.

    Link connected to: another Router (point-to-point)
     (Link ID) Neighboring Router ID: 3.3.3.3
     (Link Data) Router Interface address: 10.2.3.2
      Number of TOS metrics: 0
       TOS 0 Metrics: 65535

Traffic moves to the cost 100 link and detours while keeping both labels.

Success rate is 100 percent (20/20), round-trip min/avg/max = 17/18/24 ms
 1  10.1.2.2 7 msec  16 msec  5 msec 
 2  10.2.4.4 [MPLS: Labels 24009/24011 Exp 0] 20 msec  17 msec  17 msec 
 3  10.3.4.3 [MPLS: Labels 24009/24011 Exp 0] 19 msec  21 msec  20 msec 
 4  10.3.5.5 [MPLS: Labels 24009/24011 Exp 0] 18 msec  16 msec  17 msec 
 5  10.5.7.7 [MPLS: Label 24011 Exp 0] 18 msec  20 msec  18 msec 
 6  10.7.8.8 20 msec  *  20 msec 

It leaves via P2 (10.2.4.4), crosses back to P1 (10.3.4.3) and rejoins the original path — six hops against the five of STEP 0.

Removing the ACL in STEP 3 completes the sync and the cost returns to 1. The order — LDP declaring sync, then OSPF lowering the maximum cost — is left in the timestamps.

Sep 11 14:14:12.047 mpls/ldp/isyn 0/RP0/CPU0 t4071  [ISYNC]:2158: Intf GigabitEthernet0/0/0/1 (ifh 0x10): Nbr 3.3.3.3:0 came up, sync_achieved up

The route was then reinstalled at Installed Sep 11 14:14:12.183, 0.136 seconds after the declaration.

Restoring the link did not cause a blackhole (STEP 4 and 5)

PE1 - P1 was shut at both ends and restored 40 seconds later, without sync (STEP 4) and with it (STEP 5).

Both lost one packet and were out for about 2.1 seconds, with no difference. And those 2.1 seconds are at the shutdown, not at the restoration.

The syslog timestamps say why.

Sep 11 14:22:29.026 UTC: ospf[1035]: %ROUTING-OSPF-5-ADJCHG : Process 1, Nbr 3.3.3.3 on GigabitEthernet0/0/0/1 in area 0 from FULL to DOWN, Neighbor Down: interface down or detached, vrf default vrfid 0x60000000 
Sep 11 14:22:29.086 UTC: mpls_ldp[1178]: %ROUTING-LDP-5-NBR_CHANGE : VRF 'default' (0x60000000), Neighbor 3.3.3.3:0 is DOWN (Interface state down) 
Sep 11 14:23:23.596 UTC: mpls_ldp[1178]: %ROUTING-LDP-5-NBR_CHANGE : VRF 'default' (0x60000000), Neighbor 3.3.3.3:0 is UP (IPv4 connection) 
Sep 11 14:23:28.016 UTC: ospf[1035]: %ROUTING-OSPF-5-ADJCHG : Process 1, Nbr 3.3.3.3 on GigabitEthernet0/0/0/1 in area 0 from LOADING to FULL, Loading Done, vrf default vrfid 0x60000000 

LDP came up at 14:23:23.596 and OSPF at 14:23:28.016 — LDP was 4.4 seconds earlier. STEP 5 had the same order (LDP 3.5 seconds earlier). By the time the IGP chose the link again the labels were already usable, so there was no moment for synchronization to hold back.

Session protection (STEP 6)

session protection was added under mpls ldp and exactly the same steps as 4 and 5 were repeated. One packet lost, 2.11 seconds out — the numbers do not change. What changed is the session.

Sep 11 14:37:22.782 UTC: ospf[1035]: %ROUTING-OSPF-5-ADJCHG : Process 1, Nbr 3.3.3.3 on GigabitEthernet0/0/0/1 in area 0 from FULL to DOWN, Neighbor Down: interface down or detached, vrf default vrfid 0x60000000 
Sep 11 14:38:25.254 UTC: ospf[1035]: %ROUTING-OSPF-5-ADJCHG : Process 1, Nbr 3.3.3.3 on GigabitEthernet0/0/0/1 in area 0 from LOADING to FULL, Loading Done, vrf default vrfid 0x60000000 

The OSPF adjacency went down and came back, yet there is not a single NBR_CHANGE. The LDP session never went down. In STEP 4 the same operation logged Interface state down.

show mpls ldp discovery lines up the reason.

  Interfaces:
    GigabitEthernet0/0/0/1 : xmit/recv
          Hold time: 15 sec (local:15 sec, peer:15 sec)
          Established: Sep 11 14:38:21.013 (00:03:02 ago)
  Targeted Hellos:
    2.2.2.2 -> 3.3.3.3 (active), xmit/recv
      LDP Id: 3.3.3.3:0
          Hold time: 90 sec (local:90 sec, peer:90 sec)
          Established: Sep 11 14:36:32.378 (00:04:51 ago)

The link was down from 14:37:18 to 14:38:21. The link hello adjacency was rebuilt afterwards (14:38:21), while the targeted Hello adjacency has never broken since before the outage (14:36:32). The hold times differ too: 15 seconds for the link, 90 for the targeted one.

Where they travelled shows in the capture. For the duration of the outage only, unicast Hellos flowed on the PE1 - P2 link (the detour).

That matches the outage window exactly. The reverse direction (P1 → PE1) appeared in the same window.

    Source Address: 2.2.2.2
    Destination Address: 3.3.3.3
    Source Port: 646
    Destination Port: 646
Label Distribution Protocol
    LSR ID: 2.2.2.2
    Label Space ID: 0
    Hello Message
        Message Type: Hello Message (0x100)
        Common Hello Parameters
            TLV Type: Common Hello Parameters (0x400)
            Hold Time: 90
            1... .... .... .... = Targeted Hello: Targeted Hello
            .1.. .... .... .... = Hello Requested: Source requests periodic hellos

The destination is a unicast to the peer’s Lo0 (3.3.3.3). The Hello Requested bit is the asymmetric exchange of section 2.4.2: the initiator asks to be sent Hellos back periodically.

Comparison

Three things follow.

Synchronization made a difference only between STEP 1 and 2, where 0% became 100%. That is the mechanism doing exactly what it is for: keeping the IGP from choosing a link where LDP cannot come up.

Restoring the link showed no difference with or without sync (STEP 4 and 5), because with the default timers LDP comes up first — the same order both times. Synchronization is not a feature that speeds up recovery; it is insurance against the IGP choosing a link where LDP is unusable. It matters when a missing configuration, a filter or a mismatched password stops LDP, or when LDP converges more slowly than the IGP because there are many prefixes or the peer is slow.

Session protection did not change the loss count or the outage either (STEP 4 and 5 against 6). In this topology traffic detours via P2 the moment the link fails and returns the same way, so the cost of switching is about 2.1 seconds regardless. What protection changed is whether the session and its labels are lost — keeping the labels means neither re-establishment nor re-advertisement is needed.

Captures

Three links were captured with no filter, across every STEP.

CE1 - PE1 (the measurement point)

PE1 - P1 (the link with the ACL)

PE1 - P2 (the detour, where the targeted Hellos go)

Verification config and show output

The following file types were collected from all eight routers at every STEP, split per router. The verification config is these ..._run.txt files (the final state is the one from the last STEP).

STEP 0: Initial state

STEP 1: An ACL on PE1 - P1 drops LDP Discovery

STEP 2: Add mpls ldp sync in OSPF

STEP 3: Remove the ACL

STEP 4: Shut and restore PE1 - P1 without sync

STEP 5: The same with sync

STEP 6: The same with session protection

STEP 7: Remove the configuration (final state)

References

Book: Luc De Ghein, MPLS Fundamentals (Cisco Press, 2006), Chapter 4

Related articles

• What Is MPLS
• MPLS Labels and the Label Stack
• MPLS Label Operations (push / swap / pop) and PHP
• MPLS TTL Processing and MTU
• What Is LDP
• LDP Label Distribution Modes and Label Spaces
• LDP-IGP Synchronization and LDP Session Protection