MPLS VPN VRF (Virtual Routing and Forwarding)

What a VRF Is in MPLS VPN

A VRF (Virtual Routing and Forwarding) is an independent pair of routing and forwarding tables inside a single router. Section 3 of RFC 4364 states that a PE router maintains several forwarding tables: the default one, plus “VPN Routing and Forwarding tables”, or VRFs.

A PE holds one VRF per customer (VPN) attached to it. Customer A’s routes go into customer A’s VRF, customer B’s into customer B’s, and neither sees the other. This separation is what lets customers with overlapping IP addresses share the same PE.

This article is about the container itself. How the contents of a VRF are carried to other PEs (RD, RT, and MP-BGP) is covered from MPLS VPN RD onwards. The overall picture is in What Is MPLS VPN (L3VPN).

The Global Table and the VRF Tables

The routing table a router has to begin with is the global table (IOS XR displays it as default). Creating a VRF adds another routing table beside it.

What you want to see	IOS XR command
The global table	`show route`
The routing table of one VRF	`show route vrf <name>`
The routing tables of every VRF	`show route vrf all`
The VRFs and their settings	`show vrf all detail`
Which VRF an interface belongs to	`show ipv4 vrf all interface brief`
The forwarding table of one VRF	`show cef vrf <name>`

Because the tables are separate, reachability tests name a VRF as well. ping 10.1.1.1 looks in the global table and ping vrf CUST-A 10.1.1.1 looks in the CUST-A table. The same address reaches a different device.

An Interface Belongs to Exactly One VRF

Routes in a VRF are not enough on their own. The router also has to decide which customer an incoming packet belongs to, and what it uses for that is the VRF membership of the interface. Section 3 of RFC 4364 likewise says that a PE-CE interface is associated with exactly one VRF.

On IOS XR you write vrf under the interface.

Putting an interface into a VRF (IOS XR)
interface GigabitEthernet0/0/0/0
 vrf CUST-A
 ipv4 address 172.16.1.1 255.255.255.0
!

An interface can belong to only one VRF, but a VRF can hold many interfaces. An interface with no VRF configured stays in the global table.

On IOS XR (XRd 26.1.1) adding vrf later does not remove the IP address. Some routers drop the address when an interface moves to another VRF, so do not carry a procedure over from another platform without checking show running-config interface afterwards.

Creating a VRF Needs No MP-BGP

A VRF is a container for a routing table, so it can be created on its own, with no MPLS and no MP-BGP. The minimum configuration on IOS XR is two lines.

Defining a VRF (IOS XR)
vrf CUST-A
 address-family ipv4 unicast
!

An RD and RTs are needed when the routes of that VRF are carried to other PEs. On IOS XR the RD is configured under router bgp (inside vrf <name>), so no RD appears until BGP is running. show vrf all detail reports RD not set.

Because of this, VRFs are useful outside MPLS VPN as well (see VRF-Lite below).

Overlapping Addresses Coexisting

Customers use whatever addresses they like, so two of them may both hold 10.1.1.0/24. As long as the VRFs are separate, the same prefix can appear more than once inside the PE without colliding.

It is not only the customer LANs that can overlap. The PE-CE link addresses can overlap too: with different VRFs, two interfaces of one PE can both carry 172.16.1.1/24.

Put the same prefixes into the global table instead, and one table ends up with two routes for it. The router treats them as two equal-cost paths, and packets for customer A can be sent towards customer B. STEP 0 below creates exactly that state.

VRF-Lite (Multi-VRF CE)

Using VRFs purely for separation inside one router, with no MPLS and no MP-BGP, is called VRF-Lite (Multi-VRF CE). A customer router separating its departments into different routing tables is a typical use.

Since there is no mechanism carrying the VPN routes to other routers, the separation ends at that device. To extend it to the next router, split the link logically (with VLANs, for example) and put each side into the matching VRF at both ends.

Lab Setup

Four XRd routers, with customer A and customer B using the same addresses. The LANs of CE-A1 and CE-B1 are both 10.1.1.0/24, and in STEP 4 the PE-CE links are made to overlap on 172.16.1.0/24 as well.

STEP	Action	What it shows
0	Initial state (no VRFs, all three PE interfaces global)	One table holds two routes for the same prefix and cannot tell the customers apart
1	Create `vrf CUST-A` and `vrf CUST-B` on PE1	A VRF can be created without BGP
2	Put Gi0/0/0/0 into CUST-A and Gi0/0/0/1 into CUST-B	Membership alone moves the table; the address stays
3	Move Gi0/0/0/2 into CUST-A and re-add the static routes per VRF	The same `10.1.1.0/24` lands in both VRFs, and `ping vrf` reaches different devices
4	Overlap the PE-CE links on `172.16.1.0/24`	Two interfaces of one PE can carry the same address
5	Create `vrf DEPT-1` and `DEPT-2` on CE-A1	A CE alone can separate its tables (VRF-Lite)

PE-CE routing is static here. Setups that run a routing protocol are covered from PE-CE routing (static) onwards.

Without a VRF the Customers Cannot Be Told Apart (STEP 0)

In STEP 0 no VRF exists and all three interfaces of PE1 sit in the global table. The static routes to the LANs of customers A and B are in that same table.

STEP 0: show route on PE1 (no VRF)
Gateway of last resort is not set

S    10.1.1.0/24 [1/0] via 172.16.1.2, 00:03:05
                 [1/0] via 172.16.11.2, 00:03:05
S    10.1.2.0/24 [1/0] via 172.16.2.2, 00:03:05
L    11.11.11.11/32 is directly connected, 00:03:11, Loopback0
C    172.16.1.0/24 is directly connected, 00:03:05, GigabitEthernet0/0/0/0
L    172.16.1.1/32 is directly connected, 00:03:05, GigabitEthernet0/0/0/0
C    172.16.2.0/24 is directly connected, 00:03:05, GigabitEthernet0/0/0/2
L    172.16.2.1/32 is directly connected, 00:03:05, GigabitEthernet0/0/0/2
C    172.16.11.0/24 is directly connected, 00:03:05, GigabitEthernet0/0/0/1
L    172.16.11.1/32 is directly connected, 00:03:05, GigabitEthernet0/0/0/1

10.1.1.0/24 has two routes. The first points at customer A (CE-A1) and the second at customer B (CE-B1); to the router they are simply two equal-cost paths to the same prefix. Nothing distinguishes which customer a packet is for.

The table actually used for forwarding (CEF) looks the same. In show cef, the single line for 10.1.1.0/24 carries two recursive next hops.

STEP 0: the global CEF table on PE1
RP/0/RP0/CPU0:PE1#show cef
Sun Sep 20 04:54:09.689 UTC

Prefix              Next Hop            Interface
------------------- ------------------- ------------------
0.0.0/0           drop                default handler 
0.0.0/32          broadcast
1.1.0/24         172.16.1.2/32       <recursive>
                    172.16.11.2/32      <recursive>
1.2.0/24         172.16.2.2/32       <recursive>
11.11.11/32      receive             Loopback0
16.1.0/24       attached            GigabitEthernet0/0/0/0
16.1.0/32       broadcast           GigabitEthernet0/0/0/0
16.1.1/32       receive             GigabitEthernet0/0/0/0
16.1.2/32       172.16.1.2/32       GigabitEthernet0/0/0/0
16.1.255/32     broadcast           GigabitEthernet0/0/0/0
16.2.0/24       attached            GigabitEthernet0/0/0/2
16.2.0/32       broadcast           GigabitEthernet0/0/0/2
16.2.1/32       receive             GigabitEthernet0/0/0/2
16.2.2/32       172.16.2.2/32       GigabitEthernet0/0/0/2
16.2.255/32     broadcast           GigabitEthernet0/0/0/2
16.11.0/24      attached            GigabitEthernet0/0/0/1
16.11.0/32      broadcast           GigabitEthernet0/0/0/1
16.11.1/32      receive             GigabitEthernet0/0/0/1
16.11.2/32      172.16.11.2/32      GigabitEthernet0/0/0/1
16.11.255/32    broadcast           GigabitEthernet0/0/0/1
0.0.0/4         0.0.0.0/32          
0.0.0/24        receive
255.255.255/32  broadcast

Measuring between customer A’s sites (from CE-A1 to CE-A2’s LAN) in this state, the ping itself succeeds.

STEP 0: reachability from CE-A1 to the other site of customer A
RP/0/RP0/CPU0:CE-A1#ping 10.1.2.1 source 10.1.1.1 count 50 timeout 1
Sun Sep 20 04:52:46.415 UTC
Type escape sequence to abort.
Sending 50, 100-byte ICMP Echos to 10.1.2.1 timeout is 1 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (50/50), round-trip min/avg/max = 8/10/29 ms
RP/0/RP0/CPU0:CE-A1#traceroute 10.1.2.1 source 10.1.1.1 timeout 1 probe 2 maxttl 4
Sun Sep 20 04:52:48.040 UTC

Type escape sequence to abort.
Tracing the route to 10.1.2.1

 1   *  * 
 2   *  * 
 3   *  * 
 4   *  * 

The ping is at 100%, yet traceroute gets no answer even from the first hop. The ICMP time exceeded replies are addressed to 10.1.1.1, and PE1 sends them down one of its two routes. Those sent towards customer B arrive at CE-B1, which owns 10.1.1.1 itself, and end there instead of returning to the sender.

Customer A’s packets are being handed to customer B, which is not a working VPN.

Creating VRFs and Separating the Routes (STEP 1 to 3)

STEP 1 creates two VRFs on PE1. This is everything that was committed, and there is not one line of BGP.

STEP 1: the configuration committed
vrf CUST-A
 address-family ipv4 unicast
 !
!
vrf CUST-B
 address-family ipv4 unicast
 !
!

STEP 1: show vrf all detail on PE1 (excerpt)
RP/0/RP0/CPU0:PE1#show vrf all detail
Sun Sep 20 04:58:52.194 UTC

VRF CUST-A; RD not set; VPN ID not set
VRF mode: Regular
Description not set
Address family IPV4 Unicast
  No import VPN route-target communities
  No export VPN route-target communities
  No import route policy
  No export route policy

RD not set, and the import and export RTs are empty. The VRF is already usable as a routing table.

STEP 2 assigns the interfaces. Only the vrf line was committed, yet Vrf-Name in show ipv4 vrf all interface brief changes and the address 172.16.1.1 is still there.

STEP 2: interface membership on PE1
RP/0/RP0/CPU0:PE1#show ipv4 vrf all interface brief
Sun Sep 20 05:06:10.788 UTC

Interface                      IP-Address      Status          Protocol Vrf-Name
Loopback0                      11.11.11.11     Up              Up       default 
MgmtEth0/RP0/CPU0/0            10.19.7.11      Up              Up       Mgmt    
GigabitEthernet0/0/0/0         172.16.1.1      Up              Up       CUST-A  
GigabitEthernet0/0/0/1         172.16.11.1     Up              Up       CUST-B  
GigabitEthernet0/0/0/2         172.16.2.1      Up              Up       default 
GigabitEthernet0/0/0/3         unassigned      Shutdown        Down     default 

STEP 3 moves the remaining Gi0/0/0/2 into CUST-A, removes the static routes from the global table and re-adds them per VRF. The result is the same 10.1.1.0/24 in both VRFs, once each.

STEP 3: show route vrf CUST-A on PE1
Gateway of last resort is not set

S    10.1.1.0/24 [1/0] via 172.16.1.2, 00:02:16
S    10.1.2.0/24 [1/0] via 172.16.2.2, 00:02:16
C    172.16.1.0/24 is directly connected, 00:09:27, GigabitEthernet0/0/0/0
L    172.16.1.1/32 is directly connected, 00:09:27, GigabitEthernet0/0/0/0
C    172.16.2.0/24 is directly connected, 00:02:16, GigabitEthernet0/0/0/2
L    172.16.2.1/32 is directly connected, 00:02:16, GigabitEthernet0/0/0/2

STEP 3: show route vrf CUST-B on PE1
Gateway of last resort is not set

S    10.1.1.0/24 [1/0] via 172.16.11.2, 00:02:17
C    172.16.11.0/24 is directly connected, 00:09:28, GigabitEthernet0/0/0/1
L    172.16.11.1/32 is directly connected, 00:09:28, GigabitEthernet0/0/0/1

The forwarding tables are split per VRF in the same way. In STEP 0 one table held two recursive next hops; now each table holds one.

STEP 3: show cef vrf CUST-A on PE1
RP/0/RP0/CPU0:PE1#show cef vrf CUST-A
Sun Sep 20 05:09:48.093 UTC

Prefix              Next Hop            Interface
------------------- ------------------- ------------------
0.0.0/0           drop                default handler 
0.0.0/32          broadcast
1.1.0/24         172.16.1.2/32       <recursive>
1.2.0/24         172.16.2.2/32       <recursive>
16.1.0/24       attached            GigabitEthernet0/0/0/0
16.1.0/32       broadcast           GigabitEthernet0/0/0/0
16.1.1/32       receive             GigabitEthernet0/0/0/0
16.1.2/32       172.16.1.2/32       GigabitEthernet0/0/0/0
16.1.255/32     broadcast           GigabitEthernet0/0/0/0
16.2.0/24       attached            GigabitEthernet0/0/0/2
16.2.0/32       broadcast           GigabitEthernet0/0/0/2
16.2.1/32       receive             GigabitEthernet0/0/0/2
16.2.2/32       172.16.2.2/32       GigabitEthernet0/0/0/2
16.2.255/32     broadcast           GigabitEthernet0/0/0/2
0.0.0/4         0.0.0.0/32          
0.0.0/24        receive
255.255.255/32  broadcast

STEP 3: show cef vrf CUST-B on PE1
RP/0/RP0/CPU0:PE1#show cef vrf CUST-B
Sun Sep 20 05:09:48.451 UTC

Prefix              Next Hop            Interface
------------------- ------------------- ------------------
0.0.0.0/0           drop                default handler 
0.0.0.0/32          broadcast
10.1.1.0/24         172.16.11.2/32      <recursive>
172.16.11.0/24      attached            GigabitEthernet0/0/0/1
172.16.11.0/32      broadcast           GigabitEthernet0/0/0/1
172.16.11.1/32      receive             GigabitEthernet0/0/0/1
172.16.11.2/32      172.16.11.2/32      GigabitEthernet0/0/0/1
172.16.11.255/32    broadcast           GigabitEthernet0/0/0/1
224.0.0.0/4         0.0.0.0/32          
224.0.0.0/24        receive
255.255.255.255/32  broadcast

Both tables hold 10.1.1.0/24, with different next hops. Naming a VRF on ping sends the same address to a different device.

STEP 3: reachability from PE1 with and without a VRF
RP/0/RP0/CPU0:PE1#ping vrf CUST-A 10.1.1.1 count 50 timeout 1
Sun Sep 20 05:07:37.488 UTC
Type escape sequence to abort.
Sending 50, 100-byte ICMP Echos to 10.1.1.1 timeout is 1 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (50/50), round-trip min/avg/max = 4/9/60 ms
RP/0/RP0/CPU0:PE1#ping vrf CUST-B 10.1.1.1 count 50 timeout 1
Sun Sep 20 05:07:39.294 UTC
Type escape sequence to abort.
Sending 50, 100-byte ICMP Echos to 10.1.1.1 timeout is 1 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (50/50), round-trip min/avg/max = 4/9/58 ms
RP/0/RP0/CPU0:PE1#ping 10.1.1.1 count 50 timeout 1
Sun Sep 20 05:07:41.195 UTC
Type escape sequence to abort.
Sending 50, 100-byte ICMP Echos to 10.1.1.1 timeout is 1 seconds:
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
Success rate is 0 percent (0/50)

The two pings that name a VRF are both at 100%, but they are reaching different devices. The last one, without a VRF, returns U (unreachable) because the global table no longer holds 10.1.1.0/24.

Traffic across customers does not get through either: a ping from customer B’s CE-B1 to customer A’s 10.1.2.1 is at 0%, so the leak into customer B seen in STEP 0 is gone.

Even the Link Addresses Can Overlap (STEP 4)

STEP 4 changes CE-B1 and PE1’s Gi0/0/0/1 to 172.16.1.0/24, so that the PE-CE link uses the same addresses as customer A’s.

STEP 4: PE1 interfaces (the same address twice)
RP/0/RP0/CPU0:PE1#show ipv4 vrf all interface brief
Sun Sep 20 05:13:50.621 UTC

Interface                      IP-Address      Status          Protocol Vrf-Name
Loopback0                      11.11.11.11     Up              Up       default 
MgmtEth0/RP0/CPU0/0            10.19.7.11      Up              Up       Mgmt    
GigabitEthernet0/0/0/0         172.16.1.1      Up              Up       CUST-A  
GigabitEthernet0/0/0/1         172.16.1.1      Up              Up       CUST-B  
GigabitEthernet0/0/0/2         172.16.2.1      Up              Up       CUST-A  
GigabitEthernet0/0/0/3         unassigned      Shutdown        Down     default 

Two interfaces of one PE carry the same 172.16.1.1. A configuration that the global table would reject as a duplicate is accepted once the VRFs differ. Reachability is unchanged from STEP 3.

VRF-Lite on a CE (STEP 5)

STEP 5 creates two VRFs on the customer router CE-A1 rather than the PE, and puts one loopback in each. There is no MPLS, no MP-BGP, and no involvement of the PE.

STEP 5: the configuration committed on CE-A1
vrf DEPT-1
 address-family ipv4 unicast
 !
!
vrf DEPT-2
 address-family ipv4 unicast
 !
!
interface Loopback2
 description dept-1 LAN 10.9.1.0/24
 vrf DEPT-1
 ipv4 address 10.9.1.1 255.255.255.0
!
interface Loopback3
 description dept-2 LAN 10.9.1.0/24 (overlaps dept-1)
 vrf DEPT-2
 ipv4 address 10.9.1.1 255.255.255.0
!

STEP 5: CE-A1 interfaces
RP/0/RP0/CPU0:CE-A1#show ipv4 vrf all interface brief
Sun Sep 20 05:18:04.919 UTC

Interface                      IP-Address      Status          Protocol Vrf-Name
Loopback0                      1.1.1.1         Up              Up       default 
Loopback1                      10.1.1.1        Up              Up       default 
Loopback2                      10.9.1.1        Up              Up       DEPT-1  
Loopback3                      10.9.1.1        Up              Up       DEPT-2  
MgmtEth0/RP0/CPU0/0            10.19.7.21      Up              Up       Mgmt    
GigabitEthernet0/0/0/0         172.16.1.2      Up              Up       default 
GigabitEthernet0/0/0/1         unassigned      Shutdown        Down     default 
GigabitEthernet0/0/0/2         unassigned      Shutdown        Down     default 
GigabitEthernet0/0/0/3         unassigned      Shutdown        Down     default 

The same 10.9.1.1 sits on two loopbacks, and show route vrf DEPT-1 and show route vrf DEPT-2 each hold only their own connected routes. VRF separation works the same way on a CE as on a PE.

Verification Config and show Output

For every STEP, the following files were collected from all four routers, split per router. The verification config is the ..._run.txt file (the final state is the one from the last STEP).

File	Content
`..._show.txt`	`show version`, `show interface description`, `show route`, and the VRF-related `show` commands
`..._ping.txt`	The `ping` and `traceroute` results for that STEP
`..._log.txt`	`show logging` limited to that STEP. A marker is written with `logmsg` at the start of each STEP and its timestamp passed to `show logging start`
`..._run.txt`	`show running-config` at that STEP (that is, the verification config)
`..._commit.cfg`	Only the configuration committed in that STEP, for the routers that were changed

In the final state PE1 has two VRFs with three interfaces in them, and CE-A1 has the two VRF-Lite VRFs.

STEP 0: initial state (no VRFs, everything in the global table)

Router	show	ping	syslog	running-config
PE1	show	ping	log	run
CE-A1	show	ping	log	run
CE-B1	show	ping	log	run
CE-A2	show	ping	log	run

STEP 1: create two VRFs on PE1

Router	show	ping	syslog	running-config	committed
PE1	show	ping	log	run	commit
CE-A1	show	ping	log	run	—
CE-B1	show	ping	log	run	—
CE-A2	show	ping	log	run	—

STEP 2: put the interfaces into the VRFs

Router	show	ping	syslog	running-config	committed
PE1	show	ping	log	run	commit
CE-A1	show	ping	log	run	—
CE-B1	show	ping	log	run	—
CE-A2	show	ping	log	run	—

STEP 3: re-add the static routes per VRF

Router	show	ping	syslog	running-config	committed
PE1	show	ping	log	run	commit
CE-A1	show	ping	log	run	—
CE-B1	show	ping	log	run	—
CE-A2	show	ping	log	run	—

STEP 4: overlap the PE-CE link addresses

Router	show	ping	syslog	running-config	committed
PE1	show	ping	log	run	commit
CE-A1	show	ping	log	run	—
CE-B1	show	ping	log	run	commit
CE-A2	show	ping	log	run	—

STEP 5: VRF-Lite on CE-A1 (final state)

Router	show	ping	syslog	running-config	committed
PE1	show	ping	log	run	—
CE-A1	show	ping	log	run	commit
CE-B1	show	ping	log	run	—
CE-A2	show	ping	log	run	—

References

RFC	Title	Summary
RFC 4364	BGP/MPLS IP Virtual Private Networks (VPNs)	A PE maintains several forwarding tables (VRFs) and each interface is associated with one of them (3)

Book: Luc De Ghein, MPLS Fundamentals (Cisco Press, 2006), Chapter 7

Test environment: Cisco IOS XRd 26.1.1 (Cisco Modeling Labs)