Hub and spoke in MPLS VPN

A hub-and-spoke VPN is one where traffic between sites always goes through a central site (the hub). It is used when the sites must not reach each other directly, for example because a firewall or an inspection device at the hub has to see every packet.

Section 4.3.5 of RFC 4364 describes how to build this shape with two RTs.

Alternatively, suppose one desired, for whatever reason, to create a “hub and spoke” kind of VPN. This could be done by the use of two Route Target values, one meaning “Hub” and one meaning “Spoke”. At the VRFs attached to the hub sites, “Hub” is the Export Target and “Spoke” is the Import Target. At the VRFs attached to the spoke site, “Hub” is the Import Target and “Spoke” is the Export Target.

The hub exports “Hub” and imports “Spoke”; the spokes do the opposite. The spokes do not import each other’s export, so they cannot reach each other directly. That much is done with RTs alone, and it is covered in Controlling routes with RTs in MPLS VPN.

This article is about what comes next. Being unable to reach each other directly is not the goal: the sites still have to communicate, through the hub.

Turning a route around needs two VRFs at the hub

Importing a spoke route into the hub’s VRF does not send it back out into VPNv4. VPNv4 routes are distributed between PEs over iBGP, and BGP does not pass a route learned from one iBGP peer on to another iBGP peer. Once the route lands in the hub’s VRF, it stops there.

The only way to turn it around is to let it leave the VPN and have it handed back.

The hub PE imports the spoke routes into a VRF and advertises them to the hub CE
The hub CE advertises them back over a different session
The hub PE receives them and puts them back into VPNv4 with the RT the spokes import

Step 2 has to use a different session. BGP does not advertise a route back to the peer it learned it from, so a single session would never return it. The hub PE therefore runs two eBGP sessions to the hub CE.

On top of that, the receiving side and the sending side need opposite RTs, so the hub PE holds two VRFs: one that only receives the spoke routes (import only) and one that only sends the returned routes back out (export only). Each has its own RD.

The returning route is filtered twice on its AS_PATH

As the route travels through the hub CE it collects two AS numbers. It starts out with an empty AS_PATH at the spoke CE, but because the spoke PE and the hub PE are in the same AS (65001 in this lab), it reads 65001 65102 when the hub CE receives it and 65101 65001 65102 when the hub CE advertises it back. That 65001 is filtered in two places.

The first is the sending side. BGP does not advertise a route to a peer when that peer’s AS number is already in the AS_PATH. From the hub CE, the peer (the hub PE) is AS 65001, which is already there, so the hub CE never offers the route. The same behaviour appears in When several MPLS VPN sites share one AS number as a PE not advertising to a CE; here the roles are swapped and it is the hub CE that withholds the route.

Note that this sending-side suppression is not written in section 9.2 (Update-Send Process) of RFC 4271: it is implementation behaviour rather than a requirement, and it can differ between devices. What the RFC does specify is the receiving side below.

local-as is what opens this. Making the hub PE present a different AS number on that session changes the peer AS the hub CE sees, so it is no longer in the AS_PATH. That number is also prepended to the AS_PATH of the routes the hub PE receives on the session.

The second is the receiving side. Section 9.1.2 of RFC 4271 says that a route whose AS_PATH contains the local AS number is excluded from the decision process.

If the AS_PATH attribute of a BGP route contains an AS loop, the BGP route should be excluded from the Phase 2 decision function. AS loop detection is done by scanning the full AS path (as specified in the AS_PATH attribute), and checking that the autonomous system number of the local system does not appear in the AS path. Operations of a BGP speaker that is configured to accept routes with its own autonomous system number in the AS path are outside the scope of this document.

The route the hub CE sends back carries 65001, so the hub PE hits exactly this. The last sentence sets aside “a BGP speaker that is configured to accept routes with its own autonomous system number in the AS path” as out of scope, and that configuration is allowas-in.

The lab

Customer A has a hub site (CE-HUB) and two spoke sites (CE-SP1 and CE-SP2). The two spokes sit on different PEs: in one VRF on one PE they could reach each other whatever the RTs say. PE and CE run eBGP, and each CE has its own AS (65101, 65102 and 65103) so that as-override does not enter the picture and only the conditions for re-advertisement are on show.

Only the hub PE (PE1) holds two VRFs. CUST-A-HUBIN only imports 65001:1001, and CUST-A-HUBOUT only exports 65001:1000. The spoke VRFs do the opposite: they export 65001:1001 and import 65001:1000.

The day-0 configuration contains neither allowas-in nor local-as. Each is added in turn to see which one is needed.

STEP	Change	What it shows
0	Initial state (no `allowas-in`, no `local-as`)	The hub CE receives on (1) but sends back only its own route on (2)
1	`allowas-in 2` on the hub-out neighbor of PE1	No change: a setting on the receiving side cannot help while the sender withholds the route
2	`local-as 65011` on the hub-out neighbor of PE1, and `remote-as 65011` on the hub CE	The hub CE starts sending them back. The spokes can ping each other and the hub CE shows in the traceroute
3	`allowas-in 2` on the vpnv4 neighbors of PE2 and PE3	No change: it already worked at STEP 2, so the spoke PEs do not need it
4	Remove `allowas-in` from the hub-out neighbor of PE1	The hub CE keeps sending, but nothing arrives. The block is on the receiving side
5	Put `allowas-in` back (final state)	Back to the previous state

The hub CE receives the routes but does not send them back

In the initial state the hub CE’s BGP table holds both spoke routes, learned over link (1).

STEP 0 BGP table on CE-HUB
RP/0/RP0/CPU0:CE-HUB#show bgp
Thu Oct 15 12:57:45.797 UTC
BGP router identifier 11.11.11.11, local AS number 65101
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000000   RD version: 9
BGP main routing table version 9
BGP NSR Initial initsync version 3 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
*> 10.1.1.1/32        0.0.0.0                  0         32768 i
*> 10.1.2.1/32        172.16.1.1                             0 65001 65102 i
*> 10.1.3.1/32        172.16.1.1                             0 65001 65103 i
*> 172.16.1.0/24      172.16.1.1               0             0 65001 ?
*> 172.16.2.0/24      172.16.1.1                             0 65001 ?
*> 172.16.3.0/24      172.16.1.1                             0 65001 ?
*> 172.16.5.0/24      172.16.5.1               0             0 65001 ?

Processed 7 prefixes, 7 paths

What it advertises on link (2), however, is only its own 10.1.1.1/32.

STEP 0 Routes CE-HUB advertises to PE1 on link (2)
RP/0/RP0/CPU0:CE-HUB#show bgp neighbors 172.16.5.1 advertised-routes
Thu Oct 15 12:57:48.382 UTC
Network            Next Hop        From            AS Path
10.1.1.1/32        172.16.5.2      Local           65101i

Processed 1 prefixes, 1 paths

The AS_PATH of the spoke routes contains 65001, the AS of the PEs, and the peer it would send them to is in that same AS, so the hub CE withholds them. The connected routes (172.16.1.0/24 and the others) come from redistribute connected on the PEs and carry the same AS, so they are withheld for the same reason.

Spoke 1 cannot reach spoke 2.

STEP 0 Ping from CE-SP1 to CE-SP2
RP/0/RP0/CPU0:CE-SP1#ping 10.1.3.1 source 10.1.2.1 count 10 timeout 1
Thu Oct 15 12:54:59.412 UTC
Type escape sequence to abort.
Sending 10, 100-byte ICMP Echos to 10.1.3.1 timeout is 1 seconds:
..........
Success rate is 0 percent (0/10)

`allowas-in` on its own changes nothing

allowas-in goes on the hub PE’s neighbor on link (2).

STEP 1 Configuration committed on PE1
router bgp 65001
 vrf CUST-A-HUBOUT
  neighbor 172.16.5.2
   address-family ipv4 unicast
    allowas-in 2
   !
  !
 !
!
end

STEP 1 Routes CE-HUB advertises to PE1 on link (2)
RP/0/RP0/CPU0:CE-HUB#show bgp neighbors 172.16.5.1 advertised-routes
Thu Oct 15 13:03:37.786 UTC
Network            Next Hop        From            AS Path
10.1.1.1/32        172.16.5.2      Local           65101i

Processed 1 prefixes, 1 paths

Nothing changes. allowas-in is a setting on the receiving side, and it cannot do anything while the sending side is not offering the route.

`local-as` opens the sending side

local-as 65011 goes on the hub PE’s neighbor on link (2), and the hub CE’s remote-as for that session becomes 65011.

STEP 2 Configuration committed on PE1
router bgp 65001
 vrf CUST-A-HUBOUT
  neighbor 172.16.5.2
   local-as 65011
  !
 !
!
end

STEP 2 Configuration committed on CE-HUB
router bgp 65101
 neighbor 172.16.5.1
  remote-as 65011
 !
!
end

The peer AS the hub CE sees is now 65011, which no longer collides with the 65001 in the AS_PATH, and the spoke routes are advertised.

STEP 2 Routes CE-HUB advertises to PE1 on link (2)
RP/0/RP0/CPU0:CE-HUB#show bgp neighbors 172.16.5.1 advertised-routes
Thu Oct 15 13:10:42.637 UTC
Network            Next Hop        From            AS Path
10.1.1.1/32        172.16.5.2      Local           65101i
10.1.2.1/32        172.16.5.2      172.16.1.1      65101 65001 65102i
10.1.3.1/32        172.16.5.2      172.16.1.1      65101 65001 65103i
172.16.1.0/24      172.16.5.2      172.16.1.1      65101 65001?
172.16.2.0/24      172.16.5.2      172.16.1.1      65101 65001?
172.16.3.0/24      172.16.5.2      172.16.1.1      65101 65001?

Processed 6 prefixes, 6 paths

The capture shows the same thing. No.17 of the attached link (2) capture for STEP 2 carries six UPDATEs, and this is the one for 10.1.3.1/32.

STEP 2 No.17 UPDATE from CE-HUB to PE1 (excerpt of tshark -V)
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 62
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 0
    Total Path Attribute Length: 39
    Path attributes
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
                1... .... = Optional: Set
                .0.. .... = Transitive: Not set
                ..0. .... = Partial: Not set
                ...1 .... = Extended-Length: Set
                .... 0000 = Unused: 0x0
            Type Code: MP_REACH_NLRI (14)
            Length: 14
            Address family identifier (AFI): IPv4 (1)
            Subsequent address family identifier (SAFI): Unicast (1)
            Next hop: 172.16.5.2
                IPv4 Address: 172.16.5.2
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                10.1.3.1/32
                    MP Reach NLRI prefix length: 32
                    MP Reach NLRI IPv4 prefix: 10.1.3.1
        Path Attribute - ORIGIN: IGP
            Flags: 0x40, Transitive, Well-known, Complete
                0... .... = Optional: Not set
                .1.. .... = Transitive: Set
                ..0. .... = Partial: Not set
                ...0 .... = Extended-Length: Not set
                .... 0000 = Unused: 0x0
            Type Code: ORIGIN (1)
            Length: 1
            Origin: IGP (0)
        Path Attribute - AS_PATH: 65101 65001 65103 
            Flags: 0x40, Transitive, Well-known, Complete
                0... .... = Optional: Not set
                .1.. .... = Transitive: Set
                ..0. .... = Partial: Not set
                ...0 .... = Extended-Length: Not set
                .... 0000 = Unused: 0x0
            Type Code: AS_PATH (2)
            Length: 14
            AS Path segment: 65101 65001 65103
                Segment type: AS_SEQUENCE (2)
                Segment length (number of ASN): 3
                AS4: 65101
                AS4: 65001
                AS4: 65103

Download the pcap of the packet in the tshark output above (No.17 UPDATE)

allowas-in lets the hub PE accept this route even though its own AS is in the path. The local-as value is prepended, so the path reads 65011 65101 65001 65103.

STEP 2 Routes PE1 received on link (2)
RP/0/RP0/CPU0:PE1#show bgp vrf CUST-A-HUBOUT neighbors 172.16.5.2 received routes
Thu Oct 15 13:08:50.015 UTC
BGP VRF CUST-A-HUBOUT, state: Active
BGP Route Distinguisher: 65001:12
VRF ID: 0x60000003
BGP router identifier 1.1.1.1, local AS number 65001
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000003   RD version: 30
BGP main routing table version 30
BGP NSR Initial initsync version 13 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65001:12 (default for vrf CUST-A-HUBOUT)
Route Distinguisher Version: 30
*  10.1.1.1/32        172.16.5.2               0             0 65011 65101 i
*  10.1.2.1/32        172.16.5.2                             0 65011 65101 65001 65102 i
*  10.1.3.1/32        172.16.5.2                             0 65011 65101 65001 65103 i
*  172.16.1.0/24      172.16.5.2                             0 65011 65101 65001 ?
*  172.16.2.0/24      172.16.5.2                             0 65011 65101 65001 ?
*  172.16.3.0/24      172.16.5.2                             0 65011 65101 65001 ?

Processed 6 prefixes, 6 paths

The other spoke’s route is now in spoke 1’s BGP table.

STEP 2 BGP table on CE-SP1
RP/0/RP0/CPU0:CE-SP1#show bgp
Thu Oct 15 13:10:58.180 UTC
BGP router identifier 12.12.12.12, local AS number 65102
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000000   RD version: 11
BGP main routing table version 11
BGP NSR Initial initsync version 3 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
*> 10.1.1.1/32        172.16.2.1                             0 65001 65011 65101 i
*> 10.1.2.1/32        0.0.0.0                  0         32768 i
*> 10.1.3.1/32        172.16.2.1                             0 65001 65011 65101 65001 65103 i
*> 172.16.1.0/24      172.16.2.1                             0 65001 65011 65101 65001 ?
*> 172.16.2.0/24      172.16.2.1               0             0 65001 ?
*> 172.16.3.0/24      172.16.2.1                             0 65001 65011 65101 65001 ?
*> 172.16.5.0/24      172.16.2.1                             0 65001 ?

Processed 7 prefixes, 7 paths

The ping gets through.

STEP 2 Ping from CE-SP1 to CE-SP2
RP/0/RP0/CPU0:CE-SP1#ping 10.1.3.1 source 10.1.2.1 count 10 timeout 1
Thu Oct 15 13:08:13.957 UTC
Type escape sequence to abort.
Sending 10, 100-byte ICMP Echos to 10.1.3.1 timeout is 1 seconds:
!!!!!!!!!!
Success rate is 100 percent (10/10), round-trip min/avg/max = 22/25/35 ms

The hub CE appears in the traceroute. Hop 4, 172.16.5.2, is the hub CE, and hop 5, 172.16.1.1, is the hub PE on link (1): the packet leaves the hub PE, turns around at the hub CE and comes back on the other link.

STEP 2 Traceroute from CE-SP1 to CE-SP2
RP/0/RP0/CPU0:CE-SP1#traceroute 10.1.3.1 source 10.1.2.1 timeout 1 probe 2 maxttl 6
Thu Oct 15 13:08:16.111 UTC

Type escape sequence to abort.
Tracing the route to 10.1.3.1

 1  172.16.2.1 6 msec  5 msec 
 2  10.0.23.3 [MPLS: Labels 24000/24010 Exp 0] 13 msec  12 msec 
 3  10.0.13.1 [MPLS: Label 24010 Exp 0] 11 msec  12 msec 
 4  172.16.5.2 12 msec  12 msec 
 5  172.16.1.1 15 msec  12 msec 
 6  10.0.13.3 [MPLS: Labels 24002/24006 Exp 0] 23 msec  22 msec 

The trace stops at six hops (maxttl 6) so CE-SP2 is not shown, but the ping above shows that it is reached.

The spoke PEs do not need it

The route the hub CE sends back still carries 65001 in its AS_PATH as it crosses the core in VPNv4, and the spoke PEs that receive it are in AS 65001 too, so they might drop it as well. allowas-in goes on the vpnv4 neighbor of a spoke PE to find out.

STEP 3 Configuration committed on PE2
router bgp 65001
 neighbor 1.1.1.1
  address-family vpnv4 unicast
   allowas-in 2
  !
 !
!
end

STEP 3 Ping from CE-SP1 to CE-SP2
RP/0/RP0/CPU0:CE-SP1#ping 10.1.3.1 source 10.1.2.1 count 10 timeout 1
Thu Oct 15 13:13:52.674 UTC
Type escape sequence to abort.
Sending 10, 100-byte ICMP Echos to 10.1.3.1 timeout is 1 seconds:
!!!!!!!!!!
Success rate is 100 percent (10/10), round-trip min/avg/max = 21/25/41 ms

Nothing changes. The route was already getting through at STEP 2, so this setting is not needed: the AS loop check bites where the route is received over eBGP, not on the iBGP vpnv4 session.

Removing `allowas-in` stops it on the receiving side

allowas-in is removed from the hub PE’s link (2) neighbor, while local-as stays.

STEP 4 Configuration committed on PE1
router bgp 65001
 vrf CUST-A-HUBOUT
  neighbor 172.16.5.2
   address-family ipv4 unicast
    no allowas-in 2
   !
  !
 !
!
end

The hub CE keeps sending.

STEP 4 Routes CE-HUB advertises to PE1 on link (2)
RP/0/RP0/CPU0:CE-HUB#show bgp neighbors 172.16.5.1 advertised-routes
Thu Oct 15 13:23:41.692 UTC
Network            Next Hop        From            AS Path
10.1.1.1/32        172.16.5.2      Local           65101i
10.1.2.1/32        172.16.5.2      172.16.1.1      65101 65001 65102i
10.1.3.1/32        172.16.5.2      172.16.1.1      65101 65001 65103i
172.16.1.0/24      172.16.5.2      172.16.1.1      65101 65001?
172.16.2.0/24      172.16.5.2      172.16.1.1      65101 65001?
172.16.3.0/24      172.16.5.2      172.16.1.1      65101 65001?

Processed 6 prefixes, 6 paths

Only one route reaches the hub PE: 10.1.1.1/32, the one whose AS_PATH does not contain 65001.

STEP 4 Routes PE1 received on link (2)
RP/0/RP0/CPU0:PE1#show bgp vrf CUST-A-HUBOUT neighbors 172.16.5.2 received routes
Thu Oct 15 13:21:47.061 UTC
BGP VRF CUST-A-HUBOUT, state: Active
BGP Route Distinguisher: 65001:12
VRF ID: 0x60000003
BGP router identifier 1.1.1.1, local AS number 65001
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000003   RD version: 35
BGP main routing table version 35
BGP NSR Initial initsync version 13 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65001:12 (default for vrf CUST-A-HUBOUT)
Route Distinguisher Version: 35
*  10.1.1.1/32        172.16.5.2               0             0 65011 65101 i

Processed 1 prefixes, 1 paths

That neighbor has soft-reconfiguration inbound always, which keeps the routes as received, before any inbound policy. They are not in received routes either, so the routes are not missing: they arrive and are not accepted. What drops them is the check from section 9.1.2 of RFC 4271 quoted earlier. Note that this lab did not collect a counter for the discard itself (show bgp ... neighbors <peer> was not taken for this neighbor); what it shows is the contrast between the six routes sent and the one received.

Reachability between the spokes is gone.

STEP 4 Ping from CE-SP1 to CE-SP2
RP/0/RP0/CPU0:CE-SP1#ping 10.1.3.1 source 10.1.2.1 count 10 timeout 1
Thu Oct 15 13:20:49.884 UTC
Type escape sequence to abort.
Sending 10, 100-byte ICMP Echos to 10.1.3.1 timeout is 1 seconds:
..........
Success rate is 0 percent (0/10)

Neither local-as nor allowas-in is sufficient on its own. The wall on the sending side and the wall on the receiving side are separate, and both have to be opened before the hub CE’s re-advertisement works.

Lab configuration and show output

Every STEP is collected from all seven routers, one file per router and kind. The configuration under test is in ..._run.txt (the final state is the one from the last STEP).

File	Contents
`..._show.txt`	`show version`, `show interface description`, and the OSPF, LDP, BGP and VRF show commands
`..._log.txt`	`show logging` narrowed to that STEP
`..._run.txt`	`show running-config` at that STEP (the configuration under test)
`..._ping.txt`	`ping` and `traceroute` for that STEP (between every pair of CEs)
`..._trace.txt`	`show bgp trace` (PEs only)
`..._commit.cfg`	What was actually committed in that STEP (only the routers that changed)

STEP 0: Initial state (no allowas-in, no local-as)

Router	show	syslog	running-config	ping	trace	committed config
CE-HUB	show	log	run	ping	-	-
PE1	show	log	run	ping	trace	-
P1	show	log	run	ping	-	-
PE2	show	log	run	ping	trace	-
CE-SP1	show	log	run	ping	-	-
PE3	show	log	run	ping	trace	-
CE-SP2	show	log	run	ping	-	-

STEP 1: allowas-in 2 on the hub-out neighbor of PE1

Router	show	syslog	running-config	ping	trace	committed config
CE-HUB	show	log	run	ping	-	-
PE1	show	log	run	ping	trace	commit
P1	show	log	run	ping	-	-
PE2	show	log	run	ping	trace	-
CE-SP1	show	log	run	ping	-	-
PE3	show	log	run	ping	trace	-
CE-SP2	show	log	run	ping	-	-

STEP 2: local-as 65011 on the hub-out neighbor of PE1, and remote-as 65011 on the hub CE

Router	show	syslog	running-config	ping	trace	committed config
CE-HUB	show	log	run	ping	-	commit
PE1	show	log	run	ping	trace	commit
P1	show	log	run	ping	-	-
PE2	show	log	run	ping	trace	-
CE-SP1	show	log	run	ping	-	-
PE3	show	log	run	ping	trace	-
CE-SP2	show	log	run	ping	-	-

STEP 3: allowas-in 2 on the vpnv4 neighbors of PE2 and PE3

Router	show	syslog	running-config	ping	trace	committed config
CE-HUB	show	log	run	ping	-	-
PE1	show	log	run	ping	trace	-
P1	show	log	run	ping	-	-
PE2	show	log	run	ping	trace	commit
CE-SP1	show	log	run	ping	-	-
PE3	show	log	run	ping	trace	commit
CE-SP2	show	log	run	ping	-	-

STEP 4: Remove allowas-in from the hub-out neighbor of PE1

Router	show	syslog	running-config	ping	trace	committed config
CE-HUB	show	log	run	ping	-	-
PE1	show	log	run	ping	trace	commit
P1	show	log	run	ping	-	-
PE2	show	log	run	ping	trace	-
CE-SP1	show	log	run	ping	-	-
PE3	show	log	run	ping	trace	-
CE-SP2	show	log	run	ping	-	-

STEP 5: Put allowas-in back (final state)

Router	show	syslog	running-config	ping	trace	committed config
CE-HUB	show	log	run	ping	-	-
PE1	show	log	run	ping	trace	commit
P1	show	log	run	ping	-	-
PE2	show	log	run	ping	trace	-
CE-SP1	show	log	run	ping	-	-
PE3	show	log	run	ping	trace	-
CE-SP2	show	log	run	ping	-	-

Packet captures are taken per STEP.

STEP	PE1-CE-HUB (link 2)	PE1-P1
0	pcap	pcap
1	pcap	pcap
2	pcap	pcap
3	pcap	pcap
4	pcap	pcap
5	pcap	pcap