MPLS L3VPN – Route Distinguisher vs Route Target vs VPN label

A lot of people confuse the above 3 items. I’ll explain exactly what each of the 3 above items do, how you can see them, and how the routers use them to provide a L3VPN service.

Let’s take the following topology for this post:

Here we have 2 L3VPN customers running over our MPLS core. R5 is advertising 5.5.5.5/32. R8 is also advertising 5.5.5.5/32

Route Distinguisher:

The route distinguisher’s sole job is to keep a route unique while the PE routers advertise NLRI (Network Layer Reachability Information) to each other. If R5 and R8 both advertise 5.5.5.5/32 to R3, how will R3 advertise both of those routes to R4 while keeping them unique. The VPNV4 family itself doesn’t run in a VRF. It runs in the global routing instance and hence it needs something to distinguish a route.

Let’s take a quick look at the vrf RD config for both customers and then the vpnv4 route for 6.6.6.6/32 in the BGP table on R3:

R3#sh run | include ip vrf | rd
ip vrf CUS1
 rd 3.3.3.3:100
ip vrf CUS2
 rd 3.3.3.3:200

R3#sh bgp vpnv4 unicast rd 3.3.3.3:200 6.6.6.6
BGP routing table entry for 3.3.3.3:200:6.6.6.6/32, version 106
Paths: (1 available, best #1, table CUS2)
  Not advertised to any peer
  Local, imported path from 4.4.4.4:200:6.6.6.6/32
    4.4.4.4 (metric 4) from 4.4.4.4 (4.4.4.4)
      Origin incomplete, metric 2, localpref 100, valid, internal, best
      Extended Community: RT:100:200 OSPF DOMAIN ID:0x0005:0x000000030200
        OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:10.0.47.4:0
      mpls labels in/out nolabel/21

R3#sh bgp vpnv4 unicast rd 3.3.3.3:100 6.6.6.6
BGP routing table entry for 3.3.3.3:100:6.6.6.6/32, version 108
Paths: (1 available, best #1, table CUS1)
  Not advertised to any peer
  Local, imported path from 4.4.4.4:100:6.6.6.6/32
    4.4.4.4 (metric 4) from 4.4.4.4 (4.4.4.4)
      Origin incomplete, metric 2, localpref 100, valid, internal, best
      Extended Community: RT:100:100 OSPF DOMAIN ID:0x0005:0x000000020200
        OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:10.0.46.4:0
      mpls labels in/out nolabel/23

You can see that R3 has 2 vpnv4 routes for 6.6.6.6/32 – 3.3.3.3:200:6.6.6.6/32 and 3.3.3.3:100:6.6.6.6/32. They are unique as one contains :100: and the other contains :200: – Note that R4 does not have to match this RD in any way. It simple needs to be able to accept 2 unique routes. This is especially important when using route reflectors as RR’s will normally only advertise the best route to it’s clients. If they were not unique, the RR would only be advertising one of these routes. The RD in no way determines what VPN a route actually belongs to.

That’s all the route distinguisher does. No more.

Route Target:

The route target’s job is to tell the PE routers what VPN a route actually belongs to. Let’s take a look at the target config on R3:

R3#sh run | inc ip vrf|target
ip vrf CUS1
 route-target export 100:100
 route-target import 100:100
ip vrf CUS2
 route-target export 100:200
 route-target import 100:200

When R3 receives an advertisement from R5, not only does it change the route into a vpnv4 route with the RD to make it unique, it also adds a community value to that advertisement. This is an RT value. Once this NLRI gets to R4, R4 will ensure that only routes that have a certain RT, will be placed in their respective VRF. As an example let’s have a look at the advertisements of 5.5.5.5 from R3 to R4:

R3#sh bgp vpnv4 unicast rd 3.3.3.3:100 5.5.5.5
BGP routing table entry for 3.3.3.3:100:5.5.5.5/32, version 37
Paths: (1 available, best #1, table CUS1)
  Advertised to update-groups:
     9
  Local
    10.0.35.5 from 0.0.0.0 (3.3.3.3)
      Origin incomplete, metric 2, localpref 100, weight 32768, valid, sourced, best
      Extended Community: RT:100:100 OSPF DOMAIN ID:0x0005:0x000000020200
        OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:10.0.35.3:0
      mpls labels in/out 24/nolabel

We can see the extended community of 100:100 is encoded into this NLRI on R3. This is advertised to R4:

R4#sh bgp vpnv4 unicast rd 3.3.3.3:100 5.5.5.5
BGP routing table entry for 3.3.3.3:100:5.5.5.5/32, version 178
Paths: (1 available, best #1, no table)
  Not advertised to any peer
  Local
    3.3.3.3 (metric 4) from 3.3.3.3 (3.3.3.3)
      Origin incomplete, metric 2, localpref 100, valid, internal, best
      Extended Community: RT:100:100 OSPF DOMAIN ID:0x0005:0x000000020200
        OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:10.0.35.3:0
      mpls labels in/out nolabel/24

R4#sh run | include ip vrf | 100:100
ip vrf CUS1
 route-target export 100:100
 route-target import 100:100

R4 has an import 100:100 configuration under it’s VRF, and hence matching the community of 100:100 on the received NLRI, the PE router knows that the advertisement is meant for vrf CUS1. Note that the RD has nothing to do with this.

VPN Label:

The VPN label is to determine what VPN a packet belongs to. But hang on, surely that’s what the RT is for? No. The RT is for the control plane, while the VPN label is for the data plane. Let’s expand on that idea a bit. When R3 advertises NLRI to R4, the RT is used to determine where a route actually belongs. When it comes to R5 actually sending a packet to R6, the VPN label is used. Why? Because when a packet is sent, there is no field in the packet that the route-target is stored. Only the route advertisement contains the route-target as a community value. When R5 sends a ping to R6 from it’s loopback, it’s simply a packet with a destination address of 6.6.6.6 and a source address of 5.5.5.5.

So with L3VPNs we have two labels. The top label is the transport label and the bottom label is the VPN label. PHP will pop the transport label off the second to last router, but the VPN label will only be popped by the actual PE in question. When that frame comes in with the VPN label, R4 knows which VRF that packet belongs to.

VPN labels are advertised in the NLRI along with the RT. Let’s take a look at the 2 VPN labels that R4 is advertising to R3:

R3#sh bgp vpnv4 unicast rd 4.4.4.4:100 6.6.6.6
BGP routing table entry for 4.4.4.4:100:6.6.6.6/32, version 6
Paths: (1 available, best #1, no table)
  Not advertised to any peer
  Local
    4.4.4.4 (metric 4) from 4.4.4.4 (4.4.4.4)
      Origin incomplete, metric 2, localpref 100, valid, internal, best
      Extended Community: RT:100:100 OSPF DOMAIN ID:0x0005:0x000000020200
        OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:10.0.46.4:0
      mpls labels in/out nolabel/21
R3#sh bgp vpnv4 unicast rd 4.4.4.4:200 6.6.6.6
BGP routing table entry for 4.4.4.4:200:6.6.6.6/32, version 8
Paths: (1 available, best #1, no table)
  Not advertised to any peer
  Local
    4.4.4.4 (metric 4) from 4.4.4.4 (4.4.4.4)
      Origin incomplete, metric 2, localpref 100, valid, internal, best
      Extended Community: RT:100:200 OSPF DOMAIN ID:0x0005:0x000000030200
        OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:10.0.47.4:0
      mpls labels in/out nolabel/23

We can see that R3 will use a VPN label of 21 when sending traffic to the CUS1 VRF, while it’ll use VPN label 23 when sending to CUS2’s VRF.

Let’s run a traceroute from R5 and R8 to confirm this.
CUS1:

R5#traceroute 6.6.6.6

Type escape sequence to abort.
Tracing the route to 6.6.6.6

  1 10.0.35.3 36 msec *  52 msec
  2 10.0.13.1 [MPLS: Labels 20/21 Exp 0] 120 msec 120 msec 132 msec
  3 10.0.12.2 [MPLS: Labels 18/21 Exp 0] 92 msec 148 msec 104 msec
  4 10.0.46.4 [MPLS: Label 21 Exp 0] 104 msec 100 msec 68 msec
  5 10.0.46.6 172 msec *  140 msec

CUS2:

R8#traceroute 6.6.6.6

Type escape sequence to abort.
Tracing the route to 6.6.6.6

  1 10.0.38.3 44 msec 64 msec 40 msec
  2 10.0.13.1 [MPLS: Labels 20/23 Exp 0] 132 msec 132 msec 88 msec
  3 10.0.12.2 [MPLS: Labels 18/23 Exp 0] 124 msec 156 msec 104 msec
  4 10.0.47.4 [MPLS: Label 23 Exp 0] 192 msec 96 msec 76 msec
  5 10.0.47.7 156 msec *  116 msec

The first hop for CUS1 shows a label stack of 20/21 – 20 being transport and 21 being the VPN. CUS2 uses 20/23 – Notice that as the egress PE is the same, the same transport label is used.

On the 3rd hop on both, R2 is popping off the transport label. Both frames now get to R4. One has a VPN label of 21 and the second a label of 23. R4 knows which VRF both packets belong to and sends them on their way to the correct routers.

Hopefully this helps clear this up for some of you..

Published by

Darren

Network Architect. Dual CCIE and JNCIE-SP.

25 thoughts on “MPLS L3VPN – Route Distinguisher vs Route Target vs VPN label”

  1. Ah it definitely clears things up! :)

    Can you maybe explain something about the working of the NRLI? How does it work in the background? When you add a new site to a particular VPN, how does the NLRI get on the new router? Will it be “flooded/broadcasted” once BGP session comes up (either by RR or full mesh iBGP)? Or will it subscribe to and ask for it? And does it still store the information of a particular VPN if it is not configured on a router?

    Also, could you elaborate some cool scenario in which you would use multiple route-targets within a VRF or just limit it to e.g. import? I must be missing something, but I assume you wouldn’t configure just import+export for 1 RT on every router?

    Thanks

  2. Nice post Darren,

    Many people have this confused. They have issues understanding control plane (RT) vs data plane (VPN label).

    Mr_X

    Sometimes you might have a services VPN for like network management, SNMP and things like that. Then you would import to all VPNs from that RT and you would need to export with something that the services VPN would pick up as well.

  3. Thanks Daniel!

    Mr_X – I’ll try and do some more MPLS stuff. I’ve got a LOT of ideas for posts, so time permitting there will be more :)

  4. Although the gist of this is technically correct, it kind of skips over a key detail: the only reason a transport label is not used is because both PE’s in this case are PHP’s for any traffic forwarded between them. You mentioned they are for the VPN as well, however, penultimate label popping never . For some reason I can’t really fathom, you can’t enable MPLS forwarding on interfaces in IOS without also enabling LDP on those interfaces. On JunOS, doing the same is done by adding the MPLS address family to necessary interfaces, as well as configuring LDP on the same. What does all this mean? It means that on JunOS, if you have BGP running throughout your network (a non-“BGP free core”), you have a more graceful way to implement MPLS L3VPNs with only the VPN labels. I was contemplating using access lists to prevent LDP from distributing or installing any forwarding labels on IOS and see if the same can be accomplished, but I hadn’t had time yet to try it out.

  5. Sorry for the spam, but second sentence should read: “You mentioned they are for the VPN as well, however, penultimate label popping never occurs in that situation anyway.”

  6. Hi Mike. No worry about ‘spam’ – always happy to get comments.

    I’m not quite catching exactly what you’re saying though. Could you explain a bit more in regards to PHP and the VPN label?

    And yes I agree. I’ve never quite liked Cisco’s MPLS implementation on IOS. Brocade and Junos both allow me to have MPLS interfaces without actually running LDP on the same interface. Filtering out LDP just seems like a bit of a hack job :/

  7. Great post. I had a hard time understanding this concept until I figured out the data plane vs control plane while studying. This post helped confirm my hypothesis.
    I wish certification guides would try to explain more the “why” of things vs the “how” of things!

  8. Thanks Darren for your great explanation.

    Could you please explain for me regarding RD: “Note that R4 does not have to match this RD in any way. It simple needs to be able to accept 2 unique routes.”

    Thanks

    Abdel

  9. Thanks sir this is great explanation

    Could you please explain why do we use “address-family”

  10. Netpro000 – VPNv4 routes are a completely different address type to IPv4. BGP has been extended to support multiple address families, with VPNV4 Unicast being one of them.

  11. Could you please explain for me regarding RD: “Note that R4 does not have to match this RD in any way. It simple needs to be able to accept 2 unique routes.”

    Thanks

  12. Hi, Still I am having doubt that why can’t we use RD to find out which VPN it belongs to while receiving the route advertizements at R4 ?

    because RD has to be unique for each VPN, so won’t the Route target look redundant here ?

    (RD of CUS1, at R3, and R4 is same, so at R4, you can extract RD from each advertized route, and then find its mapping it to VPN (CUS1) as its configured..)

  13. The RT won’t be redundant. The RT is to show what VPN a prefix belongs to. The RD is to make a route unique. However, you might want to install a prefix into more than one VPN. There is still a single RD assigned to the route, but multiple RTs

  14. Great Post Darren!

    One question though, Why is a route target import and export needed in the same vrf config. This is confusing quite a bit.

  15. under the heading VPN label, 2nd paragraph, last line; you wrote “When that frame comes in with the VPN label, R6 knows which VRF that packet belongs to.”
    should it be R4 instead of R6? because the VRF itself is in R4

  16. Hi Darren,

    One query , when packets comes to ingress PE how it decides to go to labeled path(MPLS) ?

  17. Hi Rohit.

    When a packet arrives ingress on a PE, that packet will ingress on an interface inside a VRF. Because of this, the lookup table used will be the VRF table, which has the next-hops as labelled paths

  18. Great Explanation…Have few questions though,

    1. What will the receiving PE do with RD ? Why is it getting propagated in NLRI ?
    I know it uses RT to find out the corresponding VRFs in Egress PE ? Whats the use of RD in egress PE ?

    2. You said ” However, you might want to install a prefix into more than one VPN. There is still a single RD assigned to the route, but multiple RTs”, Could you please give me some example where we do this ?

    3. Why can’t we use RD itself as a VPN label ? Why introduce a new field for it ?

    Thanks,

Leave a Reply