CSC, or Carrier Supporting Carrier, takes inter-AS L3 VPN to the next level. Let’s say that you are an ISP and you are offering L3VPN MPLS services to your customers in England. You take over another ISP located in say, Australia, and two of your UK customers are also located in Australia. They would like their offices in both locations connected over the MPLS cloud.
It would be very expensive to run a new line between those locations. You do, however, still want to provide L3VPN services to your customers. The is where CSC comes in. CSC allows another ISP to connect both sides of your ISP network together. It also ensures the the core ISP doesn’t learn about any of your customer prefixes, as it doesn’t need to.
Let’s take the following diagram into consideration. (click the image for full size)
Routers 1, 7, 14, 6, 9, and 8 are all part of ISP2. There are three routers located in each geographical location. Routers 2, 3, 15, and 5 are part of the core carrier. This network stretches to both locations. The rest are customer routers injecting their loopback addresses into OSPF to test.
The core carrier is running IS-IS and LDP. ISP100 is running OSPF and LDP. I won’t go into the regular IGP+LDP config as is pretty straightforward.
With CSC, there are a few new terms to deal with. R14 and R8 are going to be regular PE routers for our ISP. R1 and R6 are going to be called CSC-CE (Customer Supporting Carrier – Customer Edge) routers. R2 and R5 are going to be CSC-PE (Customer Supporting Carrier – Provider Edge) routers. All other ISP routers are simply core routers.
The terminology above assumes that you are speaking in regards to being the core carrier (ISP 500 in this case). That is, ISP500′s edge routers are ‘PE’ and R1 and R6 is the customer’s (ISP) PE routers (Called CE in this case)
It’s a little confusing, depending on which view you take, but it’s really not that difficult.
Initial R14/R8 config – regular PE
The first thing we can start off with is to ensure R14′s PE config is correct. I am running OSPF with the CE routers and learning routes from them:
vrf definition CUS1 rd 14.14.14.14:1 route-target export 100:1 route-target import 100:1 ! address-family ipv4 exit-address-family ! vrf definition CUS2 rd 14.14.14.14:2 route-target export 100:2 route-target import 100:2 ! address-family ipv4 exit-address-family ! interface FastEthernet1/1 vrf forwarding CUS1 ip address 10.14.16.14 255.255.255.0 ip ospf network point-to-point ip ospf 3 area 0 ! interface FastEthernet2/0 vrf forwarding CUS2 ip address 10.14.15.14 255.255.255.0 ip ospf network point-to-point ip ospf 2 area 0
R8 has a similar config on the other side so I won’t put it here.
CSC-PE config
R2 and R5 are going to act as PE routers for the core carrier. They will have their AS200 facing interfaces in a VRF. R2 and R5 will be running regular VPNv4 BGP with each other.
R2:
vrf definition CSC_AS100 rd 2.2.2.2:500 route-target export 500:100 route-target import 500:100 ! address-family ipv4 exit-address-family ! interface FastEthernet1/0 vrf forwarding CSC_AS100 ip address 10.1.2.2 255.255.255.0 ! router bgp 500 no bgp default ipv4-unicast neighbor 5.5.5.5 remote-as 500 neighbor 5.5.5.5 update-source Loopback0 ! address-family vpnv4 neighbor 5.5.5.5 activate neighbor 5.5.5.5 send-community extended exit-address-family
Once again, R5 has a similar config on the other side so I’m not putting it here.
CSC-CE
Eventually we need R14 and R8 to peer with each other via VPNv4.
This is much like option C in the inter-AS config in which two PE routers peer with each other even though they are not directly connected in the same AS. In order to do so we need each of them to have a valid route to each other. As R1 and R6 each have routes to their local PE device, they need to learn routes from the other side through the core carrier. To do this I’ll be running BGP to the core carrier. I also need to ensure that I’m sending and receiving labeled BGP routes and the end LSP has to be end to end. No part of the path can be unlabelled. I need to advertise the PE’s (R14 and R8) loopback are advertised over to the core carrier:
interface FastEthernet1/0 ip address 10.1.2.1 255.255.255.0 mpls bgp forwarding ! router bgp 100 bgp log-neighbor-changes neighbor 10.1.2.2 remote-as 500 ! address-family ipv4 network 14.14.14.14 mask 255.255.255.255 neighbor 10.1.2.2 activate neighbor 10.1.2.2 allowas-in 1 neighbor 10.1.2.2 send-label no auto-summary exit-address-family
I’ll need to allowas-in 1 as both sides are running the same AS number. Without it, the CSC-CE routers would reject the BGP update.
I then need to ensure the CSC-CE routers are redistributing those learned prefixes into the IGP:
router ospf 1 redistribute bgp 100 subnets
R6 again has a similar config.
The end result of it so far is that R8 and R14 should now be able to ping each other from their respective loopbacks:
R8#ping 14.14.14.14 so lo0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 14.14.14.14, timeout is 2 seconds: Packet sent with a source address of 8.8.8.8 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 32/45/56 ms R8#traceroute 14.14.14.14 so lo0 Type escape sequence to abort. Tracing the route to 14.14.14.14 1 10.8.9.9 [MPLS: Label 24 Exp 0] 56 msec 44 msec 32 msec 2 10.6.9.6 [MPLS: Label 24 Exp 0] 60 msec 24 msec 60 msec 3 10.5.6.5 [MPLS: Label 16 Exp 0] 44 msec 44 msec 48 msec 4 10.5.13.13 [MPLS: Labels 20/34 Exp 0] 48 msec 44 msec 44 msec 5 10.3.13.3 [MPLS: Labels 19/34 Exp 0] 44 msec 44 msec 40 msec 6 10.1.2.2 [MPLS: Label 34 Exp 0] 40 msec 44 msec 44 msec 7 10.1.2.1 [MPLS: Label 19 Exp 0] 44 msec 44 msec 28 msec 8 10.1.7.7 [MPLS: Label 18 Exp 0] 56 msec 32 msec 12 msec 9 10.7.14.14 52 msec * 72 msec
Which they can.
PE config – continued
Now that the PE routers have connectivity to each other, we can set up the VPNv4 BGP session:
router bgp 100 no bgp default ipv4-unicast neighbor 8.8.8.8 remote-as 100 neighbor 8.8.8.8 update-source Loopback0 ! address-family vpnv4 neighbor 8.8.8.8 activate neighbor 8.8.8.8 send-community extended exit-address-family
Is the session up?
R14#show bgp vpnv4 unicast all summary BGP router identifier 14.14.14.14, local AS number 100 BGP table version is 1, main routing table version 1 Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 8.8.8.8 4 100 4 4 1 0 0 00:00:50 0
Yes, but no prefixes learnt. We still need to redistribute between our vrf aware OSPF processes and BGP:
router ospf 3 vrf CUS1 redistribute bgp 100 subnets ! router ospf 2 vrf CUS2 redistribute bgp 100 subnets ! router bgp 100 ! address-family ipv4 vrf CUS1 redistribute ospf 3 vrf CUS1 exit-address-family ! address-family ipv4 vrf CUS2 redistribute ospf 2 vrf CUS2 exit-address-family
Verification
As always with these types of configs, we need to ensure both the control and data planes are working correctly. First let’s see the control plane update of R16′s loopback over to R11. The PE router of R14 should be learning this as a vrf prefix:
R14#show ip route vrf CUS1 16.16.16.16
Routing Table: CUS1
Routing entry for 16.16.16.16/32
Known via "ospf 3", distance 110, metric 2, type intra area
Redistributing via bgp 100
Advertised by bgp 100
Last update from 10.14.16.16 on FastEthernet1/1, 00:18:35 ago
Routing Descriptor Blocks:
* 10.14.16.16, from 16.16.16.16, 00:18:35 ago, via FastEthernet1/1
Route metric is 2, traffic share count is 1
This prefix is converted into a VPNv4 prefix and advertised over to R8:
R8#show bgp vpnv4 un rd 14.14.14.14:1 16.16.16.16
BGP routing table entry for 14.14.14.14:1:16.16.16.16/32, version 3
Paths: (1 available, best #1, no table)
Not advertised to any peer
Local
14.14.14.14 (metric 1) from 14.14.14.14 (14.14.14.14)
Origin incomplete, metric 2, localpref 100, valid, internal, best
Extended Community: RT:100:1 OSPF DOMAIN ID:0x0005:0x000000030200
OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:10.14.16.14:0
mpls labels in/out nolabel/27
This should end up in the correct vrf table:
R8#show ip route vrf CUS1 16.16.16.16
Routing Table: CUS1
Routing entry for 16.16.16.16/32
Known via "bgp 100", distance 200, metric 2, type internal
Redistributing via ospf 2
Advertised by ospf 2 subnets
Last update from 14.14.14.14 00:07:03 ago
Routing Descriptor Blocks:
* 14.14.14.14 (default), from 14.14.14.14, 00:07:03 ago
Route metric is 2, traffic share count is 1
AS Hops 0
MPLS label: 27
MPLS Flags: MPLS Required
Finally R11 should receive that as on OSPF route:
R11#show ip route 16.16.16.16
Routing entry for 16.16.16.16/32
Known via "ospf 1", distance 110, metric 2
Tag Complete, Path Length == 1, AS 100, , type extern 2, forward metric 1
Last update from 10.8.11.8 on FastEthernet1/0, 00:06:38 ago
Routing Descriptor Blocks:
* 10.8.11.8, from 10.8.11.8, 00:06:38 ago, via FastEthernet1/0
Route metric is 2, traffic share count is 1
Route tag 3489661028
So our control plane is all good so far. Let’s check our data plane forwarding:
R11#ping 16.16.16.16 so lo0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 16.16.16.16, timeout is 2 seconds: Packet sent with a source address of 11.11.11.11 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 44/57/76 ms R11#traceroute 16.16.16.16 so lo0 Type escape sequence to abort. Tracing the route to 16.16.16.16 1 10.8.11.8 8 msec 12 msec 12 msec 2 10.8.9.9 [MPLS: Labels 24/27 Exp 0] 52 msec 56 msec 48 msec 3 10.6.9.6 [MPLS: Labels 24/27 Exp 0] 56 msec 48 msec 48 msec 4 10.5.6.5 [MPLS: Labels 16/27 Exp 0] 56 msec 40 msec 68 msec 5 10.5.13.13 [MPLS: Labels 20/34/27 Exp 0] 52 msec 44 msec 52 msec 6 10.3.13.3 [MPLS: Labels 19/34/27 Exp 0] 56 msec 48 msec 40 msec 7 10.2.3.2 [MPLS: Labels 34/27 Exp 0] 52 msec 52 msec 40 msec 8 10.1.2.1 [MPLS: Labels 19/27 Exp 0] 44 msec 44 msec 48 msec 9 10.1.7.7 [MPLS: Labels 18/27 Exp 0] 52 msec 40 msec 60 msec 10 10.14.16.14 [MPLS: Label 27 Exp 0] 40 msec 44 msec 36 msec 11 10.14.16.16 44 msec * 52 msec
No problems there :)
Carrier Supporing Carrier Supporting Carrier
So let’s be silly and go a step further. What if our final customer is actually another ISP offering L3VPN to it’s customers? Let’s change our topology slightly(click the image for full size)
I’m not going to show all the config here as it’s simply too much to fit into a blog post. However the config itself is pretty much like so. You’ll need to click the image for the larger version:
R10 and R15 are our final CE routers. Once all is configured does it all actually work?
R10#traceroute 15.15.15.15 so lo0 Type escape sequence to abort. Tracing the route to 15.15.15.15 1 10.10.11.11 12 msec 12 msec 8 msec 2 10.8.11.8 [MPLS: Labels 16/17 Exp 0] 72 msec 48 msec 68 msec 3 10.8.9.9 [MPLS: Labels 22/24/17 Exp 0] 36 msec 80 msec 44 msec 4 10.6.9.6 [MPLS: Labels 20/24/17 Exp 0] 52 msec 56 msec 52 msec 5 10.5.6.5 [MPLS: Labels 23/24/17 Exp 0] 64 msec 52 msec 52 msec 6 10.5.13.13 [MPLS: Labels 20/23/24/17 Exp 0] 48 msec 60 msec 52 msec 7 10.3.13.3 [MPLS: Labels 19/23/24/17 Exp 0] 56 msec 56 msec 48 msec 8 10.2.3.2 [MPLS: Labels 23/24/17 Exp 0] 56 msec 44 msec 56 msec 9 10.1.2.1 [MPLS: Labels 19/24/17 Exp 0] 48 msec 48 msec 64 msec 10 10.1.7.7 [MPLS: Labels 16/24/17 Exp 0] 56 msec 52 msec 60 msec 11 10.7.14.14 [MPLS: Labels 24/17 Exp 0] 44 msec 56 msec 56 msec 12 10.15.16.16 [MPLS: Label 17 Exp 0] 56 msec 56 msec 44 msec 13 10.15.16.15 56 msec * 76 msec
It does indeed. At this point we are up to a four label stack in AS500. If we were running RSVP-TE and FRR we would have even more labels sitting on top.
There is a much easier way to do this of course. The original Customer Carrier could just buy some some or virtual leased line or VPLS from AS500 and they would be directly connected over the same subnet. They could then run MPLS over that link and as far as anyone cares R1 and R6 would be directly connected to each other.
But of course this is a topic on the CCIE SP after all…
I wanted to replicate the VRF using PBR post I did over here. Anyne who has used Junos will tell you that their version of PBR, while more powerful, is a lot more complicated that Cisco’s offering.
Let’s use the following topology:
R3 is going to be my source. It will have multiple interfaces configured. When I send traffic off to R2, I would like R2 to decide which VRF the packet should go into based on the source address used by R3. R2 and R4 are going to be simple ISP PE devices. R1 and R5 are going to be in VRF’s CUS1 and CUS5 respectively.
Configuration
CPE config
All the CPE routers have their loopback configured and have a static route pointing to their connected PE.
All the CPEs are configured simiraily. I’m only going to show the configuration of one.
USER3:R3> show configuration interfaces
ae1 {
unit 12 {
vlan-id 12;
family inet {
address 10.0.4.5/30;
}
}
}
lo0 {
unit 3 {
family inet {
address 3.3.3.3/32;
}
}
}
USER3:R3> show configuration routing-options
static {
route 0.0.0.0/0 next-hop 10.0.4.6;
}
PE Config
R4 is a regular PE so nothing special:
USER4:R4> show configuration
interfaces {
fe-0/0/3 {
unit 24 {
vlan-id 24;
family inet {
address 10.0.4.9/30;
}
family mpls;
}
}
ae1 {
unit 34 {
vlan-id 34;
family inet {
address 10.0.2.5/30;
}
}
unit 45 {
vlan-id 45;
family inet {
address 10.0.8.9/30;
}
}
}
lo0 {
unit 4 {
family inet {
address 4.4.4.4/32;
}
}
}
}
protocols {
mpls {
interface fe-0/0/3.24;
}
bgp {
group L3VPN {
local-address 4.4.4.4;
family inet-vpn {
unicast;
}
peer-as 100;
neighbor 2.2.2.2;
}
}
ospf {
area 0.0.0.0 {
interface fe-0/0/3.24;
interface lo0.4;
}
}
ldp {
interface fe-0/0/3.24;
}
}
routing-instances {
CUS1 {
instance-type vrf;
interface ae1.34;
route-distinguisher 100:1;
vrf-target target:100:1;
routing-options {
static {
route 1.1.1.1/32 next-hop 10.0.2.6;
}
}
}
CUS5 {
instance-type vrf;
interface ae1.45;
route-distinguisher 100:5;
vrf-target target:100:5;
routing-options {
static {
route 5.5.5.5/32 next-hop 10.0.8.10;
}
}
}
}
routing-options {
autonomous-system 100;
}
As you can see, R4 has a static route to R1 and R5 in each respective VRF. That information is then sent off to R2 as an MP-BGP update.
R2 is where all the magic happens. I need to ensure that packets coming into interface fe-0/0/3.12 via R3 is put into various VRFs based on the source address. I also need to ensure that R2 is able to get back to either these addresses over the VRF, even though interface fe-0/0/3.12 is not in a VRF.
First, let’s create a firewall filter that will match and move packets to the right VRF:
USER2:R2> show configuration firewall
family inet {
filter VRF_FBF {
term CUS1 {
from {
address {
192.168.1.1/32;
}
}
then {
routing-instance CUS1;
}
}
term CUS5 {
from {
address {
192.168.5.5/32;
}
}
then {
routing-instance CUS5;
}
}
term ANY {
then accept;
}
}
}
If a packet comes in with a source address of 192.168.1.1/32, send that off to the CUS1 vrf. If it comes in with a source address of 192.168.5.5/32, send that off to the CUS2 vrf. I’ve then got a catch-all to ensure any other packets are not dropped. Once that filter is created. I need to apply it inbound in my interface:
USER2:R2> show configuration interfaces fe-0/0/3.12
vlan-id 12;
family inet {
filter {
input VRF_FBF;
}
address 10.0.4.6/30;
}
This is great for packets coming into R2 from R3, but what about getting back? I could create static routes in each VRF, but the actual link between R2 and R3 is not in any VRF. i.e. the next-hops will not be able to be resolved. I could have a static route pointing to the default/global vrf, but I could also use a rib-group to get that interface into both vrfs.
Let’s try the second option. I wanted to get 10.0.4.4/30 into both VRFs. I also want to ensure only this link gets into the vrf and not all the others.
USER2:R2> show configuration policy-options
policy-statement R2-R3-LINK {
term 1 {
from {
route-filter 10.0.4.4/30 exact;
}
then accept;
}
term 2 {
then reject;
}
}
USER2:R2> show configuration routing-options rib-groups
GLOBAL_TO_VRF {
import-rib [ inet.0 CUS1.inet.0 CUS5.inet.0 ];
import-policy R2-R3-LINK;
}
USER2:R2> show configuration routing-options interface-routes
rib-group inet GLOBAL_TO_VRF;
The above says to match 10.0.4.4/30 and nothing else. That is applied to interface-routes which calls a rib-group. That rib-group states that when the interface route is placed in the default/global RIB, place it in CUS1 and CUS5′s RIB at the same time.
Verification
The end result of this is from from R2′s perspective, in the CUS1 RIB I should see R1′s address, R3′s 192.168.1.1/32 address, the R2-R3 link, and the R4-R1 link:
USER2:R2> show route table CUS1
CUS1.inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
1.1.1.1/32 *[BGP/170] 01:17:21, localpref 100, from 4.4.4.4
AS path: I
> to 10.0.4.9 via ae1.24, Push 299792
10.0.2.4/30 *[BGP/170] 01:17:21, localpref 100, from 4.4.4.4
AS path: I
> to 10.0.4.9 via ae1.24, Push 299792
10.0.4.4/30 *[Direct/0] 00:56:02
> via fe-0/0/3.12
192.168.1.1/32 *[Static/5] 00:56:02
> to 10.0.4.5 via fe-0/0/3.12
If we check CUS5′s RIB, I should see R5′s address, R3′s 192.168.5.5/32 address, the R2-R3 link again, and the R4-R5 link:
USER2:R2> show route table CUS5
CUS5.inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
5.5.5.5/32 *[BGP/170] 01:23:36, localpref 100, from 4.4.4.4
AS path: I
> to 10.0.4.9 via ae1.24, Push 299808
10.0.4.4/30 *[Direct/0] 00:58:51
> via fe-0/0/3.12
10.0.8.8/30 *[BGP/170] 01:23:36, localpref 100, from 4.4.4.4
AS path: I
> to 10.0.4.9 via ae1.24, Push 299808
192.168.5.5/32 *[Static/5] 00:58:51
> to 10.0.4.5 via fe-0/0/3.12
So our control plane is working perfectly fine. Let’s check our data plane for the final verification:
USER3:R3> ping 1.1.1.1 source 192.168.1.1 rapid PING 1.1.1.1 (1.1.1.1): 56 data bytes !!!!! --- 1.1.1.1 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 1.252/1.466/1.823/0.211 ms USER3:R3> ping 1.1.1.1 source 192.168.5.5 rapid PING 1.1.1.1 (1.1.1.1): 56 data bytes ..... --- 1.1.1.1 ping statistics --- 5 packets transmitted, 0 packets received, 100% packet loss
R3 to R1 is working as expected. Let’s check R3 to R5:
USER3:R3> ping 5.5.5.5 source 192.168.5.5 rapid PING 5.5.5.5 (5.5.5.5): 56 data bytes !!!!! --- 5.5.5.5 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 1.262/1.358/1.577/0.115 ms USER3:R3> ping 5.5.5.5 source 192.168.1.1 rapid PING 5.5.5.5 (5.5.5.5): 56 data bytes ..... --- 5.5.5.5 ping statistics --- 5 packets transmitted, 0 packets received, 100% packet loss
No problems there.
VPLS is a LAN emulation service that can be run over an MPLS backbone. I do not have a ton of free devices on hand and so my acual lab will only consist of two CE devices even though it supports more than two. VPLS runs over an MPLS core constructed by RSVP-TE or LDP LSP tunnels. VPLS itself requires either LDP (RFC 4762) or BGP (RFC 4761) as the VC singnalling protocol. Note that when LDP is used, this is a targeted LDP session and has nothing to do with the protocol you use for the LSPs itself. To prove this I will be using RSVP-TE LSP tunnels with LDP and BGP on top.
Part one of this series will use LDP as the VC signalling protocol. Part two will use BGP.
I was originally going to include Cisco’s IOS as well, but you need a 6500 or 7600 and I don’t have a spare. The 7200 platform does not support VPLS.
This the the topology I’m going to use:
R2 and R3 are P routers. R1 is a Junos PE and R8 is a Netiron PE. R6 and R10 are both CPEs
The actual P router config is standard RSVP-TE which I have covered extensively on this site already. The CPE’s have both been configured to be in the same subnet (10.0.0.0/24)
PE Config
Junos
darreno> show configuration interfaces fe-0/0/2
encapsulation ethernet-vpls;
unit 0 {
family vpls;
}
darreno> show configuration protocols ldp
interface lo0.1;
darreno> show configuration routing-instances
MELLOWD-VPLS {
instance-type vpls;
interface fe-0/0/2.0;
protocols {
vpls {
vpls-id 150;
neighbor 8.8.8.8;
}
}
}
On Junos you need to enable LDP on the loopback interface, even when running RSVP. You need to ensure VPLS encapsulation is on the physical interface. Finally you need to create the VPLS instance and tie this all together. You specify neighbours under the process. This actually creates the targeted LDP sessions (i.e. there is no need to specify a T-LDP session separately)
PE Config
Netiron
Brocade’s Netiron config is actually very simple compared to the above
router mpls mpls-interface ve2 vpls MELLOWD-VPLS 150 vpls-peer 1.1.1.1 vpls-mtu 1500 vlan 550 tagged ethe 3/19
That’s it. When you enable rsvp on an interface, and then set up a VPLS with a neighbour, it automatically sets up a T-LDP session with it’s peer. Under the VLAN I’ve said tagged eth 3/19 as it’ll be receiving tagged frames from the CPE router.
Verification
Control Plane
Let’s check to see if the session is actually up:
darreno> show vpls connections
Layer-2 VPN connections:
Legend for connection status (St)
[deleted for brevity]
Legend for interface status
Up -- operational
Dn -- down
Instance: MELLOWD-VPLS
VPLS-id: 150
Neighbor Type St Time last up # Up trans
8.8.8.8(vpls-id 150) rmt Up Mar 8 13:52:45 2013 1
Remote PE: 8.8.8.8, Negotiated control-word: No
Incoming label: 800000, Outgoing label: 983040
Negotiated PW status TLV: No
Local interface: vt-0/2/0.1048579, Status: Up, Encapsulation: ETHERNET
Description: Intf - vpls MELLOWD-VPLS neighbor 8.8.8.8 vpls-id 150
SSH@XMR_R8# show mpls vpls id 150 VPLS MELLOWD-VPLS, Id 150, Max mac entries: 8192 Routing Interface Id 150 Total vlans: 1, Tagged ports: 1 (1 Up), Untagged ports 0 (0 Up) IFL-ID: n/a Vlan 550 L2 Protocol: NONE Tagged: ethe 3/19 VC-Mode: Raw Total VPLS peers: 1 (1 Operational) Peer address: 1.1.1.1, State: Operational, Uptime: 55 min Tnnl in use: tnl2(299984)[RSVP] Peer Index:0 Local VC lbl: 983040, Remote VC lbl: 800000 Local VC MTU: 1500, Remote VC MTU: 1500 Local VC-Type: Ethernet(0x05), Remote VC-Type: Ethernet(0x05) CPU-Protection: OFF Local Switching: Enabled Extended Counter: ON Multicast Snooping: Disabled
Both sessions are up. Both see the others VC labels. Can we show that LDP is actually used?
darreno> show ldp neighbor Address Interface Label space ID Hold time 8.8.8.8 lo0.1 8.8.8.8:0 36
SSH@XMR_R8#sh mpls ldp neighbor Number of link neighbors: 0 Number of targeted neighbors: 1 Nbr Transport Interface Nbr LDP ID Max Hold Time Left 1.1.1.1 (targeted) 1.1.1.1:0 45 42
Verification
Data Plane
So can our CPE’s ping each others?
R10#ping 10.0.0.6 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.0.0.6, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/12 ms
USER6:R6> ping 10.0.0.10 rapid count 5 PING 10.0.0.10 (10.0.0.10): 56 data bytes !!!!! --- 10.0.0.10 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 1.368/1.460/1.755/0.148 ms
No problems there. It’s always handy to check MAC addresses learned over the VPLS. We can check like so:
darreno> show route forwarding-table family vpls
Routing table: MELLOWD-VPLS.vpls
VPLS:
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 1007 1
fe-0/0/2.0 user 0 comp 1150 2
vt-0/2/0.1048579 user 0 comp 1258 2
00:12:f2:93:a9:00/48 dynm 0 indr 262142 5
10.0.4.14 Push 983040, Push 299920(top) 1233 2 ae1.13
00:13:19:22:8f:91/48 dynm 0 ucst 1146 4 fe-0/0/2.0
00:90:69:a5:13:f1/48 dynm 0 ucst 1146 4 fe-0/0/2.0
8c:b6:4f:63:46:b8/48 dynm 0 indr 262142 5
10.0.4.14 Push 983040, Push 299920(top) 1233 2 ae1.13
SSH@XMR_R8#sh mac vpls 150 Total MAC entries for VPLS 150: 4 (Local: 1, Remote: 3) VPLS MAC Address L/R/IB Port Vlan(In-Tag)/Peer ISID Age Type ==== =========== ====== ==== ================= ==== === ==== 150 8cb6.4f63.46b8 L 3/19 550 NA 70 NA 150 0a0a.0009.0004 R 3/7 1.1.1.1 NA 250 NA 150 0013.1922.8f91 R 3/7 1.1.1.1 NA 0 NA 150 0090.69a5.13f1 R 3/7 1.1.1.1 NA 70 NA
Both outputs show local and remote locations of MAC addresses. Both also show the neighbour ID of who has the directly connected MAC address.
Management over the VPLS
A handy new feature on the Netiron is the ability to have a layer3 interface over the VPLS. This can be handy when you need to manage the CPE devices. While in the past you may need to have a ‘break-in’ interface also attached to the VPLS, you can now do it directly on the Netiron.
interface ve 150 ip address 10.0.0.8/24 ! router mpls vpls MELLOWD-VPLS 150 router-interface ve 150
This essentially works like in SVI interface on a vlan. Let’s check if we have communication:
SSH@XMR_R8#ping 10.0.0.10 Sending 1, 16-byte ICMP Echo to 10.0.0.10, timeout 5000 msec, TTL 64 Type Control-c to abort Reply from 10.0.0.10 : bytes=16 time=1ms TTL=255 Success rate is 100 percent (1/1), round-trip min/avg/max=1/1/1 ms. SSH@XMR_R8#ping 10.0.0.6 Sending 1, 16-byte ICMP Echo to 10.0.0.6, timeout 5000 msec, TTL 64 Type Control-c to abort Reply from 10.0.0.6 : bytes=16 time=1ms TTL=64 Success rate is 100 percent (1/1), round-trip min/avg/max=1/1/1 ms.
This is part two of my blog started here: http://mellowd.co.uk/ccie/?p=3300
Same diagram as last time:
This time each CPE is going to be connected to a VRF on the PE router. I’m only using one customer for this post, but this is regular L3VPN so scale as you see fit.
Major issue with the Netiron. It doesn’t support the VPNV6 adress family :( – I’m using the latest 5.4b code and nothing. So this means this is a Junos/IOS lab only
CPE config
All the CPEs are running BGP with their directly connected PE routers. All are advertising reachability to their IPv6 loopback addresses to their PE router. I’m only showing R6′s config as the others are the same with different addresses:
interfaces {
ae1 {
unit 36 {
vlan-id 36;
family inet6 {
address 2001:db8:36::6/64;
}
}
lo0 {
unit 6 {
family inet6 {
address 2001:db8:6666::6666/128;
}
}
}
}
protocols {
bgp {
group PROVIDER {
family inet6 {
unicast;
}
export LOOPBACK;
neighbor 2001:db8:36::3 {
peer-as 100;
}
}
}
}
policy-options {
policy-statement LOOPBACK {
from {
protocol direct;
route-filter 2001:db8:6666::6666/128 exact;
}
then accept;
}
}
routing-options {
router-id 6.6.6.6;
autonomous-system 65123 loops 2;
}
You’ll need to statically define your router-id for all sites. If a router is running ONLY IPv6, or your VRF ONLY has a IPv6 address, then the router has no IPv4 address to choose it’s router-id from. This will be a common theme throughout as you’ll also need to set router-ids in IPv6-only VRF instances.
PE config
Junos
First we need to set up the VRF to the customer and run BGP. We then need to enable the VPNV6 family in BGP. I’m going to remove the old IPv6 unicast config used in part one of this series.
USER3:R3> show configuration protocols
mpls {
ipv6-tunneling;
interface ae1.13;
}
bgp {
group 6VPE {
family inet6-vpn {
unicast;
}
peer-as 100;
neighbor 4.4.4.4;
}
}
USER3:R3> show configuration routing-instances
CUSTOMER1 {
instance-type vrf;
interface fe-0/0/3.36;
route-distinguisher 3.3.3.3:1;
vrf-target target:100:1;
routing-options {
router-id 3.3.3.3;
}
protocols {
bgp {
group EXTERNAL {
advertise-peer-as;
family inet6 {
unicast;
}
neighbor 2001:db8:36::6 {
peer-as 65123;
}
}
}
}
}
IPv6 address family running with the customer. VPNv6 address family running with IOS PE R4. Note that I have to use ‘advertise-peer-as’ on R3 as Junos will not advertise a route to an AS that already has the AS number in the path by default.
IOS
The main issue with IOS is that I cannot statically definate a BGP router-id if I’m ONLY running IPv6. BGP requires a router-id on the x.x.x.x format. IOS does not give me the option to hard-code a router-id under the BGP process for the VRF, or the ipv6 unicast address family. So I had to enable the ipv4 address-family under the VRF and define a loopback address in the VRF to use as the router-id. Very silly indeed.
vrf definition CUSTOMER1 rd 4.4.4.4:100 ! address-family ipv4 exit-address-family ! address-family ipv6 route-target export 100:1 route-target import 100:1 exit-address-family ! interface Loopback4 vrf forwarding CUSTOMER1 ip address 4.4.4.4 255.255.255.255 ! router bgp 100 bgp router-id vrf auto-assign no bgp default ipv4-unicast bgp log-neighbor-changes neighbor 3.3.3.3 remote-as 100 neighbor 3.3.3.3 update-source Loopback0 ! address-family vpnv6 neighbor 3.3.3.3 activate neighbor 3.3.3.3 send-community extended exit-address-family ! address-family ipv6 vrf CUSTOMER1 no synchronization neighbor 2001:DB8:47::7 remote-as 65123 neighbor 2001:DB8:47::7 activate exit-address-family
VRF assigned to the CE-PE link. IPv6 unicast running with the CPE and VPNv6 running with the Junos PE R3 router.
Verification
Let’s first check if our VPNv6 sessions are up:
7200_SRD_R4#show bgp vpnv6 unicast all neighbors 3.3.3.3 | include state|fam$
BGP state = Established, up for 03:09:47
Address family VPNv6 Unicast: advertised and received
For address family: VPNv6 Unicast
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
USER3:R3> show bgp neighbor 4.4.4.4 | match "Estab|NLRI" Type: Internal State: Established Flags:NLRI for restart configured on peer: inet6-vpn-unicast NLRI advertised by peer: inet6-vpn-unicast NLRI for this session: inet6-vpn-unicast
Sessions are up and running the VPNv6 family.
Can the CE’s ping each other from their IPv6 loopbacks?
USER7:R7> ping 2001:db8:6666::6666 source 2001:db8:7777::7777 rapid count 5 PING6(56=40+8+8 bytes) 2001:db8:7777::7777 --> 2001:db8:6666::6666 !!!!! --- 2001:db8:6666::6666 ping6 statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/std-dev = 1.520/1.726/1.997/0.195 ms
USER6:R6> ping 2001:db8:7777::7777 source 2001:db8:6666::6666 rapid count 5 PING6(56=40+8+8 bytes) 2001:db8:6666::6666 --> 2001:db8:7777::7777 !!!!! --- 2001:db8:7777::7777 ping6 statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/std-dev = 1.533/1.706/1.968/0.147 ms
No problems there :)
-
Part 1 – Cisco IOS
Part 2 – Juniper JunOS
Part 3 – Brocade Netiron XMR/MLX
Part 4 – Cisco IOS-XR
Initially this was going to be a three part series. I finally managed to get a little lab time on an XR platform so here we go.
The version I am using is:
Cisco IOS XR Software, Version 3.9.1[00]
The issue is that I only have two XR devices. Therefore my P routers will be regular IOS and my PE routers will be XR. This is because most of the config is done on the PE routers.
Same topology as last time. Substitute AR for XR:
I won’t go in depth on the P routers again, just to note that you need mpls traffic-eng tunnels configured globally, on the core interfaces, and TE enabled on OSPF/IS-IS.
If the P routers were IOS-XR, then you would need to configure them like so:
interface Loopback0 ipv4 address 11.11.11.11 255.255.255.255 ! router ospf LAB address-family ipv4 unicast area 0 mpls traffic-eng mpls traffic-eng router-id Loopback0 ! rsvp interface GigabitEthernetx/x/x/x ! interface GigabitEthernetx/x/x/x ! ! mpls traffic-eng interface GigabitEthernetx/x/x/x ! interface GigabitEthernetx/x/x/x
IOS-XR basic config
explicit-path name TO_XR2 index 1 next-address strict ipv4 unicast 10.0.111.1 index 2 next-address strict ipv4 unicast 10.0.112.2 ! interface Loopback0 ipv4 address 11.11.11.11 255.255.255.255 ! interface Loopback111 ipv4 address 111.111.111.111 255.255.255.255 ! interface tunnel-te0 ipv4 unnumbered Loopback0 destination 22.22.22.22 path-option 5 explicit name TO_XR2 ! interface GigabitEthernet0/1/0/1.11 ipv4 address 10.0.111.11 255.255.255.0 dot1q vlan 11 ! interface GigabitEthernet0/1/0/1.122 ipv4 address 10.0.122.12 255.255.255.0 dot1q vlan 122 ! router ospf LAB address-family ipv4 unicast area 0 mpls traffic-eng interface Loopback0 ! interface GigabitEthernet0/1/0/1.11 ! interface GigabitEthernet0/1/0/1.122 ! ! mpls traffic-eng router-id Loopback0 ! router bgp 12 bgp router-id 11.11.11.11 address-family ipv4 unicast network 111.111.111.111/32 ! neighbor 22.22.22.22 remote-as 12 update-source Loopback0 address-family ipv4 unicast ! ! ! rsvp interface GigabitEthernet0/1/0/1.11 ! interface GigabitEthernet0/1/0/1.122 ! ! mpls traffic-eng interface GigabitEthernet0/1/0/1.11 ! interface GigabitEthernet0/1/0/1.122 ! ! end
XR2 has a very similar config of course. I have enabled both RSVP and MPLS-TE on all the needed interfaces. I have enabled TE under OSPF. You’ll notice that XR has a specific TE tunnel type and so there is no need to specify that it is a TE tunnel. For testing purposes I’ve created an iBGP session with a second loopback so the PE devices can ping over the MPLS core.
Have we learned anything over the BGP session?
RP/0/0/CPU0:XR1#show route ipv4 bgp Sat Mar 2 11:47:18.626 UTC B 222.222.222.222/32 [200/0] via 22.22.22.22, 00:02:15
Can we ping accross?
RP/0/0/CPU0:XR1#ping 222.222.222.222 Sat Mar 2 11:47:55.854 UTC Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 222.222.222.222, timeout is 2 seconds: U.U.U Success rate is 0 percent (0/5)
No. Like IOS, XR will not use the TE tunnel for anything unless you tell it to do so. We could have a static route, but I’ve just added it to the IGP like so:
interface tunnel-te0 autoroute announce
We should now be able to ping:
RP/0/0/CPU0:XR1#ping 222.222.222.222 Sat Mar 2 12:46:44.598 UTC Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 222.222.222.222, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
Let’s look deeper at what labels will be used:
RP/0/0/CPU0:XR1# show ip route 22.22.22.22
Sat Mar 2 11:54:59.982 UTC
Routing entry for 22.22.22.22/32
Known via "ospf LAB", distance 110, metric 3, type intra area
Installed Mar 2 11:49:25.841 for 00:05:34
Routing Descriptor Blocks
22.22.22.22, from 22.22.22.22, via tunnel-te0
Route metric is 3
No advertising protos.
RP/0/0/CPU0:XR1#show mpls forwarding tunnels detail
Sat Mar 2 11:55:19.496 UTC
Tunnel Outgoing Outgoing Next Hop Bytes
Name Label Interface Switched
-------- ----------- ------------ --------------- ------------
tt0 16 Gi0/1/0/1.11 10.0.111.1 1610
Updated Mar 2 11:17:49.661
MAC/Encaps: 18/22, MTU: 1500
Label Stack (Top -> Bottom): { 16 }
Local Label: N/A
Packets Switched: 22
To get to 222.222.222.22, we use the BGP route which has a next-hop of 22.22.22.22. To get there we’ll now use the TE tunnel. When we go over the tunnel, XR1 will impose a label of 16. R1 being the PHP router will pop that label off so the second hop would be XR2:
RP/0/0/CPU0:XR1#traceroute 222.222.222.222 Sat Mar 2 11:55:38.841 UTC Type escape sequence to abort. Tracing the route to 222.222.222.222 1 10.0.111.1 [MPLS: Label 16 Exp 0] 4 msec 5 msec 3 msec 2 10.0.112.2 5 msec * 4 msec
IOS-XR explicit paths
I actually creted an explicit path above already, but let’s force XR2 to go through R3 and R2:
explicit-path name PRI_TO_XR1 index 1 next-address strict ipv4 unicast 10.0.123.3 index 2 next-address strict ipv4 unicast 10.0.23.2 index 3 next-address strict ipv4 unicast 10.0.122.12 ! interface tunnel-te0 path-option 2 explicit name PRI_TO_XR1
Is the tunnel up?
RP/0/3/CPU0:XR2#sh mpls traffic-eng tun brief
Sat Mar 2 12:03:23.648 UTC
Signalling Summary:
LSP Tunnels Process: running
RSVP Process: running
Forwarding: enabled
Periodic reoptimization: every 3600 seconds, next in 3283 seconds
Periodic FRR Promotion: every 300 seconds, next in 279 seconds
Auto-bw enabled tunnels: 0 (disabled)
TUNNEL NAME DESTINATION STATUS STATE
tunnel-te0 11.11.11.11 up up
XR1_t0 22.22.22.22 up up
Displayed 1 (of 1) heads, 0 (of 0) midpoints, 1 (of 1) tails
Displayed 1 up, 0 down, 0 recovering, 0 recovered heads
We should now see three hops going back:
RP/0/3/CPU0:XR2#traceroute 111.111.111.111 Sat Mar 2 12:03:43.107 UTC Type escape sequence to abort. Tracing the route to 111.111.111.111 1 10.0.123.3 [MPLS: Label 16 Exp 0] 4 msec 3 msec 1 msec 2 10.0.23.2 [MPLS: Label 16 Exp 0] 2 msec 3 msec 2 msec 3 10.0.122.12 5 msec * 4 msec
I’ve not bothered to put the type-10 LSAs here as they are the same as IOS.
Comments