A big advantage of using a single IGP area is the ability to use RSVP-TE. RSVP-TE uses type 10 OSPF LSAs which have an area-wide scope. This would mean that in order to have an end-to-end RSVP-TE singnalled LSP you would need all your routers in the same area.
This is not 100% true.
I’m a firm believer in a single area in an ISP network. However there may come a time when you simple have to have an LSP span multiple areas. I’ll show how this is easily done on Junos and IOS.
RFC 4105 has all the technical details about requirements for this.
Let’s configure the following:
JR1 and JR2 are Junos routers of course.
Config
Cisco P routers
Noting too complicated here. All you need to do is ensure that MPLS traffic-eng is enabled globally and on the transit interfaces. You also need to ensure the correct areas are enabled for TE in OSPF. This is R2′s relevant config:
mpls traffic-eng tunnels ! interface Loopback0 ip address 2.2.2.2 255.255.255.255 ip ospf 1 area 0 ! interface FastEthernet0/0 ip address 10.0.23.2 255.255.255.0 ip ospf 1 area 0 mpls traffic-eng tunnels ! interface FastEthernet1/0 ip address 10.0.12.2 255.255.255.0 ip ospf 1 area 12 mpls traffic-eng tunnels ! router ospf 1 mpls traffic-eng router-id Loopback0 mpls traffic-eng area 0 mpls traffic-eng area 12
Juniper P routers
With Junos you need to enable MPLS processing on the transit interfaces, as well as enable RSVP, OSPF, and MPLS on the transit interfaces under the protocols stanza. This is JR1:
darreno@JR1> show configuration interfaces ge-0/0/1.504
vlan-id 504;
family inet {
address 10.0.16.1/24;
}
family mpls;
darreno@JR1> show configuration interfaces ge-0/0/1.503
vlan-id 503;
family inet {
address 10.0.21.1/24;
}
family mpls;
darreno@JR1> show configuration protocols
rsvp {
interface ge-0/0/1.503;
interface ge-0/0/1.504;
}
mpls {
expand-loose-hop;
interface ge-0/0/1.503;
interface ge-0/0/1.504;
}
ospf {
traffic-engineering;
area 0.0.0.124 {
interface ge-0/0/1.503;
interface ge-0/0/1.504;
interface lo0.11;
}
}
For inter-AS TE to work on Junos, you must configure expand-loose-hop under the MPLS stanza on your P routers.
Cisco PE routers
With Inter-AS TE, you need to ensure that your loose-hops point to devices that has information in the next area. This means ABRs. I’ll set up an explicit path that gets to R2, then R4, then off to JR2. That path will then be referred to in the LSP set up:
ip explicit-path name TO-JR2 enable next-address loose 2.2.2.2 next-address loose 4.4.4.4 next-address loose 200.200.200.200 ! interface Tunnel200 ip unnumbered Loopback0 tunnel mode mpls traffic-eng tunnel destination 200.200.200.200 tunnel mpls traffic-eng autoroute destination tunnel mpls traffic-eng path-option 5 explicit name TO-JR2
tunnel mpls traffic-eng autoroute destination adds a static route to 200.200.200.200 via the tunnel once it’s up
Juniper PE routers
Junos is almost the same. One difference being that you have to state to the OS that this LSP will traverse multiple areas:
darreno@JR2> show configuration protocols mpls
label-switched-path TO-R1 {
from 200.200.200.200;
to 1.1.1.1;
inter-domain;
primary TO-R1;
}
path TO-R1 {
4.4.4.4 loose;
2.2.2.2 loose;
1.1.1.1 loose;
}
interface ge-0/0/1.503;
Note the inter-domain config under the LSP config
Verification
Let’s check the LSP from R1 to JR2:
R1#sh mpls traffic-eng tunnels tunnel 200
Name: R1_t200 (Tunnel200) Destination: 200.200.200.200
Status:
Admin: up Oper: up Path: valid Signalling: connected
path option 5, type explicit TO-JR2 (Basis for Setup, path weight 1)
Config Parameters:
Bandwidth: 0 kbps (Global) Priority: 7 7 Affinity: 0x0/0xFFFF
Metric Type: TE (default)
AutoRoute announce: enabled LockDown: disabled Loadshare: 0 bw-based
auto-bw: disabled
Active Path Option Parameters:
State: explicit path option 5 is active
BandwidthOverride: disabled LockDown: disabled Verbatim: disabled
InLabel : -
OutLabel : FastEthernet0/0, 16
Next Hop : 10.0.12.2
RSVP Signalling Info:
Src 1.1.1.1, Dst 200.200.200.200, Tun_Id 200, Tun_Instance 7
RSVP Path Info:
My Address: 10.0.12.1
Explicit Route: 10.0.12.2 2.2.2.2 4.4.4.4* 200.200.200.200*
Record Route:
Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
RSVP Resv Info:
Record Route: 10.0.23.2 10.0.34.3 10.0.16.6 10.0.16.1
10.0.21.2
Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
Shortest Unconstrained Path Info:
Path Weight: UNKNOWN
Explicit Route: UNKNOWN
History:
Tunnel:
Time since created: 12 minutes, 57 seconds
Time since path change: 12 minutes, 12 seconds
Number of LSP IDs (Tun_Instances) used: 7
Current LSP: [ID: 7]
Uptime: 12 minutes, 12 seconds
All looks good. Let’s run a traceroute to JR2:
R1#traceroute 200.200.200.200 Type escape sequence to abort. Tracing the route to 200.200.200.200 1 10.0.12.2 [MPLS: Label 16 Exp 0] 52 msec 8 msec 16 msec 2 10.0.23.3 [MPLS: Label 16 Exp 0] 20 msec 8 msec 16 msec 3 10.0.34.4 [MPLS: Label 16 Exp 0] 12 msec 8 msec 12 msec 4 10.0.16.1 [MPLS: Label 299920 Exp 0] 12 msec 12 msec 24 msec 5 200.200.200.200 16 msec 24 msec 44 msec
Looks good from there. Let’s check JR2:
darreno@JR2> show mpls lsp ingress detail
Ingress LSP: 1 sessions
1.1.1.1
From: 200.200.200.200, State: Up, ActiveRoute: 0, LSPname: TO-R1
ActivePath: TO-R1 (primary)
PathDomain: Inter-domain
LSPtype: Static Configured
LoadBalance: Random
Encoding type: Packet, Switching type: Packet, GPID: IPv4
*Primary TO-R1 State: Up
Priorities: 7 0
SmartOptimizeTimer: 180
Computed ERO (S [L] denotes strict [loose] hops): (CSPF metric: 2)
10.0.21.1 S 10.0.16.6 S 2.2.2.2 L 1.1.1.1 L
Received RRO (ProtectionFlag 1=Available 2=InUse 4=B/W 8=Node 10=SoftPreempt 20=Node-ID):
10.0.21.1 10.0.34.4 10.0.23.3 10.0.12.2 10.0.12.1
Total 1 displayed, Up 1, Down 0
In Junos, Labelled end point are only used for BGP hops by default, so a traceroute won’t show it going through as labelled traffic. However a show route does show that 1.1.1.1 comes up as an RSVP route which will be used for BGP next-hops:
darreno@JR2> show route 1.1.1.1
inet.0: 22 destinations, 22 routes (22 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
1.1.1.1/32 *[OSPF/10] 00:17:23, metric 6
> to 10.0.21.1 via ge-0/0/1.503
inet.3: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
1.1.1.1/32 *[RSVP/7/1] 00:17:02, metric 6
> to 10.0.21.1 via ge-0/0/1.503, label-switched-path TO-R1
Notes
This also works with IS-IS. Instead of loose hopping to ABR routers, you simply loose hop to your L1/L2 boundary router.
Oddly enough, neither platform shows you directly what your MSS between your BGP speakers is. Quite annoying. To quickly check on Junos do this:
darren@pgr1.ixe1> show system connections extensive | match "179|mss"
tcp4 0 0 10.250.8.9.59064 10.0.1.89.179
rttmin: 1000 mss: 512
tcp4 0 0 10.0.0.1.65081 10.1.0.1.179
rttmin: 1000 mss: 492
tcp4 0 0 10.10.0.1.56298 10.20.0.1.179
rttmin: 1000 mss: 1440
For IOS its quite similar:
par8.the1#show tcp | include .179|Datagrams Foreign host: 10.20.30.1, Foreign port: 179 Datagrams (max data segment is 556 bytes): Foreign host: 10.30.20.1, Foreign port: 179 Datagrams (max data segment is 8860 bytes):
Would be nice of this was show in the actual BGP neighbourship output, but BGP does ride on top of TCP so…
EDIT: Actually IOS does show this in a neighbour command:
R2#show bgp vpnv4 unicast all neighbors 19.19.19.19 | include Data Datagrams (max data segment is 1460 bytes):
I had a couple of requests to post the .net and config files for my Carrier Supporting Carrier post.
So there it is. Click here to download.
Note that I’ve taken these configs from the end of that post, so it’s actually the CSCSC configs. There are a couple of ‘extra’ config statements in the configs that aren’t doing anything. I just didn’t feel like cleaning it up.
You might need to adjust a few things here and there in the net (mainly the config location) to get things to work
I’ve seen a fair amount of debate where people are saying Cisco ‘stole’ the commit command from Juniper. Perhaps. But Juniper wasn’t the first to have a command available for staging configs before making them active.
Let’s look at this ancient (but still live) Riverstone RS3000:
RS135# conf RS135(config)# vlan create vlan26 ip id 26 RS135(config)# vlan add ports et.2.6 to vlan26 RS135(config)# interface create ip ip26 address-netmask 10.26.0.1/30 vlan vlan26
None of this is live. The config is merely staged. In order to check what will happen when we commit we check our ‘scratchpad’
RS135(config)# show scratchpad
******** Non-committed changes in Scratchpad ********
1*: vlan create vlan26 ip id 26
2*: vlan add ports et.2.6 to vlan26
!
3*: interface create ip ip26 address-netmask 10.26.0.1/30 vlan vlan26
To commit those changes:
RS135(config)# save active %VLAN-I-CREATED, VLAN vlan26 created with VLAN ID 26. To add ports to the VLAN, use the "vlan add ports" command. %INTERFACE-I-CREATEDIF, Interface ip26 was successfully created.
One thing the Riverstone doesn’t do is save those changes to the startup config. That is a separate step:
RS135(config)# save startup Are you sure you want to overwrite the Startup configuration [no]? yes %CONFIG-I-SAVED, configuration saved to Startup configuration.
The Riverstone also had some handy ‘diff’ commands which kind of worked like a show | compare in Junos.
Alas Riverstone Networks ceased to exist in 2006. They were bought out by Alcatel Lucent so are now under their umbrella.
CSC, or Carrier Supporting Carrier, takes inter-AS L3 VPN to the next level. Let’s say that you are an ISP and you are offering L3VPN MPLS services to your customers in England. You take over another ISP located in say, Australia, and two of your UK customers are also located in Australia. They would like their offices in both locations connected over the MPLS cloud.
It would be very expensive to run a new line between those locations. You do, however, still want to provide L3VPN services to your customers. The is where CSC comes in. CSC allows another ISP to connect both sides of your ISP network together. It also ensures the the core ISP doesn’t learn about any of your customer prefixes, as it doesn’t need to.
Let’s take the following diagram into consideration. (click the image for full size)
Routers 1, 7, 14, 6, 9, and 8 are all part of ISP2. There are three routers located in each geographical location. Routers 2, 3, 15, and 5 are part of the core carrier. This network stretches to both locations. The rest are customer routers injecting their loopback addresses into OSPF to test.
The core carrier is running IS-IS and LDP. ISP100 is running OSPF and LDP. I won’t go into the regular IGP+LDP config as is pretty straightforward.
With CSC, there are a few new terms to deal with. R14 and R8 are going to be regular PE routers for our ISP. R1 and R6 are going to be called CSC-CE (Customer Supporting Carrier – Customer Edge) routers. R2 and R5 are going to be CSC-PE (Customer Supporting Carrier – Provider Edge) routers. All other ISP routers are simply core routers.
The terminology above assumes that you are speaking in regards to being the core carrier (ISP 500 in this case). That is, ISP500′s edge routers are ‘PE’ and R1 and R6 is the customer’s (ISP) PE routers (Called CE in this case)
It’s a little confusing, depending on which view you take, but it’s really not that difficult.
Initial R14/R8 config – regular PE
The first thing we can start off with is to ensure R14′s PE config is correct. I am running OSPF with the CE routers and learning routes from them:
vrf definition CUS1 rd 14.14.14.14:1 route-target export 100:1 route-target import 100:1 ! address-family ipv4 exit-address-family ! vrf definition CUS2 rd 14.14.14.14:2 route-target export 100:2 route-target import 100:2 ! address-family ipv4 exit-address-family ! interface FastEthernet1/1 vrf forwarding CUS1 ip address 10.14.16.14 255.255.255.0 ip ospf network point-to-point ip ospf 3 area 0 ! interface FastEthernet2/0 vrf forwarding CUS2 ip address 10.14.15.14 255.255.255.0 ip ospf network point-to-point ip ospf 2 area 0
R8 has a similar config on the other side so I won’t put it here.
CSC-PE config
R2 and R5 are going to act as PE routers for the core carrier. They will have their AS200 facing interfaces in a VRF. R2 and R5 will be running regular VPNv4 BGP with each other.
R2:
vrf definition CSC_AS100 rd 2.2.2.2:500 route-target export 500:100 route-target import 500:100 ! address-family ipv4 exit-address-family ! interface FastEthernet1/0 vrf forwarding CSC_AS100 ip address 10.1.2.2 255.255.255.0 ! router bgp 500 no bgp default ipv4-unicast neighbor 5.5.5.5 remote-as 500 neighbor 5.5.5.5 update-source Loopback0 ! address-family vpnv4 neighbor 5.5.5.5 activate neighbor 5.5.5.5 send-community extended exit-address-family
Once again, R5 has a similar config on the other side so I’m not putting it here.
CSC-CE
Eventually we need R14 and R8 to peer with each other via VPNv4.
This is much like option C in the inter-AS config in which two PE routers peer with each other even though they are not directly connected in the same AS. In order to do so we need each of them to have a valid route to each other. As R1 and R6 each have routes to their local PE device, they need to learn routes from the other side through the core carrier. To do this I’ll be running BGP to the core carrier. I also need to ensure that I’m sending and receiving labeled BGP routes and the end LSP has to be end to end. No part of the path can be unlabelled. I need to advertise the PE’s (R14 and R8) loopback are advertised over to the core carrier:
interface FastEthernet1/0 ip address 10.1.2.1 255.255.255.0 mpls bgp forwarding ! router bgp 100 bgp log-neighbor-changes neighbor 10.1.2.2 remote-as 500 ! address-family ipv4 network 14.14.14.14 mask 255.255.255.255 neighbor 10.1.2.2 activate neighbor 10.1.2.2 allowas-in 1 neighbor 10.1.2.2 send-label no auto-summary exit-address-family
I’ll need to allowas-in 1 as both sides are running the same AS number. Without it, the CSC-CE routers would reject the BGP update.
I then need to ensure the CSC-CE routers are redistributing those learned prefixes into the IGP:
router ospf 1 redistribute bgp 100 subnets
R6 again has a similar config.
The end result of it so far is that R8 and R14 should now be able to ping each other from their respective loopbacks:
R8#ping 14.14.14.14 so lo0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 14.14.14.14, timeout is 2 seconds: Packet sent with a source address of 8.8.8.8 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 32/45/56 ms R8#traceroute 14.14.14.14 so lo0 Type escape sequence to abort. Tracing the route to 14.14.14.14 1 10.8.9.9 [MPLS: Label 24 Exp 0] 56 msec 44 msec 32 msec 2 10.6.9.6 [MPLS: Label 24 Exp 0] 60 msec 24 msec 60 msec 3 10.5.6.5 [MPLS: Label 16 Exp 0] 44 msec 44 msec 48 msec 4 10.5.13.13 [MPLS: Labels 20/34 Exp 0] 48 msec 44 msec 44 msec 5 10.3.13.3 [MPLS: Labels 19/34 Exp 0] 44 msec 44 msec 40 msec 6 10.1.2.2 [MPLS: Label 34 Exp 0] 40 msec 44 msec 44 msec 7 10.1.2.1 [MPLS: Label 19 Exp 0] 44 msec 44 msec 28 msec 8 10.1.7.7 [MPLS: Label 18 Exp 0] 56 msec 32 msec 12 msec 9 10.7.14.14 52 msec * 72 msec
Which they can.
PE config – continued
Now that the PE routers have connectivity to each other, we can set up the VPNv4 BGP session:
router bgp 100 no bgp default ipv4-unicast neighbor 8.8.8.8 remote-as 100 neighbor 8.8.8.8 update-source Loopback0 ! address-family vpnv4 neighbor 8.8.8.8 activate neighbor 8.8.8.8 send-community extended exit-address-family
Is the session up?
R14#show bgp vpnv4 unicast all summary BGP router identifier 14.14.14.14, local AS number 100 BGP table version is 1, main routing table version 1 Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 8.8.8.8 4 100 4 4 1 0 0 00:00:50 0
Yes, but no prefixes learnt. We still need to redistribute between our vrf aware OSPF processes and BGP:
router ospf 3 vrf CUS1 redistribute bgp 100 subnets ! router ospf 2 vrf CUS2 redistribute bgp 100 subnets ! router bgp 100 ! address-family ipv4 vrf CUS1 redistribute ospf 3 vrf CUS1 exit-address-family ! address-family ipv4 vrf CUS2 redistribute ospf 2 vrf CUS2 exit-address-family
Verification
As always with these types of configs, we need to ensure both the control and data planes are working correctly. First let’s see the control plane update of R16′s loopback over to R11. The PE router of R14 should be learning this as a vrf prefix:
R14#show ip route vrf CUS1 16.16.16.16
Routing Table: CUS1
Routing entry for 16.16.16.16/32
Known via "ospf 3", distance 110, metric 2, type intra area
Redistributing via bgp 100
Advertised by bgp 100
Last update from 10.14.16.16 on FastEthernet1/1, 00:18:35 ago
Routing Descriptor Blocks:
* 10.14.16.16, from 16.16.16.16, 00:18:35 ago, via FastEthernet1/1
Route metric is 2, traffic share count is 1
This prefix is converted into a VPNv4 prefix and advertised over to R8:
R8#show bgp vpnv4 un rd 14.14.14.14:1 16.16.16.16
BGP routing table entry for 14.14.14.14:1:16.16.16.16/32, version 3
Paths: (1 available, best #1, no table)
Not advertised to any peer
Local
14.14.14.14 (metric 1) from 14.14.14.14 (14.14.14.14)
Origin incomplete, metric 2, localpref 100, valid, internal, best
Extended Community: RT:100:1 OSPF DOMAIN ID:0x0005:0x000000030200
OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:10.14.16.14:0
mpls labels in/out nolabel/27
This should end up in the correct vrf table:
R8#show ip route vrf CUS1 16.16.16.16
Routing Table: CUS1
Routing entry for 16.16.16.16/32
Known via "bgp 100", distance 200, metric 2, type internal
Redistributing via ospf 2
Advertised by ospf 2 subnets
Last update from 14.14.14.14 00:07:03 ago
Routing Descriptor Blocks:
* 14.14.14.14 (default), from 14.14.14.14, 00:07:03 ago
Route metric is 2, traffic share count is 1
AS Hops 0
MPLS label: 27
MPLS Flags: MPLS Required
Finally R11 should receive that as on OSPF route:
R11#show ip route 16.16.16.16
Routing entry for 16.16.16.16/32
Known via "ospf 1", distance 110, metric 2
Tag Complete, Path Length == 1, AS 100, , type extern 2, forward metric 1
Last update from 10.8.11.8 on FastEthernet1/0, 00:06:38 ago
Routing Descriptor Blocks:
* 10.8.11.8, from 10.8.11.8, 00:06:38 ago, via FastEthernet1/0
Route metric is 2, traffic share count is 1
Route tag 3489661028
So our control plane is all good so far. Let’s check our data plane forwarding:
R11#ping 16.16.16.16 so lo0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 16.16.16.16, timeout is 2 seconds: Packet sent with a source address of 11.11.11.11 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 44/57/76 ms R11#traceroute 16.16.16.16 so lo0 Type escape sequence to abort. Tracing the route to 16.16.16.16 1 10.8.11.8 8 msec 12 msec 12 msec 2 10.8.9.9 [MPLS: Labels 24/27 Exp 0] 52 msec 56 msec 48 msec 3 10.6.9.6 [MPLS: Labels 24/27 Exp 0] 56 msec 48 msec 48 msec 4 10.5.6.5 [MPLS: Labels 16/27 Exp 0] 56 msec 40 msec 68 msec 5 10.5.13.13 [MPLS: Labels 20/34/27 Exp 0] 52 msec 44 msec 52 msec 6 10.3.13.3 [MPLS: Labels 19/34/27 Exp 0] 56 msec 48 msec 40 msec 7 10.2.3.2 [MPLS: Labels 34/27 Exp 0] 52 msec 52 msec 40 msec 8 10.1.2.1 [MPLS: Labels 19/27 Exp 0] 44 msec 44 msec 48 msec 9 10.1.7.7 [MPLS: Labels 18/27 Exp 0] 52 msec 40 msec 60 msec 10 10.14.16.14 [MPLS: Label 27 Exp 0] 40 msec 44 msec 36 msec 11 10.14.16.16 44 msec * 52 msec
No problems there :)
Carrier Supporing Carrier Supporting Carrier
So let’s be silly and go a step further. What if our final customer is actually another ISP offering L3VPN to it’s customers? Let’s change our topology slightly(click the image for full size)
I’m not going to show all the config here as it’s simply too much to fit into a blog post. However the config itself is pretty much like so. You’ll need to click the image for the larger version:
R10 and R15 are our final CE routers. Once all is configured does it all actually work?
R10#traceroute 15.15.15.15 so lo0 Type escape sequence to abort. Tracing the route to 15.15.15.15 1 10.10.11.11 12 msec 12 msec 8 msec 2 10.8.11.8 [MPLS: Labels 16/17 Exp 0] 72 msec 48 msec 68 msec 3 10.8.9.9 [MPLS: Labels 22/24/17 Exp 0] 36 msec 80 msec 44 msec 4 10.6.9.6 [MPLS: Labels 20/24/17 Exp 0] 52 msec 56 msec 52 msec 5 10.5.6.5 [MPLS: Labels 23/24/17 Exp 0] 64 msec 52 msec 52 msec 6 10.5.13.13 [MPLS: Labels 20/23/24/17 Exp 0] 48 msec 60 msec 52 msec 7 10.3.13.3 [MPLS: Labels 19/23/24/17 Exp 0] 56 msec 56 msec 48 msec 8 10.2.3.2 [MPLS: Labels 23/24/17 Exp 0] 56 msec 44 msec 56 msec 9 10.1.2.1 [MPLS: Labels 19/24/17 Exp 0] 48 msec 48 msec 64 msec 10 10.1.7.7 [MPLS: Labels 16/24/17 Exp 0] 56 msec 52 msec 60 msec 11 10.7.14.14 [MPLS: Labels 24/17 Exp 0] 44 msec 56 msec 56 msec 12 10.15.16.16 [MPLS: Label 17 Exp 0] 56 msec 56 msec 44 msec 13 10.15.16.15 56 msec * 76 msec
It does indeed. At this point we are up to a four label stack in AS500. If we were running RSVP-TE and FRR we would have even more labels sitting on top.
There is a much easier way to do this of course. The original Customer Carrier could just buy some some or virtual leased line or VPLS from AS500 and they would be directly connected over the same subnet. They could then run MPLS over that link and as far as anyone cares R1 and R6 would be directly connected to each other.
But of course this is a topic on the CCIE SP after all…
Comments