Moving routes between a VRF and the global (default) RIB – Part 3 – Juniper Junos

Part 1 – Cisco IOS
Part 2 – Brocade Netiron
Part 3 – Juniper Junos
Part three of this series and we move onto Junos. Junos’ implementation is pretty similar to the way Brocade’s Netiron does it. You can import and export routes from different RIBs through policies. These routes do not need to be BGP.

This is the new topology:

R4 is our CPE device in it’s own vrf. R1 is acting as the server. R2 is the PE and R3 is a regular core router. As the previous posts have shown, I want R4 to be able to ping R1’s interface address from R4’s loopback. R4 is in a customer vrf, while R1 is simply in the core of my network, reachable by a regular IGP route.

For this example I’ve used OSPF again.

Copying global routes to VRF

Junos has a feature called rib-groups. This allows a protocol (including static) to place a route in more than one table at the same time. inet.0 is the global table, while our customers table is CUS1.inet.0 – I’ll create a rib group that tells OSPF to populate both tables at the same time.
R2:

routing-options {
    rib-groups {
        COPY_GLOBAL_TO_VRF {
            import-rib [ inet.0 CUS1.inet.0 ];
            import-policy R1_ADDRESS;
        }
    }
}

USER2:R2> show configuration protocols ospf
rib-group COPY_GLOBAL_TO_VRF;

USER2:R2> show configuration policy-options policy-statement R1_ADDRESS
term 1 {
    from {
        route-filter 10.0.4.12/30 exact;
    }
    then accept;
}
term 2 {
    then reject;
}

What the above configuration is saying is:

  1. Once SPF is run, select routes to move from OSPF into the RIB
  2. When moving into the RIB, populate both inet.0 and CUS1.inet.0 at the same time
  3. Only populate CUS1.inet.0 with routes matching the policy statement

We don’t want the router importing the entire inet.0 table so we push it through a policy matching only the route that we want. This policy only affects putting routes into the CUS1 table, not our generic inet.0 table. The result of this is that R2 will place the 10.0.4.12/30 range into the customer vrf table:

USER2:R2> show route table CUS1 10.0.4.12

CUS1.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.0.4.12/30       *[OSPF/10] 02:45:01, metric 2
                    > to 10.0.4.5 via fe-0/0/3.12

This route only sits on R2 though. It needs to be redistributed into the CE-PE protocol so that R4 will see it:

USER2:R2> show configuration routing-instances CUS1 protocols ospf
export R1_ADDRESS;

I’ve use the same policy-statement that matched R1’s range before. No need to configure the same policy twice. The result is that I should see 10.0.4.12/30 as an external OSPF route on the CE:

USER4:R4> show route 10.0.4.12

inet.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.0.4.12/30       *[OSPF/150] 02:48:48, metric 2, tag 0
                    > to 10.0.4.10 via fe-0/0/3.24

Copying VRF routes to global

The reverse is pretty much the same. The difference will be the rib-groups will instead be reversed. Also this rib-group will be applied to the vrf OSPF process.
R2:

routing-options {
    rib-groups {
        COPY_VRF_TO_GLOBAL {
            import-rib [ CUS1.inet.0 inet.0 ];
        }
    }
}

USER2:R2> show configuration routing-instances CUS1 protocols ospf
rib-group COPY_VRF_TO_GLOBAL;

Once again, this is the flow:

  1. Once SPF is run in the VRF, select routes to move from OSPF into the VRF RIB
  2. When moving into the VRF RIB, populate both CUS1.inet.0 and inet.0 at the same time
  3. Populate all OSPF routes, i.e. not through a policy like last time

So now R2 should have the loopback of R4 in the global table:

USER2:R2> show route 4.4.4.4

inet.0: 16 destinations, 17 routes (16 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

4.4.4.4/32         *[OSPF/10] 1d 20:32:16, metric 1
                    > to 10.0.4.9 via ae1.24

CUS1.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

4.4.4.4/32         *[OSPF/10] 1d 20:32:16, metric 1
                    > to 10.0.4.9 via ae1.24

Which it is. The command shows both tables output. We now need to redistribute that local OSPF route into the global IGP RIB:

USER2:R2> show configuration policy-options policy-statement EXPORT_CUSTOMER
term 1 {
    from {
        protocol ospf;
        route-filter 4.4.4.4/32 exact;
    }
    then accept;
}
term 2 {
    then reject;
}

USER2:R2> show configuration protocols ospf
export EXPORT_CUSTOMER;

Which would mean that R3 sees 4.4.4.4/32 as an external OSPF route:

USER3:R3> show route 4.4.4.4

inet.0: 15 destinations, 15 routes (15 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

4.4.4.4/32         *[OSPF/150] 03:06:24, metric 1, tag 0
                    > to 10.0.4.6 via ae1.12

Verification

The end result is that R4 should be able to ping R1’s interface, but only if that ping is sourced from it’s loopback:

USER4:R4> ping 10.0.4.13 rapid
PING 10.0.4.13 (10.0.4.13): 56 data bytes
.....
--- 10.0.4.13 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

USER4:R4> ping 10.0.4.13 rapid source 4.4.4.4
PING 10.0.4.13 (10.0.4.13): 56 data bytes
!!!!!
--- 10.0.4.13 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.284/1.419/1.781/0.189 ms

Final Notes

Junos has no issue with ipv6 and rib-groups. I’ve happily moved ipv6 routes between a customer vrf and the global table. You just need to substitute inet.0 with inet6.0 as well as put the rib-group config under your ipv6 IGP protocol.

Moving routes between a VRF and the global (default) RIB – Part 2 – Brocade Netiron

Part 1 – Cisco IOS
Part 2 – Brocade Netiron
Part 3 – Juniper Junos

Part 2 of this series will focus on Brocade’s Netiron implementation. I have a few XMRs here to test with.

Slightly different set-up here for this lab:

I have firewalls acting as my CE kit this time. FW2 has a loopback address of 22.22.22.22/32 that needs to be able to speak to 172.24.0.97, FW1’s IP address that sites in the global IGP table.

Copying global routes to VRF

Brocade simply calls this feature Inter-VRF routing. Though it does allow you to leak routes between a customer vrf and the ‘default’ vrf. The default vrf is in fact the global RIB. The nice thing about Brocade’s implementation is that routes do not need to be BGP routes. They can be static, IGP, or BGP learned. This gives you a lot more flexibility when it comes to doing this sort of thing. To test I am going to run OSPF as the CE-PE routing protocol between R6 and FW2.

Let’s first check the VRF table on R6 first to ensure we don’t see 172.24.0.97:

[email protected]#sh ip route vrf CUS1
Total number of IP routes: 2
Type Codes - B:BGP D:Connected I:ISIS O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
ISIS Codes - L1:Level-1 L2:Level-2
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2 s:Sham Link
STATIC Codes - d:DHCPv6
        Destination        Gateway         Port          Cost          Type Uptime src-vrf
1       10.0.222.0/24      DIRECT          ve 22         0/0           D    1m23s  -
2       22.22.22.22/32     10.0.222.22     ve 22         110/2         O    1m3s   -

Only the directly connected link and FW’s loopback are in the table as expected. I’ll create a prefix list, route-map, and then import that into the vrf:

ip prefix-list ISP_SERVER seq 5 permit 172.24.0.0/24
!
route-map MAP_ISP_SERVER permit 10
 match ip address prefix-list ISP_SERVER
!
vrf CUS1
 address-family ipv4
   import routes vrf default-vrf route-map MAP_ISP_SERVER

One thing to note is that the global RIB is named default-vrf – all lowercase. Context-sensitive help doesn’t show this. You need to either know this, or get it out the configuration guide.

Is that route now in the vrf?

[email protected]#sh ip route vrf CUS1 172.24.0.97
Type Codes - B:BGP D:Connected I:ISIS O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
ISIS Codes - L1:Level-1 L2:Level-2
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2 s:Sham Link
STATIC Codes - d:DHCPv6
        Destination        Gateway         Port          Cost          Type Uptime src-vrf
1       172.24.0.0/24      10.0.16.1       ve 16         110/4         O    1m56s  default-vrf

It is. It’s nice enough to tell us the source vrf in the same command. You’ll notice the source protocol is still OSPF. Even though the PE is running OSPF with the CE device, that route is not advertised. The route, even though it’s in the vrf table on the PE, is still from another process. You’ll need to redistribute from the global process into the customers vrf process:

router ospf vrf CUS1
 area 0
 redistribute ospf

Note that this redistribute command will only redistribute routes that are already leaked into the local vrf table on the PE device.

FW2 should now have that route as an external OSPF route:

FW2-> get route ip 172.24.0.97
 Dest for 172.24.0.97
--------------------------------------------------------------------------------------
trust-vr       : => 172.24.0.0/24 (id=14) via 10.0.222.2 (vr: trust-vr)
                    Interface ethernet1/1.1 , metric 10

Copying VRf routes to global

With Brocade, the reverse is just as easy. One of the main differences is that instead of exporting routes out of the vrf, we simply import directly into the global RIB:

ip prefix-list CUS1_RANGE seq 5 permit 22.22.22.22/32
!
route-map MAP_CUS1_RANGE permit 10
 match ip address prefix-list CUS1_RANGE
!
ip import routes vrf CUS1 route-map MAP_CUS1_RANGE

If we now check the global table for FW2’s loopback:

[email protected]#show ip route 22.22.22.22
Type Codes - B:BGP D:Connected I:ISIS O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
ISIS Codes - L1:Level-1 L2:Level-2
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2 s:Sham Link
STATIC Codes - d:DHCPv6
        Destination        Gateway         Port          Cost          Type Uptime src-vrf
1       22.22.22.22/32     10.0.222.22     ve 22         110/2         O    1m7s   CUS1

There it is. Once again it’s an OSPF route and it also shows us the source vrf. We also need to redistribute between processes in order for those vrf OSPF routes to get into the global OSPF table:

router ospf
 area 0
 redistribute ospf

Unfortunately the Netiron doesn’t allow me to specify which process to redistribute. The above command would redistribute ALL OSPF routes that are already leaked into the local table. 95% of the time that’s what I would want to do, but it’s not the most flexible solution. I could push the routes I wanted through a route-map if I wanted.

Verification

FW2 should now be able to ping 172.24.0.97, but only if it sources those pings from 22.22.22.22/32:

FW2-> ping 172.24.0.97
Type escape sequence to abort

Sending 5, 100-byte ICMP Echos to 172.24.0.97, timeout is 1 seconds
.....
Success Rate is 0 percent (0/5)
FW2-> ping 172.24.0.97 from  loopback.1
Type escape sequence to abort

Sending 5, 100-byte ICMP Echos to 172.24.0.97, timeout is 1 seconds from loopback.1
!!!!!
Success Rate is 100 percent (5/5), round-trip time min/avg/max=2/2/3 ms

Which is exactly what we see.

Final Notes

Brocade allows the use of both IPv4 and IPv6 with all the above commands. Simply substitute ip for ipv6 and you’re good to go.

Got my CCIE SP Lab ticket. CCIE SP written passed a few minutes ago

Just a quick update. Passed the CCIE SP written this morning. The service provider track is far more relevant to my day to day duties, so the written itself was not overly difficult. Lab not booked anytime soon yet. I don’t even have a date in mind yet

Moving routes between a VRF and the global (default) RIB – Part 1 – Cisco IOS

Part 1 – Cisco IOS
Part 2 – Brocade Netiron
Part 3 – Juniper Junos

I don’t think there is a standard name for the initial route-table on a router. Junos calls it inet.0, Netiron has no name, and it’s also the no name VRF in IOS. I’ll simple be calling it the ‘global’ table for these and all future posts.

It’s easy to share routes between VRFs by manipulating route-targets. It’s not so easy when you try to do this with the global table as there are no route-targets associated with the global RIB. While there is a standard for moving routes between named VRFs, there is no such standard for moving routes in and out of the global table in this way.

As there is no standard, all three vendors take different approaches to doing this.

You can do this via static routes, but this is less than ideal. What happens if the CE device is multihomed to two different PE devices? I want to be able to import/export dynamic routes.

Let’s take the following diagram as an example:

R6 is a CE device. It is multihomed to two PE routers, R4 and R5. R1 is acting as a server connected to R2. This server is not in a vrf. i.e. all routers in the core are able to get to R1 as it’s simply an address in the global IGP table.

This particular server is running a service that the customer needs to get to. They should only be able to get to R1 via 6.6.6.6, it’s loopback address. Essentially we need the PE’s to leak the dynamically learned 6.6.6.6/32 from the VRF into the global table. At the same time we need to leak R1’s address into the VRF.

Copying global routes to VRF

First, let’s check the VRF route-table on R4 to make sure we don’t have any global routes in the vrf yet. R1’s address is 10.0.12.1/24:

R4#sh ip route vrf CUS1 | beg Gate
Gateway of last resort is not set

      6.0.0.0/32 is subnetted, 1 subnets
B        6.6.6.6 [20/0] via 10.0.46.6, 00:00:31
      10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        10.0.46.0/24 is directly connected, FastEthernet1/1
L        10.0.46.4/32 is directly connected, FastEthernet1/1

You’ve been able to do this on IOS for some time. Under your vrf definition, type import map ipv4 unicast route-map:

ip prefix-list ISP_SERVER seq 5 permit 10.0.12.0/24
!
route-map MAP_ISP_SERVER permit 10
 match ip address prefix-list ISP_SERVER
!
vrf definition CUS1
 !
 address-family ipv4
  import ipv4 unicast map MAP_ISP_SERVER

Do we see the routes in the VRF?

R4#sh ip route vrf CUS1 10.0.12.0

Routing Table: CUS1
% Subnet not in table

No. Why not? Well if you read the command reference, the import map calls this the ‘BGP Support for IP Prefix Import from Global Table into a VRF Table feature’ – R1’s address is not currently a BGP route, its an internal IGP route. We need to ensure the PEs see this route as a BGP route. Note that the BGP route itself doesn’t need to be active, it just needs to be in the BGP table and valid. On R2 I’m running regular ipv4 unicast BGP and will advertise that subnet into BGP:

R2#sh run | sec router bgp
router bgp 100
!
 address-family ipv4
  network 10.0.12.0 mask 255.255.255.0

That global route should now be imported into the vrf:

R4#sh ip route vrf CUS1 10.0.12.0

Routing Table: CUS1
Routing entry for 10.0.12.0/24
  Known via "bgp 100", distance 200, metric 0, type internal
  Last update from 2.2.2.2 00:00:35 ago
  Routing Descriptor Blocks:
  * 2.2.2.2 (default), from 2.2.2.2, 00:00:35 ago
      Route metric is 0, traffic share count is 1
      AS Hops 0
      MPLS label: none

Note that this command has been with us since IOS release 12.0(5)T

Copying VRF routes to global

Copying routes this way was a LOT more tricky, until very recently. Cisco has finally given us the export ipv4 unicast map command. This command has only existed from IOS release 15.2(4)S – A very recent release. Odd since the import command has existed for many years.

The use of this command is very similar to the import. Let’s match R6’s loopback via a prefix-list, match that in a route-map, and export from the vrf to the global through the vrf defination:

ip prefix-list CUS1_PREFIX seq 5 permit 6.6.6.6/32
!
route-map MAP_CUS1_PREFIX permit 10
 match ip address prefix-list CUS1_PREFIX
!
vrf definition CUS1
 !
 address-family ipv4
  export ipv4 unicast map MAP_CUS1_PREFIX

Just like the import command, this command export the route into the global BGP table. This means I could either redistribute the BGP route into my IGP, or use the existing BGP session I have with R2. Let’s check on R2 to see if I have R6’s loopback:

R2#sh ip route 6.6.6.6
% Network not in table

It’s not there. But is it in my BGP table?

R2#show ip bgp 6.6.6.6
BGP routing table entry for 6.6.6.6/32, version 6
Paths: (1 available, no best path)
  Not advertised to any peer
  Refresh Epoch 1
  6
    10.0.46.6 (inaccessible) from 4.4.4.4 (4.4.4.4)
      Origin IGP, metric 0, localpref 100, valid, internal
      rx pathid: 0, tx pathid: 0

It’s there, but the next-hop is inaccessible. R4 is treating this as a regular eBGP update and so does not change the next-hop to itself as it would with a VPNv4 route. This can be easily fixed:

R4#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R4(config)#router bgp 100
R4(config-router)#address-family ipv4 unicast
R4(config-router-af)#neighbor 2.2.2.2 next-hop-self

We should now see the valid route on R2:

R2#show ip bgp 6.6.6.6
BGP routing table entry for 6.6.6.6/32, version 7
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Refresh Epoch 1
  6
    4.4.4.4 (metric 3) from 4.4.4.4 (4.4.4.4)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      rx pathid: 0, tx pathid: 0x0

Verification

So at the end of all of this, R6 should be able to ping R1’s interface, but only if it sources the ping from 6.6.6.6. Let’s confirm this:

CUSTOMER#ping 10.0.12.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.12.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
CUSTOMER#ping 10.0.12.1 source 6.6.6.6
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.12.1, timeout is 2 seconds:
Packet sent with a source address of 6.6.6.6
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 80/93/112 ms

Which is exactly what we see.

Final Notes

With IOS, in order to move routes between the global and vrf tables, you need to invoke the import [ipv6|ipv4] unicast command. There is another import map command under the vrf, but that command does NOT move routes between the vrf and global. That command is there to have finer control of which vrf routes are imported/exported between MP-BGP and the vrf.

The export ipv4/ipv6 unicast command is only a very recent command, but at least supports both address families. The import command has been around for years, yet oddly does not support ipv6:

R4#sh ver | include IOS
Cisco IOS Software, 7200 Software (C7200-ADVENTERPRISEK9-M), Version 15.2(4)S2, RELEASE SOFTWARE

R4(config-vrf)#address-family ipv4

R4(config-vrf-af)#export ?
  ipv4  Address family based VRF export
  map   Route-map based VRF export

R4(config-vrf-af)#import ?
  ipv4  Address family based VRF import
  map   Route-map based VRF import


R4(config-vrf-af)#address-family ipv6
R4(config-vrf-af)#export ?
  ipv6  Address family based VRF export
  map   Route-map based VRF export

R4(config-vrf-af)#import ?
  map  Route-map based VRF import

CCIE Plaque arrived

Almost forgot, I got a delivery from Cisco this past week :)