Tag Archives: Juniper

Juniper EX – Private vlans

I’ve gone over pvlans before on IOS, so I’m going to cover Juniper’s implementation today. This post will be based on the following topology:
PVLANS 11 Juniper EX   Private vlans
There are five hosts and a single router. Host1 and Host3 are in the same community vlan, while Host2, Host4, and Host5 are in isolated vlans. R1 is the default gateway for all hosts.
The vlan plan is laid out as follows:

PVLAN Table
Host Switch Vlan
1 SW1 Community 501
2 SW1 Isolated 502
3 SW2 Community 501
4 SW2 Isolated 502
5 SW2 Isolated 502
R1 SW3 Primary 500

Community vlan – SW1

The community vlan has a vlan-id and primary vlan specified. You enable the interface under tha vlan config:

set vlans HOST_COMM vlan-id 501
set vlans HOST_COMM primary-vlan HOST_PRIMARY
set vlans HOST_COMM interface ge-0/0/4.0

Isolated and Primary Van – SW1

Isolated vlans are configured directly under the primary vlan. You also specify the interfaces in this vlan under the vlans hierarchy. Finally, as this pvlans span multiple switches, you need to ensure the trunk interfaces are pvlans aware:

set vlans HOST_PRIMARY vlan-id 500
set vlans HOST_PRIMARY interface ge-0/0/1.0 pvlan-trunk
set vlans HOST_PRIMARY interface ge-0/0/3.0 pvlan-trunk
set vlans HOST_PRIMARY interface ge-0/0/5.0
set vlans HOST_PRIMARY no-local-switching
set vlans HOST_PRIMARY isolation-id 502

Personally I really don’t like the Junos way of doing isolated vlans. Interface ge-0/0/5.0 is an isolated port as its untagged and no-local-switching is configured. Configuring the promiscuous port to R1 from SW3 is configured like so:

set vlans HOST_PRIMARY vlan-id 500
set vlans HOST_PRIMARY interface ge-0/0/2.0 pvlan-trunk
set vlans HOST_PRIMARY interface ge-0/0/1.0 pvlan-trunk
set vlans HOST_PRIMARY interface ge-0/0/0.0
set vlans HOST_PRIMARY no-local-switching
set vlans HOST_PRIMARY isolation-id 502

There is no difference in the vlan configuration between an actual isolated port and a promiscuous port. What makes the difference is the interface config itself on both switches:

root@SW2> show configuration interfaces ge-0/0/5
unit 0 {
    family ethernet-switching {
        port-mode access;
    }
}

While SW3′s port to R1 is tagged:

root@SW3> show configuration interfaces ge-0/0/0
unit 0 {
    family ethernet-switching {
        port-mode trunk;
    }
}

If I wanted SW3′s like to R1 to be untagged it would change it to an isolated port. If I needed a host to send tagged traffic into an isolated vlan (like an ESX server), Junos makes that a promiscuous port. This is a lack of flexibility that I don’t like. The switches should be able to put devices in isolated or promiscuous mode by config separate to the fact that the host-facing port has a dot1q tag or not.

Verification

Show vlans extensive shows the pvlan information. It would be nice if Junos had a separate show pvlans command:

lab@SW1> show vlans extensive
VLAN: HOST_COMM, Created at: Wed Mar 19 04:00:58 2014
802.1Q Tag: 501, Internal index: 14, Admin State: Enabled, Origin: Static
Private VLAN Mode: Community, Primary VLAN: HOST_PRIMARY
Protocol: Port Mode, Mac aging time: 300 seconds
Number of interfaces: Tagged 2 (Active = 2), Untagged  1 (Active = 1)
      ge-0/0/1.0*, tagged, trunk, pvlan-trunk
      ge-0/0/3.0*, tagged, trunk, pvlan-trunk
      ge-0/0/4.0*, untagged, access

Here we see ge-0/0/4.0 is an access port in vlan HOST_COMM. ge-0/0/1.0 and ge-0/0/3.0 are pvlan trunks as expected.

LAN: HOST_PRIMARY, Created at: Wed Mar 19 04:00:58 2014
802.1Q Tag: 500, Internal index: 16, Admin State: Enabled, Origin: Static
Private VLAN Mode: Primary
Protocol: Port Mode, Mac aging time: 300 seconds
Number of interfaces: Tagged 2 (Active = 2), Untagged  2 (Active = 2)
      ge-0/0/1.0*, tagged, trunk, pvlan-trunk
      ge-0/0/3.0*, tagged, trunk, pvlan-trunk
      ge-0/0/4.0*, untagged, access
      ge-0/0/5.0*, untagged, access
Secondary VLANs: Isolated 1, Community  1, Inter-switch-isolated  1
  Isolated VLANs :
      __pvlan_HOST_PRIMARY_ge-0/0/5.0__
  Community VLANs :
      HOST_COMM
  Inter-switch-isolated VLAN :
      __pvlan_HOST_PRIMARY_isiv__

VLAN: __pvlan_HOST_PRIMARY_ge-0/0/5.0__, Created at: Wed Mar 19 04:26:16 2014
Internal index: 17, Admin State: Enabled, Origin: Static
Private VLAN Mode: Isolated, Primary VLAN: HOST_PRIMARY
Protocol: Port Mode, Mac aging time: 300 seconds
Number of interfaces: Tagged 2 (Active = 2), Untagged  1 (Active = 1)
      ge-0/0/1.0*, tagged, trunk, pvlan-trunk
      ge-0/0/3.0*, tagged, trunk, pvlan-trunk
      ge-0/0/5.0*, untagged, access

VLAN: __pvlan_HOST_PRIMARY_isiv__, Created at: Wed Mar 19 04:26:16 2014
802.1Q Tag: 502, Internal index: 18, Admin State: Enabled, Origin: Static
Private VLAN Mode: Inter-switch-isolated, Primary VLAN: HOST_PRIMARY
Protocol: Port Mode, Mac aging time: 300 seconds
Number of interfaces: Tagged 2 (Active = 2), Untagged  0 (Active = 0)
      ge-0/0/1.0*, tagged, trunk, pvlan-trunk
      ge-0/0/3.0*, tagged, trunk, pvlan-trunk

A lot of information above, buit it does show which ports are connected to the primary vlan and which are isolated. It also shows which community and isolated vlans are connected to the primary vlan.

Ultimately the end result is that Host1 should be able to ping Host3 and Router1, but nothing else:

root@MULTI_HOST> ping routing-instance HOST1 10.0.0.2 rapid
PING 10.0.0.2 (10.0.0.2): 56 data bytes
.....
--- 10.0.0.2 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

{master:0}
root@MULTI_HOST> ping routing-instance HOST1 10.0.0.3 rapid
PING 10.0.0.3 (10.0.0.3): 56 data bytes
!!!!!
--- 10.0.0.3 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.056/1.175/1.415/0.136 ms

{master:0}
root@MULTI_HOST> ping routing-instance HOST1 10.0.0.4 rapid
PING 10.0.0.4 (10.0.0.4): 56 data bytes
.....
--- 10.0.0.4 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

{master:0}
root@MULTI_HOST> ping routing-instance HOST1 10.0.0.5 rapid
PING 10.0.0.5 (10.0.0.5): 56 data bytes
.....
--- 10.0.0.5 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

{master:0}
root@MULTI_HOST> ping routing-instance HOST1 10.0.0.254 rapid
PING 10.0.0.254 (10.0.0.254): 56 data bytes
!!!!!
--- 10.0.0.254 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.007/1.189/1.638/0.229 ms

I won’t go every single possible combination a there will simply be too much text, but I’ll go over Host4 and Router1.

Host4 should only be able to ping Router1 and nothing else:

root@MULTI_HOST> ping routing-instance HOST4 10.0.0.1 rapid
PING 10.0.0.1 (10.0.0.1): 56 data bytes
.....
--- 10.0.0.1 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

{master:0}
root@MULTI_HOST> ping routing-instance HOST4 10.0.0.2 rapid
PING 10.0.0.2 (10.0.0.2): 56 data bytes
.....
--- 10.0.0.2 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

{master:0}
root@MULTI_HOST> ping routing-instance HOST4 10.0.0.3 rapid
PING 10.0.0.3 (10.0.0.3): 56 data bytes
.....
--- 10.0.0.3 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

{master:0}
root@MULTI_HOST> ping routing-instance HOST4 10.0.0.5 rapid
PING 10.0.0.5 (10.0.0.5): 56 data bytes
.....
--- 10.0.0.5 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

{master:0}
root@MULTI_HOST> ping routing-instance HOST4 10.0.0.254 rapid
PING 10.0.0.254 (10.0.0.254): 56 data bytes
!!!!!
--- 10.0.0.254 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.997/1.763/3.471/0.927 ms

Finally, Router1 should be able to ping all hosts:

root@MULTI_HOST> ping routing-instance ROUTER1 10.0.0.1 rapid
PING 10.0.0.1 (10.0.0.1): 56 data bytes
!!!!!
--- 10.0.0.1 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.011/1.168/1.301/0.122 ms

{master:0}
root@MULTI_HOST> ping routing-instance ROUTER1 10.0.0.2 rapid
PING 10.0.0.2 (10.0.0.2): 56 data bytes
!!!!!
--- 10.0.0.2 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.963/1.194/1.432/0.180 ms

{master:0}
root@MULTI_HOST> ping routing-instance ROUTER1 10.0.0.3 rapid
PING 10.0.0.3 (10.0.0.3): 56 data bytes
!!!!!
--- 10.0.0.3 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.988/1.165/1.438/0.161 ms

{master:0}
root@MULTI_HOST> ping routing-instance ROUTER1 10.0.0.4 rapid
PING 10.0.0.4 (10.0.0.4): 56 data bytes
!!!!!
--- 10.0.0.4 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.997/1.194/1.444/0.200 ms

{master:0}
root@MULTI_HOST> ping routing-instance ROUTER1 10.0.0.5 rapid
PING 10.0.0.5 (10.0.0.5): 56 data bytes
!!!!!
--- 10.0.0.5 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.015/1.174/1.324/0.123 ms

Juniper EX Virtual-Chassis notes

I’ve been deploying some EX VCs recently so this post will go over some configuration and verification commands. To start with I have two EX4200s in my lab connected via the built-in VC ports. I’m running code version 12.3R6.6

VC 1 Juniper EX Virtual Chassis notes

VC Ports

When booting this type of configuration, the switches will automatically attempt to create a virtual-chassis. i.e. without configuring anything they are already in a virtual-chassis:

root@EX4200> show virtual-chassis

Virtual Chassis ID: f365.a9c6.1714
Virtual Chassis Mode: Enabled
                                           Mstr           Mixed Neighbor List
Member ID  Status   Serial No    Model     prio  Role      Mode ID  Interface
0 (FPC 0)  Prsnt    BN0208105351 ex4200-24p 128  Backup       N  1  vcp-0
                                                                 1  vcp-1
1 (FPC 1)  Prsnt    BN0208084463 ex4200-24p 128  Master*      N  0  vcp-0
                                                                 0  vcp-1

Member ID for next new member: 2 (FPC 2)

Here we see two members, both of which are present. Both have a default priority of 128. vcp-0/1 are the purpose-made VC ports. We can drill a bit deeper to see the bandwidth offered by these ports:

root@EX4200> show virtual-chassis vc-port
fpc0:
--------------------------------------------------------------------------
Interface   Type              Trunk  Status       Speed        Neighbor
or                             ID                 (mbps)       ID  Interface
PIC / Port
vcp-0       Dedicated           2    Up           32000        1   vcp-1
vcp-1       Dedicated           1    Up           32000        1   vcp-0

fpc1:
--------------------------------------------------------------------------
Interface   Type              Trunk  Status       Speed        Neighbor
or                             ID                 (mbps)       ID  Interface
PIC / Port
vcp-0       Dedicated           2    Up           32000        0   vcp-1
vcp-1       Dedicated           1    Up           32000        0   vcp-0

VCEP Ports

VCP cables are limited in length. Juniper allows you to use the uplink module on the ex4200 to create a VC. These are called VCEP ports and requires manual configuration to do so. You can mix and match both VCP and VCEP ports at the same time. I’ve added the following switch to my topology:
VC 2 Juniper EX Virtual Chassis notes
Turning these ports into VCEP ports is done in operational mode!

root@EX4200> request virtual-chassis vc-port set pic-slot 1 port 0

Once this is done on both, I can verify that the vcep ports are created and the switched has joined the VC:

root@EX4200> show virtual-chassis vc-port
fpc0:
--------------------------------------------------------------------------
Interface   Type              Trunk  Status       Speed        Neighbor
or                             ID                 (mbps)       ID  Interface
PIC / Port
vcp-0       Dedicated           2    Up           32000        1   vcp-1
vcp-1       Dedicated           1    Up           32000        1   vcp-0
1/0         Configured         -1    Up           1000         2   vcp-255/1/0

fpc1:
--------------------------------------------------------------------------
Interface   Type              Trunk  Status       Speed        Neighbor
or                             ID                 (mbps)       ID  Interface
PIC / Port
vcp-0       Dedicated           2    Up           32000        0   vcp-1
vcp-1       Dedicated           1    Up           32000        0   vcp-0

fpc2:
--------------------------------------------------------------------------
Interface   Type              Trunk  Status       Speed        Neighbor
or                             ID                 (mbps)       ID  Interface
PIC / Port
vcp-0       Dedicated           1    Down         32000
vcp-1       Dedicated           2    Down         32000
1/0         Configured         -1    Up           1000         0   vcp-255/1/0

The advantage of using VCEP ports is that my ethernet cables can run as long as standard ethernet. The disadvantages include losing some front ports as well as much lower bandwidth. In the above the VCP ports give 32Gb while the VCEP port only gives 1Gb. Of course I could use a 10Gb module for the same duty.

Deterministic Master

Juniper has a great page showing how a master is elected right here. It basically goes like this:

  • Choose the member with the highest user-configured mastership priority (255 is the highest possible value). A switch with a mastership priority of 0 will always stay in the linecard role.
  • Choose the member that was master the last time the Virtual Chassis configuration booted.
  • Choose the member that has been included in the Virtual Chassis configuration for the longest period of time. (For this to be a deciding factor, there has to be a minimum time lapse of 1 minute between the power-ons of the individual interconnected member switches.)
  • Choose the member with the lowest MAC address.

I’d like to ensure SW1 is the master when possible:

{master:1}[edit]
root@EX4200# set virtual-chassis member 0 mastership-priority 255

I can verify that it’s now 255:

root@EX4200> show virtual-chassis

Virtual Chassis ID: f365.a9c6.1714
Virtual Chassis Mode: Enabled
                                           Mstr           Mixed Neighbor List
Member ID  Status   Serial No    Model     prio  Role      Mode ID  Interface
0 (FPC 0)  Prsnt    BN0208105351 ex4200-24p 255  Master*      N  1  vcp-0
                                                                 2  vcp-255/1/0
                                                                 1  vcp-1
1 (FPC 1)  Prsnt    BN0208084463 ex4200-24p 128  Backup       N  0  vcp-0
                                                                 0  vcp-1
2 (FPC 2)  Prsnt    BP0213260138 ex4200-48t 128  Linecard     N  0  vcp-255/1/0

Member ID for next new member: 3 (FPC 3)

Note that this mastership is pre-emptive and you cannot change that behaviour. This can be disruptive and the current best practise is to actually ensure all devices are configured with the same priority. – Virtual Chassis Technology Best Practises

Management port

Each ex4200 has an onboard ethernet management port referred to as me0.0 in the configuration. In a VC you con configure a vme port. This is a management address that moves to whichever switch is the master. Note that packets coming into ANY members me port will get directed to the master switch via the VCP ports:

root@EX4200> show configuration interfaces vme
unit 0 {
    family inet {
        address 192.168.0.0/31;
    }
}

LCD Menu

The LCD panel of switches in a VC will inform you of their current role. RE for master, BK for backup, LC for linecard.

VCCP

Juniper uses VCCP as the VC protocol. This is actually customised IS-IS and you cannot configure it. You are able to extract some information though:

root@EX4200> show virtual-chassis protocol interface
fpc0:
--------------------------------------------------------------------------
IS-IS interface database:
Interface             State         Metric
internal-0/27         Up             7
internal-1/24         Up             7
vcp-0.32768           Up             7
vcp-1.32768           Up             7
vcp-255/1/0.32768     Up             240

fpc1:
--------------------------------------------------------------------------
IS-IS interface database:
Interface             State         Metric
internal-0/27         Up             7
internal-1/24         Up             7
vcp-0.32768           Up             7
vcp-1.32768           Up             7

fpc2:
--------------------------------------------------------------------------
IS-IS interface database:
Interface             State         Metric
internal-0/24         Up             7
internal-1/25         Up             7
internal-2/24         Up             7
internal-2/27         Up             7
vcp-0.32768           Down           7
vcp-1.32768           Down           7
vcp-255/1/0.32768     Up             240

Upgrading

When upgrading, all member switches will be upgraded:

lab@EX4200> request system software add /tmp/usb/jinstall-ex-4200-13.2X50-D19.2-domestic-signed.tgz

[Mar 13 10:55:09]: Retrieving software images. This process can take several minutes. Please be patient..

[Mar 13 10:56:19]: Retrieving version and model information from /tmp/usb/jinstall-ex-4200-13.2X50-D19.2-domestic-signed.tgz

[Mar 13 10:57:37]: Checking pending install on fpc0

[Mar 13 10:57:37]: Checking pending install on fpc2

[Mar 13 10:57:38]: Checking pending install on fpc1
[Mar 13 10:58:04]: Pushing bundle to fpc0
[Mar 13 10:58:30]: Pushing bundle to fpc2

[Mar 13 10:58:59]: Validating on fpc0

[Mar 13 10:59:13]: Validating on fpc2

[Mar 13 10:59:14]: Validating on fpc1
[Mar 13 10:59:14]: Done with validate on all virtual chassis members

fpc0:
Verify the signature of the new package
Verified jinstall-ex-4200-13.2X50-D19.2-export.tgz signed by PackageProduction_13_2_0
WARNING: A reboot is required to install the software
WARNING:     Use the 'request system reboot' command immediately

fpc2:
Verify the signature of the new package
Verified jinstall-ex-4200-13.2X50-D19.2-export.tgz signed by PackageProduction_13_2_0
WARNING: A reboot is required to install the software
WARNING:     Use the 'request system reboot' command immediately

fpc1:
Verify the signature of the new package
Verified jinstall-ex-4200-13.2X50-D19.2-export.tgz signed by PackageProduction_13_2_0
WARNING: A reboot is required to install the software
WARNING:     Use the 'request system reboot' command immediately

You can reboot individual switches, but a member in the wrong version will not join the VC stack:

lab@EX4200> request system reboot member 2
Reboot the system ? [yes,no] (no) yes


Rebooting fpc2
lab@EX4200> show virtual-chassis

Virtual Chassis ID: f365.a9c6.1714
Virtual Chassis Mode: Enabled
                                           Mstr           Mixed Neighbor List
Member ID  Status   Serial No    Model     prio  Role      Mode ID  Interface
0 (FPC 0)  Prsnt    BN0208105351 ex4200-24p 255  Backup       N  1  vcp-0
                                                                 2  vcp-255/1/0
                                                                 1  vcp-1
1 (FPC 1)  Prsnt    BN0208084463 ex4200-24p 255  Master*      N  0  vcp-0
                                                                 0  vcp-1
2 (FPC 2)  Inactive BP0213260138 ex4200-48t 255  Linecard     N  0  vcp-255/1/0

The last switches remains inactive until all versions match. You can reboot all switches by issuing a request system reboot

Note, it’s possible to do NSSU on EX switches. Capabilities and versions matter. Juniper has a long document here showing how it works so I won’t repeat the information here.

NSR/NSB/GRES

A VC stack gives you multiple routing-engines of which one is active and a second is backup. Graceful routing engine switchover and non-stop bridging and routing are supported, but not enabled by default. It’s a simple matter to enable:

{master:1}[edit]
root@EX4200# set chassis redundancy graceful-switchover
{master:1}[edit]
root@EX4200# set ethernet-switching-options nonstop-bridging
{master:1}[edit]
root@EX4200# set routing-options nonstop-routing

In order to verify you need to issue show system switchover on the backup RE:

root@EX4200> show system switchover
fpc0:
--------------------------------------------------------------------------
Graceful switchover: On
Configuration database: Ready
Kernel database: Ready
Peer state: Steady State

GRE Tunneling

Juniper added support for GRE on EX in 12.1 onwards. Juniper requires you to convert a hardware port into a tunnel interface to do GRE encapsulation. The issue with this is that the gre interface is configured on the tunnel interface that was created. As this is tied to a single physical port, the loss of that port means the GRE tunnel goes down. Essentially this means if you lose the switch that is doing the tunnelling you lose the GRE tunnel. The only way around this is to create multiple tunnel interfaces with multiple GRE tunnels to systems that require it.
Configuring Generic Routing Encapsulation Tunneling

My book – MPLS for Enterprise Engineers is now available from multiple channels

I put together a beginners MPLS book for Juniper. I’ve noticed, when interviewing candidates, that often they have a good knowledge of routing protocols, but lack in MPLS. This is to be expected unless they’ve worked at an ISP. The book is targeted towards those users.

J-net

https://www.juniper.net/us/en/community/junos/training-certification/day-one/networking-technologies-series/mpls-enterprise-engineers/

Amazon

http://www.amazon.com/dp/B00IU1KCJ0
http://www.amazon.co.uk/dp/B00IU1KCJ0

iTunes

https://itunes.apple.com/us/book/day-one-mpls-for-enterprise/id836201741?mt=11

Vervante (Print version)

http://store.vervante.com/c/v/V4081705490.html

Always check the forwarding table – IOS, Junos, Netiron

Most bigger routers these days use a distributed system. One of the bigger differences is the separation on the control and forwarding plane. When troubleshooting or verifying it’s essential to view both. Too many engineers simply show the control plane output. While these should match, they don’t always. Note that the forwarding table doesn’t have to be distributed to different hardware.

For the examples below I’ll simply be viewing a default route learned through OSPF. The router in question will always have two equal costs out of the network so you would expect to see two routes.

IOS

First we check the routing table:

R1#sh ip route 0.0.0.0
Routing entry for 0.0.0.0/0, supernet
  Known via "ospf 1", distance 110, metric 1, candidate default path
  Tag 1, type extern 2, forward metric 2
  Last update from 10.0.12.2 on GigabitEthernet2/0, 00:00:33 ago
  Routing Descriptor Blocks:
  * 10.0.13.3, from 10.0.24.4, 00:00:33 ago, via GigabitEthernet1/0
      Route metric is 1, traffic share count is 1
      Route tag 1
    10.0.12.2, from 10.0.24.4, 00:00:33 ago, via GigabitEthernet2/0
      Route metric is 1, traffic share count is 1
      Route tag 1

Two ways to get to 0.0.0.0 – What does the forwarding table show? For this I’ll choose an IP that would follow the default route:

R1#sh ip cef 4.2.2.1
0.0.0.0/0
  nexthop 10.0.12.2 GigabitEthernet2/0
  nexthop 10.0.13.3 GigabitEthernet1/0

Both control plane and data plane agree.

Netiron

Routing table:

[email protected]#sh ip route 0.0.0.0
Type Codes - B:BGP D:Connected I:ISIS O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
ISIS Codes - L1:Level-1 L2:Level-2
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2 s:Sham Link
STATIC Codes - d:DHCPv6
        Destination        Gateway         Port          Cost          Type Uptime src-vrf
1       0.0.0.0/0          10.0.0.1        eth 15/1      110/110       O1   1h22m  -
        0.0.0.0/0          10.0.0.2        eth 16/1      110/110       O1   1h22m  -

In order to show the forwarding table you use show route x.x.x.x detail. Note that I’m executing this command on an XMR16 and I will get the forwarding entry for every single module. I’m going to only show the output for the first module:

[email protected]#sh ip route 4.2.2.1 detail
Type Codes - B:BGP D:Connected I:ISIS O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
ISIS Codes - L1:Level-1 L2:Level-2
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2 s:Sham Link
STATIC Codes - d:DHCPv6
        Destination        Gateway         Port          Cost          Type Uptime src-vrf
1       0.0.0.0/0          10.0.0.1        eth 15/1      110/110       O1   1h24m  -
        0.0.0.0/0          10.0.0.1        eth 16/1      110/110       O1   1h24m  -
        Nexthop Entry ID:65540, Paths: 2, Ref_Count:707/712

D:Dynamic  P:Permanent  F:Forward  U:Us  C:Connected Network E: ESI VLAN
W:Wait ARP  I:ICMP Deny  K:Drop  R:Fragment  S:Snap Encap N:CamInvalid

Module S1:
      IP Address         Next Hop        MAC              Type  Port  Vlan  Pri
      0.0.0.0/0          10.0.0.1       0012.f293.a802   PF    16/1   1     0

      OutgoingIf  ArpIndex PPCR_ID   CamLevel   Parent  DontAge Index Is_trunk
      eth 16/1    5        1:1       31              0               0 0

      U_flags   Entry_flags  Age   Cam:Index               HW_Path_count
      0000e000               0     0x0005ffff (L3, right)  2

        CAM Entry Flag: 00000001H
        PPCR : 1:1 CIDX: 0x0005ffff (L3, right) (IP_NETWORK: 0x56000)

        pram_index_programmed: ppcr[0] 0x0000014c

The output is a little cryptic so I’ll highlight the important bits. First the paths show as two:

Nexthop Entry ID:65540, Paths: 2, Ref_Count:707/712

But the actual next-hop is only showing a single:

     0.0.0.0/0          10.0.0.1       0012.f293.a802   PF    16/1   1     0

This is a cosmetic error. The most important bit is here:

      U_flags   Entry_flags  Age   Cam:Index               HW_Path_count
      0000e000               0     0x0005ffff (L3, right)  2

The hardware path count is two, which is what we expect.

Junos

Finally Junos. First up we look at the route table:

lab@Vega_SRX6> show route 0.0.0.0

inet.0: 32 destinations, 32 routes (29 active, 3 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0          *[OSPF/150] 00:00:12, metric 0, tag 0
                      to 172.30.0.17 via ge-0/0/4.126
                    > to 172.30.0.89 via ge-0/0/4.146

Two routes, our forwarding table should match?

lab@Vega_SRX6> show route forwarding-table destination 4.2.2.1
Routing table: default.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
default            user     1 0:c:29:86:21:55    ucst   584    13 ge-0/0/4.146
default            perm     0                    rjct    36     5

Routing table: __master.anon__.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
default            perm     0                    rjct   534     1

Well no, it doesn’t. While the route table shows two routes, only one is being used by the forwarding table. Junos will not install multiple next-hops into the forwarding-table unless you tell it to:

lab@Vega_SRX6> show configuration policy-options policy-statement BALANCE
then {
    load-balance per-packet;
}
lab@Vega_SRX6> show configuration routing-options forwarding-table
export BALANCE;

Let’s check again:

lab@Vega_SRX6> show route forwarding-table destination 4.2.2.1
Routing table: default.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
default            user     1                    ulst 262142     7
                              0:c:29:25:21:57    ucst   612    11 ge-0/0/4.126
                              0:c:29:86:21:55    ucst   584     9 ge-0/0/4.146
default            perm     0                    rjct    36     5

Routing table: __master.anon__.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
default            perm     0                    rjct   534     1

This time we have both in the forwarding table. Note that while the policy states load-blance per-packet, it’s actually doing per-flow load-sharing.

Conclusion

I have seen routers disagree as to what they think they are doing compared to what they are doing. You need to check both tables above to note what both are doing. This could help immensely when a router is dropping packets it’s supposed to be forwarding, due to your FIB having no entry. I might write a bit on this as I’ve seen it happen more than once.

EDIT – 04/11/13

I’ve since found another way to verify this on the Brocades. If you rconsole onto the line card itself you can see a bit more:

SSH@XMR16#rconsole 1
Remote connection to LP slot 1 established
Press CTRL-X or type 'exit' to disconnect it
LP-1>en
LP-1#sh ip network 0.0.0.0
D:Dynamic  P:Permanent  F:Forward  U:Us  C:Connected Network
W:Wait ARP  I:ICMP Deny  K:Drop  R:Fragment  S:Snap Encap N:CamInvalid
      IP Address         Next Hop        MAC              Type  Port  Vlan  Pri
      0.0.0.0/0          10.0.0.1*    0012.f293.ad02   PF    15/1*  1     0

      OutgoingIf  ArpIndex PPCR_ID   CamLevel   Parent  DontAge Index Is_trunk
      eth 15/1    4        1:1       31              0               0 0

      U_flags   Entry_flags  Age   Cam:Index               HW_Path_count
      0000e000  0x00000001   0     0x0005ffff (L3, right)  2

        CAM Entry Flag: 00000001H
        PPCR : 1:1 CIDX: 0x0005ffff (L3, right) (IP_NETWORK: 0x56000)

        pram_index_programmed: ppcr[0] 0x0000014c
use_index: 0
IP-nh-Pram 0: 0x2ebeec10, ref_count 1
n_paths = 2, type = ECMP_PHY_VE, is_default  = 1, vrf_index = 0
  path[0]: FORWARD, out_intf eth 15/1, nh 10.0.0.1, out_port 15/1, is_trunk 0
  path[1]: FORWARD, out_intf eth 16/1, nh 10.0.0.5, out_port 16/1, is_trunk 0
Pram info: alloc_count 2 use_count 2
  pram[0]: idx 0, pram_idx[0] 0x0000014c
  pram[1]: idx 1, pram_idx[0] 0x0000014d

The top half still shows a single port, but down it shows this:

n_paths = 2, type = ECMP_PHY_VE, is_default  = 1, vrf_index = 0
  path[0]: FORWARD, out_intf eth 15/1, nh 10.0.0.1, out_port 15/1, is_trunk 0
  path[1]: FORWARD, out_intf eth 16/1, nh 10.0.0.5, out_port 16/1, is_trunk 0

n paths is the number of paths. The router is also doing ECMP. It then shows which ports outbound it’ll send traffic.

On a route with only a single hop the bit above are shown as so:

n_paths = 1, type = NON_ECMP, is_default  = 0, vrf_index = 0
  path[0]: FORWARD, out_intf eth 1/20, nh 10.0.0.8, out_port 1/20, is_trunk 0

OSPF Fast Re-Route and BFD on Junos

One of the few advantages that EIGRP had over OSPF and IS-IS was that it had feasable successors. That is the router had already pre-calculated a route to a destination over a backup, non-looping, path.

OSPF and IS-Is has had this for sometime now on both IOS and Junos. It’s also supported on IOS-XR.

This post will mainly go over OSPF. The process is nearly identical for IS-IS.

To start I’ll be using the following topology:
FRR 1 OSPF Fast Re Route and BFD on Junos
R3 has two links to R4. This is going through a switch which will allow us to bring the link down without pulling the interface down. I’m configuring a cost of 100 on the first link and 1000 on the second as I don’t want to bring ECMP into play for this post.

How does a router know it’s neighbour is down? If the interface goes down the detection will be quick. If the interface stays up, but something alone the path is dropping packets, the router will take quite a long time to detect this.

If we leave OSPF to its defaults, it could be 40 seconds before R3 realises it cannot get to R4 over their primary interface (Standard dead timer on broadcast links). Until that happens R3 will be sending packets into the void.

I’ll set up standard OSPF on all interfaces. From R2 I’ll be sending pings to R5′s loopback. R3 and R4 are both tagged interfaces in different vlans. On the switch I can simply remove vlan 24 which will cause packets to be dropped over that vlan.

OSPF – No tweaking

Standard OSPF here with no tweaks. I’ll be showing R3′s config here:

darreno@M7i> show configuration protocols ospf
area 0.0.0.0 {
    interface lo0.3;
    interface fe-0/1/4.24 {
        metric 100;
    }
    interface fe-0/1/5.35 {
        metric 1000;
    }
}

I’ll now initiate a ping flood from R2 to R5. Once that starts I’ll remove vlan 24 from the switch.

Let’s see how the ping flood goes:

!!!.....................................................................!!!

Not very good at all!

OSPF – BFD

Let’s add BFD to the OSPF session on both R3 and R4:

darreno@M7i> show configuration protocols ospf
area 0.0.0.0 {
    interface all;
    interface lo0.3;
    interface fe-0/1/4.24 {
        metric 100;
        bfd-liveness-detection {
            minimum-interval 50;
            minimum-receive-interval 30;
            multiplier 3;
        }
    }
    interface fe-0/1/5.35 {
        metric 1000;
        bfd-liveness-detection {
            minimum-interval 50;
            minimum-receive-interval 30;
            multiplier 3;
        }
    }
}

Do the same test as above.

!!!!.!!!

Much much better. Note that this is a very small topology though so LSAs are very quick to flood. If you had a larger topology, especially if it spans geographic regions it could take much longer for the new route to be calculated.

OSPF – BFD & FRR

Now I’ll add FRR to OSPF on R3. I’ll protect the fe-0/1/4.0 link from R3′s point of view. R3 will run SPF for all it’s destinations through that interface and will know if it can get to any destination through any other interfaces without being looped. In this simple topology any traffic sent over the higher metric interface to R4 will still get to R5 as R4 will not send it back.

First we enable link-protection:

darreno@M7i> show configuration protocols ospf area 0 interface fe-0/1/4.24
link-protection;
metric 100;
bfd-liveness-detection {
    minimum-interval 50;
    minimum-receive-interval 30;
    multiplier 3;
}

Junos will pre-calculate the routes, but it will NOT add it to the FIB by default. You have to enable more than one next-hop in the FIB:

darreno@M7i> show configuration policy-options policy-statement BALANCE
then {
    load-balance per-packet;
}

darreno@M7i> show configuration routing-options forwarding-table
export BALANCE;

Let’s run the same test as above again:

!!!!!!!!!!!!!!!!!!!!!!

I’m simply not losing any at all. The difference between BFD alone and BFD and link-protection is most pronounced on much larger topologies. Remember FRR is a router making a local repair quickly to get packets form A to B while an alternative regular route is calculated.

You can see that enabling FRR is a piece of cake. To verify you need to dig a little deeper. First let’s see the FRR coverage on R3:

darreno@M7i> show ospf backup coverage
Topology default coverage:

Node Coverage:

Area             Covered  Total  Percent
                   Nodes  Nodes  Covered
0.0.0.0                2      3   66.67%

Route Coverage:

Path Type  Covered   Total  Percent
            Routes  Routes  Covered
Intra            5      11   45.45%
Inter            0       0  100.00%
Ext1             0       0  100.00%
Ext2             0       0  100.00%
All              5      11   45.45%

Not every single prefix can be covered as it’s quite topology dependant. If we look into the detail for specifically 5.5.5.5:

darreno@M7i> show ospf backup spf detail | find 5.5.5.5
5.5.5.5
  Self to Destination Metric: 101
  Parent Node: 10.0.8.10
  Primary next-hop: fe-0/1/4.24 via 10.0.24.4
  Backup next-hop: fe-0/1/5.35 via 10.0.35.4
  Backup Neighbor: 4.4.4.4
    Neighbor to Destination Metric: 1, Neighbor to Self Metric: 1
    Self to Neighbor Metric: 100, Backup preference: 0x0
    Eligible, Reason: Contributes backup next-hop

Here we see that fe-0/1/4.24 is the primary and fe-0/1/5.35 is the backup. The backup is also eligible. If we take a look at the route itself:

darreno@M7i> show route 5.5.5.5

inet.0: 24 destinations, 25 routes (24 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

5.5.5.5/32         *[OSPF/10] 00:03:15, metric 101
                    > to 10.0.24.4 via fe-0/1/4.24
                      to 10.0.35.4 via fe-0/1/5.35

Both routes are there, but only the first will be used until it fails.

Finally we can take a look at the FIB entry:

darreno@M7i> show route forwarding-table destination 5.5.5.5
Routing table: default.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
5.5.5.5/32         user     1                    ulst 262142     5
                              10.0.24.4          ucst  1303     2 fe-0/1/4.24
                              10.0.35.4          ucst  1304     2 fe-0/1/5.35

The backup hop is already programmed ready to take over as soon as the primary fails.