Tag Archives: ospf

SPF Delay – CCDE

SPF timers are usually one of those things that engineers don’t bother with. Hello/Dead timers are often adjusted, but not actual SPF timers themselves.

Different vendors, and even different platforms within vendors, can have dramatically different timers. Micro-loops can be even more pronounced when different vendors/platforms are involved.

SPF Timers

In OSPF, SPF is only run when certain conditions are met. One of those conditions is when a router originates a new type-1 LSA. If a router interface goes down, it will originate a new type-1 to let other routers in the area know about it. How soon after the interface goes down does the type-1 get sent? Once another router in the area receives that type-1, does it run SPF straight away? Does it flood the LSA before or after it runs SPF?
Micro-loops form when router’s FIBs do not agree on where the best path is. Two routers will bounce a packet backwards and forwards to each other until those routers agree on the forwarding path and have that path installed in their FIB.

The best way to understand this is to show the loop forming.

Let’s consider the following topology of five routers. The OSPF costs of each link is also displayed:
SPF Timers SPF Delay   CCDE

Most router interfaces have a cost of 50, while R3 has a second slower link with a cost of 200.

Under normal circumstances, any traffic from R1 to R5 with go through R2-R4.
SPF Timers2 SPF Delay   CCDE

R1#traceroute 10.0.0.5
Type escape sequence to abort.
Tracing the route to 10.0.0.5
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.12.2 12 msec 32 msec 16 msec
  2 192.168.24.4 44 msec 56 msec 16 msec
  3 192.168.45.5 68 msec 48 msec 48 msec

When the link between R2 and R4 fails, traffic should traverse the R2-R3-R4 links:
SPF Timers3 SPF Delay   CCDE
There are a number of milliseconds where this will not be the case.

In order to show how a micro-loop is formed, I’ll first need to artificially increase my SPF timers. This is because it’s very difficult to show an actual micro-loop simply with traceroute.
On R3 I’ll increase the wait time to run SPF after it receives an LSA to 10 seconds:

R3(config)#router ospf 1
R3(config-router)# timers throttle spf 10000 10000 10000

I’ll now break the link between R2 and R4 and run another traceroute from R1 to R5:

R2(config)#int gi2/0
R2(config-if)#shut
R1#traceroute 10.0.0.5
Type escape sequence to abort.
Tracing the route to 10.0.0.5
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.12.2 16 msec 16 msec 12 msec
  2  *  *
    192.168.23.3 36 msec
  3 192.168.23.2 40 msec 36 msec 68 msec
  4 192.168.23.3 44 msec 60 msec 60 msec
  5 192.168.23.2 56 msec 64 msec 60 msec
  6 192.168.23.3 100 msec 80 msec 80 msec
  7 192.168.23.2 80 msec 80 msec 84 msec
  8 192.168.23.3 80 msec 104 msec 104 msec
  9 192.168.23.2 100 msec 104 msec 100 msec
 10 192.168.23.3 128 msec 124 msec 124 msec
 11 192.168.23.2 132 msec 116 msec 124 msec
 12 192.168.23.3 152 msec 148 msec 148 msec
 13 192.168.23.2 144 msec 144 msec 148 msec
 14 192.168.23.3 152 msec
    192.168.45.5 112 msec 84 msec

Because R3 is delaying it’s SPF run until 10 seconds after it receives a relevant LSA, it still assumes the best path is through R2. R2 has run it’s SPF and it assumes the best path is through R3. This is the reason the packet bounces between both routers. The packet get to it’s destination only when R3 has run SPF and CEF updated.

Of course in the real world we don’t wait 10 seconds. But what are the actual timers? That depends a lot on which vendor and platform you’re running:

Vendor OS Initial SPF Delay (ms)
Cisco IOS & IOS-XE 5000
Cisco IOS-XR 50
Cisco NX-OS 200
Juniper Junos 200

The above list is of course not exhaustive.

The timers between vendors and platforms can be dramatically different. Even in an environment in when you are not cared about rapid convergence, it’s still important that your IGP routers all agree on their timers. Connecting an ASR1k to an ASR9k with default timers could cause traffic to loop for almost five seconds if left to the defaults. I would suggest you ensure all OSPF routers in an area, or all IS-IS routers in the same level, have identical timers.

Another option is to ensure the initial SPF delay run timer is set high enough so that LSA/LSP reaches all edges of the area/level. That way all router can run SPF at the same time and update their FIBs at the same time. The problem with this approach is that each router receives the LSA at different times. Even if they did receive them at exactly the same time, we are relying on the fact that all routers have 100% identical SPF and FIB-Update run times.

Further Reading

RFC 5715 – A Framework for Loop-Free Convergence
RFC 6976 – Framework for Loop-Free Convergence Using the Ordered Forwarding Information Base (oFIB) Approach

Splitting a module from a python app

My OSPF checker is getting a bit big. The majority of the code is the function that parses the OSPF output and returns the required values.

I’d like to continue to refine what it can pull out. I’d also like to check non-IOS devices like Junos and IOS-XR output.

A function can very easily be moved into a new file and then called as a module. The great thing about this is that others can use the same module in different applications of their own. I can also create a separate module per OS that I’m interested in. Each can be edited separately.

The IOS OSPF checker has now been split into it’s own module like so:

import re
import sys

def ospf_information(i):
    int_list = {}
    ospf = re.split(r'[\n](?=GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI|Vlan|Virtual-Access)',i)
    print(ospf)
    for o in ospf:
        properties = {}
        interface =  re.search(r'(GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI|Vlan|Virtual-Access)[0-9]{1,4}/?[0-9]{0,4}.
?[0-9]{0,4}/?[0-9]{0,3}/?[0-9]{0,3}/?[0-9]{0,3}:?[0-9]{0,3}',o)
        if not interface:
            continue
        interface = interface.group()
        ip = re.search(r'(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/\d{1,2})',o)
        if not ip:
            ip = re.search(r'Interface is unnumbered. Using address of [a-zA-Z]{1,10}[0-9]{1,5}/?[0-9]{0,5}.?[0-9]{0,5}',o)
            properties['IP'] = ip.group()
        else:
            properties['IP'] = ip.group()
        a = re.search(r'Area ([\s]{0,3}[0-9]{1,5})',o)
        properties['Area'] = a.group(1)
        n = re.search(r'Network Type ([\s]{0,3}[a-zA-Z_]{0,20})',o)
        properties['Net'] = n.group(1)
        c = re.search(r'Cost: ([0-9]{1,5})',o)
        properties['Cost'] = c.group(1)
        s = re.search(r'line protocol is[\s]([a-zA-Z]{1,4})',o)
        properties['Status'] = s.group(1)
        p = re.search(r'Passive',o)
        if p:
            properties['Neigh'] = "Passive Interface"
            properties['Adj'] = None
        else:
            ne = re.search(r'(?:Neighbor Count is )([0-9]{1,3})',o)
            if not ne:
                properties['Neigh'] = None
            else:
                properties['Neigh'] = ne.group(1)
            ad = re.search(r'(?:Adjacent neighbor count is )([0-9]{1,3})',o)
            if not ad:
                properties['Adj'] = None
            else:
                properties['Adj'] = ad.group(1)
        h = re.search(r'Hello ([0-9]{1,3})',o)
        if not h:
            properties['Hello'] = None
        else:
            properties['Hello'] = h.group(1)
        d = re.search(r'Dead ([0-9]{1,3})',o)
        if not d:
            properties['Dead'] = None
        else:
            properties['Dead'] = d.group(1)
        int_list[interface]=properties
    return int_list

if __name__ == "__main__":
    f = open(sys.argv[1])
    info = f.read()
    f.close()
    ospf = ospf_information(info)
    print("This device contains "+str(len(ospf))+" ospf enabled interfaces")
    print(ospf)

A couple of things to note here. The module now returns a dictionary. This allows any app using this module to easily extract whatever values it chooses instead of iterating through a list. The last section of code allows me to run the module directly against some raw router output directly to pull information out. This part is not run if calling as a module.

In my main application I now simply import the module and change how I call it slightly:

import ospfios
 ospf_int = ospfios.ospf_information(output)

I’ve started a preliminary Junos OSPF module which will return similar values:

import re
import sys

def ospf_information(i):
    int_list = {}
    ospf = re.split(r'[\n](?=ge|fe|lo|ae|et|fxp)',i)
    for o in ospf:
        properties = {}
        interface =  re.search(r'(ge|fe|lo|ae|et|fxp)([0-9]?)([-]?){0,1}[0-9]{1,5}/?[0-9]{0,5}/?[0-9]{0,5}/?[0-9]?[.][0-9]{1,5}',o)
        if not interface:
            continue
        interface = interface.group()
        ip = re.search(r'Address: (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})',o)
        properties['IP'] = ip.group(1)
        c = re.search(r'Cost: ([0-9]{1,5})',o)
        properties['Cost'] = c.group(1)
        ad = re.search(r'(?:Adj count: )([0-9]{1,3})',o)
        properties['Adj'] = ad.group(1)
        h = re.search(r'Hello: ([0-9]{1,3})',o)
        properties['Hello'] = h.group(1)
        d = re.search(r'Dead: ([0-9]{1,3})',o)
        properties['Dead'] = d.group(1)
        int_list[interface]=properties
    return int_list

if __name__ == "__main__":
    f = open(sys.argv[1])
    info = f.read()
    f.close()
    ospf = ospf_information(info)
    print("This device contains "+str(len(ospf))+" ospf enabled interfaces")
    print(ospf)

A quick run directly on a small Junos box:

darreno@Jumpbox:~/git/ospf_checker$ python3 ospfjunos.py junos.txt
This device contains 4 ospf enabled interfaces
{'ge-1/3/0.641': {'IP': '10.11.31.227', 'Cost': '10', 'Adj': '1', 'Hello': '10', 'Dead': '40'}, 'lo0.0': {'IP': '10.11.225.224', 'Cost': '0', 'Adj': '0', 'Hello': '10', 'Dead': '40'}, 'ge-0/0/0.643': {'IP': '10.11.31.90', 'Cost': '10', 'Adj': '1', 'Hello': '10', 'Dead': '40'}, 'ge-0/2/0.644': {'IP': '10.11.31.94', 'Cost': '10', 'Adj': '1', 'Hello': '10', 'Dead': '40'}}

Splitting up your python app into multiple functions

I’ve been working on splitting my OSPF Checker into a few different functions. This has a few benefits which I’ve gone over before. I’ve split out logging into the device and capturing information into it’s own function. In future I’ll use this function to try SSH in first, and then telnetting in if that fails. I have a separate function that gets all my OSPF information.

Logging in:

def login(i):
    try:
        tn = telnetlib.Telnet(device,23,5)
        tn.read_until(b"Username: ")
        tn.write(user.encode('ascii') + b"\n")
        tn.read_until(b"Password: ")
        tn.write(password.encode('ascii') + b"\n")
        tn.write(b"\n")
        tn.write(b"terminal length 0\n")
        tn.write(b"show ver | include IOS\n")
        tn.write(b"show ip ospf interface\n")
        tn.write(b"exit\n")
        output=(tn.read_all().decode('ascii'))
        return output
    except:
        return None

OSPF Information:

def ospf_information(i):
    ospf_int = re.search(r'(GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI|Vlan|Virtual-Access)[0-9]{1,4}/?[0-9]{0,4}.?[0-9]{0,4}/?[0-9]{0,3}/?[0-9]{0,3}/?[0-9]{0,3}:?[0-9]{0,3}',i)
    if not ospf_int:
        return None
    ospf_int = ospf_int.group()
    ip = re.search(r'(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/\d{1,2})',i)
    ip = ip.group()
    if not ip:
        ip = re.search(r'Interface is unnumbered. Using address of [a-zA-Z]{1,10}[0-9]{1,5}/?[0-9]{0,5}.?[0-9]{0,5}',i)
        ip = ip.group()
    a = re.search(r'Area ([\s]{0,3}[0-9]{1,5})',i)
    area = a.group(1)
    n = re.search(r'Network Type ([\s]{0,3}[a-zA-Z_]{0,20})',i)
    net = n.group(1)
    c = re.search(r'Cost: ([0-9]{1,5})',i)
    cost = c.group(1)
    p = re.search(r'Passive',i)
    if p:
        neighbour = "Passive"
        adjacency = None
    else:
        ne = re.search(r'(?:Neighbor Count is )([0-9]{1,3})',i)
        if not ne:
            neighbour = None
        else:
            neighbour = ne.group(1)
        ad = re.search(r'(?:Adjacent neighbor count is )([0-9]{1,3})',i)
        if not ad:
            adjacency = None
        else:
            adjacency = ad.group(1)
    h = re.search(r'Hello ([0-9]{1,3})',i)
    if not h:
        hello = None
    else:
        hello = h.group(1)
    d = re.search(r'Dead ([0-9]{1,3})',i)
    if not d:
        dead = None
    else:
        dead = d.group(1)
    return (ospf_int,ip,area,net,cost,neighbour,adjacency,hello,dead)

The great thing about the above code is that if I want to get more OSPF information, I simply add it to the ospf_information function. If I wrote another app to get other information, I can use the rest and replace ospf_information with something else.

I want to do a bit more splitting, but I’m liking the way it works thus far!

Regex – Lookahead Assertions – Python

While trying to scrape information out of a router output, I realised that sometimes I just think about things in the wrong way. Basically I end up making my life difficult when there is usually an easier way to go.

Ultimately I’m not even happy with my result at the moment. I’m trying to work our better ways to get the info I need, but it’s a learning process so I’m happy with that.

Let’s start with this block of text:

cr1.123456#sh ip ospf interface
Loopback0 is up, line protocol is up
  Internet Address 10.90.3.38/32, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type LOOPBACK, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
  Loopback interface is treated as a stub Host
GigabitEthernet0/1 is up, line protocol is up
  Internet Address 172.16.13.150/29, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type BROADCAST, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
  Transmit Delay is 1 sec, State DR, Priority 1
  Designated Router (ID) 10.90.3.38, Interface address 172.16.13.150
  No backup designated router on this network
  Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
    oob-resync timeout 40
    No Hellos (Passive interface)
  Supports Link-local Signaling (LLS)
  Cisco NSF helper support enabled
  IETF NSF helper support enabled
  Index 3/3, flood queue length 0
  Next 0x0(0)/0x0(0)
  Last flood scan length is 0, maximum is 0
  Last flood scan time is 0 msec, maximum is 0 msec
  Neighbor Count is 0, Adjacent neighbor count is 0
  Suppress hello for 0 neighbor(s)
  Message digest authentication enabled
      No key configured, using default key id 0
GigabitEthernet0/0.2561 is up, line protocol is up
  Internet Address 10.22.0.117/30, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type POINT_TO_POINT, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
  Transmit Delay is 1 sec, State POINT_TO_POINT
  Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
    oob-resync timeout 40
    Hello due in 00:00:05
  Supports Link-local Signaling (LLS)
  Cisco NSF helper support enabled
  IETF NSF helper support enabled
  Index 1/1, flood queue length 0
  Next 0x0(0)/0x0(0)
  Last flood scan length is 1, maximum is 1
  Last flood scan time is 0 msec, maximum is 0 msec
  Neighbor Count is 1, Adjacent neighbor count is 1
    Adjacent with neighbor 217.196.224.51
  Suppress hello for 0 neighbor(s)
  Message digest authentication enabled
    Youngest key id is 1

I would like to extract information from the above text. I want my app to list each OSPF enabled interface, plus its properties.

Initially I had my script look for various expressions, and then iterate through to give me what I wanted. However there was a serious issue with this:

 i = re.findall('(?:GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI)[0-9]{1,4}/?[0-9]{0,4}.?[0-9]{0,4}/?[0-9]{0,3}/?[0-9]{0,3}/?[0-9]{0,3}:?[0-9]{0,3}',output)
ip = re.findall(r'(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})',output)
A = re.findall(r'Area[\s]{0,3}[0-9]{1,5}',output)
n = re.findall(r'Network Type[\s]{0,3}[a-zA-Z_]{0,20}',output)
C = re.findall(r'Cost:[\s]{0,3}[0-9]{1,5}',output)
h = re.findall(r'Hello[\s][0-9]{1,3}',output)
D = re.findall(r'Dead[\s][0-9]{1,3}',output)
for a,b,c,d,e,f,g in zip(i,ip,A,n,C,h,D):
    print("\n"+a)
    print(b)
    print(c)
    print(d)
    print(e)
    print(f)
    print(g)

The problem with the above code is that Loopback interface don’t have hello and dead timers. This meant the output scraped ended up showing the first capture of a Hello: being matched with the Loopback0 interface. It also meant the last few interfaces were not shown. Basically the output was completely wrong:

Darrens-MacBook-Pro:Desktop darrenoconnor$ python3 1.py

Loopback0
10.90.3.38
Area 30395
Network Type LOOPBACK
Cost: 1
Hello 10
Dead 40

GigabitEthernet0/1
10.90.3.38
Area 30395
Network Type BROADCAST
Cost: 1
Hello 10
Dead 40

My first thought was to try and split the output into three separate strings. Each interface would be the start of the string and I could then search in the smaller string. Essentially like this:

cr1.123456#sh ip ospf interface

======== Begin String One ==========
Loopback0 is up, line protocol is up
  Internet Address 10.90.3.38/32, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type LOOPBACK, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
  Loopback interface is treated as a stub Host
========= End String One ===========

======== Begin String Two ==========
GigabitEthernet0/1 is up, line protocol is up
  Internet Address 172.16.13.150/29, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type BROADCAST, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
========= End String Two ===========

======== Begin String ETC ==========

At first this turned out to be a lot hardher than I thought it would be. Initially I tried to count the lines in which the interface name appeared. I would then split from 0 to the first point, then from the first point to the second point. this turned out to be a lot more difficult than I thought. Counting the lines where the interface names appeared was easy:

int = re.compile(r'GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI')
for (i,line) in enumerate(output):
   if int.match(line):
      for match in int.findall(line):
         print (i,line)

This gives me:

Darrens-MacBook-Pro:Desktop darrenoconnor$ python3 1.py
1 Loopback0 is up, line protocol is up

7 GigabitEthernet0/1 is up, line protocol is up

29 GigabitEthernet0/0.2561 is up, line protocol is up

But thinking about this, this simply cannot be very efficient. Get some output, convert to string, convert to list, convert back to string based on line numbers? Also Python doesn’t seem to make it very easy to create strings based on lines numbers of a list. At least I haven’t found.

So after trying for a few hours to get the above to work, I though there simply had to be a better way. Why not read the router output into a string, and then split that string based on the first occurence of the interface name. That should give me separte strings for each interface.

output_split = re.split(r'GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI', output)
print(len(output_split))
for i in output_split:
    print(i)

However this creates a big issue. When splitting on a matching regex group, the part matched will be removed from the string. I did get the right amount of splits at least. I would expect a list length of 4 as there are three interfaces as well as the first line of show ip ospf interfaces:

Darrens-MacBook-Pro:Desktop darrenoconnor$ python3 1.py
4

However look at my actual output:

 is up, line protocol is up
  Internet Address 10.90.3.38/32, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type LOOPBACK, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base

0/1 is up, line protocol is up
  Internet Address 172.16.13.150/29, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type BROADCAST, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name

The interface names matched are now gone!

Python’s re module allows you to do a lookahead assertion. Basically this means it can look forward and match a regex, but the matched regex is NOT removed when doing the split. This is exactly what we want.

Now I’ll match on a newline followed immidiately by my interface name. I’ll split on the newline, but do a lookahead assertion on the interface name itself. To do a lookahead assertion you match the lookahead with (?=) I still want to remote the newline so that will come before the lookahead. For now I’ll also add a couple of * and newlines so you can see where the split occurs when printed out.

output_split = re.split(r'\n(?=GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI)', output)
print(len(output_split))
for i in output_split:
    print("\n******\n"+i)

I should now have a list length of 4, and my interface names intact:

Darrens-MacBook-Pro:Desktop darrenoconnor$ python3 1.py
4

******
cr1.123456#sh ip ospf interface

******
Loopback0 is up, line protocol is up
  Internet Address 10.90.3.38/32, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type LOOPBACK, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
  Loopback interface is treated as a stub Host

******
GigabitEthernet0/1 is up, line protocol is up
  Internet Address 172.16.13.150/29, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type BROADCAST, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base

The rest of the app is now easy. I can iterate through my split list. If no hello/dead timer is found in the loopback split, it doesn’t add it to the loopback section. My interfaces will now all be shown with the correct values.

Ultimately after cleaning up some outputs my output now gives me:

Darrens-MacBook-Pro:Desktop darrenoconnor$ python3 1.py

Int:	Loopback0
IP:	10.90.3.38
Area:	30395
Type:	LOOPBACK
Cost	1

Int:	GigabitEthernet0/1
IP:	172.16.13.150
Area:	30395
Type:	BROADCAST
Cost	1
Hello:	10
Dead:	40

Int:	GigabitEthernet0/0.2561
IP:	10.22.0.117
Area:	30395
Type:	POINT_TO_POINT
Cost	1
Hello:	10
Dead:	40

I’ve moved the above script now to github

There are still a large amount of changes I’d like to make. The next step is to move the regular expressions to some definitions. For now it’s doing what I want it to do.

OSPF as the PE-CE routing protocols deep dive – Part 3 of 3 – Loop Prevention

Read part 1
Read part 2
Read part 3

 
When customer sites are single-homed, there is no possibility of a loop forming, unless of course your customer decides to set up a bunch of GRE tunnels and run OSPF over that, but I digress. If a site is multi-homed, or two sites have a back-door between them, it’s essential that route from BGP going into OSPF, do not go back into BGP.

Let’s create a slightly different diagram for this one. R3 is now also a PE router:
loop ospf OSPF as the PE CE routing protocols deep dive – Part 3 of 3 – Loop Prevention

The loop prevention used ultimately depends on whether a prefix comes in as internal or external. If a sham-link is configured and all OSPF routes are intra-area, no loop prevention is needed. Standard SPF is run everything is fine. This is because everything is seen in area 0, and SPF can run with full knowledge of the entire area.

As soon as type3s and type5s are used, OSPF becomes a little more distance vector like. ABRs/ASBRs originate new LSAs and other OSPF router believe what is told to them. This makes is possible for loops to appear when multual redistribution is occuring.

The down bit

Let’s go back to RFC 4577, specifically section 4.2.5.1

When a type 3 LSA is sent from a PE router to a CE router, the DN bit [OSPF-DN] in the LSA Options field MUST be set. This is used to ensure that if any CE router sends this type 3 LSA to a PE router, the PE router will not redistribute it further.

When a PE router needs to distribute to a CE router a route that comes from a site outside the latter’s OSPF domain, the PE router presents itself as an ASBR (Autonomous System Border Router), and distributes the route in a type 5 LSA. The DN bit [OSPF-DN] MUST be set in these LSAs to ensure that they will be ignored by any other PE routers that receive them.

There are deployed implementations that do not set the DN bit, but instead use OSPF route tagging to ensure that a type 5 LSA generated by a PE router will be ignored by any other PE router that may receive it. A special OSPF route tag, which we will call the VPN Route Tag (see Section 4.2.5.2), is used for this purpose. To ensure backward compatibility, all implementations adhering to this specification MUST by default support the VPN Route Tag procedures specified in Sections 4.2.5.2, 4.2.8.1, and 4.2.8.2. When it is no longer necessary to use the VPN Route Tag in a particular deployment, its use (both sending and receiving) may be disabled by configuration.

Essentially, if an LSA arrives at a PE with the down bit set, that will never be redistributed into BGP. This prevents the route from leaking in from one PE back into another PE.

Down Bit – IOS

R7 is advertising it’s loopback address. No sham-links are used and so R4 will originate a type3 LSA to R6:

R6#show ip ospf database summary 7.7.7.7  adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Summary Net Link States (Area 0)

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 441
  Options: (No TOS-capability, DC, Downward)
  LS Type: Summary Links(Network)
  Link State ID: 7.7.7.7 (summary Network Number)
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000003
  Checksum: 0x5636
  Length: 28
  Network Mask: /32
        MTID: 0         Metric: 2

Options state ‘Downward’ – This LSA is flooded to R6 -> R5 -> R3. R3, another PE, will have the LSA (all databases need to match remember) but it will not use the LSA. The routing bit will not be set, and it will not redistribute that into BGP either:

R3#  show ip ospf database summary 7.7.7.7  adv-router 4.4.4.4

            OSPF Router with ID (10.0.35.3) (Process ID 1)

                Summary Net Link States (Area 0)

  LS age: 597
  Options: (No TOS-capability, DC, Downward)
  LS Type: Summary Links(Network)
  Link State ID: 7.7.7.7 (summary Network Number)
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000003
  Checksum: 0x5636
  Length: 28
  Network Mask: /32
        MTID: 0         Metric: 2

The same happens vice-versa. Any LSA originated by R3 to R5, will be received but not used by R4.
loop ospf2 OSPF as the PE CE routing protocols deep dive – Part 3 of 3 – Loop Prevention

Down Bit – IOS-XR

No change in IOS-XR behaviour. You need to be sure your domain-ids match to get a type3 between IOS and IOS-XE:

R6#sh ip ospf database summary 7.7.7.7 adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Summary Net Link States (Area 0)

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 20
  Options: (No TOS-capability, DC, Downward)
  LS Type: Summary Links(Network)
  Link State ID: 7.7.7.7 (summary Network Number)
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0x5A34
  Length: 28
  Network Mask: /32
        MTID: 0         Metric: 2

Down bit set on the type3.

Route tags – IOS

Let’s go back to the RFC to see what this is all about. Section 4.2.5.2

If a particular VRF in a PE is associated with an instance of OSPF, then by default it MUST be configured with a special OSPF route tag value, which we call the VPN Route Tag. By default, this route tag MUST be included in the Type 5 LSAs that the PE originates (as the result of receiving a BGP-distributed VPN-IPv4 route, see Section 4.2.8) and sends to any of the attached CEs.

The configuration and inclusion of the VPN Route Tag is required for backward compatibility with deployed implementations that do not set the DN bit in type 5 LSAs. The inclusion of the VPN Route Tag may be disabled by configuration if it has been determined that it is no longer needed for backward compatibility.

The value of the VPN Route Tag is arbitrary but must be distinct from any OSPF Route Tag being used within the OSPF domain. Its value MUST therefore be configurable. If the Autonomous System number of the VPN backbone is two bytes long, the default value SHOULD be an automatically computed tag based on that Autonomous System number

If the Autonomous System number is four bytes long, then a Route Tag value MUST be configured, and it MUST be distinct from any Route Tag used within the VPN itself.

If a PE router needs to use OSPF to distribute to a CE router a route that comes from a site outside the CE router’s OSPF domain, the PE router SHOULD present itself to the CE router as an Autonomous System Border Router (ASBR) and SHOULD report such routes as AS-external routes. That is, these PE routers originate Type 5 LSAs reporting the extra-domain routes as AS-external routes. Each such Type 5 LSA MUST contain an OSPF route tag whose value is that of the VPN Route Tag. This tag identifies the route as having come from a PE router. The VPN Route Tag MUST be used to ensure that a Type 5 LSA originated by a PE router is not redistributed through the OSPF area to another PE router.

Note that it says the OSPF should set a route-tag when the implementation doesn’t support setting the down bit in type5 LSAs. Also note in the previous RFC quote that it did note an implementation could set the down bit in type5s if desired. At this point I’ve stopped advertising R7′s loopback directly into OSPF and simply redistributed the loopback. This ensures that the LSA is external.

Usually when an ASBR originates a type5, that type5 remains unchanged in the domain. i.e. the originating router is the same. However according to the quote above, the PE need to originate a new type5 to the attached CE. This we see on R6:

R6#show ip ospf database external 7.7.7.7  adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 38
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0x77C7
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 3489661028

Notice no down bit. Also note the originator of this type5 is R4 itself. Finally the route has an external route tag of 3489661028

Much like the down bit, if a PE router receives an external LSA with a domain tag that matches it’s own, that LSA will not be used or redistributed
loop ospf31 OSPF as the PE CE routing protocols deep dive – Part 3 of 3 – Loop Prevention

R3#show ip ospf 1 database external 7.7.7.7 adv-router 4.4.4.4

            OSPF Router with ID (10.0.35.3) (Process ID 1)

                Type-5 AS External Link States

  LS age: 744
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0x77C7
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 3489661028

No routing bit set, no redistribution happening.

Route tags – IOS-XR

R6#sh ip ospf database external 7.7.7.7 adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 11
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0xEFCE
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 3489661028

IOS-XR and IOS have the same behaviour.

IOS – 32bit AS number – Route-tag

The RFC states that when using 16bit AS numbers, the domain tag is automatically derived. When using a 32bit AS number, it should be manually configured. You are able to manually set this even when using a 16bit number with the domain-tag command. You can see above that when using a 16bit number it was automatic. Let’s move to a 32bit number and see what we see.
A quick change of the BGP sessions:

R4#sh run | sec router bgp
router bgp 4294967295
 no bgp default ipv4-unicast
 bgp log-neighbor-changes
 neighbor 2.2.2.2 remote-as 4294967295
 neighbor 2.2.2.2 update-source Loopback0
 neighbor 3.3.3.3 remote-as 4294967295
 neighbor 3.3.3.3 update-source Loopback0

Take a look at the type5 on R6. The domain-tag matches the 32bit AS number directly. This is not 100% confirming to the RFC which states it should be manually set:

R6#sh ip ospf database external 7.7.7.7 adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 76
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0x2C48
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 4294967295

Of course, R3 will not use that LSA as it’s domain-tag matches.

Considering the domain-tag matches, it stands to reason that any inter-AS VPN using OSPF would be susceptible to routing loops as each SP will have a different domain-tag. One of them could manually set it to match the other.

32bit AS number – Route-tag – IOS-XR

IOS-XR’s 32bit external behaviour is identical to IOS:

R6#sh ip ospf database external 7.7.7.7 adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 76
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0xA44F
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 4294967295

Once again, IOS and IOS-XR have the same behaviour.

Notes

  • Unlike parts 1 and 2 of this blog, IOS and IOS-XR finally show identical behaviour when it comes to loop prevention.