Tag Archives: cisco

Splitting up your python app into multiple functions

I’ve been working on splitting my OSPF Checker into a few different functions. This has a few benefits which I’ve gone over before. I’ve split out logging into the device and capturing information into it’s own function. In future I’ll use this function to try SSH in first, and then telnetting in if that fails. I have a separate function that gets all my OSPF information.

Logging in:

def login(i):
    try:
        tn = telnetlib.Telnet(device,23,5)
        tn.read_until(b"Username: ")
        tn.write(user.encode('ascii') + b"\n")
        tn.read_until(b"Password: ")
        tn.write(password.encode('ascii') + b"\n")
        tn.write(b"\n")
        tn.write(b"terminal length 0\n")
        tn.write(b"show ver | include IOS\n")
        tn.write(b"show ip ospf interface\n")
        tn.write(b"exit\n")
        output=(tn.read_all().decode('ascii'))
        return output
    except:
        return None

OSPF Information:

def ospf_information(i):
    ospf_int = re.search(r'(GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI|Vlan|Virtual-Access)[0-9]{1,4}/?[0-9]{0,4}.?[0-9]{0,4}/?[0-9]{0,3}/?[0-9]{0,3}/?[0-9]{0,3}:?[0-9]{0,3}',i)
    if not ospf_int:
        return None
    ospf_int = ospf_int.group()
    ip = re.search(r'(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/\d{1,2})',i)
    ip = ip.group()
    if not ip:
        ip = re.search(r'Interface is unnumbered. Using address of [a-zA-Z]{1,10}[0-9]{1,5}/?[0-9]{0,5}.?[0-9]{0,5}',i)
        ip = ip.group()
    a = re.search(r'Area ([\s]{0,3}[0-9]{1,5})',i)
    area = a.group(1)
    n = re.search(r'Network Type ([\s]{0,3}[a-zA-Z_]{0,20})',i)
    net = n.group(1)
    c = re.search(r'Cost: ([0-9]{1,5})',i)
    cost = c.group(1)
    p = re.search(r'Passive',i)
    if p:
        neighbour = "Passive"
        adjacency = None
    else:
        ne = re.search(r'(?:Neighbor Count is )([0-9]{1,3})',i)
        if not ne:
            neighbour = None
        else:
            neighbour = ne.group(1)
        ad = re.search(r'(?:Adjacent neighbor count is )([0-9]{1,3})',i)
        if not ad:
            adjacency = None
        else:
            adjacency = ad.group(1)
    h = re.search(r'Hello ([0-9]{1,3})',i)
    if not h:
        hello = None
    else:
        hello = h.group(1)
    d = re.search(r'Dead ([0-9]{1,3})',i)
    if not d:
        dead = None
    else:
        dead = d.group(1)
    return (ospf_int,ip,area,net,cost,neighbour,adjacency,hello,dead)

The great thing about the above code is that if I want to get more OSPF information, I simply add it to the ospf_information function. If I wrote another app to get other information, I can use the rest and replace ospf_information with something else.

I want to do a bit more splitting, but I’m liking the way it works thus far!

Regex – Lookahead Assertions – Python

While trying to scrape information out of a router output, I realised that sometimes I just think about things in the wrong way. Basically I end up making my life difficult when there is usually an easier way to go.

Ultimately I’m not even happy with my result at the moment. I’m trying to work our better ways to get the info I need, but it’s a learning process so I’m happy with that.

Let’s start with this block of text:

cr1.123456#sh ip ospf interface
Loopback0 is up, line protocol is up
  Internet Address 10.90.3.38/32, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type LOOPBACK, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
  Loopback interface is treated as a stub Host
GigabitEthernet0/1 is up, line protocol is up
  Internet Address 172.16.13.150/29, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type BROADCAST, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
  Transmit Delay is 1 sec, State DR, Priority 1
  Designated Router (ID) 10.90.3.38, Interface address 172.16.13.150
  No backup designated router on this network
  Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
    oob-resync timeout 40
    No Hellos (Passive interface)
  Supports Link-local Signaling (LLS)
  Cisco NSF helper support enabled
  IETF NSF helper support enabled
  Index 3/3, flood queue length 0
  Next 0x0(0)/0x0(0)
  Last flood scan length is 0, maximum is 0
  Last flood scan time is 0 msec, maximum is 0 msec
  Neighbor Count is 0, Adjacent neighbor count is 0
  Suppress hello for 0 neighbor(s)
  Message digest authentication enabled
      No key configured, using default key id 0
GigabitEthernet0/0.2561 is up, line protocol is up
  Internet Address 10.22.0.117/30, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type POINT_TO_POINT, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
  Transmit Delay is 1 sec, State POINT_TO_POINT
  Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
    oob-resync timeout 40
    Hello due in 00:00:05
  Supports Link-local Signaling (LLS)
  Cisco NSF helper support enabled
  IETF NSF helper support enabled
  Index 1/1, flood queue length 0
  Next 0x0(0)/0x0(0)
  Last flood scan length is 1, maximum is 1
  Last flood scan time is 0 msec, maximum is 0 msec
  Neighbor Count is 1, Adjacent neighbor count is 1
    Adjacent with neighbor 217.196.224.51
  Suppress hello for 0 neighbor(s)
  Message digest authentication enabled
    Youngest key id is 1

I would like to extract information from the above text. I want my app to list each OSPF enabled interface, plus its properties.

Initially I had my script look for various expressions, and then iterate through to give me what I wanted. However there was a serious issue with this:

 i = re.findall('(?:GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI)[0-9]{1,4}/?[0-9]{0,4}.?[0-9]{0,4}/?[0-9]{0,3}/?[0-9]{0,3}/?[0-9]{0,3}:?[0-9]{0,3}',output)
ip = re.findall(r'(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})',output)
A = re.findall(r'Area[\s]{0,3}[0-9]{1,5}',output)
n = re.findall(r'Network Type[\s]{0,3}[a-zA-Z_]{0,20}',output)
C = re.findall(r'Cost:[\s]{0,3}[0-9]{1,5}',output)
h = re.findall(r'Hello[\s][0-9]{1,3}',output)
D = re.findall(r'Dead[\s][0-9]{1,3}',output)
for a,b,c,d,e,f,g in zip(i,ip,A,n,C,h,D):
    print("\n"+a)
    print(b)
    print(c)
    print(d)
    print(e)
    print(f)
    print(g)

The problem with the above code is that Loopback interface don’t have hello and dead timers. This meant the output scraped ended up showing the first capture of a Hello: being matched with the Loopback0 interface. It also meant the last few interfaces were not shown. Basically the output was completely wrong:

Darrens-MacBook-Pro:Desktop darrenoconnor$ python3 1.py

Loopback0
10.90.3.38
Area 30395
Network Type LOOPBACK
Cost: 1
Hello 10
Dead 40

GigabitEthernet0/1
10.90.3.38
Area 30395
Network Type BROADCAST
Cost: 1
Hello 10
Dead 40

My first thought was to try and split the output into three separate strings. Each interface would be the start of the string and I could then search in the smaller string. Essentially like this:

cr1.123456#sh ip ospf interface

======== Begin String One ==========
Loopback0 is up, line protocol is up
  Internet Address 10.90.3.38/32, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type LOOPBACK, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
  Loopback interface is treated as a stub Host
========= End String One ===========

======== Begin String Two ==========
GigabitEthernet0/1 is up, line protocol is up
  Internet Address 172.16.13.150/29, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type BROADCAST, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
========= End String Two ===========

======== Begin String ETC ==========

At first this turned out to be a lot hardher than I thought it would be. Initially I tried to count the lines in which the interface name appeared. I would then split from 0 to the first point, then from the first point to the second point. this turned out to be a lot more difficult than I thought. Counting the lines where the interface names appeared was easy:

int = re.compile(r'GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI')
for (i,line) in enumerate(output):
   if int.match(line):
      for match in int.findall(line):
         print (i,line)

This gives me:

Darrens-MacBook-Pro:Desktop darrenoconnor$ python3 1.py
1 Loopback0 is up, line protocol is up

7 GigabitEthernet0/1 is up, line protocol is up

29 GigabitEthernet0/0.2561 is up, line protocol is up

But thinking about this, this simply cannot be very efficient. Get some output, convert to string, convert to list, convert back to string based on line numbers? Also Python doesn’t seem to make it very easy to create strings based on lines numbers of a list. At least I haven’t found.

So after trying for a few hours to get the above to work, I though there simply had to be a better way. Why not read the router output into a string, and then split that string based on the first occurence of the interface name. That should give me separte strings for each interface.

output_split = re.split(r'GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI', output)
print(len(output_split))
for i in output_split:
    print(i)

However this creates a big issue. When splitting on a matching regex group, the part matched will be removed from the string. I did get the right amount of splits at least. I would expect a list length of 4 as there are three interfaces as well as the first line of show ip ospf interfaces:

Darrens-MacBook-Pro:Desktop darrenoconnor$ python3 1.py
4

However look at my actual output:

 is up, line protocol is up
  Internet Address 10.90.3.38/32, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type LOOPBACK, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base

0/1 is up, line protocol is up
  Internet Address 172.16.13.150/29, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type BROADCAST, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name

The interface names matched are now gone!

Python’s re module allows you to do a lookahead assertion. Basically this means it can look forward and match a regex, but the matched regex is NOT removed when doing the split. This is exactly what we want.

Now I’ll match on a newline followed immidiately by my interface name. I’ll split on the newline, but do a lookahead assertion on the interface name itself. To do a lookahead assertion you match the lookahead with (?=) I still want to remote the newline so that will come before the lookahead. For now I’ll also add a couple of * and newlines so you can see where the split occurs when printed out.

output_split = re.split(r'\n(?=GigabitEthernet|FastEthernet|Serial|Tunnel|Loopback|Dialer|BVI)', output)
print(len(output_split))
for i in output_split:
    print("\n******\n"+i)

I should now have a list length of 4, and my interface names intact:

Darrens-MacBook-Pro:Desktop darrenoconnor$ python3 1.py
4

******
cr1.123456#sh ip ospf interface

******
Loopback0 is up, line protocol is up
  Internet Address 10.90.3.38/32, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type LOOPBACK, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base
  Loopback interface is treated as a stub Host

******
GigabitEthernet0/1 is up, line protocol is up
  Internet Address 172.16.13.150/29, Area 303953, Attached via Network Statement
  Process ID 1, Router ID 10.90.3.38, Network Type BROADCAST, Cost: 1
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           1         no          no            Base

The rest of the app is now easy. I can iterate through my split list. If no hello/dead timer is found in the loopback split, it doesn’t add it to the loopback section. My interfaces will now all be shown with the correct values.

Ultimately after cleaning up some outputs my output now gives me:

Darrens-MacBook-Pro:Desktop darrenoconnor$ python3 1.py

Int:	Loopback0
IP:	10.90.3.38
Area:	30395
Type:	LOOPBACK
Cost	1

Int:	GigabitEthernet0/1
IP:	172.16.13.150
Area:	30395
Type:	BROADCAST
Cost	1
Hello:	10
Dead:	40

Int:	GigabitEthernet0/0.2561
IP:	10.22.0.117
Area:	30395
Type:	POINT_TO_POINT
Cost	1
Hello:	10
Dead:	40

I’ve moved the above script now to github

There are still a large amount of changes I’d like to make. The next step is to move the regular expressions to some definitions. For now it’s doing what I want it to do.

OSPF as the PE-CE routing protocols deep dive – Part 3 of 3 – Loop Prevention

Read part 1
Read part 2
Read part 3

 
When customer sites are single-homed, there is no possibility of a loop forming, unless of course your customer decides to set up a bunch of GRE tunnels and run OSPF over that, but I digress. If a site is multi-homed, or two sites have a back-door between them, it’s essential that route from BGP going into OSPF, do not go back into BGP.

Let’s create a slightly different diagram for this one. R3 is now also a PE router:
loop ospf OSPF as the PE CE routing protocols deep dive – Part 3 of 3 – Loop Prevention

The loop prevention used ultimately depends on whether a prefix comes in as internal or external. If a sham-link is configured and all OSPF routes are intra-area, no loop prevention is needed. Standard SPF is run everything is fine. This is because everything is seen in area 0, and SPF can run with full knowledge of the entire area.

As soon as type3s and type5s are used, OSPF becomes a little more distance vector like. ABRs/ASBRs originate new LSAs and other OSPF router believe what is told to them. This makes is possible for loops to appear when multual redistribution is occuring.

The down bit

Let’s go back to RFC 4577, specifically section 4.2.5.1

When a type 3 LSA is sent from a PE router to a CE router, the DN bit [OSPF-DN] in the LSA Options field MUST be set. This is used to ensure that if any CE router sends this type 3 LSA to a PE router, the PE router will not redistribute it further.

When a PE router needs to distribute to a CE router a route that comes from a site outside the latter’s OSPF domain, the PE router presents itself as an ASBR (Autonomous System Border Router), and distributes the route in a type 5 LSA. The DN bit [OSPF-DN] MUST be set in these LSAs to ensure that they will be ignored by any other PE routers that receive them.

There are deployed implementations that do not set the DN bit, but instead use OSPF route tagging to ensure that a type 5 LSA generated by a PE router will be ignored by any other PE router that may receive it. A special OSPF route tag, which we will call the VPN Route Tag (see Section 4.2.5.2), is used for this purpose. To ensure backward compatibility, all implementations adhering to this specification MUST by default support the VPN Route Tag procedures specified in Sections 4.2.5.2, 4.2.8.1, and 4.2.8.2. When it is no longer necessary to use the VPN Route Tag in a particular deployment, its use (both sending and receiving) may be disabled by configuration.

Essentially, if an LSA arrives at a PE with the down bit set, that will never be redistributed into BGP. This prevents the route from leaking in from one PE back into another PE.

Down Bit – IOS

R7 is advertising it’s loopback address. No sham-links are used and so R4 will originate a type3 LSA to R6:

R6#show ip ospf database summary 7.7.7.7  adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Summary Net Link States (Area 0)

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 441
  Options: (No TOS-capability, DC, Downward)
  LS Type: Summary Links(Network)
  Link State ID: 7.7.7.7 (summary Network Number)
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000003
  Checksum: 0x5636
  Length: 28
  Network Mask: /32
        MTID: 0         Metric: 2

Options state ‘Downward’ – This LSA is flooded to R6 -> R5 -> R3. R3, another PE, will have the LSA (all databases need to match remember) but it will not use the LSA. The routing bit will not be set, and it will not redistribute that into BGP either:

R3#  show ip ospf database summary 7.7.7.7  adv-router 4.4.4.4

            OSPF Router with ID (10.0.35.3) (Process ID 1)

                Summary Net Link States (Area 0)

  LS age: 597
  Options: (No TOS-capability, DC, Downward)
  LS Type: Summary Links(Network)
  Link State ID: 7.7.7.7 (summary Network Number)
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000003
  Checksum: 0x5636
  Length: 28
  Network Mask: /32
        MTID: 0         Metric: 2

The same happens vice-versa. Any LSA originated by R3 to R5, will be received but not used by R4.
loop ospf2 OSPF as the PE CE routing protocols deep dive – Part 3 of 3 – Loop Prevention

Down Bit – IOS-XR

No change in IOS-XR behaviour. You need to be sure your domain-ids match to get a type3 between IOS and IOS-XE:

R6#sh ip ospf database summary 7.7.7.7 adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Summary Net Link States (Area 0)

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 20
  Options: (No TOS-capability, DC, Downward)
  LS Type: Summary Links(Network)
  Link State ID: 7.7.7.7 (summary Network Number)
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0x5A34
  Length: 28
  Network Mask: /32
        MTID: 0         Metric: 2

Down bit set on the type3.

Route tags – IOS

Let’s go back to the RFC to see what this is all about. Section 4.2.5.2

If a particular VRF in a PE is associated with an instance of OSPF, then by default it MUST be configured with a special OSPF route tag value, which we call the VPN Route Tag. By default, this route tag MUST be included in the Type 5 LSAs that the PE originates (as the result of receiving a BGP-distributed VPN-IPv4 route, see Section 4.2.8) and sends to any of the attached CEs.

The configuration and inclusion of the VPN Route Tag is required for backward compatibility with deployed implementations that do not set the DN bit in type 5 LSAs. The inclusion of the VPN Route Tag may be disabled by configuration if it has been determined that it is no longer needed for backward compatibility.

The value of the VPN Route Tag is arbitrary but must be distinct from any OSPF Route Tag being used within the OSPF domain. Its value MUST therefore be configurable. If the Autonomous System number of the VPN backbone is two bytes long, the default value SHOULD be an automatically computed tag based on that Autonomous System number

If the Autonomous System number is four bytes long, then a Route Tag value MUST be configured, and it MUST be distinct from any Route Tag used within the VPN itself.

If a PE router needs to use OSPF to distribute to a CE router a route that comes from a site outside the CE router’s OSPF domain, the PE router SHOULD present itself to the CE router as an Autonomous System Border Router (ASBR) and SHOULD report such routes as AS-external routes. That is, these PE routers originate Type 5 LSAs reporting the extra-domain routes as AS-external routes. Each such Type 5 LSA MUST contain an OSPF route tag whose value is that of the VPN Route Tag. This tag identifies the route as having come from a PE router. The VPN Route Tag MUST be used to ensure that a Type 5 LSA originated by a PE router is not redistributed through the OSPF area to another PE router.

Note that it says the OSPF should set a route-tag when the implementation doesn’t support setting the down bit in type5 LSAs. Also note in the previous RFC quote that it did note an implementation could set the down bit in type5s if desired. At this point I’ve stopped advertising R7′s loopback directly into OSPF and simply redistributed the loopback. This ensures that the LSA is external.

Usually when an ASBR originates a type5, that type5 remains unchanged in the domain. i.e. the originating router is the same. However according to the quote above, the PE need to originate a new type5 to the attached CE. This we see on R6:

R6#show ip ospf database external 7.7.7.7  adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 38
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0x77C7
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 3489661028

Notice no down bit. Also note the originator of this type5 is R4 itself. Finally the route has an external route tag of 3489661028

Much like the down bit, if a PE router receives an external LSA with a domain tag that matches it’s own, that LSA will not be used or redistributed
loop ospf31 OSPF as the PE CE routing protocols deep dive – Part 3 of 3 – Loop Prevention

R3#show ip ospf 1 database external 7.7.7.7 adv-router 4.4.4.4

            OSPF Router with ID (10.0.35.3) (Process ID 1)

                Type-5 AS External Link States

  LS age: 744
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0x77C7
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 3489661028

No routing bit set, no redistribution happening.

Route tags – IOS-XR

R6#sh ip ospf database external 7.7.7.7 adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 11
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0xEFCE
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 3489661028

IOS-XR and IOS have the same behaviour.

IOS – 32bit AS number – Route-tag

The RFC states that when using 16bit AS numbers, the domain tag is automatically derived. When using a 32bit AS number, it should be manually configured. You are able to manually set this even when using a 16bit number with the domain-tag command. You can see above that when using a 16bit number it was automatic. Let’s move to a 32bit number and see what we see.
A quick change of the BGP sessions:

R4#sh run | sec router bgp
router bgp 4294967295
 no bgp default ipv4-unicast
 bgp log-neighbor-changes
 neighbor 2.2.2.2 remote-as 4294967295
 neighbor 2.2.2.2 update-source Loopback0
 neighbor 3.3.3.3 remote-as 4294967295
 neighbor 3.3.3.3 update-source Loopback0

Take a look at the type5 on R6. The domain-tag matches the 32bit AS number directly. This is not 100% confirming to the RFC which states it should be manually set:

R6#sh ip ospf database external 7.7.7.7 adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 76
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0x2C48
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 4294967295

Of course, R3 will not use that LSA as it’s domain-tag matches.

Considering the domain-tag matches, it stands to reason that any inter-AS VPN using OSPF would be susceptible to routing loops as each SP will have a different domain-tag. One of them could manually set it to match the other.

32bit AS number – Route-tag – IOS-XR

IOS-XR’s 32bit external behaviour is identical to IOS:

R6#sh ip ospf database external 7.7.7.7 adv-router 4.4.4.4

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 76
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0xA44F
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 4294967295

Once again, IOS and IOS-XR have the same behaviour.

Notes

  • Unlike parts 1 and 2 of this blog, IOS and IOS-XR finally show identical behaviour when it comes to loop prevention.

Cisco LNS-LAC basic configuration

I’ve been testing some vpdn features and thought I would put the topology up here for others to play with.
Click the image to see full size.
LAC LNS2 300x135 Cisco LNS LAC basic configuration
R1 and R2 are acting as P routers. R10 is the LNS controlled by you, the ISP. R5 and R6 and LACs located in the carrier’s network (BT in my example) R8 and R9 are customer devices connected to their local exchange.

I’ll show some relevant relevant configuration on the basics of the CPE, LAC and LNS. This won’t be an in depth post. Also there are many ways to configure this, I’m simply showing a quick and easy working config for you to play with.

CPE

Customer config is standard config. It will submit a username and password over PPP. It will also create a default route to the PPP interface:

interface Serial1/0
 ip address negotiated
 encapsulation ppp
 ppp pap sent-username [email protected] password 0 password
!
ip route 0.0.0.0 0.0.0.0 Serial1/0

LAC

This will be the device that will be forming l2tp tunnels to your LNS. In order for these l2tp tunnels to come up your LNS and LAC will need to have ip reachability to each other. In this network they are on the same subnet. Generally you would use BGP for this.

aaa new-model
!
vpdn enable
vpdn search-order domain
!
vpdn-group TESTNETWORK
 request-dialin
  protocol l2tp
  domain network.com
 initiate-to ip 10.0.0.10
 local name 21CN
 l2tp tunnel password 0 password
!
interface Serial1/0
 no ip address
 encapsulation ppp
 ppp authentication pap callin
 serial restart-delay 0
!
interface GigabitEthernet2/0
 ip address 10.0.0.5 255.255.255.0

Any PPP request coming in will be sent to the correct LNS. It determines this by domain name. The domain used above is network.com – I can add more groups, each using a different domain and terminating on a different LNS.

LNS

Finally the LNS itself. I’ve set this one up to authenticate the PPP session and give out IP addresses locally. In the real world this would be sent off to two or more radius servers. I’ve got a separate post on how to use radius to do this.

vpdn enable
!
vpdn-group 21CN
 accept-dialin
  protocol l2tp
  virtual-template 1
 terminate-from hostname 21CN
 l2tp tunnel password 0 password
!
username [email protected] password 0 password
!
interface Loopback0
 ip address 10.10.10.10 255.255.255.255
!
interface GigabitEthernet2/0
 ip address 10.0.0.10 255.255.255.0
!
interface Virtual-Template1
 ip unnumbered Loopback0
 peer default ip address pool POOL
 ppp authentication pap callin
!
ip local pool POOL 192.168.1.1 192.168.2.254
!
router ospf 1
 redistribute connected subnets

Testing

When R8 starts up, it will bring up its PPP interface and send that towards The LAC. When the LAC receives that PPP request, it will note the domain name and call up the correct vpdn group. It will create a l2tp tunnel towards the correct LNS. That PPP request is then passed to the LNS who is currently configured to authenticate locally. It will then hand an IP address over PPP to the CPE. The LNS will then have a connected route to that CPE. The LNS is redistributing connected routes into OSPF currently.

The end result is that devices in the core network will be able to reach the CPEs and vice-versa. The hop from the LNS to the CPE will look like a single hop as its going over an l2tp tunnel.

CPE

R8#sh ip int s1/0
Serial1/0 is up, line protocol is up
  Internet address is 192.168.1.3/32
  Broadcast address is 255.255.255.255
  Address determined by IPCP
  Peer address is 10.10.10.10

LNS

R10#sh ip route 192.168.1.3
Routing entry for 192.168.1.3/32
  Known via "connected", distance 0, metric 0 (connected, via interface)
  Redistributing via ospf 1
  Advertised by ospf 1 subnets
  Routing Descriptor Blocks:
  * directly connected, via Virtual-Access2.1
      Route metric is 0, traffic share count is 1

R10#sh ip int Virtual-Access2.1
Virtual-Access2.1 is up, line protocol is up
  Interface is unnumbered. Using address of Loopback0 (10.10.10.10)
  Broadcast address is 255.255.255.255
  Peer address is 192.168.1.3

P router

R1#sh ip route 192.168.1.3
Routing entry for 192.168.1.3/32
  Known via "ospf 1", distance 110, metric 20, type extern 2, forward metric 1
  Last update from 10.0.13.3 on FastEthernet1/0, 00:13:12 ago
  Routing Descriptor Blocks:
  * 10.0.13.3, from 10.10.10.10, 00:13:12 ago, via FastEthernet1/0
      Route metric is 20, traffic share count is 1
R1#traceroute 192.168.1.3
Type escape sequence to abort.
Tracing the route to 192.168.1.3
VRF info: (vrf in name/id, vrf out name/id)
  1 10.0.13.3 28 msec 44 msec 40 msec
  2 192.168.1.3 80 msec 44 msec 44 msec

LAC

You’ll see single vpdn tunnel currently used:

R5#sh vpdn tunnel

L2TP Tunnel Information Total tunnels 1 sessions 1

LocTunID   RemTunID   Remote Name   State  Remote Address  Sessn L2TP Class/
                                                           Count VPDN Group
25801      55713      R10           est    10.0.0.10       1     TESTNETWORK

Download the initial configs and gns3 topology here: http://mellowd.co.uk/ccie/wp-content/uploads/2014/02/BT_LNS.zip – You will need to adjust folder paths and images as needed.

Always check the forwarding table – IOS, Junos, Netiron

Most bigger routers these days use a distributed system. One of the bigger differences is the separation on the control and forwarding plane. When troubleshooting or verifying it’s essential to view both. Too many engineers simply show the control plane output. While these should match, they don’t always. Note that the forwarding table doesn’t have to be distributed to different hardware.

For the examples below I’ll simply be viewing a default route learned through OSPF. The router in question will always have two equal costs out of the network so you would expect to see two routes.

IOS

First we check the routing table:

R1#sh ip route 0.0.0.0
Routing entry for 0.0.0.0/0, supernet
  Known via "ospf 1", distance 110, metric 1, candidate default path
  Tag 1, type extern 2, forward metric 2
  Last update from 10.0.12.2 on GigabitEthernet2/0, 00:00:33 ago
  Routing Descriptor Blocks:
  * 10.0.13.3, from 10.0.24.4, 00:00:33 ago, via GigabitEthernet1/0
      Route metric is 1, traffic share count is 1
      Route tag 1
    10.0.12.2, from 10.0.24.4, 00:00:33 ago, via GigabitEthernet2/0
      Route metric is 1, traffic share count is 1
      Route tag 1

Two ways to get to 0.0.0.0 – What does the forwarding table show? For this I’ll choose an IP that would follow the default route:

R1#sh ip cef 4.2.2.1
0.0.0.0/0
  nexthop 10.0.12.2 GigabitEthernet2/0
  nexthop 10.0.13.3 GigabitEthernet1/0

Both control plane and data plane agree.

Netiron

Routing table:

[email protected]#sh ip route 0.0.0.0
Type Codes - B:BGP D:Connected I:ISIS O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
ISIS Codes - L1:Level-1 L2:Level-2
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2 s:Sham Link
STATIC Codes - d:DHCPv6
        Destination        Gateway         Port          Cost          Type Uptime src-vrf
1       0.0.0.0/0          10.0.0.1        eth 15/1      110/110       O1   1h22m  -
        0.0.0.0/0          10.0.0.2        eth 16/1      110/110       O1   1h22m  -

In order to show the forwarding table you use show route x.x.x.x detail. Note that I’m executing this command on an XMR16 and I will get the forwarding entry for every single module. I’m going to only show the output for the first module:

[email protected]#sh ip route 4.2.2.1 detail
Type Codes - B:BGP D:Connected I:ISIS O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
ISIS Codes - L1:Level-1 L2:Level-2
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2 s:Sham Link
STATIC Codes - d:DHCPv6
        Destination        Gateway         Port          Cost          Type Uptime src-vrf
1       0.0.0.0/0          10.0.0.1        eth 15/1      110/110       O1   1h24m  -
        0.0.0.0/0          10.0.0.1        eth 16/1      110/110       O1   1h24m  -
        Nexthop Entry ID:65540, Paths: 2, Ref_Count:707/712

D:Dynamic  P:Permanent  F:Forward  U:Us  C:Connected Network E: ESI VLAN
W:Wait ARP  I:ICMP Deny  K:Drop  R:Fragment  S:Snap Encap N:CamInvalid

Module S1:
      IP Address         Next Hop        MAC              Type  Port  Vlan  Pri
      0.0.0.0/0          10.0.0.1       0012.f293.a802   PF    16/1   1     0

      OutgoingIf  ArpIndex PPCR_ID   CamLevel   Parent  DontAge Index Is_trunk
      eth 16/1    5        1:1       31              0               0 0

      U_flags   Entry_flags  Age   Cam:Index               HW_Path_count
      0000e000               0     0x0005ffff (L3, right)  2

        CAM Entry Flag: 00000001H
        PPCR : 1:1 CIDX: 0x0005ffff (L3, right) (IP_NETWORK: 0x56000)

        pram_index_programmed: ppcr[0] 0x0000014c

The output is a little cryptic so I’ll highlight the important bits. First the paths show as two:

Nexthop Entry ID:65540, Paths: 2, Ref_Count:707/712

But the actual next-hop is only showing a single:

     0.0.0.0/0          10.0.0.1       0012.f293.a802   PF    16/1   1     0

This is a cosmetic error. The most important bit is here:

      U_flags   Entry_flags  Age   Cam:Index               HW_Path_count
      0000e000               0     0x0005ffff (L3, right)  2

The hardware path count is two, which is what we expect.

Junos

Finally Junos. First up we look at the route table:

lab@Vega_SRX6> show route 0.0.0.0

inet.0: 32 destinations, 32 routes (29 active, 3 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0          *[OSPF/150] 00:00:12, metric 0, tag 0
                      to 172.30.0.17 via ge-0/0/4.126
                    > to 172.30.0.89 via ge-0/0/4.146

Two routes, our forwarding table should match?

lab@Vega_SRX6> show route forwarding-table destination 4.2.2.1
Routing table: default.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
default            user     1 0:c:29:86:21:55    ucst   584    13 ge-0/0/4.146
default            perm     0                    rjct    36     5

Routing table: __master.anon__.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
default            perm     0                    rjct   534     1

Well no, it doesn’t. While the route table shows two routes, only one is being used by the forwarding table. Junos will not install multiple next-hops into the forwarding-table unless you tell it to:

lab@Vega_SRX6> show configuration policy-options policy-statement BALANCE
then {
    load-balance per-packet;
}
lab@Vega_SRX6> show configuration routing-options forwarding-table
export BALANCE;

Let’s check again:

lab@Vega_SRX6> show route forwarding-table destination 4.2.2.1
Routing table: default.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
default            user     1                    ulst 262142     7
                              0:c:29:25:21:57    ucst   612    11 ge-0/0/4.126
                              0:c:29:86:21:55    ucst   584     9 ge-0/0/4.146
default            perm     0                    rjct    36     5

Routing table: __master.anon__.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
default            perm     0                    rjct   534     1

This time we have both in the forwarding table. Note that while the policy states load-blance per-packet, it’s actually doing per-flow load-sharing.

Conclusion

I have seen routers disagree as to what they think they are doing compared to what they are doing. You need to check both tables above to note what both are doing. This could help immensely when a router is dropping packets it’s supposed to be forwarding, due to your FIB having no entry. I might write a bit on this as I’ve seen it happen more than once.

EDIT – 04/11/13

I’ve since found another way to verify this on the Brocades. If you rconsole onto the line card itself you can see a bit more:

SSH@XMR16#rconsole 1
Remote connection to LP slot 1 established
Press CTRL-X or type 'exit' to disconnect it
LP-1>en
LP-1#sh ip network 0.0.0.0
D:Dynamic  P:Permanent  F:Forward  U:Us  C:Connected Network
W:Wait ARP  I:ICMP Deny  K:Drop  R:Fragment  S:Snap Encap N:CamInvalid
      IP Address         Next Hop        MAC              Type  Port  Vlan  Pri
      0.0.0.0/0          10.0.0.1*    0012.f293.ad02   PF    15/1*  1     0

      OutgoingIf  ArpIndex PPCR_ID   CamLevel   Parent  DontAge Index Is_trunk
      eth 15/1    4        1:1       31              0               0 0

      U_flags   Entry_flags  Age   Cam:Index               HW_Path_count
      0000e000  0x00000001   0     0x0005ffff (L3, right)  2

        CAM Entry Flag: 00000001H
        PPCR : 1:1 CIDX: 0x0005ffff (L3, right) (IP_NETWORK: 0x56000)

        pram_index_programmed: ppcr[0] 0x0000014c
use_index: 0
IP-nh-Pram 0: 0x2ebeec10, ref_count 1
n_paths = 2, type = ECMP_PHY_VE, is_default  = 1, vrf_index = 0
  path[0]: FORWARD, out_intf eth 15/1, nh 10.0.0.1, out_port 15/1, is_trunk 0
  path[1]: FORWARD, out_intf eth 16/1, nh 10.0.0.5, out_port 16/1, is_trunk 0
Pram info: alloc_count 2 use_count 2
  pram[0]: idx 0, pram_idx[0] 0x0000014c
  pram[1]: idx 1, pram_idx[0] 0x0000014d

The top half still shows a single port, but down it shows this:

n_paths = 2, type = ECMP_PHY_VE, is_default  = 1, vrf_index = 0
  path[0]: FORWARD, out_intf eth 15/1, nh 10.0.0.1, out_port 15/1, is_trunk 0
  path[1]: FORWARD, out_intf eth 16/1, nh 10.0.0.5, out_port 16/1, is_trunk 0

n paths is the number of paths. The router is also doing ECMP. It then shows which ports outbound it’ll send traffic.

On a route with only a single hop the bit above are shown as so:

n_paths = 1, type = NON_ECMP, is_default  = 0, vrf_index = 0
  path[0]: FORWARD, out_intf eth 1/20, nh 10.0.0.8, out_port 1/20, is_trunk 0