Private Vlans – Control Plane/Data Plane

On December 9, 2012, in CCIE, by Darren

Private vlans are actually very easy to configure. But what’s actually happening at the data plane level? Knowing this helps when we need to span our pvlans through switches that do not support private vlans.

 

Let’s use the following topology for these tests. The routers are just acting as hosts. I also have a laptop connected to port fa0/48 on SW3 to capture some frames. All routers are in the 10.0.0.x/24 range.

data pvlan 1 Private Vlans   Control Plane/Data Plane

 

SW1 is a 3750 and SW2 is a 3560. Both support pvlans. SW3 is a 3550 which has no concept of private vlans.

The first thing I’m going to do is create the primary private vlan with an id of 50. I’ll then create a secondary community vlan with a vlaue of 24.  I’ll then put R2′s and R4′s port into the pvlan.

R1:

vlan 24
  private-vlan community
!
vlan 50
  private-vlan primary
  private-vlan association 24
!
interface FastEthernet1/0/4
 switchport private-vlan host-association 50 24
 switchport mode private-vlan host

At this point if I ping R4 from R2 the ping fails:

R2#ping 10.0.0.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

If two community ports are trying to speak to each other, SW1 and SW2 will use the community vlan value id for traffic in both ways. The community vlan is 24, so let’s try creating a regular vlan 24 on SW3:

SW3(config)#vlan 24
SW3(config-vlan)#end

Can we ping now?

R2#ping 10.0.0.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/6/20 ms

It doesn’t matter that SW3 doesn’t support pvlans at this point. SW1 and SW2 simply use a generic vlan tag of 24. This can be shown in the wireshark capture for both the ping request and response:
pvlan 1 Private Vlans   Control Plane/Data Plane

pvlan 2 Private Vlans   Control Plane/Data Plane
SW3 doesn’t know this is a private-vlan. It simply sees traffic for vlan 24 passing through it’s trunk ports. Does this mean that we can put R3′s port into vlan 24 and it’ll be able to ping both R2 and R4? Well it should, but let’s find out.
SW3:

interface FastEthernet0/1
 switchport access vlan 24
 switchport mode access

R2:

R2#ping 10.0.0.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/7/16 ms

No problems there. Wireshark shows the same regular vlan 24 being used between the switches.

Now let’s add a promiscuous port. I’ll make port fa1/0/1 on SW1 a promiscuous port which will represent the gateway on this subnet.
SW1:

interface FastEthernet1/0/1
 switchport private-vlan mapping 50 24
 switchport mode private-vlan promiscuous

Can R4 and R2 ping R1?

R4#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/6/16 ms
R2#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

R4 can ping, R2 can’t. Why it can’t will become clear very shortly. Let’s add vlan 50 on SW3 and try again on R2.

SW3#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
SW3(config)#vlan 50
SW3(config-vlan)#end
R2#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/8/20 ms

Now R2 can ping. To figure out why, let’s have a look at what wireshark is telling us.
pvlan 3 Private Vlans   Control Plane/Data Plane

pvlan 4 Private Vlans   Control Plane/Data Plane
It’s quite clear from the captures above. When traffic needs to get from a community port to an promiscuous port, SW1 and SW2 will use vlan 24 unidirectionally to get to the promiscuous port. Traffic going from the promiscuous port back to the community ports will use the primary vlan, which is id 50. (Isolated ports are the same in this regard)
As soon as vlan 50 is allowed through SW3, R2 can speak to the segment gateway. Once again SW3 has no idea that vlan 50 is any sort of private vlan.

However, this does cause a problem if R3 tries to get to the default gateway. R3 belongs to vlan 24 only, and so will never receive any reply from R1:

R3#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

R3 is sending out an ARP request which R1 does  receive. This can be proved with a debug arp on R1:

R1#debug arp
ARP packet debugging is on
R1#
*Dec  9 13:04:55.339: IP ARP: rcvd req src 10.0.0.3 ca02.0b28.0008, dst 10.0.0.1 FastEthernet0/0
*Dec  9 13:04:55.339: IP ARP: sent rep src 10.0.0.1 ca00.0b28.0008,
                 dst 10.0.0.3 ca02.0b28.0008 FastEthernet0/0

The problem is that the reply is getting sent over vlan 50, of which R3 is not a part of. What happens if we make R3 part of vlan 50 on SW3?

SW3(config)#int fa0/1
SW3(config-if)#swit acc vlan 50
SW3(config-if)#end
R3#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/7/16 ms

That worked. Surely SW1 would assume that incoming traffic on vlan 50 can’t be correct as traffic from the associated ports should come in on the secondary vlan? Well there is a reason this works which I’ll cover in a bit. Wireshark proves that both the request and replys were send using vlan 50. Of course R3 no longer has access to R2 or R4:

R3#ping 10.0.0.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
R3#ping 10.0.0.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

What if we wanted to create an SVI on all 3 of our switches so that each had a layer3 interface in the vlan? Let’s try to create int vlan 24 on SW1:

SW1(config)#int vlan 24
SW1(config-if)#
*Mar  1 02:16:48.839: %PV-6-PV_SVI_DOWN: Vlan 24's interface remains down because this vlan is a secondary vlan.
*Mar  1 02:16:48.848: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan24, changed state to down

This doesn’t work. SVI’s will only work when you use the primary vlan id:

interface Vlan50
 ip address 10.0.0.11 255.255.255.0

This SVI is not part of the community though. This means it should be able to ping R1, R3, but not R2 or R4:

SW1#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/208/1015 ms
SW1#ping 10.0.0.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
SW1#ping 10.0.0.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/206/1007 ms
SW1#ping 10.0.0.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

Which is what we see. In fact if I ping from SW2 to R1, the vlans in both directions should be vlan 50. Are they?
pvlan 5 Private Vlans   Control Plane/Data Planepvlan 6 Private Vlans   Control Plane/Data Plane
Yes they are. This is why the ping from R3 worked earlier. When the frame entered SW1′s trunk interface encapsulated with vlan 50, it has no idea this isn’t coming from a pvlan switch. It simply sees a source of vlan 50, which is valid. Yes community and isolated ports will be coming inbound with their respective secondary vlans, but a frame coming in tagged with the primary vlan is still a valid frame.

What happens if I don’t want to use R1 as the gateway. Rather I would like to use SW2′s SVI interface. At the moment none of the community ports can actually ping the SVIs as they are not promiscuous. We can’t specifically make them promiscuous, but we can map vlans to an SVI interface:
SW2:

interface Vlan50
 ip address 10.0.0.12 255.255.255.0
 private-vlan mapping 24
R2#ping 10.0.0.12

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.12, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/4 ms
R4#ping 10.0.0.12

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.12, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/7/20 ms

For now, this means that SW3′s SVI should be able to ping SW1, SW2, R1, but neither R2 nor R4:

SW3#ping 10.0.0.11

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.11, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/4/8 ms
SW3#ping 10.0.0.12

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.12, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/4 ms
SW3#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/202/1000 ms
SW3#ping 10.0.0.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
SW3#ping 10.0.0.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

There are a couple of interesting things to note. If I remove the associations in my vlan config, then any interface using pvlans associated with the wrongly configured pvlan will go down:

SW1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
SW1(config)#vlan 50
SW1(config-vlan)#no private-vlan association 24
SW1(config-vlan)#end
SW1#
*Mar  1 03:07:08.604: %PV-6-PV_MSG: Purged a private vlan mapping, Primary 50, Secondary 24
*Mar  1 03:07:08.621: %SYS-5-CONFIG_I: Configured from console by console
*Mar  1 03:07:09.619: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/1, changed state to down
*Mar  1 03:07:09.619: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/4, changed state to down

This can be a bit tricky to troubleshoot, as there is no reason given for the port being down:

SW1#sh int fa1/0/1
FastEthernet1/0/1 is up, line protocol is down (notconnect)

The only way to figure that out (at least I can find) is to have a look at the vlans configured, or check the switchport command:

SW1#sh int fa1/0/1 switchport | include Administrative|Operational
Administrative Mode: private-vlan promiscuous
Operational Mode: down
Administrative Trunking Encapsulation: negotiate
Administrative Native VLAN tagging: enabled
Administrative private-vlan host-association: none
Administrative private-vlan mapping: 50 (VLAN0050) 24 (VLAN0024)
Administrative private-vlan trunk native VLAN: none
Administrative private-vlan trunk Native VLAN tagging: enabled
Administrative private-vlan trunk encapsulation: dot1q
Administrative private-vlan trunk normal VLANs: none
Administrative private-vlan trunk associations: none
Administrative private-vlan trunk mappings: none
Operational private-vlan: none

The administrative mode shows private-vlan, but the operational mode shows none. Not that easy to find.

Conclusions

  • It’s not essential to have a switch that suports pvlan as a transport switch between 2 pvlan switches
  • While you could add a host to the transport switch, it’s either going to be able to speak to the hosts, or the gateway. Not both at the same time
  • Just keep hosts or the routing SVI off the transport switch. Keep these limited to the pvlan switches
  • In the data-plane, the only distinguishing feature of the packet is the dot1q header. This is why you can use non pvlan switches in the middle. Your transport switches are simply forwarding based on the MAC address in the vlans configured over the trunk links
  • SVIs can only be created in the primary vlan id. A Secondary SVI id will not come up
  • SVIs are not promiscuous by default. You need to map secondary vlans to an SVI port
  • If you need to monitor pvlan traffic on a non-pvlan switch, you’ll need to monitor both the primary and secondary vlan id.
Tagged with:  

Saving and running TCL scripts from flash

On August 23, 2012, in CCIE, by Darren

Anyone who is studying for the CCIE knows a lot about TCL scripting. Most certainly a big timesaver.

Of course TCL scripts can also be used in the real world. You don’t want to have to type our your script all the time, so is there a better way?

Over at networking-forum.com I asked if someone could come up with a script that would check for a route in any given subnet and return a ‘No route found’ for any IP address not in my global route table. I forgot who came up with it for me (sorry) but here is an example. Let’s say I wanted to check what routes I had for the range 10.20.16.0/20 – Of course usually this would be for a public range, but work with me here. This is the script:

#set the starting IP address octets
set firstoctet 10
set secondoctet 20
set thirdoctet 16
set fourthoctet 0

#while we're less than 10.20.31.255
while { $thirdoctet <= 31 } {
      while { $fourthoctet <= 255 } {
            set ipaddr $firstoctet.$secondoctet.$thirdoctet.$fourthoctet
            set results [exec "show ip route $ipaddr"]
            
            if { [regexp "not in table" $results] } {
                  puts "No route found for $ipaddr"
            }
            
            incr fourthoctet
      }
      
      #if out of the inner loop, reset the fourth back to 0 and increment the second octet
      set fourthoctet 0
      incr thirdoctet
}

This script works wonders, but it's pretty big to have to even copy and paste in each time I want to run it.

What I've now done is save the above script into a simple text file and renamed it to 10_20_16_0.tcl - I then tftp'd is to the flash on my router

There are now a couple of ways to run this file. The first way is to manually run it when I need to. How? Like so:

Router#tclsh flash:/10.20.16.0.tcl
No route found for 10.20.16.40
No route found for 10.20.16.41

The source will simply load and execute the tcl script straight away. This is certainly handy for those scripts you only want to run now and then.

What if you want to run a script automatically? Why not use EEM :)

event manager applet TCL_RUN
 event none
 action 1.0 cli command "enable"
 action 1.1 cli command "tclsh flash:/10_20_16_0.tcl"

The beauty of using EEM is that it can track all kinds of things, and based on those events it can run tcl scripts for you.

To actually run the EEM script yourself you can do so like this:

event manager run TCL_RUN

Nice and easy :)

Tagged with:  

Native vlan subinterfaces

On July 27, 2012, in CCIE, by Darren

The subinterface is an invaluable tool in your belt. I pretty must use subinterfaces whenever I connect routers to each other via ethernet, even if initially they are only going to have a single point-to-point connection to each other.

Why? Well a subinterface allows you to stick a dot1q header on your frames. Just like running a trunk between 2 switches, it allows you to run multiple logical links between 2 routers. As noted above, even if you only have a single point-to-point connection between 2 routers I still run them. In the past I’ve had to change or add to a design. If I was running just the physical interfaces and needed to change it, I would require downtime and possibly even going to site to actually reconfigure the CPE device. If I used subinterfaces from the start I can just add another subinterface to the mix. There is ways to get around this though.

You can also mix regular and subiterfaces together on the same interface. You can also tag a subinterface as the native vlan. Let’s look at a couple of different examples and see what we see.

dot1qnative Native vlan subinterfaces

Let’s just start with a simple physical interface setup.
R1:

interface FastEthernet0/0
 ip address 192.168.1.1 255.255.255.0

R2:

interface FastEthernet0/0
 ip address 192.168.1.2 255.255.255.0
R1#ping 192.168.1.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/25/68 ms

dot1qnative 11 Native vlan subinterfaces

Let’s say now that for whatever reason we need to run another logical link between these 2 routers. Let’s also say that R2 happens to be on the other side of the country and we need to do this now. Well I did note above that there is a workaround for this as you can run both a physical and subinterface on the router at the same time. Let’s leave the above config in and add this:
R1:

interface FastEthernet0/0.10
 encapsulation dot1Q 10
 ip address 10.0.0.1 255.255.255.0
interface FastEthernet0/0.10
 encapsulation dot1Q 10
 ip address 10.0.0.2 255.255.255.0
R1#ping 10.0.0.2 repeat 1

Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds:
!
Success rate is 100 percent (1/1), round-trip min/avg/max = 28/28/28 ms
R1#ping 192.168.1.2 repeat 1

Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!
Success rate is 100 percent (1/1), round-trip min/avg/max = 24/24/24 ms

Both still work. Now you could log onto R2 via the 10.0.0.2 link and remove the 192.168.1.0/24 address if you wanted without worry. Wireshark shows traffic going over the physical interface has no dot1q tag, while the subinterface has a dot1q tag of 10:
dot1qnative 2 Native vlan subinterfaces
dot1qnative 3 Native vlan subinterfaces

Now let’s mix it up a bit. I can say that a subinterface is tagged with the native vlan. i.e. no vlan tag:
R1:

interface FastEthernet0/0.20
 encapsulation dot1Q 20 native
 ip address 20.20.20.1 255.255.255.0

R2:

interface FastEthernet0/0
 ip address 20.20.20.2 255.255.255.0 secondary
 ip address 192.168.1.2 255.255.255.0
R1#ping 20.20.20.2 repeat 1

Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 20.20.20.2, timeout is 2 seconds:
!
Success rate is 100 percent (1/1), round-trip min/avg/max = 24/24/24 ms

dot1qnative 4 Native vlan subinterfaces

On R1 I created a subinterface with the native vlan. On R2 I added a secondary IP address. Traffic for the 20.20.20.0/24 range goes over the wire without a dot1q tag. Why not just use a secondary address? Well you can stick a subinterface into a vrf, while you can’t do that with a secondary address. Your IGP behavior also changes somewhat.

I don’t quite like doing the above though. I don’t like have layer2 seperation between networks that should be separated. However it proves the point that a native subinterface’s traffic outbound is exactly the same format as a physical interface. Just a standard ethernet layer2 header with no dot1q tag.

There is another advantage to using a native subinterface. If I want to shut down the link for 20.20.20.0/24, on R1 I can shut down the subinterface. All other subinterfaces stay up. If I do this on R2 however, all subinterfaces go down:
R1:

R1(config)#int fa0/0.20
R1(config-subif)#shut
R1(config-subif)#end
R1#
*Mar  1 00:33:31.023: %SYS-5-CONFIG_I: Configured from console by console
R1#sh ip int brief
Interface                  IP-Address      OK? Method Status                Protocol
FastEthernet0/0            192.168.1.1     YES manual up                    up
FastEthernet0/0.10         10.0.0.1        YES manual up                    up
FastEthernet0/0.20         20.20.20.1      YES manual administratively down down
R2(config)#int fa0/0
R2(config-if)#shut
R2(config-if)#end
R2#
*Mar  1 00:34:51.955: %SYS-5-CONFIG_I: Configured from console by console
R2#sh ip int
*Mar  1 00:34:53.347: %LINK-5-CHANGED: Interface FastEthernet0/0, changed state to administratively down
*Mar  1 00:34:54.347: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to down
R2#sh ip int brief
Interface                  IP-Address      OK? Method Status                Protocol
FastEthernet0/0            192.168.1.2     YES manual administratively down down
FastEthernet0/0.10         10.0.0.2        YES manual administratively down down

So while the native subinterface and physical interface send traffic the same, I can shut a native subinterface without affecting any other subinterface, while the shutting down of a physical interface shuts down all the subinterfaces attached to it

Now let’s get deeper…

R1:

interface FastEthernet0/0.10
 encapsulation dot1Q 10 native
 ip address 10.10.10.1 255.255.255.0

R2:

interface FastEthernet0/0.10
 encapsulation dot1Q 10
 ip address 10.10.10.2 255.255.255.0

Here I have R1 configured with a native subinterface, while R2 is configured with a dot1q header of 10. You would expect there would be no communication between them, but you would be incorrect. I’m going to turn on debug ip packet on both routers for this to see who sees what
R1:

R1#ping 10.10.10.2 repeat 1

Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 10.10.10.2, timeout is 2 seconds:

*Mar  1 00:08:07.763: IP: tableid=0, s=10.10.10.1 (local), d=10.10.10.2 (FastEthernet0/0.10), routed via FIB
*Mar  1 00:08:07.763: IP: s=10.10.10.1 (local), d=10.10.10.2 (FastEthernet0/0.10), len 100, sending.
Success rate is 0 percent (0/1)

dot1qnative 5 Native vlan subinterfaces
So the ping failed, but look more closely. R1 was able to encapsulate the packet, which means it has a MAC address. Let’s look at the ARP table:

R1#sh arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  10.10.10.1              -   c200.3e78.0000  ARPA   FastEthernet0/0.10
Internet  10.10.10.2              2   c201.3e78.0000  ARPA   FastEthernet0/0.10

Let’s try the other way around now.
R2:

R2#ping 10.10.10.1 repeat 1

Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 10.10.10.1, timeout is 2 seconds:

*Mar  1 00:11:20.807: IP: tableid=0, s=10.10.10.2 (local), d=10.10.10.1 (FastEthernet0/0.10), routed via RIB
*Mar  1 00:11:20.811: IP: s=10.10.10.2 (local), d=10.10.10.1 (FastEthernet0/0.10), len 100, sending
*Mar  1 00:11:20.815: IP: s=10.10.10.2 (local), d=10.10.10.1 (FastEthernet0/0.10), len 100, encapsulation failed.
Success rate is 0 percent (0/1)

dot1qnative 6 Native vlan subinterfaces
Let’s have a look at the ARP cache:

R2#sh arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  10.10.10.1              0   Incomplete      ARPA
Internet  10.10.10.2              -   c201.3e78.0000  ARPA   FastEthernet0/0.10

Somehow R1 learned R2′s MAC address so communication was there, but it seems it’s only one way. Let’s clear the ARP cache on both and debug ARP to see what happens.

The first thing you notice when you clear the ARP cache is that the local router will send a gratuitous ARP out to let everyone else know about it. Let’s see what we see on R1 when we clear R2′s cache:
R2:

R2#clear arp
R2#
*Mar  1 00:18:45.135: ARP: flushing ARP entries for all interfaces
*Mar  1 00:18:45.143: IP ARP: sent rep src 10.10.10.2 c201.3e78.0000,
                 dst 10.10.10.2 ffff.ffff.ffff FastEthernet0/0.10

R1:

R1#
*Mar  1 00:18:31.231: IP ARP: rcvd rep src 10.10.10.2 c201.3e78.0000, dst 10.10.10.2 FastEthernet0/0.10

R1 receives this gratuitous ARP and installs the MAC address into it’s ARP cache. Wireshark shows that R2 is sending this gratuitous ARP with a dot1q tag:
dot1qnative 7 Native vlan subinterfaces

What about the other way?
R1:

R1#clear arp
R1#
*Mar  1 00:20:53.847: ARP: flushing ARP entries for all interfaces
*Mar  1 00:20:53.855: IP ARP: sent rep src 10.10.10.1 c200.3e78.0000,
                 dst 10.10.10.1 ffff.ffff.ffff FastEthernet0/0.10

R2:

R2#
*Mar  1 00:21:07.831: IP ARP rep filtered src 10.10.10.1 c200.3e78.0000, dst 10.10.10.1 ffff.ffff.ffff wrong cable, interface FastEthernet0/0

dot1qnative 8 Native vlan subinterfaces

R2 complains that this ARP is coming in on ‘the wrong cable’ while wireshark shows the gratuitous ARP being sent without a dot1q header as expected.

What can we get from this? Well if you have a native subinterface, it will accept both untagged AND tagged frames in the vlan, vlan 10 in my example above. However it will only SEND untagged traffic. Good to know. It can also explain why you sometimes see an ARP entry on one side, while not on the other.

Tagged with:  

Sometimes it’s required that you have a number of static routes on a router, maybe for management or some other reason. If the static route point to a next-hop, but the exit interface stays up, there is no way for the router to know that it’s sending traffic down a black hole. Let’s show the following diagram as an example:
reliable static Reliable static routing without the need for the data license

R2 is a CPE on site. It has a primary link on fa0/0 connected to both R3 and R4 through a switch/VPLS. R2 is running OSPF with R3 and R4 and also has a floating static default route to R4′s fa0/0 interface. If this link goes down, the floating static route should come into play and take over. While the link is up we have a static route on R2 that sends our management traffic (10.0.0.0/24) to R4′s fa0/0 interface.

R3 is originating a default route via OSPF.

But does this actually work? Let’s configure it up quickly first and then break R2′s primary link.

R3:

interface FastEthernet0/0
 ip address 192.168.234.3 255.255.255.0
 ip ospf 1 area 0
!
router ospf 1
 default-information originate always

R4:

interface FastEthernet0/0
 ip address 192.168.234.4 255.255.255.0
 ip ospf 1 area 0
!
interface Serial0/0
 ip address 24.24.24.4 255.255.255.0

R2

interface FastEthernet0/0
 ip address 192.168.234.2 255.255.255.0
 ip ospf 1 area 0
!
interface Serial0/0
 ip address 24.24.24.2 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 24.24.24.4 200
ip route 10.0.0.0 255.255.255.0 192.168.234.4

Let’s have a look at R2′s routing table:

R2#   sh ip route | begin Gate
Gateway of last resort is 192.168.234.3 to network 0.0.0.0

C    192.168.234.0/24 is directly connected, FastEthernet0/0
     24.0.0.0/24 is subnetted, 1 subnets
C       24.24.24.0 is directly connected, Serial0/0
     10.0.0.0/24 is subnetted, 1 subnets
S       10.0.0.0 [1/0] via 192.168.234.4
O*E2 0.0.0.0/0 [110/1] via 192.168.234.3, 00:01:22, FastEthernet0/0

All looks good. I have my OSPF default route and I also have my management range route to R4.

Now for some reason, R2′s primary link fails. The fa0/0 interface stays up however.The link is dodgy, r there is a mess-up with the VPLS, it doesn’t really matter. What happens then?

R2 loses it’s adjacency with R4, but what about our management traffic?

R2#   sh ip route | begin Gate
Gateway of last resort is 24.24.24.4 to network 0.0.0.0

C    192.168.234.0/24 is directly connected, FastEthernet0/0
     24.0.0.0/24 is subnetted, 1 subnets
C       24.24.24.0 is directly connected, Serial0/0
     10.0.0.0/24 is subnetted, 1 subnets
S       10.0.0.0 [1/0] via 192.168.234.4
S*   0.0.0.0/0 [200/0] via 24.24.24.4

The problem is that we are still sending management traffic off to R4. This is the problem with a static route, it’s static! R2 has a next-hop of 192.168.234.4 – It’s interface in this subnet is still up, and so the router is trying to ARP for 192.168.234.4. Of course R4 never responds but the router will continue to try. It’ll never fail over to the backup.

Now with reliable static routing you are able to generate an IP Sla object which consistently pings another interface. If you get no response you cause the track object to go down and hence the static route goes down. The problem with this is that you need an expensive data license for the privilege of doing this.

But track objects can track a lot more than just IP Sla objects. You can also track routes. So why not track the default route considering we are learning that through the primary link? If the primary fails and OSPF times out, we will remove the OSPF default. Let’s try and see what happens:

R2:

track 1 ip route 0.0.0.0 0.0.0.0 reachability
!
ip route 10.0.0.0 255.255.255.0 192.168.234.4 track 1

Some of you may see a problem here, but bear with me.

Let’s see if this has fixed the problem:

R2#   sh ip route | begin Gate
Gateway of last resort is 24.24.24.4 to network 0.0.0.0

C    192.168.234.0/24 is directly connected, FastEthernet0/0
     24.0.0.0/24 is subnetted, 1 subnets
C       24.24.24.0 is directly connected, Serial0/0
     10.0.0.0/24 is subnetted, 1 subnets
S       10.0.0.0 [1/0] via 192.168.234.4
S*   0.0.0.0/0 [200/0] via 24.24.24.4

Hmm, we are still sending traffic to R4′s fa0/0 interface. Why is this?

R2#sh track 1
Track 1
  IP route 0.0.0.0 0.0.0.0 reachability
  Reachability is Up (static)
    1 change, last change 00:04:22
  First-hop interface is Serial0/0
  Tracked by:
    STATIC-IP-ROUTING 0

The problem is that our floating static route went live. As soon as it did we had a default route again and hence the track object is now UP.

But you don’t HAVE to track a default route. Why don’t we simply inject a phantom prefix on R3? One that will simply be used for tracking?

R3:

interface Loopback1
 ip address 3.3.3.3 255.255.255.255
 ip ospf 1 area 0

R2:

R2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R2(config)#no track 1
R2(config)#track 1 ip route 3.3.3.3/32 reachability

R2 is now tracking the loopback route from R3:

R2#sh track 1
Track 1
  IP route 3.3.3.3 255.255.255.255 reachability
  Reachability is Up (OSPF)
    2 changes, last change 00:00:03
  First-hop interface is FastEthernet0/0
  Tracked by:
    STATIC-IP-ROUTING 0
R2#   sh ip route | begin Gate
Gateway of last resort is 192.168.234.3 to network 0.0.0.0

     3.0.0.0/32 is subnetted, 1 subnets
O       3.3.3.3 [110/11] via 192.168.234.3, 00:00:24, FastEthernet0/0
C    192.168.234.0/24 is directly connected, FastEthernet0/0
     24.0.0.0/24 is subnetted, 1 subnets
C       24.24.24.0 is directly connected, Serial0/0
     10.0.0.0/24 is subnetted, 1 subnets
S       10.0.0.0 [1/0] via 192.168.234.4
O*E2 0.0.0.0/0 [110/1] via 192.168.234.3, 00:00:24, FastEthernet0/0

R2 now loses OSPF adjacency:

R2#
*Mar  1 00:38:51.919: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.234.3 on FastEthernet0/0 from FULL to DOWN, Neighbor Down: Dead timer expired
R2#
*Mar  1 00:38:55.239: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.234.4 on FastEthernet0/0 from FULL to DOWN, Neighbor Down: Dead timer expired
R2#
*Mar  1 00:39:05.307: %TRACKING-5-STATE: 1 ip route 3.3.3.3/32 reachability Up->Down
R2#
R2#sh track 1
Track 1
  IP route 3.3.3.3 255.255.255.255 reachability
  Reachability is Down (no route)
    3 changes, last change 00:00:11
  First-hop interface is unknown
  Tracked by:
    STATIC-IP-ROUTING 0
R2#   sh ip route | begin Gate
Gateway of last resort is 24.24.24.4 to network 0.0.0.0

C    192.168.234.0/24 is directly connected, FastEthernet0/0
     24.0.0.0/24 is subnetted, 1 subnets
C       24.24.24.0 is directly connected, Serial0/0
S*   0.0.0.0/0 [200/0] via 24.24.24.4

R2 loses the track route, removes the static and install the floating static route. All is good :)

I know there are better ways of doing the above. As in advertise management ranges via OSPF or running BFD, but not all of these are always available, especially over back up links.

Tagged with:  

SPAN, RSPAN, Layer 2 control packets and VLANS

On June 4, 2012, in CCIE, by Darren

I know the title is quite a mouthful, but I did want to cover all the above in this post. Daniel asked me to check a few things as I have ready access to real switches.

You learn in your studies that layer 2 control packets are ‘special’ – Special in the way that traffic going over the trunk between 2 switches does not follow the standard practice. Let’s use wireshark to see exactly what is going on in a bunch of scenarios. It’ll also give me the opportunity to do a bit of testing with SPAN and RSPAN.

Let’s use the basic topology:
SPAN RSPAN CONTROL SPAN, RSPAN, Layer 2 control packets and VLANS

Let’s first set up a span session on the 3750. I will monitor port gi1/0/9 in both directions and send that traffic to gi1/0/24 to be picked up by the laptop.

monitor session 1 source interface Gi1/0/9
monitor session 1 destination interface Gi1/0/24

The first thing I noticed when I plug in my laptop however is that Windows of course is very noisy. Already my capture is filling up with stuff that Windows is sending out. And so I’ve downloaded an NST ISO which I’ll listen with on the laptop.

So now that I’ve booted up into NST and got Wireshark running, I hardly see anything at all happening between the 2 switches. Where is all the layer 2 control traffic? Well the problem is that control traffic is not automatically replicated to a SPAN port. You need to enable encapsulation replication in order for it to work. Let’s do so:

C3750#conf t
C3750(config)#monitor session 1 destination interface gi1/0/24 encapsulation replicate

Let’s verify:

C3750#sh monitor session 1
Session 1
---------
Type                   : Local Session
Source Ports           :
    Both               : Gi1/0/9
Destination Ports      : Gi1/0/24
    Encapsulation      : Replicate
          Ingress      : Disabled

I can now see lot’s of control traffic in my Wireshark capture.

Both switches are already connected to each other. By default they’ll create a trunk link and vlan 1 will be the native vlan. I’ll then configure the switches to tag the native vlan and see what happens.

Let’s ensure the native vlan is not currently tagged:

C3750#sh vlan dot1q tag native
dot1q native vlan tagging is disabled

So what does my CDP/STP/DTP control packets look like in Wireshark? Note that I’m running the default mode of STP on the switch for now
I do see something odd. I am seeing an STP packet that has a dot1q tag of 1, 10 and an untagged packet. 10 I can understand because I have created vlan 10 and it has a separate STP instance. But why would the main one be tagged with vlan 1 if vlan 1 is the native vlan?
dot1q vlan 1
native 1 SPAN, RSPAN, Layer 2 control packets and VLANS
dot1q vlan 10
native 2 SPAN, RSPAN, Layer 2 control packets and VLANS
no dot1q tag
native 3 SPAN, RSPAN, Layer 2 control packets and VLANS

What about DTP and CDP?
DTP
native 4 SPAN, RSPAN, Layer 2 control packets and VLANS
CDP
native 5 SPAN, RSPAN, Layer 2 control packets and VLANS

Both CDP and DTP are currently sent with no vlan tag at all. CDP does carry information about the native vlan in it’s packet which is why CDP does complain when these don’t match on either end. But the important thing is that both are untagged.

Let’s now tag the native vlan and see what happens.

C3750(config)#vlan dot1q tag native
C3750#sh vlan dot1q tag native
dot1q native vlan tagging is enabled

Let’s bring up the interfaces again and see what we see.
DTP
native 6 SPAN, RSPAN, Layer 2 control packets and VLANS
CDP
native 7 SPAN, RSPAN, Layer 2 control packets and VLANS

Interesting. DTP has not tag, but CDP is using a tag of 1.

What about STP?
native 8 SPAN, RSPAN, Layer 2 control packets and VLANS
native 9 SPAN, RSPAN, Layer 2 control packets and VLANS
native 10 SPAN, RSPAN, Layer 2 control packets and VLANS
I’m seeing the same as what I saw above. I see a tagged vlan 1 STP frame, a tagged vlan 10 STP frame, and finally an untagged STP frame

What happens if I change the native vlan to 10? Well no need to paste output because it’s exactly the previous example. i.e. vlan 10 is now the native vlan and tagged, but CDP is still using a tag value of 1. STP and DTP remain unchanged.

Now let’s try something else. Let’s keep vlan 10 as the tagged native, but let’s remove vlan 1 from the trunk:

interface GigabitEthernet1/0/9
 switchport trunk encapsulation dot1q
 switchport trunk native vlan 10
 switchport trunk allowed vlan 2-4094

DTP is unchanged. i.e. it’s still sending untagged traffic. CDP however is sending tagged traffic. In vlan 1!
native 11 SPAN, RSPAN, Layer 2 control packets and VLANS
As a quick test I created int vlan 1 on both switches in the same subnet and tried to ping accross. I could not. Therefore it looks like Cisco will use vlan 1 tagged to send certain control data, even if vlan 1 is pruned, but no user data will be allowed on that vlan.

For STP I still have the same 3 outputs. A tagged vlan 1 STP frame, A tagged vlan 10 STP frame, and an untagged STP frame.

One last thing I wanted to test now was RSPAN config. I’ve always been a little confused as to the correct config on a switch that is the RSPAN end-point, and is also sending traffic to be monitored. i.e. Let’s say that the 3550 above is monitoring traffic on vlan 2 with a destination of remote span vlan 500. The 3750 is the rspan endpoint who monitoring rspan vlan 500 and sends it out to a local port on the switch. What happens if the 3750 is also monitoring vlan 2 on it’s own ports and sending out. Do we configure the destination to vlan 500 or straight to a local port?

Let’s configure it like so:

C3750#sh monitor session all
Session 1
---------
Type                   : Remote Source Session
Source Ports           :
    Both               : Gi1/0/9
Dest RSPAN VLAN        : 500


Session 2
---------
Type                   : Remote Destination Session
Source RSPAN VLAN      : 500
Destination Ports      : Gi1/0/24
    Encapsulation      : Replicate
          Ingress      : Disabled

I’ve tested sending it to RSPAN vlan 500 and I don’t see any traffic at all. As soon as I change it to send traffic directly to the port it works.

EDIT (05/06/12) – I’ve uploaded my captures to Cloudshark so you can take them apart to do your own research
Untagged native vlan 1
Tagged native vlan 1
Tagged native vlan 10
Tagged native vlan 10 with vlan 1 removed from trunk

Tagged with:  

© 2009-2014 Darren O'Connor All Rights Reserved