Private Vlans – Control Plane/Data Plane

Private vlans are actually very easy to configure. But what’s actually happening at the data plane level? Knowing this helps when we need to span our pvlans through switches that do not support private vlans.

 

Let’s use the following topology for these tests. The routers are just acting as hosts. I also have a laptop connected to port fa0/48 on SW3 to capture some frames. All routers are in the 10.0.0.x/24 range.

 

SW1 is a 3750 and SW2 is a 3560. Both support pvlans. SW3 is a 3550 which has no concept of private vlans.

The first thing I’m going to do is create the primary private vlan with an id of 50. I’ll then create a secondary community vlan with a vlaue of 24.  I’ll then put R2’s and R4’s port into the pvlan.

R1:

vlan 24
  private-vlan community
!
vlan 50
  private-vlan primary
  private-vlan association 24
!
interface FastEthernet1/0/4
 switchport private-vlan host-association 50 24
 switchport mode private-vlan host

At this point if I ping R4 from R2 the ping fails:

R2#ping 10.0.0.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

If two community ports are trying to speak to each other, SW1 and SW2 will use the community vlan value id for traffic in both ways. The community vlan is 24, so let’s try creating a regular vlan 24 on SW3:

SW3(config)#vlan 24
SW3(config-vlan)#end

Can we ping now?

R2#ping 10.0.0.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/6/20 ms

It doesn’t matter that SW3 doesn’t support pvlans at this point. SW1 and SW2 simply use a generic vlan tag of 24. This can be shown in the wireshark capture for both the ping request and response:


SW3 doesn’t know this is a private-vlan. It simply sees traffic for vlan 24 passing through it’s trunk ports. Does this mean that we can put R3’s port into vlan 24 and it’ll be able to ping both R2 and R4? Well it should, but let’s find out.
SW3:

interface FastEthernet0/1
 switchport access vlan 24
 switchport mode access

R2:

R2#ping 10.0.0.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/7/16 ms

No problems there. Wireshark shows the same regular vlan 24 being used between the switches.

Now let’s add a promiscuous port. I’ll make port fa1/0/1 on SW1 a promiscuous port which will represent the gateway on this subnet.
SW1:

interface FastEthernet1/0/1
 switchport private-vlan mapping 50 24
 switchport mode private-vlan promiscuous

Can R4 and R2 ping R1?

R4#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/6/16 ms
R2#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

R4 can ping, R2 can’t. Why it can’t will become clear very shortly. Let’s add vlan 50 on SW3 and try again on R2.

SW3#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
SW3(config)#vlan 50
SW3(config-vlan)#end
R2#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/8/20 ms

Now R2 can ping. To figure out why, let’s have a look at what wireshark is telling us.


It’s quite clear from the captures above. When traffic needs to get from a community port to an promiscuous port, SW1 and SW2 will use vlan 24 unidirectionally to get to the promiscuous port. Traffic going from the promiscuous port back to the community ports will use the primary vlan, which is id 50. (Isolated ports are the same in this regard)
As soon as vlan 50 is allowed through SW3, R2 can speak to the segment gateway. Once again SW3 has no idea that vlan 50 is any sort of private vlan.

However, this does cause a problem if R3 tries to get to the default gateway. R3 belongs to vlan 24 only, and so will never receive any reply from R1:

R3#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

R3 is sending out an ARP request which R1 does  receive. This can be proved with a debug arp on R1:

R1#debug arp
ARP packet debugging is on
R1#
*Dec  9 13:04:55.339: IP ARP: rcvd req src 10.0.0.3 ca02.0b28.0008, dst 10.0.0.1 FastEthernet0/0
*Dec  9 13:04:55.339: IP ARP: sent rep src 10.0.0.1 ca00.0b28.0008,
                 dst 10.0.0.3 ca02.0b28.0008 FastEthernet0/0

The problem is that the reply is getting sent over vlan 50, of which R3 is not a part of. What happens if we make R3 part of vlan 50 on SW3?

SW3(config)#int fa0/1
SW3(config-if)#swit acc vlan 50
SW3(config-if)#end
R3#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/7/16 ms

That worked. Surely SW1 would assume that incoming traffic on vlan 50 can’t be correct as traffic from the associated ports should come in on the secondary vlan? Well there is a reason this works which I’ll cover in a bit. Wireshark proves that both the request and replys were send using vlan 50. Of course R3 no longer has access to R2 or R4:

R3#ping 10.0.0.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
R3#ping 10.0.0.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

What if we wanted to create an SVI on all 3 of our switches so that each had a layer3 interface in the vlan? Let’s try to create int vlan 24 on SW1:

SW1(config)#int vlan 24
SW1(config-if)#
*Mar  1 02:16:48.839: %PV-6-PV_SVI_DOWN: Vlan 24's interface remains down because this vlan is a secondary vlan.
*Mar  1 02:16:48.848: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan24, changed state to down

This doesn’t work. SVI’s will only work when you use the primary vlan id:

interface Vlan50
 ip address 10.0.0.11 255.255.255.0

This SVI is not part of the community though. This means it should be able to ping R1, R3, but not R2 or R4:

SW1#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/208/1015 ms
SW1#ping 10.0.0.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
SW1#ping 10.0.0.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/206/1007 ms
SW1#ping 10.0.0.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

Which is what we see. In fact if I ping from SW2 to R1, the vlans in both directions should be vlan 50. Are they?

Yes they are. This is why the ping from R3 worked earlier. When the frame entered SW1’s trunk interface encapsulated with vlan 50, it has no idea this isn’t coming from a pvlan switch. It simply sees a source of vlan 50, which is valid. Yes community and isolated ports will be coming inbound with their respective secondary vlans, but a frame coming in tagged with the primary vlan is still a valid frame.

What happens if I don’t want to use R1 as the gateway. Rather I would like to use SW2’s SVI interface. At the moment none of the community ports can actually ping the SVIs as they are not promiscuous. We can’t specifically make them promiscuous, but we can map vlans to an SVI interface:
SW2:

interface Vlan50
 ip address 10.0.0.12 255.255.255.0
 private-vlan mapping 24
R2#ping 10.0.0.12

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.12, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/4 ms
R4#ping 10.0.0.12

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.12, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/7/20 ms

For now, this means that SW3’s SVI should be able to ping SW1, SW2, R1, but neither R2 nor R4:

SW3#ping 10.0.0.11

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.11, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/4/8 ms
SW3#ping 10.0.0.12

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.12, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/4 ms
SW3#ping 10.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/202/1000 ms
SW3#ping 10.0.0.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
SW3#ping 10.0.0.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

There are a couple of interesting things to note. If I remove the associations in my vlan config, then any interface using pvlans associated with the wrongly configured pvlan will go down:

SW1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
SW1(config)#vlan 50
SW1(config-vlan)#no private-vlan association 24
SW1(config-vlan)#end
SW1#
*Mar  1 03:07:08.604: %PV-6-PV_MSG: Purged a private vlan mapping, Primary 50, Secondary 24
*Mar  1 03:07:08.621: %SYS-5-CONFIG_I: Configured from console by console
*Mar  1 03:07:09.619: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/1, changed state to down
*Mar  1 03:07:09.619: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/4, changed state to down

This can be a bit tricky to troubleshoot, as there is no reason given for the port being down:

SW1#sh int fa1/0/1
FastEthernet1/0/1 is up, line protocol is down (notconnect)

The only way to figure that out (at least I can find) is to have a look at the vlans configured, or check the switchport command:

SW1#sh int fa1/0/1 switchport | include Administrative|Operational
Administrative Mode: private-vlan promiscuous
Operational Mode: down
Administrative Trunking Encapsulation: negotiate
Administrative Native VLAN tagging: enabled
Administrative private-vlan host-association: none
Administrative private-vlan mapping: 50 (VLAN0050) 24 (VLAN0024)
Administrative private-vlan trunk native VLAN: none
Administrative private-vlan trunk Native VLAN tagging: enabled
Administrative private-vlan trunk encapsulation: dot1q
Administrative private-vlan trunk normal VLANs: none
Administrative private-vlan trunk associations: none
Administrative private-vlan trunk mappings: none
Operational private-vlan: none

The administrative mode shows private-vlan, but the operational mode shows none. Not that easy to find.

Conclusions

  • It’s not essential to have a switch that suports pvlan as a transport switch between 2 pvlan switches
  • While you could add a host to the transport switch, it’s either going to be able to speak to the hosts, or the gateway. Not both at the same time
  • Just keep hosts or the routing SVI off the transport switch. Keep these limited to the pvlan switches
  • In the data-plane, the only distinguishing feature of the packet is the dot1q header. This is why you can use non pvlan switches in the middle. Your transport switches are simply forwarding based on the MAC address in the vlans configured over the trunk links
  • SVIs can only be created in the primary vlan id. A Secondary SVI id will not come up
  • SVIs are not promiscuous by default. You need to map secondary vlans to an SVI port
  • If you need to monitor pvlan traffic on a non-pvlan switch, you’ll need to monitor both the primary and secondary vlan id.

© 2009-2018 Darren O'Connor All Rights Reserved -- Copyright notice by Blog Copyright