Fun with QinQ

On July 4, 2012, in Brocade, CCIE, Design, by Darren

Not all service providers have blanket coverage of an entire country. When a service provider gives service to a customer, more than likely the customer will be using a line from an external carrier of the ISP’s choosing.

Metro Ethernet is becoming more and more common these days and it does give us as designers a lot of flexibility. However each time a customer purchases a leased line, that requires another port in your core. That’s fine if the circuit is a nice gig, but quite often a lot of office will have anything from 2Mb to 1Gb. Do you really want to waste your core gig ports for a 2Mb circuit? Not really.

A lot of carriers are now offering aggregated ethernet links. Essentially this means that each customer site has a separate port (of course) but these are all aggregated when the carrier hands off to us. We get a single link carrying a bunch of customer circuits. Now you can sell hundreds of 2Mb circuits and only use a single port.
QinQ VPLS 1 Fun with QinQ

But how do we separate traffic then? Well you’ll come to an agreement with the carrier and each circuit ordered will have a vlan tag on the core side. This means customer 1 site 1′s traffic will arrive on vlan 1000. customer2 site 1′s traffic will arrive on vlan 1001. Now you just need to stick each tag into an MPLS solution and all is good.
QinQ VPLS 2 Fun with QinQ

But now what happens when you need to run multiple virtual circuits to a single customer site over their leased line? The vlan tag is already used by the carrier. What if I need to run 2 vrf’s for the customer and I need a WAN interface in each vrf?

We could do QinQ, but let’s think about this. In order to do QinQ I need another device at the customer site to pop another vlan tag on. I then need to get that tag into the core, pop off the tag and stick it back into the core. This could get messy.
Another problem with regular Cisco switches is that I can only do dot1q encapsulation on a port based. This means I need to use a port for every customer again, negating the advantage of the aggregated port to begin with. Or I can send traffic to another switch on a per-port basis, then get the second switch to aggregate the vlans back into the core. This will work, but what a nightmare to support.
Option1:
QinQ VPLS 3 Fun with QinQ

Option2:
QinQ VPLS 4 Fun with QinQ

Instead of a regular switch I could use a ME3400G and do selective QinQ. This allows me to specify that vlan 10 and 20 gets vlan 1000 popped onto it, and vlan 15 and 25, on the same port, gets vlan 1001 popped onto it.

It’s still another device that we could do without.

Let’s tackle the problem at the customer site first. Ideally I would like to do away with a separate device that does my QinQ. I can’t send double-tagged frames directly out my Cisco. Are we sure about this?

interface GigabitEthernet0/1.500
 encapsulation dot1Q 1000 second-dot1q 500
 ip address 172.16.255.1 255.255.255.0
end

The second-dot1q command allows you to both send and receive QinQ traffic directly from a router interface. The first tag is the metro tag, while the second tag is the inner tag. Let’s create another interface in another vrf.

interface GigabitEthernet0/1.600
 encapsulation dot1Q 1000 second-dot1q 600
 ip vrf forwarding 1
 ip address 192.168.1.2 255.255.255.0

Perfect. I can create as many sub-interfaces as I need using a single metro tag.

What about my core? In my core I’m using Brocade Netirons. Let’s see if I can terminate a double-tagged frame:

vpls Test_Customer 500
  vpls-peer 192.168.1.1
   vlan 1000 inner-vlan 500
   tagged ethe 15/1
!
 vpls Test_Customer 600
  vpls-peer 192.168.1.50
   vlan 1000 inner-vlan 600
   tagged ethe 15/1

No problems there. I can create 2 completely separate VPLS instances using the same single metro tag.

So far so good…

Let’s say for whatever reason I now need to run a point-to-point link directly from my core to the CPE at the customer site. I don’t want to terminate this point-to-point link into a VLL/VPLS. This could be for management, to provide voice or internet access, whatever. Unfortunately my Brocade device cannot terminate a double-tagged frame both in the local table as well as in a VPLS. Now we’re close to going back to our earlier switch examples to pop off that pesky outer tag.

But never fear, there is always a way. I did note above that we can’t terminate a double-tagged frame on both. How about sending some traffic as double-tagged and certain traffic as single? Can we do this?

Let’s start again with the CPE and show the full relevant config:

interface GigabitEthernet0/1.1000
 encapsulation dot1Q 1000
 ip address 10.0.0.1 255.255.255.0
!
interface GigabitEthernet0/1.600
 encapsulation dot1Q 1000 second-dot1q 600
 ip vrf forwarding 1
 ip address 192.168.1.2 255.255.255.0
!
interface GigabitEthernet0/1.500
 encapsulation dot1Q 1000 second-dot1q 500
 ip address 172.16.255.1 255.255.255.0

Works just fine on the Cisco. What about my Brocade?

vlan 1000 name Test
 tagged ethe 15/1
 router-interface ve 1000
!
interface ve 1000
 ip address 10.0.0.2/24
!
router mpls
!
vpls Test_Customer 500
  vpls-peer 192.168.1.1
   vlan 1000 inner-vlan 500
   tagged ethe 15/1
!
 vpls Test_Customer 600
  vpls-peer 192.168.1.50
   vlan 1000 inner-vlan 600
   tagged ethe 15/1

Everything tested and everything works perfectly. We’ve managed to remove all kinds of kit and also managed to simply the solution.

flattr badge large Fun with QinQ
Tagged with:  

2 Responses to “Fun with QinQ”

  1. Leo says:

    Hi mellowd,
    Pretty good writing on this one. I was trying to look for a sample config on how to implement this on Cisco devices? But I was unable to find anything under Cisco’s website.
    Would be able to provide some insight on how the config would look like for this scenario? Or any documents?

    Thank you very much

  2. I have read your post.your post is so great.Consider wan optimization to reduce your bandwidth requirements while also gaining the ability to granularly manage application performance, if those are relevant issues for our organization.Thanks for your posting.I hope you will continue your posting.

Leave a Reply

© 2009-2014 Darren O'Connor All Rights Reserved