802.
1Q Tunnelling (QinQ)
LAB
In this lab we are going to join 2 sites of a customer network using QinQ through our Service
Provider (SP). The result will be that the two sites are logically trunked, meaning that they are
able to send VLAN’s across to each other through the service providers dedicated QinQ VLAN.
In this example, the provider is going to be using VLAN34 as the QinQ VLAN.
Let’s start by configuring both customer & carrier switches
Cpe1.3560#
vlan 3
name vlan3
!
interface GigabitEthernet0/1
description Trunk to [Link].Sw2
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet0/5
description Link to PC1
switchport access vlan 3
cpe2.3560#
interface GigabitEthernet0/1
description Trunk to BT.Sw2
switchport trunk encapsulation dot1q
switchport mode trunk
!
vlan 3
name vlan3
!
interface GigabitEthernet0/5
description PC2 is connected here
switchport access vlan 3
switchport mode access
[Link].Sw2#
interface GigabitEthernet0/1
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet0/2
switchport trunk encapsulation dot1q
witchport mode trunk
BT.Sw2#
interface GigabitEthernet0/1
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet0/2
description Trunk to edge.sw2.3560
switchport trunk encapsulation dot1q
switchport mode trunk
Nothing fancy, we just trunk to the carrier, and make an access port to our PC’s. On the carrier
switches we just trunk both ends.
The config for our edge switches is shown below.
Edge.Sw1.3560#
vlan 34
name vlan34
!
interface FastEthernet0/1
description Trunk to [Link].Sw2
switchport access vlan 34
switchport mode dot1q-tunnel
!
interface FastEthernet0/2
description link to QinQ.Sw1.3560
switchport trunk encapsulation dot1q
switchport mode trunk
Edge.Sw2.3560#
vlan 34
name vlan34
!
interface FastEthernet0/1
description feed to BT.Sw2
switchport access vlan 34
switchport mode dot1q-tunnel
!
interface FastEthernet0/2
description feed to QInQ.Sw2.3560
switchport trunk encapsulation dot1q
switchport mode trunk
When I put the configuration on the tunnel ports, the switch prompted me to change my MTU to
1504 & reload the switch. If your switch doesn’t do that then I suggest you check your
system MTU using #show system MTU, and ensure that it’s using 1504 bytes or more. If not,
you should manually set it using #system mtu 1504. The reason for it is because QinQ adds 4
bytes when adding an extra Tag & EType field in the frame (for a more
elaborated explanation see the bottom of this page – under “QinQ technical bits”).
The config for the QinQ.sw1.3560 & QinQ.sw2.3560 are shown below. Remember, you must
check the system MTU on them, as they will also need to be 1504 to be able to process double
tagged frames in the QinQ ring.
QinQ.Sw1.3560#
vlan 34
name vlan34
!
interface FastEthernet0/1
description Trunk to QinQ.Sw2.3560
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface FastEthernet0/2
description Trunk to Edge.Sw1.3560
switchport trunk encapsulation dot1q
switchport mode trunk
!
vlan 34
name vlan34
QinQ.SW2.3560#
!
vlan 34
name vlan34
!
interface GigabitEthernet0/1
description Trunk to qinq.sw1.3560
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet0/2
description Trunk to edge.sw2.3560
switchport trunk encapsulation dot1q
switchport mode trunk
Let’s shove some PC’s on port 5 at both Cpe1.3560 and Cpe2.3560 to test this actually works!
And PC2
Notice that we can see the MAC of the PC’s at either side of the tunnel by checking the arp table
(arp -a). The reason we can see these MACs is because we’ve formed a layer 2 network between
both sites.
Tips/Considerations
Use one dot1q tunnel interface/VLAN per customer
Be sure to tag your customers native VLAN traffic into the QinQ VLAN (I forgot to do
that in my lab). If you don’t, and the customer decides he wants to use VLAN 100 as his
management vlan (as a native VLAN), when he trys to SSH or manage his switch at his
other site, his packets could get dropped. This is because the native VLAN is left
untagged, and when it hits the provider switchport in the dot1q-tunnel mode, it doesn’t
double tag the packet (as there is no original VLAN tag to start with). To avoid
complications, just tag his native VLAN using #vlan dot1q tag native when inserting his
traffic into the QinQ VLAN.
You can’t detect the received CoS value from frames in different customer side VLANs
because you added a 2 byte Tag & Etype field in front of his Tag & Etype fields when
you received the frame on the tunnel port. So you can’t prioritise his traffic according
to his VLANs. You can, however, prioritise your QinQ VLAN if you needed to.
Encapsulating frames into a QinQ VLAN adds 4 bytes of overhead (from the Tag &
Etype fields) to every frame. Switches sending/receiving QinQ frames must increase
their system MTU to 1504 for VLAN tunneling work. You can check your MTU
using #show system mtu. You can increase your MTU by issuing #system mtu 1504,
however, this change requires a reboot.
You can also tunnel the customer’s CDP, STP, VTP, and CoS values by using the
following commands:
#l2protocol-tunnel stp
#l2protocol-tunnel cdp
#l2protocol-tunnel vtp
#l2protocol-tunnel cos
QinQ Technical bits
When a frame arrives at a trunk port, it looks like the 802.1Q frame in the diagram below. As
you can see, it adds a Etype & Tag field (compared to the Original Ethernet Frame). The Etype
field (sometimes referred to as a Tag Protocol IDentifier: TPID) will have a value of 0x8100,
indicating a VLAN-tagged frame (IEEE 802.1Q). The Tag field (sometimes referred to as the
Tag Control Information :TCI) will identify a VLAN ID & and any 802.1p priority (i.e. QoS)
attached to the frame.
IEEE 802.1Q Frame (Cisco, 2006)
The last frame in the image above is a QinQ frame. Now you can see why it’s double tagged! A
tunnel interface adds another Tag (sometimes referred to as an outer tag) and Etype field.
References
Cisco (2006) IEEE 802.1q Frame [Online] Available from:
[Link]
[Accessed 08/02/2011]
CCIE: 802.1q (QinQ tunneling/802.1ad)
Jan.8.2012 [Link] Comments 0 Comment
One of my personal favorite L2 subjects to discuss. When I was at the service provider this was a
very cost effective Metro solution to extend customer VLANs. No routing protocols on the CPE
and no expensive EoMPLS hardware required from the SP perspective. Simple and effective.
You will hear many names for the outer tag (S-TAG, metro tag, etc…) just remember that the
outer tag is the unique SP ID for that customer and the inner tags are the customers tags. This
extra instance of .1q requires an additional 4 bytes so, make sure your system/global MTU is at
least 1504 bytes to transmit a data frame size of 1500 bytes.
Combine this with L2TP and you can easily tunnel VTP,STP,and CDP frames.
Here is a brief example:
SW1:
interface FastEthernet0/1
des connection to Customer_SITE_A
switchport access vlan 100
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
l2protocol-tunnel stp
l2protocol-tunnel vtp
no cdp enable
SW4:
interface FastEthernet0/4
des Cucstomer_SITE_B
switchport access vlan 100
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
l2protocol-tunnel stp
l2protocol-tunnel vtp
no cdp enable
Now to expand upon this concept you can create an Etherchannel/Portchannel
between the customer CE devices using separate SP tag for each link.
SW1: CPE_Site_A
interface fa 0/12
sw tr en dot
sw mo tr
channel-protocol LACP
channel-group 1 mode active
interface fa 0/13
sw tr en dot
sw mo tr
channel-protocol LACP
channel-group 1 mode
SW2: SP_SW1
int fa 0/12
switchport access vlan 100
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
l2protocol-tunnel lldp
l2protocol-tunnel stp
l2protocol-tunnel vtp
l2protocol-tunnel point-to-point lacp
int fa 0/13
switchport access vlan 200
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
l2protocol-tunnel lldp
l2protocol-tunnel stp
l2protocol-tunnel vtp
l2protocol-tunnel point-to-point lacp
int fa 0/24
sw tr en dot
sw mo tru
SW3: SP_SW2
int fa 0/12
switchport access vlan 100
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
l2protocol-tunnel lldp
l2protocol-tunnel stp
l2protocol-tunnel vtp
l2protocol-tunnel point-to-point lacp
int fa 0/13
switchport access vlan 200
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
l2protocol-tunnel lldp
l2protocol-tunnel stp
l2protocol-tunnel vtp
l2protocol-tunnel point-to-point lacp
int fa 0/24
sw tr en dot
sw mo tru
SW4: CPE_Site_B
interface fa 0/12
sw tr en dot
sw mo tr
channel-protocol LACP
channel-group 1 mode active
interface fa 0/13
sw tr en dot
sw mo tr
channel-protocol LACP
channel-group 1 mode active
Fun with QinQ tunnels - Part 1
A QinQ tunnel extends a VLAN across the network, or the internet. The usual way this is
done is by having a standard VLAN in your network connecting to a QinQ tunnel in the
service provider network at both ends.
This allows mutiple VLANs in your network to be encapsulated within another VLAN
across the demarc boundaries, and back into your network at another site.
There are some prerequisites with setting up a QinQ, in that the MTU size must be
increased to accommodate the larger packet size. You also need a switch that supports
them, for this I used a 3560 and a 3750, both running Advanced IP Services, the Inside
switches (in the first diagram are 3550s, and in the final diagram we used the same
3750).
The basic diagram looks like this:
A standard Trunk port from the Inside Switch (e0/1) connects to the QinQ trunk on the
provider switch (e0/1), which then connects to the other provider switch via another
standard trunk (VLAN 10, e0/10 - e0/10), and finally a QinQ tunnel port on the other
Provider switch (e0/1) connects to a standard trunk port (e0/1) on the other Inside
switch. The dotted line shows how the switches, and the end user, see the link.
The configuration of the switches would be as follows:
Inside switch (left hand side)
vlan 4
name Client_VLAN
vlan 5
name Server_VLAN
vlan 6
name Other_VLAN
int e0/1
description **** Link to QinQ ****
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 4,5,6
switchport mode trunk
int e0/4
switchport access vlan 4
int e0/5
switchport access vlan 5
int e0/6
switchport access vlan 6
Provider Switch (left hand side)
system mtu 1998
system mtu jumbo 9000
vlan 10
name QinQ_VLAN
int e0/1
description **** QinQ VLAN ****
switchport access vlan 10
switchport trunk encapsulation dot1q
switchport mode dot1q-tunnel
no keepalive
l2protocol-tunnel cdp
l2protocol-tunnel stp
int e0/10
description **** Provider to Provider link ****
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 10
switchport mode trunk
Provider Switch (right hand side)
system mtu 1998
system mtu jumbo 9000
vlan 10
name QinQ_VLAN
int e0/1
description **** QinQ VLAN ****
switchport access vlan 10
switchport trunk encapsulation dot1q
switchport mode dot1q-tunnel
no keepalive
l2protocol-tunnel cdp
l2protocol-tunnel stp
int e0/10
description **** Provider to Provider link ****
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 10
switchport mode trunk
Inside switch (right hand side)
vlan 4
name Client_VLAN
vlan 5
name Server_VLAN
vlan 6
name Other_VLAN
int e0/1
description **** Link to QinQ ****
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 4,5,6
switchport mode trunk
int e0/4
switchport access vlan 4
int e0/5
switchport access vlan 5
int e0/6
switchport access vlan 6
Now if you attach a laptop to the same ports on both sides and assign an IP address to
both laptops (say [Link]/24 and [Link]/24) they should be able to ping each other.
The above is an in-an-ideal-world scenario. Really you just want to be able to configure
standard trunk links on your equipment and have the service provider take care of all the
QinQ configuration. But sometimes what you get is slightly different. And what we got
was this:
Now our options were to either (A) purchase a new switch so we can replicate the layout
in the first picture, or (B) try and find a way of having the QinQ settings and the trunk
settings on the same switch. Option A would cost quite a bit of money, but is option B
possible? Can a QinQ tunnel exist on the same switch as the trunk? The dangers are
that it won't work due to loopguard and bpduguard. But it's worth a shot, right?
Turns out that it is, and all it takes is one little ethernet cable, now connected from port
e0/1 to e0/2. Port e0/10 is then used to link up to the provider switch at the other site.
The settings for the switches on the left hand side remain the same, but what we have
done is loop a cable from one port back into another port.
So now all of the config for both the right hand side switches goes into the one switch
(we used a 3750):
system mtu 1998
system mtu jumbo 9000
vlan 10
name QinQ_VLAN
vlan 4
name Client_VLAN
vlan 5
name Server_VLAN
vlan 6
name Other_VLAN
int e0/1
description **** Link to QinQ ****
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 4,5,6
switchport mode trunk
int e0/2
description **** QinQ VLAN ****
switchport access vlan 10
switchport trunk encapsulation dot1q
switchport mode dot1q-tunnel
no keepalive
l2protocol-tunnel cdp
l2protocol-tunnel stp
int e0/4
switchport access vlan 4
int e0/5
switchport access vlan 5
int e0/6
switchport access vlan 6
int e0/10
description **** Provider to Provider link ****
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 10
switchport mode trunk
Traffic will (effectively) come in on e0/1, get encapsulated in e0/2 and traverse to the
other side via e0/10, again we tested via ping and all was good.
So it turns out that you can have your QinQ trunk and the VLAN trunk living on the same
switch.
Part two answers the question "Can we route different subnets across a QinQ link?"
Fun with QinQ tunnels - Part 2 (Routing
different subnets)
From part one, we know that the purpose of a QinQ tunnel is to extend a VLAN across
the WAN or network. But what if site 1 is connected to site 2 and they use different IP
schemes (such as [Link]/24 and [Link]/24)? Well, making the two talk is
actually very simple.
The way I have configured this is to set up a loopback interface on each side to emulate
the different network, and set up routes between them. If you recall from part 1 I am
using a 3550 on one side and a 3750 on the other.
We start by enabling IP routing on both sides:
3550(config)#ip routing
3750(config)#ip routing
Then we assign a virtual IP address to the VLAN:
3550(config)#int vlan 501
3550(config-if)#ip address [Link] [Link]
3750(config)#int vlan 501
3750(config-if)#ip address [Link] [Link]
And then we create our loopback interfaces to emulate the different networks:
3550(config)#interface Loopback2
3550(config-if)#ip address [Link] [Link]
3750(config)interface Loopback2
3750(config-if)#ip address [Link] [Link]
Lastly we set a routing statement with the destination network and the destination IP
address, which is the virtual IP address of the vlan at the other side
3550(config)#ip route [Link] [Link] [Link]
3750(config)#ip route [Link] [Link] [Link]
Now we should be able to ping the loopback in each site:
Dot1Q Tunnel / q-in-q Tunneling
Dot1Q tunnel or q-in-q tunneling is a technique generally used by service providers to extend customer's
VLANs to different locations. It is done by encapsulating the customer VLAN inside another 802.1q
encapsulation and because of this we must increase the system mtu on the service provider switches to
something greater than 1500. In this blogtorial, we will take a look at a very simple topology with easy to
follow configurations.
Here is the topology.
Our objective:
VLAN 10 in a Customer Site in California (SW1) needs to be extended to a Customer Site in Illinois (SW4).
First we need to configure the customer equipment to put the interface facing the service provider on
VLAN 10.
SW1#sh run int fa1/1
interface FastEthernet1/1
description SW1 to SP1
switchport access vlan 10
end
SW1#sh run int vlan 10
interface vlan10
ip address [Link] [Link]
end
Next we need to get SP1 interface facing the customer in a q-in-q mode and assign a metro-tag. In this
case I assigned metro-tag 510.
SP1#sh run int fa1/1
interface FastEthernet1/1
description SW1 to SW4 dot1q
switchport access vlan 510
!!VLAN 510 above is commonly referred to as metro-tag or customer tag.
switchport mode dot1q-tunnel
!!Notice that we also tunnel cdp, stp, and vtp as well.
!!Normally they would not be by default
l2protocol-tunnel cdp
l2protocol-tunnel stp
l2protocol-tunnel vtp
no cdp enable
end
Now we need to configure the interfaces between the Service Providers which is just a basic trunk.
SP1#sh run int fa1/2
interface FastEthernet1/2
description connected to SP2
switchport trunk encapsulation dot1q
switchport mode trunk
end
SP2#sh run int fa1/2
interface FastEthernet1/1
description connected to SP1
switchport trunk encapsulation dot1q
switchport mode trunk
end
The interface on SP2 facing the customer side is assigned the same metro-tag of 510.
SP2#sh run int fa1/3
interface FastEthernet1/3
description SW4 to SW1 dot1q
switchport access vlan 510
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
l2protocol-tunnel stp
l2protocol-tunnel vtp
no cdp enable
end
We need to configure the customer equipment to put the interface facing the service provider on VLAN
10.
SW4#sh run int fa1/3
interface FastEthernet1/3
description connected to SP2
switchport access vlan 10
end
SW#sh run int vlan 10
interface vlan10
ip add [Link] [Link]
end
Let's see what happens when we ping from SW1 to SW4.
SW1#ping [Link]
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to [Link], timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/10/12 ms
SW1#
Let's see what CDP shows.
SW1#show cdp neighbors
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone
Device ID Local Intrfce Holdtme Capability Platform Port ID
SW4 Fas 1/1 142 R 3550 Fas 1/3
SW1#
As you can see it appears as though SW4 is connected directly to SW1. What is really happening is that
VLAN 10 traffic is double tagged with VLAN 510. In other words VLAN 10 is encapsulated in VLAN 510
and sent across to SP2 hence the q-in-q tunneling.
Side note -- You will have to set the system mtu greater than 1500 on the SP1 and SP2 switches which
does require a reboot to take effect. In the Cisco 3750s that I used the command to increase the system
mtu is "system mtu 1546".
802.1q Tunneling
December 25, 2012 9 Comments
By taking traffic that is already 802.1q tagged and adding another tag on top of it we can create
an 802.1q, or Q-in-Q, tunnel. This is a handy feature that service providers may use in a metro
ethernet.
802.1q tunneling allows service providers to use a single VLAN to support multiple VLANs of
customers, while preserving customer VLAN IDs and keeping traffic in different customer
VLANs segregated.
Traffic from R1 enters SW1 with an 802.1q tag. SW1 adds another 802.1q tag and forwards the
frame across the network. When the frame reaches SW4 it strips the outer 802.1q tag and leaves
the inner one. When R4 receives the frame with the first 802.1q tag R4 processes the frame as if
it came directly from R1. I have created a Layer 2 tunnel through the switches. From the routers
perspective they are directly connected and will pass L2 traffic, like CDP. Here’s how the config
looks for it
R1(config)#int fa0/0
R1(config-if)#no shut
R1(config-if)#int fa0/0.14
R1(config-if)#encapsulation dot1q 14
R1(config-if)#ip address [Link] [Link]
R4(config)#int fa0/0
R4(config-if)#no shut
R4(config-if)#fa0/0.14
R4(config-if)#encapsulation dot1q 14
R4(config-if)#ip address [Link] [Link]
With the routers setup and ready to send and receive 802.1q tagged traffic, let’s use vlan 100. So
we will create the vlan and create trunks between the switches
SW1(config)#vlan 100
SW1(config-vlan)#int fa1/12
SW1(config-if)#switchport trunk encapsulation dot1q
SW1(config-if)#switchport mode trunk
SW2(config)#vlan 100
SW2(config-vlan)#int fa1/12
SW2(config-if)#switchport trunk encapsulation dot1q
SW2(config-if)#switchport mode trunk
SW2(config-if)#int fa1/23
SW2(config-if)#switchport trunk encapsulation dot1q
SW2(config-if)#switchport mode trunk
SW3(config)#vlan 100
SW3(config-vlan)#int fa1/23
SW3(config-if)#switchport trunk encapsulation dot1q
SW3(config-if)#switchport mode trunk
SW3(config-if)#int fa1/34
SW3(config-if)#switchport trunk encapsulation dot1q
SW3(config-if)#switchport mode trunk
SW4(config)#vlan 100
SW4(config-vlan)#int fa1/34
SW4(config-if)#switchport trunk encapsulation dot1q
SW4(config-if)#switchport mode trunk
Now let’s configure routers connections
SW1(config)#int fa1/0
SW1(config-if)#switchport access vlan 100
SW1(config-if)#switchport mode dot1q-tunnel
SW1(config-if)l2protocol-tunnel cdp
SW1(config-if)#no cdp enable
SW4(config)#int fa1/0
SW4(config-if)#switchport access vlan 100
SW4(config-if)#switchport mode dot1q-tunnel
SW4(config-if)#l2protocol-tunnel cdp
SW4(config-if)#no cdp enable
what we’ve done is :
–>access ports that connect to the routers we placed them in vlan 100
–>even though the router is already sending traffic with frames tagged with vlan 14, we are
appending ( or adding on top) this new vlan.
–>We told the switches that these ports at the beginning and ending of an 802.1q tunnel.
–>we want to send cdp traffic across this tunnel.
–>we don’t want to send or receive cdp traffic on this interface. The cdp traffic can pass through
the interface, but we aren’t participating in it.
Let’s see what the routers see.
R1#sho cdp neighbors
Capability Codes: R – Router, T – Trans Bridge, B – Source Route Bridge
S – Switch, H – Host, I – IGMP, r – Repeater
Device ID Local Intrfce Holdtme Capability Platform Port ID
R4 Fas 0/0 135 R S I 3750 Fas 0/0
