Virtual Port Channel (vPC)

In this module, the following steps will be executed in order to configure and
verify vPC feature:
Enable vPC
Configure vPC domain
Configure vPC role and system priority
Configure vPC peer-keepalive link
Configure vPC peer-link
Verify vPC peer status
Configure vPC on vPC member ports
Verify vPC Operation
vPC Feature Enhancement Peer gateway, Peer Switch
Additional Tasks Auto-recovery, dual active exclude interface VLAN,
Peer-gateway exclude VLAN, vPC failure testing

NOTE: The vPC number and allowed VLAN documented in this manual may
be different from the one used in your setup. Please interpret the output
accordingly.

1.1 Enable vPC Feature

(config)# feature vpc

1.2 Configure vPC Domain

vPC domain ID is a unique number (from 1 to 1000); each Nexus VDC
instance can only belong to one vPC domain at a time. Configure terminal
monitor again, ensuring all syslog are capture in this terminal line.
(config)# terminal monitor
(config)# vpc domain ?
(config)# vpc domain <vpc_domain>
(config-vpc-domain)# show vpc
1.3 Configure vPC Role and System Priority
Each vPC member has a role (primary or secondary), it is calculated by the
role priority value plus local system mac, and the lowest value will be elected
as primary. The default role priority is 32768.

It is recommended to manually configure the vPC system priority when

running LACP to ensure that the vPC peer devices are the primary devices on
LACP. Make sure same priority value on both vPC peer devices, otherwise,
vPC will not be activated. The range of values is 1 to 65535. The default
value is 32667.

Configure N7K-1 as primary and N7K-2 as secondary by setting role priority.

Once the election is completed, the vPC role will not change unless the vPC
peer link connection is reset.
(config-vpc-domain)# role priority <vpc_role_pri>
(config-vpc-domain)# system-priority 1

1.4 Configure vPC Peer-keepalive Link

Configure vPC peer-keepalive between two Nexus. It is recommended to use
a separate L3 link for vPC keepalive exchange, as well as a separate vrf.
NOTE: You may also use management interface as keepalive
(config-vpc-domain)# vrf context vpc-keepalive
(config-vrf)# interface port-channel 2
(config-if)# vrf member vpc-keepalive
(config-if)# ip address <ip_3>/30
(config-if)# ping <remote_1> vrf vpc-keepalive
(config-if)# vpc domain <vpc_domain>
(config-vpc-domain)# peer-keepalive destination <remote_1> source <ip_3> vrf vpc-keepalive

1.5 Configure vPC Peer-Link

Configure vPC peer-link. The peer-link must be a 10GE link between the vPC
members. For this exercise, configure interface port-channel 1 as the peer-
link. The port-channel interface is configured using LACP in the previous
exercise. The supported channeling mode is ON or LACP. The port mode for
interface port-channel 1 is configured as trunk (all VLANs are allowed). The
supported port mode is trunk and access.
(config-vpc-domain)# int port-channel 1
(config-if)# vpc peer-link
1.6 Verify vPC Peer Status
Verify vPC peer status.
show vpc
show vpc consistency-parameters global
show system internal vpcm info global

WHAT TO LOOK FOR?

From your POD, execute show vpc:

# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 4
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 0
Peer Gateway : Disabled
Dual-active excluded VLANs : -

vPC Peer-link status

---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po1 up 1,11-12,99

What is the vPC domain ID?

What is the peer status?

What is the peer keep-alive status?

What is your Nexus vPC role?

What is vPC peer-link status? Is it UP? How many

active VLANs are there?

WHAT TO LOOK FOR? If the local and the peer

value is different, this
would consider
parameters
inconsistency.
From your POD, execute show vpc User must resolve all
consistency-
parameters global: Type-1 inconsistency
before vPC can be
operational.

For Type-2, vPC will still

functional using the
lower denominator
value.
# show vpc consistency-parameters global

Legend:
Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value

------------- ---- ---------------------- -----------------------
STP Mode 1 Rapid-PVST Rapid-PVST
STP Disabled 1 None None
STP MST Region Name 1 "" ""
STP MST Region Revision 1 0 0
STP MST Region Instance to 1
VLAN Mapping
STP Loopguard 1 Disabled Disabled
STP Bridge Assurance 1 Enabled Enabled
STP Port Type, Edge 1 Normal, Disabled, Normal, Disabled,
BPDUFilter, Edge BPDUGuard Disabled Disabled
STP MST Simulate PVST 1 Enabled Enabled
Interface-vlan admin up 2 11-12,99 11-12,99
Interface-vlan routing 2 1,11-12,99 1,11-12,99
capability
Allowed VLANs - 1,11-12,99 1,11-12,99
Local suspended VLANs - - -

Do you see any difference between local and peer value?

If so, is it Type-1 or Type-2 parameters inconsistency?

How to resolve it?
From you POD, execute show system internal vpcm info global
Check VPCM for status

# show system internal vpcm info global

Global Info:
Configuration Parameters:
Local System ID
Domain id: 1

OOB Configuration:
OOB VRF: vpc-keepalive Peer System ID
OOB Port: 3200
OOB Interval in ms: 1000
OOB Timeout in seconds: 5
Source Address: 1.1.1.1
Destination Address: 1.1.1.2
OOB Tos Value: 192 MCEC System ID
OOB Flush Timeout Value: 3

System Priority: 32667

System MAC: 00:00:00:00:00:00
Role Priority: 1
Tracking Object: 0
User configured interface-vlan: 0
User config interface vlan to exclude:
System Priority: 1
System MAC: d8:67:d9:04:1b:c2
Peer System id:
System Priority: 32667
System MAC: d8:67:d9:03:ef:42
MCEC System id:
System Priority: 32667
System MAC: 00:23:04:ee:be:01
<snip>

Also check show system internal vpcm info interface po11 -List
important sections of output that you see
1.7 Configure vPC on vPC Member Ports

Configure vPC port-channel. The port-channel number on the Nexus

switches does not have to be the same, which allow for easy migration. The
channel-mode will be set to LACP active.
(config-if)# interface <eth_4>,<eth_5>
(config-if-range)# no shut
(config-if-range)# switchport
(config-if-range)# channel-group 11 mode active
(config-if-range)# interface port-channel 11
(config-if)# switchport
(config-if)# switchport mode trunk
(config-if)# mtu 9216

Assign the vPC channel to the port channel that connects to the access
switch. It is best practice to match vPC number with PO number.
(config-if)# interface port-channel 11
(config-if)# vpc 11

1.8 Verify vPC Operation

show vpc
show run vpc all
show vpc brief vpc 11
show vpc consistency-parameters vpc 11
Verify vPC operation by pinging the IP address of the access switch.
ping <remote_2>

WHAT TO LOOK FOR?

From your POD, execute show vpc:

# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 4
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -

vPC Peer-link status

---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po1 up 1,11-12,99

vPC status
----------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
-- ---- ------ ----------- ------ ------------
11 Po11 up success success 1,11-12,99

Once again, make sure you work with your partner,

confirming the peer has vPC configured.

Verify vPC 11 is UP and you can reach the access switch

SVI.

Examine vPC local system-mac versus vPC system-mac. The vPC local
system-mac is same as the system LACP system identifier (same as VDC
mac address). It is recommended to assign unique vPC domain-ID for each
pair of vPC peer devices in the same L2 domain. Each vPC peer has its
own vPC local system-mac.
 show vpc role
show lacp system-identifier

The vPC system-mac is tied to the vPC domain ID. The Nexus pair within the
same vPC domain has the same vPC system-mac. The vPC system-mac is
being used for the LACP negotiation between the vPC switches and the
downstream switch, therefore, the downstream switch will see upstream
logical switch has a single vPC system-mac.
show vpc role
show lacp internal info interface <eth_4> | i lag

WHAT TO LOOK FOR?

# sh vpc role
This is used by LACP
vPC Role status internally
----------------------------------------------------
vPC role : primary
Dual Active Detection Status : 0
vPC system-mac : 00:23:04:ee:be:04
vPC system-priority : 1 This is same as LACP
vPC local system-mac : 00:22:55:7a:58:42 system identifier
vPC local role-priority : 12288

WHAT TO LOOK FOR?

# show lacp system-identifier
32768,0-22-55-7a-58-42

WHAT TO LOOK FOR?

From your POD, execute show vpc role, show lacp

system-identifier and show lacp internal info interface <eth_4>
| i lag:
# show lacp internal info interface ethernet 1/33
Interface Ethernet1/33(0x1a020000) info
--------------------------------------
port_pr 0x8000
rid type IF-Rid: ifidx 0x1a020000: ch_num 0
cfg_pc_if_idx 0x1600000a: oper_pc_if_idx 0x1600000a
lag [(1, 0-23-4-ee-be-4, 800b, 8000, 121), (8000, 0-a-f4-5e-11-0, 1, 8000, 39)]
aggr_id 0x0

Does the vPC system-mac match with the LACP internal

info?

Does the vPC local system-mac same as the lacp system-

identifier?

Check with your vPC peer, do both peers have the same
vPC system-mac and vPC system-priority?

WHAT TO LOOK FOR?

From your POD execute show system internal vpcm info

interface port-channel X and check for
 # show system internal vpcm info interface port-channel 11

port-channel11 - if_index: 0x1600000A

--------------------------------------------------------------------------------

Interface configured as vPC

IF Elem Information:

if_index: 0x1600000A
is_mcec: TRUE
mcec_num : 11
Number of allowed vlans(cfg_vlans): 3, Bitset: 1-3
<SNIP>
vPC number: 11
vPC state: Up Old Compat Status: Pass
Compat Status: Pass
Reason Code: SUCCESS
Hardware prog state: No R2 prog
Flags : 0x0 Number of members: 0

Number of Up Vlans: 3, Bitset: 1-3

Number of Suspended Vlans: 0, Bitset:

vPC Peer Information:

Peer number: 11
Peer state: Up
Number of configured VLANs on peer: 3, Bitset: 1-3
Number of Up VLANs on peer: 3, Bitset: 1-3

<SNIP>

Local Parameters::
==============================================
SAP APP-Name Param-Name Param-Type Param-Value
---- --------- ----------- ----------- ------------
171 STP STP Port Type Type-1 Default
171 STP STP Port Guard Type-1 None
<SNIP>

PEER Parameters::
==============================================
SAP APP-Name Param-Name Param-Type Param-Value
---- --------- ----------- ----------- ------------
171 STP STP Port Type Type-1 Default
171 STP STP Port Guard Type-1 None
<SNIP>

Allowed VLAN information, suspended VLAN and VLANs

that are up
Look for Local Parameters and Peer Parameters
1.9 vPC Feature Enhancement
Configure peer-gateway to enable vPC peer devices to act as the gateway
for packets destined to the vPC peer device's MAC address. This is necessary
to support NAS devices, load-balancers, and other devices which reply to
senders mac-address instead of HSRP virtual mac-address. Disable IP
redirects on all SVIs of the vPC VLANs to avoid generating IP redirect
messages if peer-gateway is configured
(config-if)# vpc domain <vpc_domain>
(config-vpc-domain)# peer-gateway
(config-vpc-domain)# interface vlan X
(config-if)# no ip redirects
(config-if)# show vpc | i "Peer Gateway"

Also, each of the Nexus switch assumes ownership of its peers mac address.
This enables a Nexus to proxy route the packet on behalf of its peer.

What to look for

# show mac address-table vlan X
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
G 299 0024.986f.bac4 static - F F sup-eth1(R)
G 299 0026.51ce.0f44 static - F F vPC Peer-Link(R)

G gateway bit is set

Beginning in NX-OS release 5.0.2, vPC peer-switch feature allows a pair of

vPC peer devices to behave as a single STP device and send BPDUs from
both vPC devices. This improves vPC convergence during vPC primary
switch failure.

(config-if)# vpc domain <vpc_domain>

(config-vpc-domain)# peer-switch
(config-vpc-domain)# spanning-tree vlan 1-4094 priority <peer-switch_pri>
(config)# show spanning-tree summary | i peer
(config)# show spanning-tree vlan 11
 WHAT TO LOOK FOR?
# show spanning-tree vlan 11 This is captured from
vPC primary
VLAN0011
Spanning tree enabled protocol rstp
Root ID Priority 12299 Both peers are shown as root
Address 0023.04ee.be04
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 12299 (priority 12288 sys-id-ext 11)

Address 0023.04ee.be04 vPC primary is
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec designated port

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------
Po1 Desg FWD 1 128.4096 (vPC peer-link) Network P2p
Po11 Desg FWD 1 128.4106 (vPC) P2p

WHAT TO LOOK FOR?

# show spanning-tree vlan 11
This is captured from
vPC secondary
VLAN0011
Spanning tree enabled protocol rstp
Root ID Priority 12299
Address 0023.04ee.be04 Both peers are shown as root
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 12299 (priority 12288 sys-id-ext 11)

Address 0023.04ee.be04 vPC secondary
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec is root port

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------
Po1 Root FWD 1 128.4096 (vPC peer-link) Network P2p
Po11 Desg FWD 1 128.4106 (vPC) P2p

Also check show spanning-tree interface etx/y detail to see the Bridge ID
and look at which interfaces are sending out BPDUs.

1.10Additional Tasks
Configure the following features
VPC Auto-Recovery
VPC Dual Active Exclude interface-vlan
VPC Peer-Gateway exclude-vlan
Check health of CFS
 (config-if)# vpc domain <vpc_domain>
(config-vpc-domain)# auto-recovery
(config-vpc-domain)# dual-active exclude interface-vlan
(config-vpc-domain)# peer-gateway exclude-vlan <LIST>

vPC failover testing :

Perform the following test and note down what you observer

Shut down keep-alive link and observer vPC status

A message is logged and vPC operates without any change

Bring the keep-alive up, remove auto-recovery if enabled and shut down the peer-link and observe what happens
The secondary vPC peer will suspend its vPCs

Now shut down the keep-alive link as well, what do you observe?
The secondary vPC peer will keep its vPCs suspended

Recover the system completely and repeat the procedure with auto-recovery enabled, what is different?
When the peer-link also is shut with keep alive down, the secondary will unsuspend its vPCs

CFS:

Nexus# show cfs internal ethernet-peer statistics | i Trans|Rece

Nexus# show cfs status
Number of Segments Transmitted : 218
Distribution : Enabled
Number of Acks Transmitted : 223
Distribution over IP : Disabled
Maximum Segment Size Transmitted :0

IPv4 multicast address : 239.255.70.83

Number of Transmission Timeouts :0

IPv6 multicast address : ff15::efff:4653 Number of segments in Transmit Queue :0

Distribution over Ethernet : Enabled Number of segments in Re-Transmit Queue :0

Nexus# show cfs peers Total Number of Segments Received : 441

Physical Fabric Number of Acks Received : 217

Number of Duplicate Messages Received :0

---------------------------------------------
Number of Unexpected Segments Received :0
Switch WWN IP Address

---------------------------------------------

TX/RX counters should

move when VPC is active
or coming up
Remote peer should be
seen
Nexus# sh cfs internal notification log name vpc
Shows timestamps for
Sun Nov 14 15:27:22 2010: Peer add 20:00:00:1b:54:c2:42:44 when CFS
communication for VPC
Sun Nov 14 19:05:25 2010: Peer gone 20:00:00:1b:54:c2:42:44 was interrupted (peer-