- Basics of vPC : Virtual Port Channel (vPC)- Part 1
- vPC Inconsistency and Control Plane
- vPC Failure Scenarios: vPC – Part III
- vPC with HSRP : vPC with HSRP – Part IV
- vPC Design Variations vPC Design Variations
In Virtual Port Channel (vPC)- Part 1, we discussed about the various vPC components and the order of operations. Now its time to dig deeper into vPC.
Topics discussed in this blog:
- Consistency Check
- Type-1 Inconsistency
- Type-2 Inconsistemcy
- Control Plane
- ARP Synchronization
- Duplicate Frame Control
Consistency check is performed to make sure both the switches have identical configuration. We have two types of Consistency checks and action is taken based on what sort of inconsistency is detected.
- Type 1 : In case of Type-1 inconsistency, vPC fails to form adjacency.
- Type 2: It results in a syslog message informing that we have encountered a Type 2 inconsistency, however will not impact the vPC adjacency. This may result in data plane failures.
These are further categories into two types:
- Global: The consistency check is performed on the global configuration and under vPC domain. Type 1 global inconsistency will suspend the vPC. To verify which parameters are compulsory to be matched , use below command.
For instance, STP related configuration is considered to be Type-1
Let me change the Spanning Tree Mode from Rapid-PVST to MST.
Now we will see that even though the peer is alive, the Member ports are suspended due to Type-1 inconsistency
“show vpc” tells that there is inconsistency in STP configuration.
“show vpc consistency-parameters global” will indicate what actually is misconfigured. So here, the local and Peer STP modes are different.
- Interface: Any sort of inconsistency due to misconfiguration on interface level. For instance, if STP port type is different , it is considered as Type 1 interface inconsistency and can be viewed via “show vpc consistency-parameters interface <>”
- There is one more check that is performed which is allowed VLANs on the interfaces. In the above snip, we can see that as of now allowed vlans on both ends are 1,10,20.
Lets configure Vlan 30 on Leaf-1. Though VLAN 30 is not configured on Leaf-2, still vPC peer and Member ports are up. This new configuration din’t have any impact.
Now lets verify the consistency.
Did you see it???
The allowed VLANs list on local device is now 1,10,20,30. Since VLAN 30 is not present in allowed list for the peer, local switch suspended that particular Vlan and no warning was displayed in “show vpc” output.
Also in the above snip, there is a Type-2 inconsistency reported as well for Interface-Vlan. There are two SVIs configured on the local device, not on peer device. “show vpc” will not indicate anything about Type-2 inconsistency
When the traffic is intended for the switch, it is control plane traffic and data plane traffic is when it gets switched. Both the switches forward the data traffic but only one of them is responsible for processing the control plane traffic.
This is decided by the vPC role. The switch with lower priority will become the active and in case of same priority, the one with lower MAC address will take over the role of Primary vPC.
Cisco Fabric Services over Ethernet (CFSoE) is used to synchronize the control plane between Primary and Secondary vPC peers. Once the vpc feature is enabled, CFSoE is enabled automatically. No need to configure it explicitly. To verify if CFS is running, enter the “show cfs application” command and it should display “Physical-eth” as Scope.
Suppose, the connected device chose the link towards Secondary switch based on hashing algorithm for sending lets say ARP request. When Secondary receives the control plane traffic, it forwards the same over the Peer Link to Primary peer. The primary peer will then process and respond to the ARP request.
Once the primary switch learns and updates its MAC address table, CFS will synchronize the same information to Secondary switch.
CFS messages are encapsulated in Ethernet frames between peers on Peer-Link. CFS status can be checked using “show cfs status”.
The default IPv4 multicast address used by CFS for distribution over IP is 126.96.36.199, however can be configured manually as well.
BPDU’s are also processed by Primary vPC only and both the switches share the same Bridge ID generated by vPC System MAC. As per the below packet capture, Bridge ID is the one generated by vPC system MAC but the Root Id is the system’s local MAC address as whenever the packet is generated, it is from system’s local MAC address.
Snip from Server point:
Snip from vPC Root:
The default behavior of only primary processing BPDU’s can be changed using “peer-switch” command under vpc domain to allow both the switches to process BPDUs. Lets configure it on both the switches:
See the change in output as soon as “peer-switch” is configured at both the end. We are now able to see that both switches now share the same Bridge Id.
Packet capture of BPDU now has only vPC virtual MAC as root and Bridge ID:
The main advantage of using “peer-switch” is that both the switches process BPDU and during failure scenarios, will reduce BPDUs loss.
The synchronization of ARP is disabled by default and status can be checked using “show ip arp vpc-statistics”.
This can be enabled by using “ip arp synchronize” under “vpc domain”.
Once the vPC link is flapped, CFSoE will synchronize the ARP entries as well to reduce the packet loss during ARP resolution for unicast traffic.
Duplicate Frame Control:
vPC has a major rule to avoid duplicate frames being received by the server.: “If a member port receives a frame, it is forwarded across peer link with a tag which identifies that this frame was sourced from member port. When vPC peer receives that frame and if destination is another member port, frame is dropped. However if destination is other than vPC member port, frame is forwarded.”
This prevents peer link from carrying the East-West traffic which should be switched North-South.
Lets take a look at our topology and suppose there is a frame received by Leaf-1 from Server-1 destined for Server-3.
- Leaf-1 forwards the frame to Server-3 and also to Leaf-2.
- Leaf-2 will check that this packet is destined to Server-3.
- It will think Why should I forward this packet to Server-3, it makes no sense as I received this packet from my vPC peer who in turn received this from member port.
- This will result in duplicate frame being received by Server-3 and unwanted BW utilization of member port.
- So Leaf-2 drops this frame.
This rule is not applicate for orphan ports.
The Duplicate Frame Control rule has one disadvantage as well. When a vPC primary peer responds to ARP request, the source MAC in ARP response is the vPC Virtual MAC and the source MAC from which response is sent is the vPC’s local system MAC, popularly known as Burned in MAC address (BIA)
Some devices consider the Source MAC of the packet rather the source MAC in ARP response field, hence they end up sending the packets destined to Primary peer’s local system MAC.
As the vPC peers act as a single switch, so Server can chose any of the link from bundle to send the packet depending upon the hashing algorithm. In case packet ends up landing at the Secondary peer and it will see that this is supposed to be sent out on vPC peer link as the destination belongs to primary peer.
When primary peer receives the frame, Duplicate Frame Control Rule kicks in and packet is dropped.
The solution to this issue is “peer-gateway“. This will allow both the vPC peer switches to act as a Gateway and reply to each other’s BIA address on each other’s behalf. So unnecessary flooding on peer link is reduced.
As per the “show mac address-table”, only the local MAC address is tagged as Gateway MAC
Configure peer-gateway under vpc domain:
Now both the peer’s BIA address will act as Gateway MAC in the MAC table:
We will be posting further topics on vPC in the upcoming series. Please do let us know if there is any specific topic you want us to touch.