While looking at some common troubleshooting our TAC team has done I came across this issue of a customer who was unable to get their vmk port on an ESXi host to talk to the SVI on a Nexus 7000 Series switch which was attached to a leaf within the ACI fabric. The N7K actually had a vPC connection to two of the leaf switches in the spine/leaf architecture.
When we looked at the mac-address table on the switch we saw it was not learning mac addresses. We also saw that all the interfaces seemed to be in discovery mode. While the N7Ks seemed to be passing VLAN information to the ACI fabric it was weird that we couldn’t ping the ESXi server from the N7K.
The interface on the leaf switch will actually stay out of service unless the following conditions are met:
1. An Attachable Entity Profile (AEP) must be associated wtih the Policy Group
2. A VLAN pool must be configured in the domain
3. CDP Traffic is being learned on the leaf (and CDP should be enabled both in the Service Profile if you’re using UCS B series servers as well as on the vSwitch/DVS which can be configured in vCenter).
Once the VLAN pool was created and mapped to the physical domain, as well as add the AEP to the interface policy group, the interface went into EPG mode and mac addresses were learned when we checked the mac-address table. At this point the customer was able to ping back and forth from the N7K and the ESXi host.