Cisco ACI - Unable to ping default gateway in a bridge domain
I've gotten to work with Cisco TAC and a few other groups on common ACI calls they're receiving. In this particular issue the customer was uanble to ping a default gateway that was configured within a bridge domain from a VM associated with an EPG that was also associated with the bridge domain.
After checking the VMM domain configurations we were able to verify that everything was done correctly. The APIC had pushed the proper portgroups to the DVS in vSphere and appeared to be up. However, when we looked at the leaf switch it was connected to we saw that it was receiving packets on the port to which the host was connected, but there were not packets being transmitted from that port. However, we could verify that the port physically seemed to work, so the problem was actually with the default gateway configured via the APIC.
When we checked the firmware version of the APIC compared to the leaf and spine switches they were in fact on different firmware versions. Once we upgraded the leaf and spine switches to the same version the APIC was on, traffic started flowing properly. Having different versions of firmware on the APIC than the switches is not a supported configuration from Cisco, and as seen here, can cause major problems within the environment. So, if you plan to upgrade, plan on upgrading the entire system, which can be done seamlessly if the system is designed with the proper redundancy.