Introduction
The Application Policy Infrastructure Controller is the brains of the Cisco ACI solution. It’s where you do all of the management for this SDN solution, but does not actually forward the packets. The data plane is all handled by the switches. In ACI, you will have at least 3 APICs set up, which right now are hardware appliances. You need three APICs to keep up redundancy, though the fabric would still work even if all the APICs go down, and you would not be able to make, change, or delete policies. More specifically they are Cisco UCS C220 M3s with a hardened APIC image built in. While you can use the CIMC to manage it still, these UCS appliances will not be managed by the UCS Manager as most UCS servers are.
Let’s get to the meat of this article, setting up your first APIC. After racking and stacking all your Nexus 9000 switches as well as your APICs, you would at this point have everything connected to the network. You can either use the CIMC address that you have setup on the APIC to connect to it over the network, or you can plug in a monitor and keyboard to the APIC itself. After it boots you’ll see the following screen:
Figure 1
Cluster Configuration
Name:
The first thing it asks for is the fabric name. This is somewhat arbitrary, but if you have multiple ACI fabrics you’ll want to make sure you use a name that will help you recognize the fabric you’re working with. Perhaps you would even use a location within the name.
Number of Controllers:
Usually you will have three controllers, especially when starting out. If you have a larger environment, more APICs may be necessary.
Controller ID:
This is actually very important. If you’re configuring your first APIC, you have to put 1. You cannot configure the APICs out of order for the first time. You must boot them and set them up in order. After they are all set up it doesn’t matter what boot order you might use later on.
Controller Name:
This is the name you’re giving to each APIC. You may want to specify that it’s APIC1 or something indicating that it’s the first controller to match the controller ID configured above. Other than that it’s pretty much an arbitrary name that will help you identify your APIC.
TEP Address Pool:
A TEP is a Tunnel End Point. This is used for VXLAN traffic and you must have TEPs to send traffic over the VXLANs. The ACI fabric uses VXLAN exclusively within the infrastructure space, or the space between the spine and leaf switches. This pool will assign each TEP an IP address on your leaf and spine switches.
While this is in the infrastructure space, you may run into issues if you use a subnet that’s already being used within your environment. If you’re not using the 10.x.x.x/16 subnet feel free to use the default. Cisco recommends not being too stingy when it comes to assigning this subnet as well. So, using a /16 is suggested.
VLAN ID for Infra Network:
Again the Infra Network (or Infrastructure Network) is the space between your leaf and spine switches. We only need to use one VLAN ID because it will use VXLAN to allow you to scale. In the image above it uses 4093 as 4094 is reserved for other communication. However, if you’re using UCS within your environment you may want to take care not to use VLANs 3968-4047 as this range of VLANs is also reserved. For more information check out the UCS Manage Configuration Guide. So in this case you’ll want to use VLAN ID 3967. Of course, check with your vendors for other VLAN reservations.
BD Multicast Addresses (GIPO):
The BD in this case stands for Bridge Domain, which in the case of ACI is essentially a container for subnets. All of your End Point Groups will be associated with a Bridge Domain within the ACI fabric and some traffic forwarding behavior will be dictated in how these Bridge Domains are configured. In this case, we’re assigning multicast addresses to the bridge domains. You may accept the default which is 225.0.0.0/15, but if you’re already using that the following range is acceptable: 225.0.0.0/15 – 231.254.0.0/15. A /15 must be used which will give you 128,000 IPs. And in case you were wondering what GIPO stands for it is Global IP Outside.
Out-of-Band Management Configuration
The OOB management is pretty simple and you’ve probably configured this type of thing dozens of times. Out of Band simply means that it’s not in the ACI fabric, but using another network, likely a dedicated management network. You can use in-band management as well, if you like. For this you need an IP address, default gateway, and speed specification.
IP Address:
Give it a free IP address from your management network. Keep in mind this is not the same as your CIMC IP address. This will actually be the address you’ll browse to in order to manage the APIC GUI. However, it may be in the same network as your CIMC.
Default Gateway:
This is the router IP that it will use to connect to other networks in your environment or to get out to the public Internet depending on how you have things set up.
Speed:
Most likely you’ll leave this at auto, unless there is some reason you’re specifying a speed on your interfaces (100MB, 1GB, 10GB, etc.).
Admin User Configuration
This is just a username and password for your APIC. This will turn out to be the fabric admin which has the rights and privileges to manage your entire ACI environment. We would recommend turning on strong passwords, especially for a production environment.
Finally it will ask you if you’d like to edit the configuration, and if you’re happy with the setup you can go ahead and say No. At that point the APIC will reboot and you’ll see a login prompt instead of the startup script. You may either login here to test your credentials or you can browse to the management IP and start managing the APIC through the GUI.
Conclusion
Though there’s a lot of explanation here you really only need to answer about 10 questions to get your first APIC set up. For the second and third APICs you can match the questions accordingly, but of course change the names and IP addresses for the individual APIC. It will know it’s part of the cluster. At this point you can start provisioning your fabric, which will happen automatically as shown in this article. Truly, it’s possible to get all your APICs set up and your fabric provisioned in about 15 minutes at which time you’re ready to start creating policies!
If you have any questions about this article, ACI, SDN, or virtualization in general please place your comments below or reach out to be on Twitter @Malhoit.
