Creating Layer 3 Outside Access in Cisco ACI

Last month I wrote about creating layer 2 outside access in Cisco ACI. Perhaps even more necessary is creating layer 3 outside access or even doing both layer 2 and layer 3. It’s likely you’re going to need to connect your ACI fabric to your current network just to get traffic between the two environments. Keep in mind you can also apply policies even to the outside network (outside of ACI) using the APIC, as I talked about in the previous article.

In this article we’ll assume that we’re using OSPF as the dynamic routing protocol, though other routing protocols are allowed with ACI, too. We’ll be creating a connection to a Nexus 5000 switch, with an NSSA area type for the connection to the ACI fabric.

One of the jobs of the spines in the ACI fabric are to act as BGP Route Reflectors. There’s a lot of information out there on Route Reflection, but briefly it is a way to share routing information among all the switches in the fabric. Not every spine needs to be configured as a Route Reflector, but it is a best practice to have two configured for redundancy purposes.

To configure BGP Route Reflectors:

1. Login to the APIC
2. Click on the Fabric tab
3. Expand Fabric Policies
4. Expand Pod Policies
5. Expand Policies
6. Select BGP Route Reflectors Default
7. For the Autonomous System Number enter 1 in the pane on the right.
8. Click the + button to add Route Reflector Nodes
9. Click on the dropdown box to select a spine node ID
10. Click Submit twice

To create a pod policy:

1. Select Policy Groups under Pod Policy
2. Right click on Policy Groups and create a new one
3. Give it a name
4. Next to BGP Route Reflector Policy select BGP Route Reflectors Default, which is what we selected up in step 6 above.
5. Click Submit
6. Click on Default under Pod Policies in the tree on the left
7. In the dropdown box on the right, select the name that you specified in step 3 for the Fabric Policy Group
8. Click Submit

We can now go ahead and configure our layer 3 connection. Keep in mind this will be done on a per tenant basis. So at the top, click on the tenant in which you’d like to create this layer 3 external connection. For our example, we’ll go in to the Production Tenant and we’ll create a connection to a Nexus 5548 switch that lives in our already existing or legacy network.

1. In the APIC, click on Tenants
2. Click on Production
3. Expand Networking in the tree on the left
4. Right click on External Routed Networks and select Create Routed Outside
5. Enter a name for it such as Production-L3-Out-Prof
6. Select OSPF, which is what we’re using in this example
7. It by default enters 1 for the OSPF area
8. Click the + sign to pick which nodes you’d like to connect

Figure 1

9. Click the dropdown next to Private Network and choose the VRF you’ve created
10. Click the + sign under Nodes and Interfaces Protocol Profiles

Here you’ll create a Node Profile. When you click on the + sign a new window will pop up.

1. Enter a name such as Production-L3-Out-Prof
2. Click the + sign under OSPF Interface Profiles to create a new Interface Profile

Figure 2

3. Here you’ll use the dropdown menu to show the leaf nodes. Expand the leaf nodes to select the port on the leaf node which will connect to the router in your currently existing network.
4. Supply an OSPF Router ID
5. Click OK
6. Now click on the + sign under OSPF Interface Profiles
7. Give it a name such as Production-L3-Out-IntProf
8. Click on the + sign to assign routed interfaces

Figure 3

Figure 4

9. A new window will appear and you will select a path from the dropdown menu once again. Make sure you expand the leaf and choose an actual port here
10. Specify an IP address
11. Specify the MTU as well, whether these are regular or jumbo frames
12. Click OK twice to complete the tasks

At this point you’ll be back at the initial wizard. You can click Next to specify External EPG Networks. Essentially create this outside connection as an actual End Point Group, which means you can assign policies to it as you would to any EPG within the ACI Fabric. This is a pretty cool thing, because now we can even assign policies to outside objects while we’re migrating from our current network to the ACI fabric.

1. Click on the + sign to add and external EPG Network
2. Give it a name such as Production-L3-Out-EPG
3. Choose the subnet 0.0.0.0/0.0.0.0 to allow any connection. You can narrow this down as needed by subnet
4. Click Finish

Now we can go back and check that the Layer 3 external connection has been created by looking under External Routed Networks in the tree on the left.

Figure 5

The layer 3 external access EPG has now been created. With the case of OSPF timers can be very important. Generally speaking, OSPF timers need to be set to the same interval on all devices for communication to happen.

To set OSPF timers in the APIC:

1. Expand Private Networks located under Networking in the tree on the left
2. Select Production-VRF
3. In the right hand pane, under OSPF timers you can enter the correct interval there. You may also select default if your other OSPF timers are set to default in your environment

Next assign your new network to a bridge domain. If you’re unfamiliar with bridge domains in ACI, please see this article.

1. Expand Bridge Domains under Networking in the tree on the left
2. Select the bridge domain to which you’d like to assign it
3. In the pane on the right click the + sign next to Associated L3 Outs
4. In the dropdown menu, choose Production-L3-Out
5. Click Update
6. Click Submit to complete this task

We’ve now completed the layer 3 external connection. Keep in mind no traffic will flow in or out of the ACI fabric until a policy, or Contract, has been created because we’re still using a white list model within ACI. A contract is necessary between two EPGs, even if that EPG is an external Layer 3 EPG. For more information on creating contracts, please see this article.

If you have any questions, as always, please leave them in the comments section or reach out to me on Twitter @Malhoit.