Chapter 9 L3Outs

Skip to content
 Home

o Your O'Reilly
 Profile
 History
 Playlists
 Highlights
o Answers
o Explore
 All Topics
 Most Popular Titles
 Recommended
 Early Releases
 Shared Playlists
 Resource Centers
o Live Events
 All Events
 Architectural Katas
 AI & ML
 Data Sci & Eng
 Programming
 Infra & Ops
 Software Arch
o Interactive
 Scenarios
 Sandboxes
 Jupyter Notebooks
o Certifications
o Settings
o Support
o Newsletters
o Sign Out
Table of Contents for

CCNP Data Center Application Centric Infrastructure 300-620 DCACI
Official Cert Guide
CLOSE
CCNP Data Center Application
Centric Infrastructure 300-620 DCACI Official Cert Guideby Ammar
AhmadiPublished by Cisco Press, 2021
1. Cover Page (01:09 mins)
2. About This eBook (01:09 mins)
3. Title Page (01:09 mins)
4. Copyright Page (03:27 mins)
5. About the Author (01:09 mins)
6. About the Technical Reviewers (01:09 mins)
7. Dedication (01:09 mins)
8. Acknowledgments (01:09 mins)
9. Contents at a Glance (01:09 mins)
10. Reader Services (01:09 mins)
11. Contents (13:48 mins)
12. Icons Used in This Book (01:09 mins)
13. Command Syntax Conventions (01:09 mins)
14. Introduction (12:39 mins)
15. Figure Credit (01:09 mins)
16. Part I Introduction to Deployment (01:09 mins)
o Chapter 1 The Big Picture: Why ACI? (32:12 mins)
o Chapter 2 Understanding ACI Hardware and Topologies (42:33 mins)
o Chapter 3 Initializing an ACI Fabric (93:09 mins)
o Chapter 4 Exploring ACI (59:48 mins)
17. Part II ACI Fundamentals (01:09 mins)
o Chapter 5 Tenant Building Blocks (44:51 mins)
o Chapter 6 Access Policies (55:12 mins)
o Chapter 7 Implementing Access Policies (92:00 mins)
o Chapter 8 Implementing Tenant Policies (97:45 mins)
18. Part III External Connectivity (01:09 mins)
o Chapter 9 L3Outs (125:21 mins)
o Chapter 10 Extending Layer 2 Outside ACI (60:57 mins)
19. Part IV Integrations (01:09 mins)
o Chapter 11 Integrating ACI into vSphere Using VDS (54:03 mins)
o Chapter 12 Implementing Service Graphs (69:00 mins)
20. Part V Management and Monitoring (01:09 mins)
o Chapter 13 Implementing Management (35:39 mins)
o Chapter 14 Monitoring ACI Using Syslog and SNMP (51:45 mins)
o Chapter 15 Implementing AAA and RBAC (63:15 mins)
21. Part VI Operations (01:09 mins)
o Chapter 16 ACI Anywhere (26:27 mins)
22. Part VII Final Preparation (01:09 mins)
o Chapter 17 Final Preparation (10:21 mins)
23. Appendix A Answers to the “Do I Know This Already?” Questions (27:36 mins)
24. Appendix B CCNP Data Center Application Centric Infrastructure DCACI 300-620
Exam Updates (02:18 mins)
25. Glossary (23:00 mins)
26. Index (69:00 mins)
27. Appendix C Memory Tables (32:12 mins)
28. Appendix D Memory Tables Answer Key (34:30 mins)
29. Appendix E Study Planner (04:36 mins)
30. Where are the companion content files? - Register (01:09 mins)
31. Inside Front Cover (01:09 mins)
32. Inside Back Cover (01:09 mins)
33. Code Snippets (05:45 mins)
 Search in book...

 Toggle Font Controls

o
o
o
o
PREV Previous Chapter
Part III External Connectivity
NEXT Next Chapter

Chapter 10 Extending Layer 2 Outside ACI
Chapter 9
L3Outs
This chapter covers the following topics:
L3Out Fundamentals: This section covers concepts related to L3Outs, the

subobjects that make up L3Outs, interface types, and BGP route reflection.
Deploying L3Outs: This section details the process of establishing routing to
the outside world via user tenant L3Outs.
Implementing Route Control: This section covers the basics of route profiles
and some use cases for route profiles in ACI.
This chapter covers the following exam topic:

 3.2 Implement Layer 3 Out
Previous chapters address how ACI access policies control the configuration of
switch downlink ports and the level of tenant access to such ports. This chapter
expands on that information by addressing the implementation of L3Outs on
switch downlinks to communicate subnet reachability into and out of ACI user
VRF instances.
This chapter tackles a number of issues related to L3Outs, such as classifying
external endpoints for contract enforcement, implementing BGP route
reflection, and basic route filtering and route manipulation.
“DO I KNOW THIS ALREADY?” QUIZ

The “Do I Know This Already?” quiz allows you to assess whether you should
read this entire chapter thoroughly or jump to the “Exam Preparation Tasks”
section. If you are in doubt about your answers to these questions or your own
assessment of your knowledge of the topics, read the entire chapter. Table 9-
1 lists the major headings in this chapter and their corresponding “Do I Know
This Already?” quiz questions. You can find the answers in Appendix A,
“Answers to the ‘Do I Know This Already?’ Questions.”
Table 9-1 “Do I Know This Already?” Section-to-Question Mapping
Foundation Topics Section Questio
L3Out Fundamentals 1–4
Deploying L3Outs 5–9
Implementing Route Control 10

Caution
The goal of self-assessment is to gauge your mastery of the topics in this chapter. If you
do not know the answer to a question or are only partially sure of the answer, you should
mark that question as wrong for purposes of the self-assessment. Giving yourself credit
for an answer you correctly guess skews your self-assessment results and might provide
you with a false sense of security.
1. An ACI administrator wants to deploy an L3Out to ASR 1000 Series routers
in a user VRF while ensuring that future L3Outs in other tenants can also reuse
the same physical connectivity to the outside routers. Which interface type
should be used?
1. Routed subinterfaces
2. Routed interfaces
3. SVIs
4. Floating SVIs
2. An L3Out has been created in a user VRF and border leaf switches have
learned external subnet 10.0.0.0/8 via dynamic routing and added it to their
routing tables. Which of the following explains why a compute leaf that has
deployed the same user VRF instance has not received that route?
1. A user has configured a route profile for interleak and applied it to the
L3Out.
2. A default import route profile has been added to the L3Out.
3. Routing protocol adjacencies between ACI and external routers never
formed.
4. BGP route reflection has not been configured for the fabric.
3. Which interface type is ideal for L3Out deployment when a physical firewall
appliance needs to be dual-homed to a pair of leaf switches and establish
routing protocol adjacencies directly with an ACI fabric?
1. Routed subinterfaces
2. Routed interfaces
3. SVIs
4. Floating SVIs
4. When configuring BGP route reflection in ACI, what are the two critical
parameters that administrators need to define?
1. BGP ASN and cluster ID
2. Border leaf switches and L3Outs from which ACI should import routes
3. BGP ASN and the spines in each pod that should function as route
reflectors
4. BGP ASN and border leafs in each pod that should function as a route
reflector client
5. An administrator learns that when she modifies a bridge domain subnet scope
to Advertised Externally and adds the bridge domain for advertisement to an
OSPF L3Out, the subnet also gets advertised out an EIGRP L3Out. Which of
the following options would allow advertisement of the BD subnet out a single
L3Out? (Choose all that apply.)
1. Remove the bridge domain from the EIGRP L3Out.
2. Move one of the L3Outs to a different switch or set of switches.
3. Switch the bridge domain back to Private to VRF.
4. Use BGP for a dedicated route map per L3Out.
6. True or false: The same infra MP-BGP ASN used for route reflectors is also
used to establish connectivity out of BGP L3Outs unless ACI uses a local-
as configuration.
1. True
2. False
7. Regarding configuration of BGP L3Outs, which of the following statements
are true? (Choose all that apply.)
1. When establishing BGP connectivity via loopbacks, BGP peer connectivity
profiles should be configured under the node profile.
2. ACI allows EIGRP to be configured on a BGP L3Out for BGP peer
reachability in multihop scenarios.
3. ACI tries to initiate BGP sessions with all IP addresses in a subnet as a result of
the dynamic neighbor establishment feature involving prefix peers.
4. ACI implements BGP subnet advertisement to outside as a redistribution.
8. Which statements about OSPF support and configuration in ACI are correct?
(Choose all that apply.)
1. The administrative distance for OSPF can be modified at the node profile level.
2. OSPF authentication is supported in ACI and can be configured under an OSPF
interface profile.
3. Border leaf L3Outs support VRF-lite connectivity to external routers.
4. ACI does not support OSPFv3 for IPv6.
9. Which statements are correct regarding ACI support for BFD? (Choose all
that apply.)
1. BFD is supported for EIGRP, OSPF, and BGP in ACI.
2. BFD is supported on L3Out loopback interfaces.
3. BFD is supported for BGP prefix peers (dynamic neighbors).
4. BFD is supported on routed interfaces, routed subinterfaces, and SVIs.
10. True or false: Route profiles are a little different from route maps on NX-OS
and IOS switches and routers because they may merge configurations between
implicit route maps and explicitly configured route profile match statements.
1. True
2. False
FOUNDATION TOPICS
L3OUT FUNDAMENTALS
An ACI Layer 3 Out (L3Out) is the set of configurations that defines
connectivity into and out of an ACI fabric via routing. The following sections
discuss the key functions other than routing that an L3Out provides and the
objects that comprise an L3Out.
Stub Network and Transit Routing

To better understand L3Outs, it helps to first understand the difference between
a stub network and a transit network. ACI was originally built to function as a
stub network. As a stub, the intent was for ACI to house all data center
endpoints but to not be used to aggregate various routing domains (for example,
core layer, firewalls, WAN, campus, mainframe) within a data center.
The community of network engineers then banded together and told Cisco that
ACI as a stub was not enough. Shortly afterward, Cisco began to support ACI as
a transit. The idea is that ACI performs route redistribution, mostly behind the
scenes, to interconnect multiple routing domains (L3Outs) and transit routes as
well as traffic between L3Outs.
Figure 9-1 compares ACI as a stub and ACI as a transit. On the left-hand side,
ACI is depicted as a stub. The fabric has only a single L3Out. In this case, ACI
does not need to do anything to campus subnet 10.10.0.0/16 except to learn it.
On the right-hand side, however, the campus core layer has been depicted
connecting directly into ACI using an L3Out. Data center core and edge
infrastructure connect to a separate L3Out. Because the data center core and
campus layers do not have any direct connectivity with one another, ACI needs
to be configured to transit routes between the L3Outs for machines in the
10.10.0.0/16 subnet to be able to reach the Internet. It is important to understand
that it is not the existence of multiple L3Outs that implies transit routing; it is
the expectation that ACI functions as a hub and routes traffic from one L3Out to
another that necessitates transit routing.
Figure 9-1 Understanding ACI as a Stub Versus ACI as a Transit
Note
Route redistribution between Layer 3 domains is a large topic, and it is not something
that the majority of ACI engineers deal with on a day-to-day basis. Therefore, transit
routing is beyond the scope of the Implementing Cisco Application Centric
Infrastructure DCACI 300-620 exam.
Types of L3Outs
Chapter 5, “Tenant Building Blocks,” provides a basic definition of L3Outs. It
may be possible to interpret that definition as suggesting that each L3Out is
only able to advertise subnets that exist within a single VRF. The following
points expand on the previous definition by describing the numerous categories
of L3Outs you can create in ACI:
 L3Outs in the infra tenant: This type of L3Out is actually more
of a family of L3Outs that typically associate with the overlay-1 VRF
instance and enable either integration of an ACI fabric with other fabrics
that are part of a larger ACI Multi-Site solution or expansion of a fabric
into additional pods as part of an ACI Multi-Pod or vPod solution. The
overlay-1 VRF can also be used to interconnect a fabric with its Remote
Leaf switches. These solutions should all be familiar from Chapter 2,
“Understanding ACI Hardware and Topologies.” One particular infra
tenant L3Out, however, has not been mentioned before. A GOLF L3Out
uses a single BGP session to extend any number of VRF instances out an
ACI fabric to certain OpFlex-capable platforms, such as Cisco ASR 1000
Series routers, Cisco ASR 9000 Series routers, and Nexus 7000 Series
switches. One common theme across the infra tenant family of L3Outs is
that traffic for multiple VRFs can potentially flow over these types of
L3Outs. Another common theme is that routing adjacencies for these
L3Outs are sourced from the spines.
 User VRF L3Outs (VRF-lite): This type of L3Out extends Layer
3 connectivity out one or more border leaf switches. It is the most
commonly deployed L3Out in ACI. Administrators typically deploy
these L3Outs in user tenants, but they can also be deployed in VRFs in
the common tenant or any other VRF used for user traffic. Each L3Out of
this type is bound to a single user VRF and supports a VRF-lite
implementation. Through the miracle of subinterfaces, a border leaf can
provide Layer 3 outside connections for multiple VRFs over a single
physical interface. This VRF-lite implementation requires one protocol
session per tenant. A shared service L3Out is the name given to any user
VRF L3Out that has additional configuration to leak routes learned from
external routers into a VRF other than the VRF to which the L3Out is
bound. This allows external routes to be consumed by EPGs in another
VRF. This feature is also referred to as shared L3Out because a service
behind the L3Out is being shared with another VRF.
If all this seems overwhelming, don’t worry. The DCACI 300-620 exam only
focuses on implementation of user VRF L3Outs. Route leaking and shared
service L3Outs are also beyond the scope of the exam. Nonetheless, you need to
be aware of the different types of L3Outs so you can better recognize which
configuration knobs are relevant or irrelevant to the type of L3Out being
deployed.
Note
The term shared service is used here to refer to a user tenant using a component from the
common tenant. This term can also apply to consumption of common tenant objects by
other tenants, but use of the term to imply route leaking is now more prevalent.
Key Functions of an L3Out

For an L3Out to be effective in connecting an ACI fabric to outside Layer 3
domains, it needs to be able to perform the following five critical functions:
 Learn external routes: This entails one or more border leafs
running a routing protocol and peering with one or more external devices
to exchange routes dynamically. Alternatively, static routes pointing to a
next-hop device outside the fabric can be configured on an L3Out.
 Distribute external routes: This involves ACI distributing
external routes learned on an L3Out (or static routes) to other switches in
the fabric using Multiprotocol BGP (MP-BGP) with VPNv4 configured
in the overlay-1 VRF (tenant infra). ACI automates this distribution of
external routes in the background. This distribution requires BGP route
reflection.
 Advertise ACI bridge domain subnets out an L3Out: For
external devices to have reachability to servers in the fabric, an ACI
administrator needs to determine which BD subnets should be advertised
out of an L3Out. ACI then automates creation of route maps in the
background to allow the advertisement of the specified BD subnets via
the selected L3Outs.
 Perform transit routing: Advertising external routes between
Layer 3 domains can be achieved by using the L3Out EPG subnet scope
Export Route Control Subnet. (The DCACI 300-620 exam does not
cover transit routing, so neither does this book.)
 Allow or deny traffic based on security policy: Even with ACI
exchanging routes with the outside world, there still needs to be some
mechanism in place for ACI to classify traffic beyond an L3Out and
determine whether a given source should be allowed to reach a particular
destination. L3Outs accomplish this by using special EPGs referred to as
external EPGs. EPGs classifying traffic beyond an L3Out are configured
on the L3Outs themselves using the L3EPG scope External Subnets for
External EPG.
The Anatomy of an L3Out

Each L3Out is structured to include several categories of configuration. Figure
9-2 shows a sample L3Out and its subobjects.
Figure 9-2 The Anatomy of an ACI L3Out
The following points summarize the structure and objects within an L3Out:
 L3Out root: As shown in Figure 9-2, the most critical L3Out

configurations that are of a global nature can be found under the Policy >
Main page at the root of the L3Out. Configurations you apply here
include the routing protocol to enable on the L3Out, the external routed
domain (L3 domain) linking the L3Out with the underlying access
policies, the VRF where the L3Out should be deployed, the autonomous
system number in the case of EIGRP, and the area type and area ID to be
used for OSPF.
 Logical node profiles: The main function of a logical node
profile is to specify which switches should establish routed connectivity
to external devices for a given L3Out. ACI creates two subfolders under
each logical node profile: Logical Interface Profiles and Configured
Nodes.
 Logical interface profiles: An administrator can configure one or
more logical interface profiles for each set of interface configurations. It
is under logical interface profiles that interface IP addresses and MTU
values for routing protocol peering can be configured. Protocol-specific
policies such as authentication and timers for EIGRP, OSPF, and BGP
can also be configured under logical interface profiles. Bidirectional
Forwarding Detection (BFD) and custom policies for QoS, data plane
policing, NetFlow, and IGMP can also be applied at the logical interface
profile level.
 Configured nodes: The Configured Nodes folder includes a single
node association entry for each switch node that is part of an L3Out.
Static routes, router IDs, and loopback IP addresses are all configured in
node association entries in this folder.
 External EPGs: Any EPG that is defined directly on an L3Out can
be called an external EPG. The only external EPGs of concern for the
DCACI 300-620 exam are those used to classify traffic from external
endpoints.
 Route map for import and export route control: These special
route maps are applied on the entire L3Out and associated bridge
domains. The two route maps that can be applied are default-import and
default-export.
Planning Deployment of L3Out Node and Interface

Profiles
As mentioned earlier in this chapter, logical node profiles and logical interface
profiles for user VRF L3Outs together define which switches are border leaf
switches and what interface-level configurations and policies the L3Out should
use for route peering with the outside world. But how are these two object types
deployed side-by-side?
The first thing to remember is that logical interface profiles fall under the
Logical Node Profiles folder. On the left-hand side of Figure 9-3, an
administrator has configured two leaf switches with node IDs 101 and 102 as
border leaf switches using a single logical node profile. A single logical
interface profile under the logical node profile then defines configurations for
interfaces on both of these nodes. In the second design pattern, the administrator
has created a single node profile but then deployed one interface profile for each
switch. In the iteration on the right, separate logical node profiles and interface
profiles have been created for each individual border leaf switch. All three of
these design patterns are correct.
Figure 9-3 Logical Node Profile and Logical Interface Profile Design Patterns
Some of the confusion that occurs for engineers when deciding between these
options is due to the fact that most engineers think in terms of design patterns
for access policies. Deployment of an interface configuration under an interface
profile that is bound to multiple switches under the Access Policies menu can
lead to the simultaneous deployment of interface configurations on multiple
switches. In contrast, each interface or port channel added under an L3Out
configuration requires explicit configuration before routing can be enabled on
the port. Therefore, logical node profiles and logical interface profiles are used
more to achieve organizational hierarchy than for automating simultaneous
interface configuration.
Understanding L3Out Interface Types

The following types of interfaces can be part of an L3Out:
 Routed subinterfaces
 Routed interfaces
 Switch virtual interfaces (SVIs)
 Floating SVIs
The design considerations for the first three options listed here are mostly the
same as what you would expect when configuring connectivity between any set
of Layer 3 devices.
Use of routed subinterfaces is very common in multitenancy environments
because one VLAN ID and subinterface can be allocated for each user VRF
L3Out, enabling multiple tenant L3Outs to flow over a single set of physical
ports.
Use of router interfaces is most common in single-tenant environments or in
instances when the fabric is expected to have dedicated physical connectivity to
a specific switch block or segregated environment via a single L3Out.
If an administrator has trunked an EPG out a port or port aggregation, the same
port or port channel cannot be used as for a routed subinterface or routed
interface because the port has already been configured as a (Layer 2)
switchport. Such a port or port aggregation can still be configured as part of an
L3Out that leverages SVIs or floating SVIs.
Note
It is has become very common for engineers to build ACI L3Outs using pure Layer 3
solutions such as routed interfaces and routed subinterfaces, but there are many great use
cases for building L3Outs using SVIs. One common use case is establishing a redundant
peering with firewalls over a vPC from a pair of border leaf switches.
The new floating SVI option, first introduced in ACI Release 4.2(1), with
further enhancements in ACI Release 5.0(1), enables users to configure an
L3Out without locking down the L3Out to specific physical interfaces. This
feature enables ACI to establish routing adjacencies with virtual machines
without having to build multiple L3Outs to accommodate potential VM
movements.
The floating SVI feature is only supported for VMM integrated environments in
ACI Release 4.2(1) code. Enhancements in ACI Release 5.0(1) allow floating
SVIs to be used with physical domains, eliminating the requirement for VMM
integration.
Understanding L3Out Bridge Domains

When instantiating an L3Out SVI, ACI creates an L3Out bridge domain
(BD) internally for the SVI to provide a Layer 2 flooding domain. This BD is
called an L3Out BD or external BD and is not visible to ACI administrators.
ACI creates a different L3Out BD for each encapsulation used in an SVI-based
L3Out. If an administrator uses a common VLAN encapsulation for SVIs on
multiple border leaf nodes in a single L3Out, ACI spans the L3Out BD and the
associated flooding domain across the switches. In Figure 9-4, L3Out SVI
encapsulation 10, deployed to both border leafs 101 and 102, prompts ACI to
place all interfaces associated with the SVI in a common flooding domain.
Meanwhile, selection of encapsulation 20 for another SVI on the same L3Out
triggers ACI to create a new L3Out BD with a different Layer 2 flooding
domain.
Figure 9-4 Significance of SVI VLAN Encapsulation Settings in an L3Out
Effectively, what happens as a result of this configuration is that the border leaf
switches and any other routers in the stretched flooding domain establish
somewhat of a full mesh of routing protocol adjacencies with one another (in
the case of OSPF or EIGRP). Figure 9-5 shows the neighbor adjacencies
established for the VLAN 10 L3Out BD. Meanwhile, the single router
connecting to the L3Out via encapsulation 20 forms only a single adjacency
with ACI.
Figure 9-5 Impact of L3Out BD Flooding Domains on Neighbor Relationships
While this full mesh of adjacencies is generally a good thing, in some cases it
can lead to ACI unintentionally transiting traffic between external routers. This
is especially true when the routers connecting to ACI each have different routes
in their routing tables. Because these routers are able to establish routing
adjacencies with one another through the L3Out BD Layer 2 flooding domain,
they are then able to use ACI as a transit to forward data plane traffic to each
other.
But what happens if an administrator tries to instantiate a common SVI
encapsulation in multiple L3Outs? It depends. If the administrator is attempting
to deploy the SVI encapsulation to different ports across different L3Outs on the
same border leaf, ACI allows use of the encapsulation for one L3Out but
generates a fault when the second instance of the encapsulation is used,
indicating that the encapsulation is already in use. Because of this, multiple
L3Outs that need to use the same encapsulation cannot coexist on the same
border leaf. However, this behavior can be changed with the SVI Encap Scope
option under the L3Out SVI.
If, on the other hand, an administrator attempts to reuse an encapsulation in a

new L3Out and on a different border leaf switch, ACI accepts the configuration
and deploys a new L3Out BD for the second L3Out. This is the case in Figure
9-6. The assumption, of course, is that the two L3Out BDs are intended to
represent different subnets if deployed in the same VRF.
Figure 9-6 Common SVI Encapsulation Used for L3Outs on Different Switches
Understanding SVI Encap Scope

When configuring L3Outs using SVIs, one important setting is the Encap
Scope. The acceptable values for this setting are VRF and Local.
The only reason to modify this setting from its default value of Local is to be
able to reuse an SVI in other L3Outs within a VRF. There are two specific use
cases for this:
 Establishing adjacencies using multiple routing protocols from a
single leaf using a common SVI
 Establishing granular route control over each BGP peer on the
same leaf by using a dedicated L3Out for each BGP peer
Figure 9-7 compares the two values for this setting. Let’s say that an engineer
needed to run both OSPF and EIGRP to an external device. In ACI, each L3Out
can be configured for only one routing protocol; therefore, two L3Outs are
needed to fulfill this requirement. The engineer determines that it would not be
feasible to run multiple physical connections between ACI and the external
device and that routed subinterfaces cannot be used in this design. In this case,
an SVI encapsulation and IP address on a border leaf needs to be shared
between the two L3Outs. If Encap Scope is set to Local for the SVI, ACI
expects a unique external SVI for each L3Out and generates a fault when an
L3Out SVI encapsulation is reused for a secondary L3Out on the switch. On the
other hand, setting Encap Scope to VRF for an SVI tells ACI to expect a unique
SVI encapsulation for each VRF and the two L3Outs are therefore allowed to
share the encapsulation.
Figure 9-7 Using SVI Encap Scope to Deploy an L3Out BD Across Multiple
L3Outs
There is one exception to the aforementioned rule that each L3Out enable only a
single routing protocol: OSPF can be enabled on a BGP L3Out to provide IGP
reachability for BGP. When OSPF is enabled in the same L3Out as BGP, OSPF
is programmed to only advertise logical node profile loopback addresses and
interface subnets.
Understanding SVI Auto State
Under default configurations, SVIs on an ACI leaf are always up, even if
associated physical ports are down. Although this is typically not a problem, it
could pose a problem when using static routes.
Figure 9-8 illustrates that a static route pointing out an L3Out SVI will remain
in the routing tables of the border leaf switches and will continue to be
distributed to other switches in the fabric if SVI Auto State is set at its default
value, Disabled.
Figure 9-8 Static Route Not Withdrawn on Downlink Failure with Auto State
Disabled
Figure 9-9 shows what happens to the static route when SVI Auto State is
toggled to Enabled. In this case, the border leaf disables the L3Out SVI once it
detects that all member ports for the L3Out SVI have gone down. This, in turn,
prompts the static route to be withdrawn from the routing table and halts
distribution of the static route to the rest of the fabric.
Figure 9-9 Static Route Withdrawn on Downlink Failure with Auto State Set to
Enabled
Note that the implementation of BFD for the static route could also resolve this
problem.
Understanding Prerequisites for Deployment of L3Outs

Before going into a tenant and creating an L3Out, access policies should be
implemented for any physical switch ports that will be part of the L3Out. This
includes assignment of an AAEP to an interface policy group and assignment of
the interface policy group to an interface profile. The interface profile needs to
be bound to the intended border leaf node IDs for the access policies to work. If
access policies for the underlying physical ports are not in place to begin with,
Layer 3 adjacencies will never be established.
Let’s say that you have created several L3 domains and have already associated
a VLAN pool with each of them. It is also important to ensure that an L3
domain is associated with the AAEP controlling connectivity to the underlying
physical ports. It is the L3 domain that an L3Out references. If an L3 domain
has not been associated with the AAEP that governs connectivity for the
intended L3Out ports, you may be unable to deploy the L3Out.
L3 Domain Implementation Examples

Say that the business unit (BU) from earlier chapters wants to deploy three
L3Outs in its tenant, as shown in Figure 9-10. One of the L3Outs uses EIGRP
and connects to a dedicated switch on the floor where most of the BU
employees work. An OSPF connection to a firewall and a BGP connection to a
router toward a partner network are also needed.
Figure 9-10 Hypothetical L3Out Connectivity Desired for a VRF

Before deploying these L3Outs, the L3 domain objects would need to be
configured. Figure 9-11 shows how an L3 domain for connectivity to the
firewall might be configured. Because the firewall will peer with ACI using an
SVI, a VLAN pool does need to be defined.
Figure 9-11 Sample L3 Domain Configuration That Includes VLAN Pool
Assignment
Figure 9-12 shows the configuration of an L3 domain for connectivity to a
router via subinterfaces. Notice that a VLAN pool has not been assigned to the
L3 domain. This is a valid configuration. Administrators do not need to assign
VLAN pools to L3 domains that will be used solely for routed interfaces or
routed subinterfaces.
Figure 9-12 Creation of an L3 Domain Without VLAN Pool Assignment
Finally, Figure 9-13 shows the AAEP for connectivity to all appliances owned
by the gaming BU. There was no need for the AAEP to be specific to the
gaming BU systems since AAEP objects are designed with multitenancy in
mind. Note here that a physical domain has also been associated with this
AAEP. This makes sense if there is a need for the firewall to simply trunk some
VLANs down to ACI using its port channel.
Figure 9-13 Associating Physical Domains and L3 Domains Simultaneously

with an AAEP
Understanding the Need for BGP Route Reflection

To distribute external routes across the fabric, ACI leaf and spine switches
establish iBGP peerings with one another within a user-defined BGP
autonomous system number (ASN).
Whenever iBGP peerings are involved, BGP split-horizon rules apply. These
rules state that a BGP router that receives a BGP route from an iBGP peer shall
not advertise that route to another router that is an iBGP peer.
While BGP split horizon works wonders in preventing intra-ASN route loops, it
requires that all routers in a BGP ASN form a full mesh to ensure that all routes
propagate correctly between all iBGP peers. This can impact scalability given
that a full-mesh design involves N(N – 1)/2 unique iBGP sessions, where N is
the number of routers. Imagine an ACI fabric with 50 switches, all having to
form BGP relationships with one another. The fabric would need to manage
50(49)/2 = 1225 BGP sessions to make this possible. Full-mesh iBGP is not
scalable and therefore is not used in ACI fabrics.
The scalable alternative to an iBGP full mesh is the deployment of route
reflectors. A route reflector is a BGP speaker that is allowed to advertise iBGP-
learned routes to certain iBGP peers. Route reflection bends the rules of BGP
split horizon just a little by introducing a new set of BGP attributes for route
loop prevention. In modern Clos fabrics such as ACI, spines make ideal route
reflectors because they often have direct connections with all leaf switches.
Using BGP route reflection, the number of unique iBGP peerings in a single-
pod ACI fabric drops down to RR × RRC, where RR is the number of route
reflectors and RRC is the number of route reflector clients. In a hypothetical 50-
switch single-pod ACI fabric consisting of 2 spines that have been configured as
route reflectors and 48 leaf switches (route reflector clients), the sum total
number of unique iBGP sessions needed stands at 96.
None of these points may be critical DCACI trivia. What should be important
for DCACI 300-620 exam candidates is first and foremost to recognize the
symptoms of forgetting to implement BGP route reflection within a fabric.
Next, you need to know how to configure BGP route reflection in the first place.
If an ACI fabric has not been configured for BGP route reflection, border leaf
switches can learn routes from external routers, but ACI does not distribute such
routes to other nodes in the fabric. External routes therefore never appear in the
routing tables of other leaf switches in the fabric.
Implementing BGP Route Reflectors

Administrators usually configure BGP route reflection during fabric
initialization or when they deploy the first L3Out in the fabric.
An administrator needs to do two things for BGP route reflection to work in
ACI:
 Enter the BGP ASN the fabric should use internally.

 Select the spines that will function as route reflectors.
Not all spines need to be configured as route reflectors, but it usually makes
sense to have at least two for redundancy.
To configure route reflection in ACI, navigate to System > System Settings >
BGP Route Reflector. Figure 9-14 shows the selection of two spines with node
IDs 201 and 202 as BGP route reflectors for Pod 1. The fabric uses BGP ASN
65000 in this example. Note that route reflectors for additional pods can also be
configured on this page.
Figure 9-14 Configuring BGP Route Reflectors Within a Fabric

You might be wondering whether you can deploy an alternative BGP route
reflector policy. The answer is that you cannot do so in ACI Release 4.2. Figure
9-15 indicates that even when creating a new pod policy group, ACI does not
allow modification of the BGP route reflector policy named default shown
earlier, in Figure 9-14.
Figure 9-15 Assigning a BGP Route Reflector Policy to a Pod Policy Group
Understanding Infra MP-BGP Route Distribution

Once BGP route reflection has been configured, ACI deploys MP-BGP on all
leaf and spine switches. The following steps summarize what takes place for
external routes to propagate from border leaf switches to all other switches in
the fabric:
Step 1.The BGP IPv4/IPv6 address family (AF) is deployed on all leaf switches
(both border and non-border leaf switches) in all user VRF instances.
Step 2.The BGP VPNv4/VPNv6 AF is also deployed on all leaf and route
reflector spine switches in the infra VRF (overlay-1 VRF). All leaf switches
establish iBGP sessions with route reflector spine switches in the infra VRF and
are then able to exchange their VPNv4/VPNv6 routes.
Step 3.Once an L3Out is deployed on a leaf, the BGP IPv4/IPv6 AF on the
same border leaf automatically creates a redistribution rule for all the routes
from the routing protocol of the L3Out within the same user VRF. This
redistribution is called interleak. If the L3Out is using BGP, no redistribution
(interleak) is required for routes learned via BGP because the BGP process for
the L3Out and for the infra MP-BGP is the same.
Step 4.The redistributed IPv4/IPv6 routes are exported from the user VRF to
the infra VRF as VPNv4/VPNv6.
Step 5.On other leaf switches, the VPNv4/VPNv6 routes distributed through
route reflector spines are imported from the infra VRF to the user VRF as
IPv4/IPv6.
Figure 9-16 recaps this route distribution process.
Figure 9-16 Infra MP-BGP Architecture and Route Distribution

Those interested in learning more about the MP-BGP route distribution process
can begin their exploration process using the commands show bgp process
detail vrf all, show bgp ipv4 unicast vrf all, and show bgp vpnv4 unicast vrf
overlay-1.
For those not interested in the route distribution process, perhaps the only thing
of importance on this issue is to be able to verify spine-to-leaf BGP
peerings. Example 9-1 shows how to verify the number of BGP adjacencies on
a leaf switch. It also demonstrates how you can correlate neighbor router IDs
with their hostnames, node IDs, and TEP addresses.
Example 9-1 Validating BGP Peerings Between a Leaf and Route Reflector Spines
Click here to view code image

LEAF101# show bgp sessions vrf overlay-1
Total peers 5, established peers 5
ASN 65000
VRF overlay-1, local ASN 65000
peers 2, established peers 2, local router-id 10.233.46.32

State: I-Idle, A-Active, O-Open, E-Established, C-Closing, S-Shutdown
Neighbor ASN Flaps LastUpDn|LastRead|LastWrit St Port(L/R) Notif(S/R)
10.233.46.33 65000 0 16w06d |never |never E 60631/179 0/0
10.233.46.35 65000 0 16w06d |never |never E 44567/179 0/0
LEAF101# acidiag fnvread | grep spine
201 1 SPINE201 FDOXXXX1 10.233.46.33/32 spine active 0
202 1 SPINE202 FDOXXXX2 10.233.46.35/32 spine active 0
The external route distribution process in ACI is extremely reliable. If all

expected BGP peerings appear to be in an established state, you should see
external routes pop up in the routing table of all leaf switches where the relevant
user VRF has been deployed.
DEPLOYING L3OUTS
ACI Release 4.2 has a streamlined wizard for setting up L3Outs. While use of
the wizard is required when configuring an L3Out via the GUI, administrators
can make modifications to L3Outs afterward.
This section demonstrates the process of going through the streamlined wizard
to create L3Outs for each routing protocol and interface type. It also touches on
deployment of external EPGs on L3Outs and some configuration changes you
might want to make manually.
Configuring an L3Out for EIGRP Peering

To launch the L3Out creation wizard, navigate to Tenants, select the tenant in
question, open the Networking folder, right-click L3Outs, and select Create
L3Out.
Figure 9-17 shows the first page of the wizard. Following completion of the
wizard, the settings on this page all appear at the root of the L3Out on the Main
subtab of the Policy page. Enter a name for the L3Out, select the VRF instance
with which the L3Out should be associated, and select the L3 domain created
for this individual L3Out. Then select the routing protocol. When EIGRP has
been selected as the protocol of choice, the wizard disables the OSPF and BGP
checkboxes, and the Autonomous System Number text box appears. Entry of an
autonomous system number is mandatory. Click Next to continue.
Figure 9-17 Entering the EIGRP and ASN Configuration in the L3Out Creation
Wizard
Note
If you are unfamiliar with domains in general, see Chapter 6, “Access Policies.” For
coverage of VRF configuration, see Chapter 5.
In Figure 9-17, notice the checkbox Use for GOLF. Selecting the Use for GOLF
checkbox on a user VRF L3Outs tells ACI that the L3Out will be a mere
placeholder for external EPGs and other policies for a GOLF L3Out. There is
no reason to select this option when the intent is to establish Layer 3 peerings
for border leaf switches.
The next page in the wizard pertains to logical node profiles and logical
interface profiles. The wizard selects a node profile name by default. Deselect
Use Defaults, as shown in Figure 9-18, if you need to customize the logical
node profile name. Next, select an interface type. This example shows a routed
interface selected with the port option in line with the L3Out designs presented
in Figure 9-10, earlier in this chapter. Select a node ID and enter a router ID for
this border leaf switch. Do not use CIDR notation for router IDs. Entering a
loopback address is required only if you expect to implement BGP Multihop by
sourcing a loopback address. Finally, enter information for the interfaces on the
border leaf that need to be enabled for EIGRP. CIDR notation for these
interfaces is required. Click Next to move on to the next page.
Figure 9-18 Entering Node and Interface Information for an EIGRP L3Out
The Protocol Associations page, shown in Figure 9-19, allows association of a
custom EIGRP interface policy with the EIGRP interface profile for this L3Out.
By default, ACI selects an EIGRP interface policy called default from the
common tenant. New protocol interface policies cannot currently be created in
the L3Out creation wizard. Custom EIGRP interface policies are discussed and
applied to L3Outs later in this chapter. Click Next to continue.
Figure 9-19 Associating an EIGRP Interface Policy with the EIGRP Interface
Profile
Figure 9-20 shows the final page of the L3Out creation wizard. This page
allows you to define external EPGs for the L3Out. When the Default EPG for
All External Networks checkbox is enabled, ACI automatically generates an
EPG that matches all traffic not matched by a more specific external EPG.
Disable this checkbox if you want to manually create external EPGs after the
L3Out has been deployed. Click Finish to deploy the L3Out.
Figure 9-20 External EPG Creation Page in the L3Out Creation Wizard
Once the L3Out is deployed, if you check to see whether EIGRP adjacencies
have been established, you will find that they have not. ACI does not attempt to
establish route peerings out an L3Out until at least one external EPG has been
deployed on the L3Out.
Deploying External EPGs

Chapter 5 describes the significance of external EPGs in classifying traffic
behind L3Outs, and Chapter 8, “Implementing Tenant Policies,” covers contract
implementation. These topics are not repeated here, but Figure 9-21 outlines
hypothetical connectivity that needs to be whitelisted between external
endpoints and internal EPGs. In this figure, Public-Access, Internal-Access, and
Admin-Access are external EPGs.
Figure 9-21 External EPG Design in a Specific VRF
To create an external EPG on an L3Out, drill down into the subfolders of the
L3Out and select the External EPG folder. Then right-click the Tools menu and
select Create External EPG (see Figure 9-22).
Figure 9-22 Kicking Off the External EPG Creation Wizard
Enter a name for the external EPG. Chapter 10, “Extending Layer 2 Outside
ACI,” briefly touches on the idea of the preferred group member. When
contracts are used to allow communication, the Preferred Group Member
parameter can be left at the default value, Exclude. To define subnets for the
external EPG, click the + sign in the Subnet part of the screen shown in Figure
9-23.
Figure 9-23 Main Page of the Create External EPG Wizard
Note that this figure shows configuration of the Admin-Access external EPG.
Two subnets have been called out for association with this particular external
EPG: 10.200.1.0/24 and 10.200.100.0/24.
Figure 9-24 shows the addition of the subnet 10.200.100.0/24 to Admin-Access.

The Name field on this page reflects the function of the particular subnet being
added. Several groups of checkboxes exist on this page. These checkboxes are
called Scope options, and they determine the function(s) of each external EPG.
The checkboxes in the Route Control section predominantly relate to transit
routing scenarios. The Shared Security Import Subnet checkbox relates to
shared service L3Outs. The only checkbox of interest for DCACI candidates is
External Subnets for External EPG. When enabled, this checkbox tells ACI that
this external EPG should be used to classify external traffic matching the subnet
for contract enforcement. Enable this checkbox and click Submit.
Figure 9-24 Adding Subnets as External Subnets for an External EPG

After you add a subnet to an external EPG, the Create External EPG page
reappears so you can assign additional subnets to the external EPG. When you
finish this, click Submit. Figure 9-25 shows the General tab for an external EPG
that has been created. Notice in this figure that the external EPGs Internal-
Access and Public-Access have been created in the background. This view is
particularly useful because it verifies that the external EPG has been deployed
and also that all intended subnets have been assigned to it, using the proper
scope.
Figure 9-25 The General Tab of an External EPG

Now that at least one external EPG has been deployed on the L3Out, ACI
should attempt to establish routing adjacencies with external routers.
Verifying Forwarding Out an L3Out

Example 9-2 shows how to verify that an EIGRP adjacency has been
established in a VRF.
Example 9-2 Verifying EIGRP Adjacencies Resulting from L3Out Configuration

LEAF301# show ip eigrp neighbors vrf Gaming-BU:Multiplayer-Prod
EIGRP neighbors for process 100 VRF Gaming-BU:Multiplayer-Prod
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 10.233.75.161 eth1/9 13 07:30:37 1 50 0 19

Note
Keep in mind that you have not seen any switch ports configured in this chapter. L3Out
adjacencies cannot form unless access policies for underlying switch ports have been
configured.
Following adjacency verification, it makes sense to confirm whether ACI has
learned any routes. Example 9-3 shows the routing table of node 301 and node
302. Notice the highlighted routes learned by node ID 301, where the L3Out has
been deployed. These all appear to have been learned via EIGRP. This is
expected because the L3Out runs EIGRP. Node 302, on the other hand, has
learned these same routes from bgp-65000. This is because node 302 learned
these routes from MP-BGP route distribution, and route reflectors in this
particular fabric have been configured with BGP ASN 65000. Notice that these
route entries on node 302 all have next-hop addresses pointing to the TEP
address of node 301. This is expected behavior.
Example 9-3 Verifying the Routing Table for a Specific VRF

LEAF301# show ip route vrf Gaming-BU:Multiplayer-Prod
IP Route Table for VRF "Gaming-BU:Multiplayer-Prod"
(...output truncated for brevity...)
10.199.90.0/24, ubest/mbest: 1/0
*via 10.233.75.161, eth1/9, [90/128576], 22:35:22, eigrp-default, internal
10.200.1.0/24, ubest/mbest: 1/0
10.200.100.0/24, ubest/mbest: 1/0
LEAF302# show ip route vrf Gaming-BU:Multiplayer-Prod
10.199.90.0/24, ubest/mbest: 1/0
*via 10.233.60.234%overlay-1, [200/128576], 22:43:13, bgp-65000, internal, tag 65000

10.200.1.0/24, ubest/mbest: 1/0
10.200.100.0/24, ubest/mbest: 1/0
Finally, recall from Chapter 8 that ACI does not store information about
external endpoints learned via an L3Out in endpoint tables. ARP is used to keep
track of next-hop MAC-to-IP address bindings for endpoints behind
L3Outs. Example 9-4 shows the ARP table for the VRF from the perspective of
border leaf 301.
Example 9-4 Checking the ARP Table for Next-Hop MAC-to-IP Address Binding

LEAF301# show ip arp vrf Gaming-BU:Multiplayer-Prod
Flags: * - Adjacencies learnt on non-active FHRP router
+ - Adjacencies synced via CFSoE
# - Adjacencies Throttled for Glean
D - Static Adjacencies attached to down interface
IP ARP Table for context Gaming-BU:Multiplayer-Prod
Total number of entries: 1
Address Age MAC Address Interface
10.233.75.161 00:02:30 a0e0.af66.c5a1 eth1/9
Unless data in one of these tables is inaccurate, ACI should be able to forward
traffic to external devices without issue.
Advertising Subnets Assigned to Bridge Domains via an

L3Out
When ACI is learning subnets behind an L3Out, it is time to advertise ACI
subnets out of the fabric. The most basic and yet common form of bridge
domain subnet advertisement involves a two-step process. First, navigate to the
desired BD and add one or more L3Out in the Associated L3Outs view. Then,
drill down into an individual subnet that needs to be advertised and update its
scope to Advertised Externally. These two configurations do not need to be
done in the order specified, but together, they tell ACI to internally create a
route map rule on the border leaf switches to redistribute the desired BD subnets
into the routing protocol of the associated L3Out.
Figure 9-26 shows the addition of a bridge domain named BD-Production to the
L3Out named Multiplayer-Prod-L3Out.
Figure 9-26 Marking a BD as a Candidate for Subnet Redistribution into an

L3Out
Figure 9-27 shows a BD subnet scope with Advertised Externally selected.
Figure 9-27 Marking a BD Subnet for Redistribution into One or More L3Outs
Figure 9-28 shows the L3 Configuration tab for a BD after an associated subnet
has been marked for advertisement out an L3Out. Notice that this view shows
not only the scope of associated subnets but also the fact that Unicast Routing
has been enabled. One issue that prevents advertisement of a subnet out an
L3Out is Unicast Routing being disabled in the first place.
Figure 9-28 Verifying That Unicast Routing Is Enabled for BDs with Subnets
Advertised Out an L3Out
Enabling Communications over L3Outs Using Contracts

Once routing into and out of a fabric has been enabled, the next step is to
whitelist desired endpoint communication. Figure 9-21 earlier in this chapter
provides a list of desired contracts. This section takes a look at how you might
go about creating the contract named Admins-to-Fabric-Systems. To match
interesting traffic, first create a filter, as shown in Figure 9-29. The only thing
that’s new here should be the filter entry matching ICMP traffic. As indicated
earlier, ICMP filters do not require definition of ports in either direction.
Figure 9-29 A Filter That Matches Interesting Traffic
Figure 9-30 shows that the filter has been added to a subject under the desired
contract that permits forwarding of all matched traffic as well as all return
traffic.
Figure 9-30 Contract Subject Allowing Traffic Matching the Previous Filter
Finally, you see that the contract Admins-to-Fabric-Access is ready to be

created. Note that the default value, VRF, has been selected as the contract
scope in Figure 9-31. It is important to understand that even though this contract
is between devices external to the fabric and internal devices, no route leaking
between ACI user VRFs is taking place. This is why the scope of VRF is
sufficient when the L3Out is not being used as a shared service L3Out.
Figure 9-31 Contract Scope Shown in the Create Contract Wizard
Next, the contract needs to be assigned between external EPGs and internal
EPGs. The easiest way to understand provider and consumer directionality for
external EPGs is to first realize that external EPGs represent external systems.
Will these external systems be initiating requests, or will they be waiting for
data center systems to send traffic? Because administrators need to initiate
connections such as SSH and HTTPS, Admin-Access should be configured as a
consumer of the contract, as indicated in Figure 9-32.
Figure 9-32 An External EPG Consuming a Contract
Meanwhile, internal EPGs need to be configured as providers of services to the
administrators. Figure 9-33 shows the Admins-to-Fabric-Systems being added
to an internal EPG in the provided direction.
Figure 9-33 An Internal EPG Configured to Provide a Service to

Administrators
After the desired contracts are put in place, all endpoints should have the
desired connectivity.
Before moving on to the next section, revisit Figure 9-21. An internal EPG
called EPG-Patch-Servers is shown consuming a service from the Internet.
Basically, for the only EPG with access to initiate communication with Internet
systems, the desire is to ensure that endpoints in this EPG can initiate TCP
sessions destined to any port, but Internet systems cannot initiate sessions to
these patch servers. This is a good example of where you might avoid use of
bidirectional filter application and instead use a return filter matching the TCP
established bits.
Deploying a Blacklist EPG with Logging

Sometimes, IT teams identify the need to block or log certain traffic that should
be able to enter the data center but should not be allowed to reach servers within
ACI. In such cases, you can deploy an external EPG that either has no contracts
associated or has a contract that denies and perhaps logs the traffic.
Because external EPGs use a longest-prefix match, all you need to do to classify
the intended traffic is to add the relevant subnets or host routes to the blacklist
EPG and make sure more specific IP addresses in the blacklisted range(s) have
not been allocated to other external EPGs.
Figure 9-34 shows creation of an external EPG for this purpose. Note that one
particular host, 10.200.1.5/32, falls within the administrator subnet range. This
is completely valid. In this case, ACI would always classify traffic from
10.200.1.5 into this new EPG and not into the Admin-Access external EPG.
Figure 9-34 Creation of an External EPG to Classify Specific Endpoints and
Ranges
If all you want is to drop traffic from these ranges, you have already succeeded
because no contracts have been enforced on this external EPG. However, if you
also want to log the traffic, you can create a specific contract that has a subject
whose filter matches all traffic. The directive in this case should be Log, and the
action should be Deny, as shown in Figure 9-35.
Figure 9-35 Creation of a Contract Subject to Log and Drop All Traffic
As indicated in Figure 9-36, the contract scope VRF is sufficient for this
contract.
Figure 9-36 Contract for Blacklisting Traffic from and to Certain External

Sources
Next, you need to apply the contract on the new external EPG in both the
consumed and provided directions. Figure 9-37 shows its application as a
consumed contract.
Figure 9-37 Applying a Contract to an External EPG to Blacklist Traffic
Navigate to Tenants and open the tenant in question. Go to the tenant’s
Operational tab, select Flows, and select L3 Drop. As shown in Figure 9-38, if
traffic has been initiated by any of the blacklisted external devices or if any
traffic in the VRF was destined to the blacklisted devices, ACI should have a
deny log for the traffic flow. Note that contract logging uses processor resources
and is rate limited in ACI.
Figure 9-38 Verifying Dropped Traffic Logged as a Result of a Contract
Advertising Host Routes Out an ACI Fabric
One of the configuration knobs under bridge domains in recent ACI code
revisions is Advertise Host Routes. When this option is enabled, ACI advertises
not only BD subnets but also any host routes that ACI has learned within any
BD subnet ranges selected for external advertisement. Figure 9-39 shows this
configuration checkbox.
Figure 9-39 Advertising Host Routes Out an L3Out
Implementing BFD on an EIGRP L3Out
A common desire in modern data centers is to implement BFD to enable

subsecond routing failover capabilities. To implement BFD on an EIGRP
L3Out, enable BFD under the associated EIGRP interface policy. If further
customization of BFD settings and timers is required, you can create a BFD
interface policy and associate it with the EIGRP L3Out.
Figure 9-40 shows an EIGRP interface policy that is active on an L3Out. This
particular L3Out has the default EIGRP interface policy from the common
tenant set. If BFD were to be enabled on this particular EIGRP interface policy,
all L3Out interfaces in other tenants that consume this policy would also
attempt to establish BFD neighbor relationships out their respective L3Outs.
This is likely not what you want to do if you are configuring this in a production
fabric. Instead, you can create and associate a new EIGRP interface policy with
the L3Out EIGRP interface profile.
Figure 9-40 Verifying an Operational EIGRP Interface Policy on an L3Out

Figure 9-41 shows the Create EIGRP Interface Policy page. Notice that this
page also allows tuning of EIGRP hello and hold intervals as well as tuning of
bandwidth and delay for route metric manipulation. Enable the BFD checkbox
and click Submit.
Figure 9-41 Configuring a New EIGRP Interface Policy with BFD Enabled
ACI comes preconfigured with a default global BFD policy located
under Fabric > Access Policies > Policies > Switch > BFD > BFD IPv4/v6 >
default. To customize timers or to enable subinterface optimization for BFD for
an individual L3Out, navigate to a logical interface profile on the L3Out and
select Create BFD Interface Profile. The Create BFD Interface Profile window
shown in Figure 9-42 appears. Select Create BFD Interface Policy.
Figure 9-42 Launching the BFD Interface Profile Creation Wizard
Then, on the Create BFD Interface Policy page, shown in Figure 9-43, select the
desired settings and click Submit. If neighboring devices that peer with ACI
over the L3Out also have BFD enabled, BFD should become fully operational.
Figure 9-43 L3Out BFD Timer and Policy Customization

BFD interface policies are not protocol specific and so are not revisited in future
sections.
Note
In addition to BFD, the following EIGRP interface policy Control State options are
available:
 Self Nexthop: This option is enabled by default. By default, EIGRP sets its local
IP address as the next hop when advertising routes. When you disable this option, the border
leaf does not overwrite the next hop and keeps the original next-hop IP address.
 Passive: This option is used to configure the interfaces as an EIGRP passive
interface. This option is disabled by default.
 Split Horizon: Split horizon is a feature that helps prevent routing loops by not
sending EIGRP updates or queries to the interface where it was learned. This option is
enabled by default.
It is very uncommon for engineers to need to modify any of these three settings from
their defaults.
Configuring Authentication for EIGRP

ACI supports authentication of EIGRP peers using MD5, but routing protocol
authentication is not very common in data centers today. EIGRP authentication,
therefore, does not warrant extensive coverage here.
To enable authentication with EIGRP neighbors, navigate to the EIGRP
interface profile under the EIGRP L3Out and select Enable Authentication.
Then either select the default keychain policy from the common tenant from the
EIGRP KeyChain Policy drop-down or define a new one.
You can define EIGRP keychain policies by navigating to Tenant > Policies >
Protocol > EIGRP > EIGRP KeyChains.
A keychain policy is a collection of key policies. Each key policy consists of a
key ID, a key name, a pre-shared key, a start time, and an end time.
The only caveat to point out is that because EIGRP authentication is
implemented at the logical interface profile level, the use of multiple logical
interface profiles becomes necessary if some EIGRP peers are required to
authenticate over the L3Out while others are not.
EIGRP Customizations Applied at the VRF Level

If you select a VRF within a tenant and click the Policy menu, one of the
configuration sections you see on the Policy page is EIGRP Context per
Address Family. This is where you can modify certain VRF-wide settings for
EIGRP by deploying a custom EIGRP address family context policy for IPv4 or
IPv6. The configuration settings that together form an EIGRP address family
context policy are described in Table 9-2.
Table 9-2 Customizable Settings for an EIGRP Address Family Context Policy

Configuration Description
Parameter
Active Interval The interval the border leaf waits after an EIGRP query is sent before d
(min) active (SIA) situation and resetting the neighborship. The default is 3 m
External Distance The administrative distance (AD) for external EIGRP routes. The defa
routes is 170.
Internal Distance The AD for internal EIGRP routes. The default AD for internal routes
Parameter
Maximum Path The maximum number of equal-cost multipathing (ECMP) next-hop ad

Limit install into the routing table for a prefix. The default is eight paths.
Metric Style EIGRP calculates its metric based on bandwidth and delay along with
However, the original 32-bit implementation cannot differentiate interf
Gigabit Ethernet. This original implementation is called the classic, or
solve this problem, a 64-bit value with an improved formula was introd
this is called the wide metric. Valid values for metric style are narrow m
metric. The default is the narrow metric.
Configuring an L3Out for OSPF Peering

Let’s revisit the requirements from Figure 9-10. The next L3Out that needs to
be provisioned is Firewall-L3Out, which will leverage SVIs over a vPC to peer
with a firewall via OSPF. Figure 9-44 shows the first page of the L3Out
creation wizard. Enter parameters for name, VRF, and L3 domain. Enable
OSPF by selecting the OSPF checkbox. In this example, the OSPF area ID 1 has
been selected, and NSSA Area is selected for OSPF Area Type. Select the
parameters that meet your requirements and click Next. Note that each OSPF
L3Out can place interfaces in a single area. If peerings in multiple areas are
required, you need to deploy additional L3Outs.
Figure 9-44 Select Global Routing Protocol Parameters for OSPF L3Out
On the Nodes and Interfaces page, shown in Figure 9-45, select the interface
type that will be deployed for the L3Out. In this case, SVIs will be used. When
deploying SVIs on a vPC in ACI, select different primary IP addresses for each
vPC peer. If the external device behind the L3Out needs to point to a common
IP address that both border leaf switches respond to, you can
define a secondary IP address by using the IP Address: Secondary field. That is
not required in this case due to the reliance on OSPF, but this same L3Out will
also be used to demonstrate static routing in ACI. Also, note the Encap field.
Enter the VLAN ID that needs to be trunked out the L3Out to the external
device here. The VLAN ID selected here must be in the VLAN pool associated
with the L3 domain for the L3Out. Click Next when you’re finished making
selections.
Figure 9-45 Configuring L3Out SVIs Trunked onto a vPC with Secondary IP
Addresses
The Protocol Associations page for OSPF, shown in Figure 9-46, allows you to
associate an OSPF interface policy to the L3Out OSPF interface profile. The
default OSPF interface policy resides in the common tenant. Click Next.
Figure 9-46 Associating an OSPF Interface Policy with the OSPF Interface
Profile
The final step in creating the L3Out is to configure an external EPG. A critical
thing to understand about external EPGs used for traffic classification is that
their scope is VRF-wide. This means that if you have already configured all
your desired subnets for classification using the scope External Subnets for
External EPG on another L3Out associated with a VRF, there is no need to
duplicate these configurations on a secondary L3Out in the same VRF. In fact,
if you try to add a subnet previously classified by another external EPG in the
VRF to an external EPG on a second L3Out, ACI raises an error and does not
deploy the erroneous configuration. However, one problem remains. ACI L3Out
still expects to see at least one external EPG before it attempts to form routing
protocol adjacencies with external devices. Figure 9-47 shows the creation of a
dummy external EPG called Placeholder. No subnets need to be associated with
this external EPG, whose only function is to ensure that ACI enables the L3Out.
Click Finish to deploy the L3Out.
Figure 9-47 Configuring a Dummy External EPG to Enable the L3Out

To verify OSPF adjacency establishment via the leaf CLI, you can execute the
command show ip ospf neighbors vrf <vrf name>. Example 9-5 shows that
ACI has learned a default route from the adjacent firewall.
Example 9-5 Verifying Routes Learned via OSPF
LEAF301# show ip route ospf vrf Gaming-BU:Multiplayer-Prod
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
0.0.0.0/0, ubest/mbest: 1/0
*via 10.198.10.2, vlan76, [110/5], 00:01:02, ospf-default, inter
A Route Advertisement Problem for OSPF and EIGRP

L3Outs
One problem commonly experienced by engineers who dedicate a pair of

switches to border leaf functions and need to create both OSPF and EIGRP
L3Outs on these switches within the same VRF is that any BD subnets marked
for advertisement out one L3Out also gets advertised out the other L3Out.
The reason for this behavior is that the route map ACI automatically generates
as a result of BD subnet advertisements is common across OSPF and EIGRP.
There are two common and recommended solutions for avoiding this behavior:
 Deploy OSPF and EIGRP L3Outs for a given VRF on different

border leaf switches.
 Use BGP instead of OSPF and EIGRP. This recommendation is
due to the fact that ACI generates route maps for BGP on a per-L3Out
basis.
Implementing BFD on an OSPF L3Out

To enable BFD on an OSPF L3Out, select the OSPF interface profile for the
L3Out. Either edit the default interface policy associated with the common
tenant or create a new OSPF interface policy to associate with the L3Out OSPF
interface profile. Figure 9-48 shows creation of a new OSPF interface policy
that enables BFD.
Figure 9-48 Enabling BFD on a Custom OSPF Interface Policy

Note
In addition to BFD, the following OSPF interface policy Control State options are
available:
 Advertise Subnet: This allows OSPF to advertise a loopback IP address with its
subnet instead of /32 without requiring that the network type be changed from loopback to
point-to-point. However, in ACI, a loopback IP address is always configured with /32.
Hence, at the time of writing, this option does not do anything in particular.
 MTU Ignore: This option allows the OSPF neighborship to form even with a
mismatching MTU. This option is intended to be enabled on an OSPF interface with a lower
MTU. Use of this option is not recommended in general.
 Passive Participation: This option configures the interfaces as OSPF passive
interfaces.
Modifying any of these settings from their defaults is very uncommon in ACI.
OSPF Customizations Applied at the VRF Level

Just as with EIGRP, there are some VRF-wide settings for OSPF. A number of
these settings are documented in Table 9-3. All OSPF timer values have been
removed to limit coverage. A custom OSPF timer policy can be applied either to
all address families within a VRF using the OSPF Timers drop-down box on the
VRF Profile menu or to an individual address family (IPv4 or IPv6).
Table 9-3 Customizable Settings Besides Timers in an OSPF Timer Policy
Parameter
Bandwidth Specifies the reference bandwidth used to calculate the default metr
Reference (Mbps) interface. The default is 40,000 Mbps (40 Gbps).
Admin Distance Specifies the administrative distance (AD) for OSPF routes. The def
Preference
Maximum ECMP Specifies the maximum number of ECMP that OSPF can install into
The default is 8 paths.
Enable Name Prompts ACI to display router IDs as DNS names in OSPF show co
Lookup for Router disabled by default.
IDs
Prefix Suppression Reduces the number of Type 1 (router) and Type 2 (network) LSAs
routing table. This option is disabled by default.
Graceful Restart Keeps all the LSAs that originated from the restarting router during
Helper period. ACI border leaf switches do not themselves perform OSPF g
ACI enables this option by default.
Adding Static Routes on an L3Out

Say that members of an IT team think that a firewall or an appliance they have
procured and enabled for dynamic routing is not very reliable and that the
routing protocol in the current appliance code may crash. Or say that they want
to disable an appliance altogether and just leverage static routing. Sometimes,
an appliance may lack the capability to leak a default route into a specific
dynamic routing protocol. In such cases, static routes can be deployed on an
L3Out either alongside a dynamic routing protocol or by themselves, and they
can be distributed throughout the fabric via MP-BGP.
If an L3Out does not require dynamic routing, you can leave the checkboxes for
EIGRP, OSPF, and BGP within the L3Out unchecked. Static routing does not
need to be configured alongside dynamic routing protocols.
Static routes in ACI are configured on individual fabric nodes. Select the node
of interest under the Configured Nodes folder and click the + sign under the
Static Routes view. Figure 9-49 shows the configuration of a static default
route. The Preference value ultimately determines the administrative distance of
the static route. There are two places where such values can be defined. Each
next hop can assign its own preference. If next-hop preference values are left at
0, the base Preference setting in the Prefix field takes effect. Configure the
Prefix, Preference, and Next Hop IP settings for a static route and click Submit.
If you create a route without setting a next-hop IP address, ACI assigns the
NULL interface as the next hop.
Figure 9-49 Adding a Static Route on an L3Out

Note the BFD checkbox. Enabling BFD for a static route leads to the subsecond
withdrawal of the static route from the routing table in the event that the BFD
session on the egress interface for the static route goes down.
Implementing IP SLA Tracking for Static Routes

With IP service-level agreement (SLA) tracking for static routes, introduced in
ACI Release 4.1(1), you have the option to collect information about network
performance in real time by tracking an IP address using ICMP or TCP probes;
you can influence routing tables by allowing a static route to be removed when
tracking results are negative and returning the routes to the table when the
results become positive again. IP addresses tracked can be next-hop IP
addresses, external addresses several hops away, or internal endpoints.
To implement IP SLA tracking for static routes, perform the following steps:
Step 1.Create an IP SLA monitoring policy if custom probe frequency and

multipliers are desired.
Step 2.Define the IP addresses to be probed. These are called track members.
Step 3.Create and associate a track list with either an individual static route or a
next-hop address for the static route.
To create an IP SLA monitoring policy, navigate to the tenant in which the

policy should be created, double-click the Policies folder, double-click the
Protocol folder, right-click the IP SLA folder, and select Create IP SLA
Monitoring Policy. Figure 9-50 shows creation of an IP SLA policy using
default settings. The SLA Frequency field specifies the frequency, in seconds,
to probe the track member IP address. Acceptable values range from 1 second to
300 seconds. The Detect Multiplier setting defines the number of missed probes
in a row that moves the track object into a down state. The SLA Type setting
defines which protocol is used for probing the track member IP. For L3Out
static routes, the supported options are either ICMP or TCP with a specified
destination port.
Figure 9-50 Creating an IP SLA Monitoring Policy
Define each IP address that needs to be probed in a track member object by

right-clicking the Track Members or IP SLA folder and selecting Create Track
Member. Figure 9-51 shows the track member IP address entered in the
Destination IP field. The Scope of Track Member drop-down allows you to
define the component (L3Out or BD) on which the destination IP address
should exist. The IP SLA monitoring policy should also be specified in the IP
SLA Policy drop-down unless the default IP SLA policy in the common tenant
is desired.
Figure 9-51 Creating a Track Member
Create a track list by right-clicking either the IP SLA folder or the Track Lists
folder and selecting Create Track List. Alternatively, you can create a new track
list and simultaneously associate it with a static route or next-hop address on the
Static Route page. The Type of Track List field has Threshold Percentage and
Threshold Weight as options. Use the percentage option if all track members are
at the same level of importance. Use the weight option to assign a different
weight to each track member for more granular threshold conditions. With
Threshold Percentage selected, ACI removes the associated static route(s) from
the routing table when probes to the percentage of track members specified by
the Percentage Down field become unreachable. ACI then reenables the static
route(s) only when probes to the percentage of track members specified by
Percentage Up become reachable. Similarly, the Weight Up Value and Weight
Down Value fields, which are exposed when Threshold Weights is selected,
determine the overall weight that needs to be reached for associated static routes
to be removed or re-introduced into the routing table.
Figure 9-52 shows a track list which demands that probes be sent against two
track member IP addresses. Because two objects have been included in the track
list, reachability to each destination contributes 50% to the overall threshold of
the track list. The Percentage Up value 51 indicates that both track members
need to be up for the static route to be added to the routing table. In this
example, ACI withdraws the associated route(s) even if one track member
becomes unreachable due to the Percentage Down value 50.
Figure 9-52 Creating a Track List

Example 9-6 shows that when this track list is assigned to the static default
route created earlier, ACI implements a track object for each track member and
track list. The overall status of the track list is down because the Percentage
Down condition has been hit due to lack of reachability to one track member.
This prompts the associated route prefix to be withdrawn from the routing tables
of all ACI switches.
Example 9-6 Verifying Tracking of Static Routes

LEAF301# show track
Track 2
IP SLA 2256
reachability is down
1 changes, last change 2020-07-11T19:23:25.179+00:00
Tracked by:
Track List 1
Track 3
IP SLA 2288
reachability is up
Tracked by:
Track List 1
Track 1
List Threshold percentage
Threshold percentage is down
Threshold percentage up 51% down 50%
Tracked List Members:
Object 3 (50)% up
Object 2 (50)% down
Attached to:
Route prefix 0.0.0.0/0
As noted earlier, track lists can also be associated with a next-hop IP address for
a static route. Figure 9-53 shows configurable options besides the next-hop IP
address that are available on the Next Hop Profile page. Next Hop Type can be
set to None or Prefix. None is used to reference the NULL interface as the next
hop of a static route. ACI accepts None as a valid next-hop type only if
0.0.0.0/0 is entered as the prefix. Alternatively, a static route without a next-hop
entry is essentially a route to the NULL interface. Prefix is the default option for
Next Hop Type and allows users to specify the actual next-hop IP address.
Figure 9-53 Options Available on the Next Hop Profile Page
Applying an IP SLA policy on a next-hop entry for a static route is functionally
equivalent to creating a track list with the next-hop address as its only track
member and applying the resulting track list to the next-hop entry. In other
words, referencing an IP SLA policy on a next-hop entry provides a shortcut
whereby the APIC internally creates a track list with the next-hop IP address as
the probe IP address.
Note
One difference between applying a track list to a static route and applying it to a next-
hop entry is apparent when a backup floating static route for the same prefix exists on
the same leaf switch(es) even when configured on a different L3Out. In this case, a track
list applied to the primary static route (lower preference value) prevents the floating
static route (higher preference value) from being added to the routing table. This
behavior is not experienced when the track list is applied to the next-hop entry of the
primary static route.
Configuring an L3Out for BGP Peering

ACI supports both iBGP and eBGP peerings with external routers. A BGP-
enabled L3Out automatically belongs to the same BGP ASN configured for
route reflection. You need to define each BGP peering under a BGP peer
connectivity profile. If there is a need for the fabric to appear as an ASN other
than the one configured for route reflection, you can tweak the local-as settings
in the BGP peer connectivity profile for the intended BGP neighbor.
Given that BGP designs often require peering with neighbors that may be
several hops away, an important topic with BGP is establishing IP reachability
to potential neighbor peering addresses. Supported methods for BGP peering IP
reachability in ACI are direct connections, static routes, and OSPF.
Figure 9-54 shows that the first step in creating a BGP L3Out via the GUI is to
enable the BGP checkbox. Notice that this does not disable the OSPF checkbox.
Enter the L3Out name, VRF instance, and L3 domain and click Next.
Figure 9-54 The L3Out Configuration Wizard Identity Page with BGP Enabled
Figure 9-55 indicates that the Use Defaults checkbox has been enabled for this
L3Out. Because of this checkbox, ACI does not show the logical node profile or
logical interface profile names it intends to create. Even though there is little
benefit in creating a loopback for EIGRP and OSPF L3Outs, the Loopback
Address field in this case has been populated. This is recommended for BGP
because a design change at any time may require the addition of redundant
multihop peerings. Enter interface configuration parameters and click Next to
continue.
Figure 9-55 Configuration Options on the Nodes and Interfaces Page
Note
In L3Outs with interface type SVI, configuration of secondary addresses is an option.
However, BGP sessions can only be sourced from the primary IP address of each
interface.
For BGP L3Outs, the Protocol Association page is where all the action is. Each
row completed on this page leads to the creation of a BGP peer connectivity
profile. While BGP peer connectivity profiles contain many options, the wizard
allows configuration of only the minimum number of required settings, which in
the case of eBGP consist of the neighbor IP address (Peer Address parameter),
the remote ASN, and the maximum number of hops to the intended peer (EBGP
Multihop TTL parameter). BGP peers need to be defined either under logical
interface profiles or under logical node profiles. When the intent is to source a
BGP session from the node loopback interface, you can configure the BGP peer
under a logical node profile. The first row in Figure 9-56 represents a
configuration that will be placed under the logical node profile. BGP peers
whose sessions should be sourced from non-loopback interfaces need to be
configured under a logical interface profile. The second row in this figure
represents a configuration object that will be deployed under a logical interface
profile sourcing the subinterface earlier defined for interface Ethernet1/7. Click
Next to continue.
Figure 9-56 Defining a BGP Peer with Session Sourced from a Routed
Subinterface
Note
If you decide to populate only the Peer Address field, ACI builds a BGP peer
connectivity profile suitable for a directly attached iBGP peer.
Note
Instead of creating multiple BGP peer connectivity profiles for neighbors that are all in a
single subnet and that require the same set of policies, you can use a feature called
Dynamic Neighbor. With this feature, you can have ACI dynamically establish BGP
peerings with multiple neighbors by configuring a subnet instead of an individual IP
address in the Peer Address field. When BGP is configured with dynamic neighbor
configuration, ACI does not attempt to initiate sessions to IP addresses in the peer
subnet. The other side needs to explicitly configure the ACI border leaf IP address to
start the BGP session. The Dynamic Neighbor feature was called Prefix Peers in
standalone NX-OS, and this term is also used in some Cisco ACI documentation.
Finally, Figure 9-57 shows how to create a new external EPG straight from the
L3Out creation wizard. In this case, the external EPG has the scope External
Subnets for External EPG set, which means the external EPG can be used for
classification of outside traffic. Because in this example remote testers need to
log in to systems in EPG-Client-VMs, created in Chapter 8, the external EPG is
consuming a contract that grants such access. Click Finish to deploy the BGP
L3Out. BGP peerings should come up if equivalent configuration has been
deployed on the peer router.
Figure 9-57 Creating an External EPG Classifying External Traffic in the
L3Out Wizard
Implementing BGP Customizations at the Node Level

Once a BGP L3Out has been created, you may want to customize certain BGP
settings at the border leaf level. Assuming that different logical node profiles
have been used for each border leaf, different node-specific policies such as
BGP timers can be applied to each switch.
To configure a node-specific BGP protocol profile, right-click the node profile
of interest and select Create BGP Protocol Profile. The Create Node Specific
BGP Protocol Profile page contains two drop-downs at the time of writing. One
is for BGP timer customization, and the other is for AS_PATH policy
customizations. Recent versions of ACI code no longer support AS_PATH
policy customizations to enable ECMP across eBGP peers in different
AS_PATHs; therefore, this second set of policies is not addressed in this
book. Figure 9-58 shows the creation of a custom BGP timer policy.
Figure 9-58 Creating a Custom BGP Timer Policy
None of the settings in Figure 9-58 should be new for you if you deal with BGP
on a regular basis, but Table 9-4 provides a review of the options.
Table 9-4 Configuration Parameters for BGP Timer Policies
Parameter
Keepalive Interval Specifies the interval at which keepalive messages are sent after a BGP
(sec) established. The default value for this setting is 60 seconds.
Hold Interval Specifies the interval at which keepalive messages must be received fo
(sec) the BGP peer operational. The default value for this setting is 180 seco
Stale Interval Specifies when to delete stale routes in case a session is not reestablish
(sec) established interval. When a graceful restart is in progress, the routes p
from the peer are still used for forwarding but marked as stale. Once th
two routers is reestablished and route information is synced again, all t
deleted and the routes from the latest exchange are used. This interval
The default value is 300 seconds.
Graceful Restart Specifies the restarting router triggered when a graceful restart is in pro
Helper likely simply helping the graceful restart operation with the restarting r
provides only graceful restart helper capability because ACI does not s
Parameter
supervisor switchover within each individual switch node. Only a cold

Instead, routing protocol high availability (HA) should be achieved by
switch nodes.
Maximum AS When nonzero, prompts ACI to discard eBGP routes received if the nu
Limit segments exceeds the stated limit. The default value of zero implies no
limit.
Note that configured intervals take effect only after a new BGP session is
established.
BGP timer policies can also be applied at the VRF level.
Implementing Per-Neighbor BGP Customizations

Figure 9-59 shows the majority of settings that can be customized on a per-
BGP-neighbor basis. Two additional settings, Admin State and Route Control
Profile, have been left out of this screenshot. Depicted customizations include
modification of the Local-AS Number setting 65600 for establishing an eBGP
session to a router in remote ASN 65700. The Local-AS Number Config
parameter has been left blank, prompting ACI to advertise routes to this external
neighbor by appending 65600 to the fabric route reflector ASN.
Figure 9-59 Customization of the Local AS Number for a Specific BGP Peer
Table 9-5 describes the customization options available in BGP peer
connectivity profiles.
Table 9-5 Configuration Parameters in BGP Peer Connectivity Profiles

Parameter
Allow Self AS Allows ACI to receive routes from eBGP neighbors when the rou
BGP AS number in the AS_PATH. This option is valid only for e
AS override Allows ACI to overwrite a remote AS in the AS_PATH with the A

is typically used when performing Transit Routing from an eBGP
eBGP L3Out with the same AS number. Otherwise, an eBGP peer
accept the route from ACI because of AS_PATH loop prevention.
is enabled, Disable Peer AS Check also needs to be enabled. This
for eBGP peers.
Disable Peer AS Allows ACI to advertise a route to the eBGP peer even if the mos
Check AS_PATH of the route is the same as the remote AS for the eBGP
is valid only for eBGP peers.
Parameter
Next-hop Self Allows ACI to update the next-hop address when advertising a ro
peer to an iBGP peer. By default, route advertisement between iB
original next-hop address of the route, and the one between eBGP
updates the next-hop address with a self IP address.
Send Community When enabled, allows ACI L3Out to advertise routes with a BGP
attribute, such as AS2:NN format. Otherwise, the BGP Communit
stripped when routes are advertised to the outside.
Send Extended When enabled, allows ACI L3Out to advertise routes along with t
Community Community attribute, such as RT:AS2:NN, RT:AS4:NN, and so o
BGP Extended Community attribute is stripped when routes are a
outside.
Password/Confirm When configured, allows the BGP peering to use MD5 authentica
Password TCP session. The password can be reset by right-clicking the BGP
profile and selecting Reset Password.
Allowed Self AS Sets the maximum count for the Allow Self AS option under BGP
Count
Bidirectional Enables BFD on the BGP neighbor.

Forwarding Detection
Disable Connected Provides an alternative to increasing the eBGP multihop TTL in c

Check a security concern about increasing TTL unnecessarily. For eBGP
checks whether the neighbor IP is on the same subnet as any of its
see if the neighbor IP is directly connected. If it is not, BGP autom
that the TTL needs to be larger than 1. Hence, when BGP is peerin
with directly connected routers, the BGP peering is rejected witho
Multihop TTL being set to 2 or larger, even though TTL 1 is techn
Weight for routes from Sets the default value of a Cisco proprietary BGP path attribute w
this neighbor routes learned from the border leaf by the configured peer.
Remove Private AS In outgoing eBGP route updates to this neighbor, removes all priv
from the AS_PATH when the AS_PATH has only private AS num
Parameter
is not applied if the neighbor remote AS is in the AS_PATH.
Remove All Private In outgoing eBGP route updates to this neighbor, removes all priv
AS from the AS_PATH, regardless of whether a public AS number is
AS_PATH. This feature does not apply if the neighbor remote AS
AS_PATH. To enable this option, Remove Private AS needs to be
Replace Private AS In outgoing eBGP route updates to this neighbor, replaces all priv
with Local AS the AS_PATH with ACI local AS, regardless of whether a public
remote AS is included in the AS_PATH. To enable this option, Re
AS needs to be enabled.
BGP Peer Prefix Defines an action to take when the number of received prefixes fr
Policy exceeds the configured maximum number. This option is activated
BGP peer prefix policy to the BGP peer connectivity profile.
Local-AS Number Disguises the ACI BGP ASN with the configured local ASN to pe
neighbor. When this feature is used, it looks like there is one more
between the ACI BGP AS and the external neighbor. Hence, the n
the configured local ASN instead of the real ACI BGP ASN. In su
the local ASN and the real ACI BGP ASN are added to the AS_PA
advertised to the neighbor. The local ASN is also prepended to rou
the neighbor.
Local-AS Number Allows granular control over how the local ASN and the fabric A
Config AS_PATHs of routes advertised to external routers or received by
The no-prepend option prevents ACI from prepending the local A
AS_PATHs of routes learned from this neighbor.
The no-prepend, replace-as option allows ACI to add only a local
both a local ASN and a real ACI BGP ASN, to the AS_PATHs of
this neighbor on top of the no-prepend option effect.
The no-prepend, replace-as, dual-as option allows the neighbor to
local ASN and a real ACI BGP ASN on top of the no-prepend and
effect.
Admin State Enables a BGP session with a peer to be turned off or on.
Parameter
Route Control Profile Allows application of a route profile to a specific BGP neighbor.
Implementing BFD on a BGP L3Out

One of the most common tweaks in BGP peer connectivity profiles is to
implement BFD with neighbors. BFD is not supported on loopback interfaces
since there is no support for BFD multihop in ACI at the time of writing. BFD
is also not supported for dynamic neighbors (prefix peers).
To enable BFD on a BGP L3Out, navigate to the desired BGP peer connectivity
profile and enable the Bidirectional Forwarding Detection checkbox, which is
visible in the Peer Controls section of Figure 9-59, shown earlier.
Customization of BFD timers on a BGP L3Out requires application of a custom
BFD policy and BFD interface profile. To apply a previously created BFD
interface policy or to create a new one, right-click the desired logical interface
profile under the L3Out and select Create BFD Interface Profile. Then select or
create a BFD interface policy.
Implementing BGP Customizations at the VRF Level

Aside from assigning a BGP timer policy at the VRF level, you can create a
custom BGP address family context policy for VRF-wide application to the
IPv4 unicast address family or the IPv6 unicast address family. Figure 9-
60 shows options that can be tweaked for a BGP address family context policy.
Figure 9-60 Creating a BGP Address Family Context Policy
Table 9-6 explains these configuration options.
Table 9-6 Configuration Parameters for BGP Timer Policies
Parameter
eBGP Distance Specifies the administrative distance for eBGP-learned routes. The d
routes is 20.
iBGP Distance Specifies the administrative distance for eBGP-learned routes. The d
routes is 200.
Local Distance Is used for aggregate discard routes. The default AD for such routes
eBGP/iBGP Max Configures the maximum number of equal-cost paths a switch adds t
ECMP for eBGP-learned and iBGP-learned routes. The default value in ACI
16.
Enable Host Route Is used only for the GOLF feature and is therefore beyond the scope
Leak 620 exam.
Once it is configured, a BGP address family context policy needs to be applied

to an address family for the policy to take effect. Figure 9-61 shows a BGP
timer policy being applied VRF-wide to Multiplayer-Prod and a BGP address
family context policy being applied to the IPv4 unicast address family for the
VRF.
Figure 9-61 Applying BGP Policies at the VRF Level

Note
Although it is not impossible, it is improbable that DCACI candidates will need to know
the name of a setting that modifies the administrative distance of routes learned by a
specific routing protocol. DCACI 300-620 exam candidates should, however, know that
enforcement of policies around protocol timers and administrative distances often take
place at the VRF level.
Implementing OSPF for IP Reachability on a BGP

L3Out
To use OSPF for dynamic IP reachability advertisement on a BGP L3Out, select
the root of the L3Out, click the Policy menu, and navigate to the Main
submenu. Toward the bottom of the page, you see options for enabling OSPF.
The available configuration options, as indicated in Figure 9-62, are exactly the
same as those available for OSPF L3Outs. But when OSPF and BGP are
enabled side-by-side in the same L3Out, OSPF is programmed only to advertise
its L3Out loopback and interfaces IP addresses.
Figure 9-62 Enabling OSPF to Advertise IP Reachability for BGP Sessions
Implementing Hot Standby Router Protocol (HSRP)

ACI supports HSRP on L3Out routed interfaces and routed subinterfaces. When
deploying HSRP, external devices must provide Layer 2 connectivity between
the ACI border leaf switches that run HSRP. This allows the exchange of Hello
messages over external Layer 2 connections. HSRP Hello messages do not pass
through spine switches.
To configure HSRP in ACI, navigate to the logical interface profile of interest
and select the Create HSRP Interface Profile checkbox. The process of
configuring an HSRP interface policy should be straightforward for those
familiar with HSRP. Note that the HSRP virtual IP address must be in the same
subnet as the interface IP address.
Because SVIs on L3Outs can leverage secondary addresses, use of HSRP in
ACI L3Outs for connectivity to appliances that only support static routing is not
common. In such cases, static routes on appliances should point to the
secondary addresses configured in the L3Out to enable next-hop high
availability.
IPv6 and OSPFv3 Support

IPv6 is not discussed in this book because ACI attempts to make the transition
to IPv6 natural by not delineating between IPv6 and IPv4.
To enable OSPFv3 on an L3Out, enable the OSPF checkbox on the L3Out Main
page and assign an IPv6 address to the L3Out and witness the border leaf bring
up the OSPFv3 process.
IMPLEMENTING ROUTE CONTROL

Sometimes it may be necessary to filter certain routes or make changes to
metrics and other attributes of routes advertised out of a fabric or learned by a
fabric. Where granular route control is a requirement, ACI route profiles can
help.
Route Profile Basics

As you have seen, ACI creates route maps behind the scenes to accomplish
certain tasks. For example, when an administrator associates a bridge domain
with an L3Out and updates the scope of a BD subnet to Advertised Externally,
ACI creates a route map to redistribute the subnet out the specified L3Out.
ACI uses route maps internally for quite a few different purposes, such as infra
MP-BGP route distribution, BD subnet advertisement to the outside world, and
transit routing.
Route profiles give users the ability to add user-defined match rules or set rules
for route filtering or route manipulation. Route profiles can alternatively be
referred to as route control profiles. The term route map is also sometimes used
interchangeably with route profile, but in terms of implementation, each route
profile is more a collection of implicit and explicit route maps.
Some of the use cases for this feature that are inside the scope of the DCACI
300-620 exam are as follows:
 A route profile to allow advertisement (export) of BD subnets to
the outside world via L3Outs
 A route profile to limit learning (importing) of external routes from
the outside world via L3Outs
 A route profile to set certain attributes on routes exiting or entering
the fabric if they meet a certain criterion
A route profile can be associated with any of the following objects/components:
 A bridge domain
 A bridge domain subnet
 An EPG on an L3Out
 A subnet of an EPG on an L3Out
 A route peering neighbor
 An entire L3Out
Table 9-7 defines the various components of a route profile in ACI.
Table 9-7 Components of a Route Profile

Component Description
Route The route profile type is specific to ACI. There are two route profile types. O
Profile Type Prefix AND Routing Policy, combines prefixes from the component that the
associated with and the match criteria configured in the route profile. Compo
profiles can be associated to include bridge domains, bridge domain subnets,
EPGs, and L3Out EPG subnets. The other type, Match Routing Policy Only,
routes based on criteria configured in the route profile and ignores prefixes f
components with which the route profile is associated.
Context In a sense, each entry in a route profile includes two context options: Order a
equivalent to a sequence number in a normal route map with the caveat that s
merge internal route maps of components with statements explicitly entered
changing the actual applicable sequence of rules. Action consists of permit o
equivalent in function to permit or deny in a normal route map.
Match Rules Route profile match rules are similar to match clauses in route maps. Clause
against include prefixes, community attributes, and regular expressions.
Set Rule Set rules are equivalent to set clauses in a route map. ACI can set parameters
community attributes, weight, OSPF types, and AS_PATH.
The next few subsections provide examples of route profile implementation.

Note
The coverage of route profiles in this book is just the tip of the iceberg when it comes to
route control in ACI. The content and examples in this chapter reflect the scope of the
DCACI 300-620 exam. Where deploying these solutions in environments with route
leaking or transit routing, additional caveats may apply.
Modifying Route Attributes to All Peers Behind an

L3Out
One of the most prevalent use cases for route maps in traditional routers and
Layer 3 switches is to manipulate route metrics or to add attributes to routes.
Let’s take a look at an example of how route profiles address this use case. Say
that a fabric has multiple L3Outs, and an IT team wants to assign different BGP
communities to routes advertised out of each L3Out. The idea is that the BGP
communities could then be used by the external devices for traffic engineering
or any type of policy enforcement.
To address any route profile requirement, the implementation team needs to
fully understand the required match rules and set rules. Since the idea is that the
interesting prefixes are any ACI routes egressing a specific L3Out, and routes
advertised have already been marked with the scope Advertised Externally and
have been associated with the relevant L3Out, there should be an easy way to
match such prefixes. It turns out there is. Application of a route profile at the
default export level of an L3Out automatically satisfies this requirement without
the need for Match statements.
The next part of the equation is configuration of set rules. To configure a set
rule, go to the tenant in question, double-click Policies, double-click Protocol,
right-click Set Rules, and select Create Set Rules for a Route Map. Figure 9-
63 shows a set rule that assigns BGP community 65000:100 to any matched
routes. Note also the other potential options for set rules.
Figure 9-63 Creating Set Rules for a Route Profile

By default, ACI strips BGP communities in route advertisements to external
peers. To address this, as shown in Figure 9-64, you can enable the Send
Community checkbox in the BGP peer connectivity profile for the single BGP
neighbor currently attached to the L3Out. If advertising extended BGP
communities, you need to enable the Send Extended Community.
Figure 9-64 Enabling Community Value Advertisements to a Neighboring

Router
The next step is to actually configure the route profile on the L3Out. At a
minimum, each route profile requires at least one context. Each context is
similar to a sequence number in a traditional route map. As indicated in Figure
9-65, creation of a context requires entry of a numeric value indicating the
priority or order of the context, whether to permit (advertise) or deny (drop) the
route, any match criteria (Match Rule drop-down) to determine prefixes of
interest, and any modifications (Set Rule drop-down) ACI should apply to
matched routes.
Figure 9-65 Creating a Context Within a Route Profile
Note in Figure 9-65 that a match rule, in this instance, was not required. This is
specifically because the route profile type Match Prefix AND Routing Policy
has been selected, as indicated in Figure 9-66. The term Match Prefix Policy
can be understood as a directive for ACI to consider any prefixes associated
with the object(s) to which the route profile is associated to be implicit match
rules. In this case, the default-export route profile is associated with the entire
L3Out. Therefore, Match Prefix Policy implicitly matches all BD subnets
associated with the L3Out with the scope Advertised Externally, even though
there is no explicit match statement in the route profile. You can read the term
Match Routing Policy to refer to the type of explicit match statements engineers
are used to adding to route maps. The phrase Match Prefix AND Routing
Policy, therefore, is a merging of the implicit route map(s) and match rules
explicitly configured in the route profile.
Figure 9-66 Setting Route Profile Type to Match Prefix AND Routing Policy
Once these settings are implemented, all routes out of the specified L3Out
should be advertised out to all peers behind the L3Out using the community
65000:100, as shown in Example 9-7.
Example 9-7 Validating Community Values on a Route on Nexus Platforms

Router# show ip bgp 10.233.58.0/24 vrf LAB-BGP
BGP routing table information for VRF LAB-BGP, address family IPv4 Unicast
BGP routing table entry for 10.233.58.0/24, version 14
Paths: (1 available, best #1)
AS-Path: 65000 , path sourced external to AS
10.197.1.1 (metric 0) from 10.197.1.1 (10.233.75.170)
Origin incomplete, MED 0, localpref 100, weight 0
Community: 65000:100
Path-id 1 not advertised to any peer
Earlier in this example configuration, the text indicated that the Deny action
drops matched routes and does not advertise them out the L3Out. This is not
always true, especially when using route profiles of the type Match Prefix AND
Routing Policy. Because this particular type of route profile merges multiple
route maps together, it can sometimes be difficult to determine whether an
implicit route map has already permitted advertisement of the prefix.
Modifying Route Attributes to a Specific Peer Behind an

L3Out
Now let’s say that the objective is a little different from the objective we’ve
been considering. Assume that multiple BGP peers have been deployed behind
an L3Out, and different community values need to be applied to outbound
routes to each peer behind the L3Out. In such a scenario, the default-export
route profile would be of little help. Instead, route profiles would need to be
applied to the BGP peer connectivity profiles. But, in addition, explicit match
rules would be needed to specify the prefixes of interest. This is because BD
subnets do not have a direct association with BGP peers.
Figure 9-67 shows an explicit match statement for the subnet assigned to BD-
Production. Note that the Aggregate flag has been set to False, indicating that
this rule matches only 10.233.58.0/24 and not host routes within that range.
Figure 9-67 An Example of a Match Rule Determining an Explicit Prefix as the

Only Match
If the expectation is that several conditions need to be met for a prefix to be
matched, all the conditions can be specified under a single Match Rule object.
For example, if the requirement for a match is for a prefix to be in the
10.5.0.0/16 range AND have a community value of 65600:100, these two
conditions should both be included in a single match rule. On the other hand, if
the goal is to say that either prefixes under 10.5.0.0/16 OR community value
65600:100 matches the criteria for advertisement with a particular set rule,
separate route map sequences (separate context policies) should be configured
for these two requirements, each referencing a different match rule.
Once the match rule has been defined, a route profile needs to be created
under Tenants > the tenant in question > Policies > Protocol > Route Maps
for Route Control. As indicated in the background in Figure 9-68, tenant-level
route profiles do not display the route profile type because route profiles defined
here are for use cases such as MP-BGP interleak and BGP route dampening,
which do not benefit from Match Prefix AND Routing Policy. Notice in Figure
9-68 that both a match rule and a set rule have been assigned to the route profile
context.
Figure 9-68 Creating a Route Profile with an Explicit Prefix List Assigned

After the route profile has been created, it needs to be assigned to the desired
BGP peer. Figure 9-69 shows that the route has been assigned to a BGP peer
connectivity profile in the export direction. In traditional route map terms, this
should be understood as the out direction.
Figure 9-69 Applying a Custom Route Profile in the Outbound (Export)
Direction
With this change, the defined community values should only be applied to the
one BGP peer.
One thing that might not be apparent from the configuration shown here is that
the act of matching 10.233.58.0/24 in a custom outbound route profile or
default-export route map is an alternative method to redistribute BD subnets
outside the fabric. In other words, as long as BD subnet 10.233.58.0/24 has been
marked Advertised Externally, this configuration would work to advertise the
subnet out of the fabric, even without a BD association with the L3Out.
Note
At the time of writing, there is no mechanism to assign a route profile on a per-neighbor
basis for OSPF and EIGRP.
Assigning Different Policies to Routes at the L3Out Level

Let’s say there is a need to be more granular and assign different community
values to routes as they leave the fabric on an L3Out. Figure 9-70 shows that a
new match rule and a new set rule have been associated with one another under
a route profile context. Notice that the order 1 has been assigned, given that
multiple requirements exist.
Figure 9-70 Configuring Multiple Route Profile Contexts
Figure 9-71 shows how contexts can be used together to match different subnets
with community values or other set rules. One very important question here is
whether it makes sense to use Match Prefix AND Routing Policy in instances
like this. The problem with using this particular route profile type is that the
merging of the explicit route profile rules with implicit route maps may place
implicit matches into the route map sequence specified with the order 0. This, in
turn, could assign 65000:100 to all BD subnets, even though this might not be
the intention. In any instance, where the intention is to only apply explicit
policy, the route profile type can be toggled to Match Routing Policy Only. In
this example, subnets in the 10.233.58.0/23 supernet range are assigned the
BGP community 65000:100, while subnets in the 10.233.60.0/23 supernet range
are assigned to 65000:101. Any subnets not in the ranges specified by match
rules hit an implicit deny rule and are not advertised.
Figure 9-71 Reliance on Explicit Match Rules Using Match Routing Policy
Only
Note
Although not apparent in this case, the policy type depicted here blocks exit of any
routes not matched with an explicit Permit action. Can this be useful if a border leaf
needs to deploy OSPF and EIGRP L3Outs side-by-side within a VRF? If there is a
requirement to advertise different subnets out of each L3Out and sidestep the issue
discussed earlier in this chapter, the answer is yes. Use of a route profile with Match
Routing Policy Only on one L3Out and leveraging regular BD subnet advertisement
mechanisms for the second L3Out is a valid (but probably not recommended) way to
overcome limitations related to OSPF and EIGRP L3Outs in the same VRF on the same
border leaf advertising the same subnets due to application of common implicit route
maps.
Configuring Inbound Route Filtering in ACI

ACI learns all routes inbound by default. If this behavior is not desired, navigate
to Policy > Main for an L3Out and enable the Route Control Enforcement
Import checkbox, as shown in Figure 9-72. This setting enables administrators
to enforce inbound route profiles.
Figure 9-72 Enabling Route Control in the Inbound Direction on an L3Out
As shown in Figure 9-73, the default-import route profile on the L3Out can be
used in conjunction with explicit prefix lists and match rules to determine what
routes can be imported into the fabric.
Figure 9-73 Creating a default-import Route Profile to Filter Specific Inbound

Routes
Note
The goal of this section on route profiles was not to provide step-by-step guidance on
configuring route profiles but to demonstrate route profiles and route profile types in
context. The coverage is by no means complete but shows that what is within the scope
of DCACI is very limited.
EXAM PREPARATION TASKS

As mentioned in the section “How to Use This Book” in the Introduction, you
have a couple of choices for exam preparation: Chapter 17, “Final Preparation,”
and the exam simulation questions in the Pearson Test Prep Software Online.
REVIEW ALL KEY TOPICS

Review the most important topics in this chapter, noted with the Key Topic icon
in the outer margin of the page. Table 9-8 lists a reference of these key topics
and the page numbers on which each is found.
Table 9-8 Key Topics for Chapter 9

Key Topic Description
Element
List Details the five critical functions of an L3Out
List Describes the anatomy and important components of a typical L3Out
List Lists the types of interfaces that can be configured under an L3Out logical
interface profile
Paragraph Describes why a port or port aggregation with an EPG mapping can no lon
function as an L3Out routed interface or routed subinterface
Figure 9-5 Details the impact in terms of route peerings when using the same SVI
encapsulation across interfaces than when different encapsulations are used
Paragraph Describes ACI behavior when an SVI with a particular encapsulation is

deployed on a second L3Out on a different border leaf switch
Paragraph Describes the significance of the SVI Encap Scope setting

Element
Figure 9-7 Compares use of SVI Encap Scope of a VRF versus Local on L3Outs on a
particular leaf switch
Paragraph Describes default ACI behavior with SVI Auto State set to Disabled
Figure 9-8 Illustrates the impact on static routes during failover with ACI Auto State
Disabled
Figure 9-9 Illustrates the impact on static routes during failover with ACI Auto State
Enabled
Paragraph Details what happens if no BGP route reflector has been configured in an A
fabric
List Lists the two configuration parameters needed for implementing BGP rout
reflection in ACI
Figure 9-14 Shows configuration of BGP route reflection under the BGP route reflecto
policy object
Figure 9-17 Shows enablement of EIGRP and entry of autonomous system
Figure 9-18 Illustrates entry of node and interface information using routed interfaces
Figure 9-20 Shows the external EPG creation page of the L3Out wizard
Paragraph Describes the significance of the External Subnets for External EPG subne
scope
Figure 9-25 Shows the General tab in an external EPG, which provides a quick view in
subnets and scopes
Figure 9-26 Shows how to mark a BD as a candidate for subnet redistribution
Figure 9-27 Shows toggling a BD subnet with Advertised Externally

Element
Figure 9-29 Illustrates a filter for communication over an L3Out
Figure 9-30 Illustrates a contract subject for filter
Paragraph Explains contract directionality in the context of external EPGs
Paragraph Describes the Advertise Host Routes setting
Paragraph Describes implementation of BFD on an EIGRP L3Out
Table 9-2 Details customizable settings for EIGRP applied at the VRF level
Figure 9-44 Shows OSPF configuration options and area types supported in ACI
Paragraph Describes a use case for secondary IP addresses on L3Out SVIs
Figure 9-45 Illustrates configuration of L3Out SVIs with secondary IP addresses
Paragraph Describes some scope implications for external EPGs that classify traffic
Paragraph Describes a common problem related to deployment of OSPF and EIGRP

L3Outs side-by-side within a VRF on the same leaf switches
List Calls out the most common and recommended solutions where OSPF and
EIGRP need to be deployed on the same border leaf while advertising diffe
subnets
Table 9-3 Details customizable parameters under OSPF timer policies
Paragraph Explains that static routing L3Outs do not need to deploy any dynamic rou
protocols
Paragraph Explains the Preference field for static routes
Figure 9-49 Demonstrates adding a static route to an L3Out

Element
List Describes the process for implementing IP SLA tracking for static routes
Paragraph Describes the various fields in the IP SLA configuration wizard
Paragraph Describes the process of creating a track member
Paragraph Describes the significance of configuration options for track lists
Paragraph Explains the difference between deploying BGP configurations at the node
profile level versus the interface profile level
Figure 9-58 Shows the custom BGP timer policy page
Paragraph Explains that BGP timer policies can be applied not just at the node level b
the VRF level
Table 9-5 Defines configuration parameters in BGP peer connectivity profiles
Table 9-7 Describes the various configuration components that make up a route prof
COMPLETE TABLES AND LISTS FROM

MEMORY
Print a copy of Appendix C, “Memory Tables” (found on the companion
website), or at least the section for this chapter, and complete the tables and lists
from memory. Appendix D, “Memory Tables Answer Key” (also on the
companion website), includes completed tables and lists you can use to check
your work.
DEFINE KEY TERMS

Define the following key terms from this chapter and check your answers in the
glossary:
logical node profile
logical interface profile
floating SVI
L3Out bridge domain (BD)
route reflector
interleak
secondary IP address
route profile
Find answers on the fly, or master something new. Subscribe today. See pricing
options.
 Support

 Sign Out
© 2021 O'Reilly Media, Inc. Terms of Service / Privacy Policy

Chapter 9 L3Outs

Uploaded by

Copyright:

Available Formats

Chapter 9 L3Outs

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 9 L3Outs

Uploaded by

Copyright:

Available Formats

What are L3Outs and what are their main components?

What are L3Outs and what are their main components?

How are external subnets and EPGs related in ACI?

How are external subnets and EPGs related in ACI?

Skip to content

Table of Contents for

Part III External Connectivity

NEXT Next Chapter

This chapter covers the following topics:

L3Out Fundamentals: This section covers concepts related to L3Outs, the

This chapter covers the following exam topic:

“DO I KNOW THIS ALREADY?” QUIZ

L3Out Fundamentals 1–4

Deploying L3Outs 5–9

Implementing Route Control 10

Stub Network and Transit Routing

Key Functions of an L3Out

The Anatomy of an L3Out

 L3Out root: As shown in Figure 9-2, the most critical L3Out

Planning Deployment of L3Out Node and Interface

Understanding L3Out Interface Types

Understanding L3Out Bridge Domains

If, on the other hand, an administrator attempts to reuse an encapsulation in a

Figure 9-6 Common SVI Encapsulation Used for L3Outs on Different Switches

Understanding SVI Encap Scope

Understanding SVI Auto State

Understanding Prerequisites for Deployment of L3Outs

L3 Domain Implementation Examples

Figure 9-10 Hypothetical L3Out Connectivity Desired for a VRF

Figure 9-13 Associating Physical Domains and L3 Domains Simultaneously

Understanding the Need for BGP Route Reflection

Implementing BGP Route Reflectors

 Enter the BGP ASN the fabric should use internally.

Figure 9-14 Configuring BGP Route Reflectors Within a Fabric

Understanding Infra MP-BGP Route Distribution

Figure 9-16 Infra MP-BGP Architecture and Route Distribution

Click here to view code image

Total peers 5, established peers 5

VRF overlay-1, local ASN 65000

peers 2, established peers 2, local router-id 10.233.46.32

Neighbor ASN Flaps LastUpDn|LastRead|LastWrit St Port(L/R) Notif(S/R)

10.233.46.33 65000 0 16w06d |never |never E 60631/179 0/0

10.233.46.35 65000 0 16w06d |never |never E 44567/179 0/0

LEAF101# acidiag fnvread | grep spine

201 1 SPINE201 FDOXXXX1 10.233.46.33/32 spine active 0

202 1 SPINE202 FDOXXXX2 10.233.46.35/32 spine active 0

The external route distribution process in ACI is extremely reliable. If all

Configuring an L3Out for EIGRP Peering

Deploying External EPGs

Figure 9-24 shows the addition of the subnet 10.200.100.0/24 to Admin-Access.

Figure 9-24 Adding Subnets as External Subnets for an External EPG

Figure 9-25 The General Tab of an External EPG

Verifying Forwarding Out an L3Out

Click here to view code image

EIGRP neighbors for process 100 VRF Gaming-BU:Multiplayer-Prod

H Address Interface Hold Uptime SRTT RTO Q Seq

(sec) (ms) Cnt Num

0 10.233.75.161 eth1/9 13 07:30:37 1 50 0 19

Click here to view code image

IP Route Table for VRF "Gaming-BU:Multiplayer-Prod"