03-Brkens-2023 (2021)

#CiscoLive
The value and details of

Multidomain Pairwise
Integration between SDA-ACI
Markus Harbeck, Principal Architect, CXPM XDA

@mhgrisu
BRKENS-2023
CCIE #8087
CCDE #20130015
#CiscoLive
Agenda
Multidomain
#CiscoLive BRKENS-2023 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Agenda
• SDA – ACI overview – THE WHY!
• What is needed for SDA – ACI
integration?
• Policy across SDA and ACI
• How is the integration achieved?
• Summary
#CiscoLive BRKENS-2023 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public
Short Hint:
“My English might be bad
but although sexy”
Source: Henning Bornemann –
“Thank you for Deutsche Bahn”
Do you recognize your network?
Can you automate this Network?
Transformation  End 2 End IT
Copyright by Hanna
Copyright by Saskia
Better together gives you end to end identity
Mobility in the past Mobility with cars
Autonomous driving
Horse drawn today
Source: www. pinterest.de Source: www.zeit.de
Source: www.welt.de
Note: We are here !

Who is Markus Harbeck?
Personal:
 Location: Eschborn, Germany (near Frankfurt) but lives in Bavaria
 Interests: My family, my 2 kids, Horse back riding, motor cycling,
hiking
My Background:
 CLI Junkie since 1996 for all Routing and Switching
 Joined CISCO October 2010
 Before; 12 years, operations, engineering, application engineering
at Lufthansa Systems (Airline)
 Book Author – Cisco DNA Assurance 2018
 Close Customer relations and interlock
 Tiger Teams and BU Advisor CCIE #8087
CCDE #20130015
Current Projects:
 X – Domain / Architecture
 Orchestration and X Domain
 as a Service
 Analytics, Assurance, Migration, Automation
 Services, Automation X, NG Analytics
 SDA, ACI, SDWAN, ITSM, Secure Remote
Workforce
Copyright by Hanna
 Cisco DNA Center since day1 in 2014
My Kids view on Cisco DNA Center and

Network Design
Copyright by Saskia
Cisco provides Pairwise Domain Integrations
Today
APIC Data Center (ACI)
Users
vManage
Cisco DNA Center
Public Cloud
SD-Access SaaS
Devices SD-WAN
Campus/Branch
Internet
Integration of Controllers
SD SDN
Access DC
End-Users (Cisco DNA Center) (Cisco APIC) Data &
Applications
SD
WAN
Users & Devices Data & Applications
(Cisco vManage)
Multi-Cloud &
WAN
What is Cisco SD-Access?
Campus Fabric + Cisco DNA Center (Automation & Assurance)
 SD-Access
APIC-EM
Automation
1.X
GUI approach provides automation & assurance
ISE Analytics
PI
of all Fabric configuration, management and
Cisco DNA group-based policy
Center
Cisco DNA Center integrates multiple
management systems, to orchestrate LAN,
Wireless LAN and WAN access
B B
 Campus Fabric
CLI or API approach to build a LISP + VXLAN +
C C
CTS Fabric overlay for your enterprise Campus
networks
Campus CLI provides backwards compatibility, but
Fabric management is box-by-box. API provides
device automation via NETCONF/YANG
Separate management systems
Fabric Roles & Terminology
 Cisco DNA Automation – provides simple
Cisco DNA
GUI management and intent based
Identity Automation Automation
automation (e.g. NCP) and context sharing
Services
ISE Analytics  Cisco DNA Assurance – Data Collectors
Cisco DNA (e.g. NDP) analyze Endpoint to App flows
Cisco DNA
Center Assurance and monitor fabric status
 Identity Services – NAC & ID Systems
(e.g. ISE) for dynamic Endpoint to Group
Fabric Border Fabric Wireless mapping and Policy definition
Nodes Controller
B B  Control-Plane Nodes – Map System that
manages Endpoint to Device relationships
Intermediate Control-Plane
C C Nodes  Fabric Border Nodes – A Fabric device
Nodes (Underlay) (e.g. Core) that connects External L3
network(s) to the SDA Fabric
Campus  Fabric Edge Nodes – A Fabric device

(e.g. Access or Distribution) that connects
Fabric Edge
Nodes Fabric Wired Endpoints to the SDA Fabric
E E E E  Fabric Wireless Controller – A Fabric device

(WLC) that connects APs and Wireless
Endpoints to the SDA Fabric
ACI
ACI Spines
ACI Leafs
L4 -7 Services
External L2 / Servers
L3
APIC APIC APIC APIC Cluster
Two Level Segmentation/Label Model
Network
1 1 Virtual Network (VN)

First level Segmentation that ensures
zero Communication between
different VNs (aka VRFs)
Building Management Campus Users
Context Context
2 2 Scalable / End-Point Group

Second level Segmentation within a
VN that ensures role-based access
Security Policy
control between End-Point Groups
The Intent of the Infrastructure
Contract
Source Group Destination Group
@ VN-X @ VN-Y
Allow Access for these users Priority Application for these users
SDN-ACI
SD
Access
End-Users Data &

Applications
SD-WAN
Transport Identity end 2 end

SDA and ACI together
SDA ACI
Management Cisco DNA Center APIC
Control Plane LISP CooP and BGP
Underlay Based on RLOC ISIS VTEP
Data Plane VXLAN VXLAN

Macro VN VN + Tenant
Segment Virtual Network Virtual Network
Micro SGT EPG
Segment Scalable Group Tag End Point Groups
What is SDA –
ACI Integration
SDA and ACI
Solution 1.0 Solution 2.0

Policy Plane Data Plane
#CiscoLive © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public
Day 0 Operations for 1.0
Pre-requirements:
pxGrid between DNAC and ISE is running
Admin DNAC Border ISE Border Leaf APIC

Node
Enable ISE-APIC integration
Ise will share. APICs

EPG and BL info to
L3 Border handoff with DNAC
IP Transit and VNs
/32 loopback on border
instead of SVIs Establish per VN SXP
peering(automated)
Download SG-EPG translation table
ACI and SDA Pairwise Integration Phase 1
SDA-ACI: Group/Identity Mapping Federation
DNAC ISE
MANAGEMENT API Based Group and Group Membership
&
SGT/IP and EPG/VM Cross {IP, group} communicated
POLICY between the domains via APIC
Domain
Identity API
Federation
C ISE acts as the API initiator and
registers for ACI attach/detach
events
SD-Access
Fabric Site ACI
B BL
Border
E E E L L
Control Single VRF BGP/OSPF/EIGRP
Plane (MP-BGP is not supported in this
CONTROL-PLANE LISP
BGP/OSPF/EIGRP
BGP & COOP
phase)
(VRF-Lite)
VXLAN SGT (16 bits) 802.1Q Header iVXLAN EPG(16 bits)

DATA-PLANE
12 1 Header VNID (24 bits) Header VNID (24 bits) Data 802.1Q between the domains,
VLAN ID (12 bits)
Plane VXLAN with 2 labels (VNID and
ClassID) internal to the domains
• DNA Version 1.2.10 and later • ACI 3.2 and later
• ISE Version 2.4 Patch 6 and later
Policy Applied on ACI Border (Option 1)
Create Create EPG

Create SGT
External EPG & Contract
Application
SecOps Admin (DC) Admin
Notify APIC of new

IP/Group Binding
Create EPG, Contract and

Filter Entries via DME
C
Learn IP/SGT
Binding
SD-Access
Fabric Site ACI
B BL
Border
E E E L L
Policy applied on the

ACI BL node
Policy Applied on Fusion Router/Firewall (Option 2)
Create SGT ISE Learns EPG and creates

corresponding SGT Create EPG
and SGACL Application
SecOps Admin (DC) Admin
Notify ISE of new

IP/Group Binding
Radius/SXP/PXGrid
Create EPG, Contract and
Filter Entries via DME
C
SD-Access FW/Router
Fabric Site B
Border BL
ACI
E E E L L
Policy applied on the

Fusion FW/Router
Day 0 Operations Solution 2.0
Admin DNAC Border ISE Border Leaf APIC

Kafka registration (APIC)
Enable ISE to APIC integration
SDA Domain registration & peering request to ACI domain
Accept Peering
Route peering information Request
Configure policy (VNs, SGTs)

Execute policy migration (ACA) Optional if previously done
Configure L3 border handoff

with ACI Transit and VNs,
Automate SXP peering config
Bring up dataplane
Provide services
Extend campus VRFs, route leaking
Download SG-EPG
Share EPG, Class ID#
translation table
#CiscoLive © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public
SDA and ACI 2.0
Cross Domain Messaging (VN, SGT/EPG Group, Contract, … Exchange)
Kafka Virtual Network, Group and Group
Kafka Kafka Membership {VN, IP, group},
Broker/Client Broker/Client BUS Tenant
Based and applied and available policies
ISE Policy NNI setup including Route Targets
Plane exchange between ACI and SDA
SDA ACI (ISE acts as broker client for DNAC
Border Border on the BUS)
Switch Leaf
E
SDA B
BL
ACI L
Control BGP-EVPN type-5 routes
BGP/EVPN (includes /32 or /128 host routes if
1 VXLAN
Plane configured)
LISP BGP & LISP/COOP
1 EPG-SGT SGT-EPG
Translation Translation Data Group Policy VXLAN Encapsulation
(contains both VNID and Group
VXLAN
Header
SGT (16 bits)
VNID (24 bits)

iVXLAN
Header
SGT/EPG (16 bits)
iVXLAN
Header
EPG (16 bits)
VNID (24 bits)

Plane Policy ID, Tenant System Interface,
VNID (24 bits)
labels)
SDA Domain ACI Domain
End to End VN Management
Campus VRF Extension into ACI
Tenant C-Wireless Tenant Sales
Campus VN Wireless (VNID 10) Shadow C-VRF Wireless (VNID 100)
Sales-VRF-1
Engineers (SGT 110) Engineers (EPG 10) Mail
(EPG 80)
Tenant C-Guest
Sales-VRF-2
Campus VN Guest (VNID 20) Shadow C-VRF Guest (VNID 200) Web
(EPG 90)
Customers (SGT 220) Customers (EPG 20)
C-VRF 1
(VNID 100)
• Campus exposes multiple VRFs to DC and ACI expose apps from multiple VRFs to campus
• SDA/DNAC admin initiates a “Remote Tenant” setup in the ACI domain for each Campus VRF
• For each defined Campus VRF there is a corresponding Shadow C-VRF created on ACI Border Leaf
• ACI VRFs are not exposed in the SDA campus
End to End VN Management Contracts Trigger Leaking of
Route-Leaking Occurs in ACI Fabric Campus Subnets
Tenant C-Wireless Tenant Sales

Campus VN Wireless (VNID 10) Shadow C-VRF Wireless (VNID 100)
Sales-VRF-1
Engineers (SGT 110) Engineers (EPG 10) Mail
(EPG 80)
Tenant C-Guest
Sales-VRF-2
Campus VN Guest (VNID 20) Shadow C-VRF Guest (VNID 200) Web
(EPG 90)
Customers (SGT 220) Customers (EPG 20)
C-VRF 1
(VNID 100)
Contracts Trigger Leaking ACI Services Subnets
• Campus SGT consuming an ACI Service: in ACI is represented as a “shared service” contract between C-VRF and the
VRF(s) of the different Application EPGs representing the ACI services
• The subnets representing the ACI services will be leaked into C-VRF on the ACI Border Leaf nodes and advertised toward
the Campus through BGP EVPN
• Similarly, the campus Subnets are advertised from the SDA border nodes into the C-VRF in ACI through BGP EVPN and
leaked into one or more application VRFs
ACI and SDA Integration Phase 2
Automation for end to end Networking
Automation for establishing connectivity
Cross Domain Messaging (VN/VRF, RT/RD, …)
ISE APIC • Automation that uses Kafka BUS to

bring up the domain to domain
EVPN RID-1 L
internetworking
TEP-1 EVPN RID-3
SDA
B BL • BGP peers and the tunnels are
ACI fully automated
vPC TEP
B
BGP EVPN peer used to exchange
BL
E
BL
•
EVPN RID-2 EVPN RID-4

prefix information for Campus and
TEP-2
L3Out Application subnets and
User
subnet
automated
BGP EVPN for communicating reachability (routing once
the peers and tunnels built)
At FCS – SDA/ACI Deployment options
Single ACI Fabric and Single SDA Domain
SDA FS SDA-ACI peering ACI
Campus / DC Co located B C Border Leaf

SDA
This can be an IP
network with
SDA FS x SDA Fabric Domain appropriate MTU
E B C
SDA
C
SDA FS
Multi SDA ‘Sites’ to DC SDA
ACI
Multiple FD to ACI possible Transit
E B C B B C
SDA SDA
MUST be the
SDA FS y same BN Border Leaf
SDA-ACI peering
SDA – ACI Deployment options
SDA FS SDA-ACI peering ACI
B C
Campus / DC Co located SDA
Border Leaf
SDA FS 2 Box SDA FS SDA-ACI peering ACI
Remote Site using E B C SDW B B C

SDA SDA
SD WAN Transit Transit
Can be the
cEdge cEdge same BN Border Leaf
SDA FS x SDA Fabric Domain

E B C
SDA
C
SDA FS SDA-ACI peering
Multi SDA FS to DC SDA
ACI
Multiple FD to ACI possible Transit
E B C B B C
SDA SDA
MUST be the
SDA FS y same BN Border Leaf
Multi-Domain Policy Exchange
Use of Kafka Communication Bus
Cross Domain Messaging (VN, SGT/EPG Group, Contract, … Exchange) • The Kafka cluster (broker) does not
Kafka Client Kafka Client
need to run in a specific location
PXGrid
2.0
For Phase 2.0 the APIC controller cluster
has a Kafka cluster running as an App
Cluster
In the future this broker functionality may be
Yang (EVPN)
SXP, moved elsewhere (MSO for example)
RADIUS, DME
xlate table
programming • Kafka clients running on APIC and ISE
SDA B
BL
ACI PXGrid still used for DNAC-ISE
communication
SDA ACI • Kafka Bus is used to join Domains: ACI

Border Border
Switch Leaf
is one domain (Datacenter), DNAC/ISE
is another domain (Campus)
Multi-Domain Policy Exchange
Cross-Domains Services and Operations Messaging BUS
Cross Domain Messaging (VN, SGT/EPG Group, Contract, … Exchange)
Message Message
• DNAC/ISE subscribe to DC • ACI publishes services DC VNs,
Publish Publish
services (EPGs, VNs, …) tenants, service name, EPGs,
bindings, contracts,
• DNAC/ISE publishes Campus provider/consumer, protocol,
SGTs that represent the ports
consumer and IP to SGT Message Message
bindings Subscribe Subscribe • ACI consumes SGT/IP
• What it is:
• Cross Domain Kafka pub/sub messaging bus model to publish/consume
services between operational domains
• Used also for the automation of infra configuration (MP-BGP EVPN peering, for
example)
Multi-Cloud Data Center is Multi-Domain Capable
Connect any user to all applications
Cross Domain Messaging
EBGP/EVPN
SD-Access EBGP/EVPN
VXLAN ACI VXLAN
& SDWAN B
BL
IPSec VPN
User Tunnel Application
Campus and WAN Infrastructure Administrative Domain Data Center and Cloud Infrastructure Administrative Domain
Demo Time
Cisco SDA – ACI
Integration
Cisco DNA Center
Policy Automation Analytics

How does
everything else
fit?
It is a little step with huge impact

Extending the Integrations
DNA ISE
Center Controller MSO Multi-Cloud vManage
App Groups App resource request

MSO
User Groups Path selection to meet
business intent
LA Campus SDA Fabric Dallas Campus SDA Fabric
Cisco SDWAN network New York

San Francisco ACI Data
ACI Data Center
Center
DCNM DCNM DCNM

Azure East
Los Angeles AWS Region East

Branch AWS Region
West Chicago
branch
Denver N9K DC
Summary
SD-Access Resources
Would you like to know more?
cisco.com/go/dna
cisco.com/go/sdaccess cisco.com/go/dnacenter
cs.co/en-cvds
• SD-Access At-A-Glance • Cisco DNA Center At-A-Glance
• SD-Access Ordering Guide • Cisco DNA ROI Calculator
• SD-Access Solution Data Sheet Validated Architectures, Prescriptive • Cisco DNA Center Data Sheet
• SD-Access Solution White Paper Guidance, Confidence to Deploy • Cisco DNA Center 'How To' Video Resources
• 4 Validated Design Guides

• 11 Prescriptive Deployment Guides
Call to action!
 1. Experience the Integrated network – try it out
 2. Enable End 2 End Automation and Analytics
 3. Enable Security End 2 End
 4. Leverage your installed base

1
1
1
1
User – Connection - Application
Thank you
#CiscoLive
#CiscoLive

03-Brkens-2023 (2021)

Uploaded by

Copyright:

Available Formats

03-Brkens-2023 (2021)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

03-Brkens-2023 (2021)

Uploaded by

Copyright:

Available Formats

#CiscoLive

The value and details of

Markus Harbeck, Principal Architect, CXPM XDA

Can you automate this Network?

Transformation  End 2 End IT

Source: www. pinterest.de Source: www.zeit.de

Note: We are here !

My Kids view on Cisco DNA Center and

APIC Data Center (ACI)

Separate management systems

Campus  Fabric Edge Nodes – A Fabric device

E E E E  Fabric Wireless Controller – A Fabric device

APIC APIC APIC APIC Cluster

1 1 Virtual Network (VN)

2 2 Scalable / End-Point Group

End-Users Data &

Transport Identity end 2 end

Management Cisco DNA Center APIC

Control Plane LISP CooP and BGP

Underlay Based on RLOC ISIS VTEP

Data Plane VXLAN VXLAN

Solution 1.0 Solution 2.0

Admin DNAC Border ISE Border Leaf APIC

Ise will share. APICs

VXLAN SGT (16 bits) 802.1Q Header iVXLAN EPG(16 bits)

Create Create EPG

Notify APIC of new

Create EPG, Contract and

Policy applied on the

Create SGT ISE Learns EPG and creates

Notify ISE of new

Policy applied on the

Admin DNAC Border ISE Border Leaf APIC

Configure policy (VNs, SGTs)

Configure L3 border handoff

VNID (24 bits)

VNID (24 bits)

Tenant C-Wireless Tenant Sales

Cross Domain Messaging (VN/VRF, RT/RD, …)

ISE APIC • Automation that uses Kafka BUS to

EVPN RID-2 EVPN RID-4

Campus / DC Co located B C Border Leaf

SDA FS 2 Box SDA FS SDA-ACI peering ACI

Remote Site using E B C SDW B B C

SDA FS x SDA Fabric Domain

SDA ACI • Kafka Bus is used to join Domains: ACI

Cross Domain Messaging (VN, SGT/EPG Group, Contract, … Exchange)

Cross Domain Messaging

Cisco DNA Center

Policy Automation Analytics

It is a little step with huge impact

App Groups App resource request

LA Campus SDA Fabric Dallas Campus SDA Fabric

Cisco SDWAN network New York

DCNM DCNM DCNM

Los Angeles AWS Region East

• 4 Validated Design Guides

 2. Enable End 2 End Automation and Analytics

 3. Enable Security End 2 End