Teccrs 3800

Campus Fabric
Technical Seminar
TECCRS-3800
Campus Fabric
Abstract
Is your Campus network facing some, or all, of these challenges?
• Host Mobility (w/o stretching VLANs)

• Network Segmentation (w/o implementing MPLS)
• Role-based Access Control (w/o end-to-end TrustSec)
Using Cisco technologies available today, you can overcome these challenges
and build an “evolved” Campus Network to better meet your business objectives.
Come to this session to get a deeper insight into the Key Technologies, Designs and
Configurations (e.g. LISP, VXLAN, and TrustSec) that brings this evolution to life!
TECCRS-3800 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Campus Fabric
Related Sessions
We recommend the following sessions:

1. BRKCRS-1800: DNA Campus Fabric – An Introduction
• 21/02/17 (Tuesday) @ 11:15 – 1.5 hours
2. BRKCRS-3800: DNA Campus Fabric – A Look Under the Hood

• 22/02/17 (Wednesday) @ 09:00– 2 hours
3. BRKCRS-2801: DNA Campus Fabric - How to Integrate with Your Existing Network
• 22/02/17 (Wednesday) @ 11:30 – 1.5 hours
4. BRKCRS-2802: DNA Campus Fabric – Monitoring & Troubleshooting

• 22/02/17 (Wednesday) @ 14:30 – 1.5 hours
5. BRKCRS-2803: DNA Campus Fabric – Connecting Outside the Fabric

• 22/02/17 (Wednesday) @ 16:30 – 1.5 hours
6. BRKACI-2400: DNA Campus Fabric – Integration with Data Center Architectures

• 23/02/17 (Thursday) @ 14:30 – 1.5 hours
7. BRKEWN-2300: Virtualize Your Wired and Wireless Network (w/ Campus Fabric)
• 24/02/17 (Friday) @ 09:00 – 2 hours
Campus Fabric Team
Who are we?
Shawn Sanjay Satish Karthik Vimarsh Kedar

Wargo Hooda Kondalam Thatikonda Puneet Karmarkar
Technical Development Technical Technical Technical Technical
Marketing Engineering Marketing Marketing Marketing Marketing
Campus Fabric
Agenda
09:00 – 09:30
1 09:30 – 11:00
30 min
90 min
Welcome and Introduction
Concepts and Fundamentals
11:00 – 11:15 15 min BREAK TIME 
11:15 – 12:45 90 min External Connectivity

2 12:45 – 13:15 30 min External Services
13:15 – 14:15 60 min LUNCH TIME 
14:15 – 15:15
3
60 min DC Integration
15:15 – 16:15 60 min Monitor and Troubleshoot
16:15 – 16:30 15 min BREAK TIME 
16:30 – 18:00 90 min Design and Migration

4 18:00 – 18:30 30 min Q&A and Take-Away
It’s a
Journey …
Agenda
1 Key Benefits
Why do I care?
2 Key Concepts
What is a Fabric?
3 Solution Overview
How does it work?
4 Putting It Together
Where do things go?
5 Take-Away
When to get started?
Key Benefits
Why do I care?
Cisco Digital Network Architecture
Overview
Network-enabled Applications
Cloud Service Management

Policy | Orchestration
Open APIs | Developers Environment Insights &

Experiences
Automation Analytics
Principles Automation
Abstraction & Policy Control Network Data,
from Core to Edge Contextual Insights & Assurance
Open & Programmable | Standards-Based

Security &
Virtualization Compliance
Physical & Virtual Infrastructure | App Hosting
Cloud-enabled | Software-delivered
What is Campus Fabric?
Foundational Technologies
Programmable Custom ASICs Converged Software Services
Industry Leading
Wired & Wireless | Stacking | TrustSec | SDN + Network Enabled Applications
Collaboration | Mobility | IoT | Security
`
Advanced Functionality Automation and Analytics
Programmable Pipeline | Flexibility | Encapsulation Controller | Visible | Programmable | Open
Optimized for Campus Virtualization

Integrated Stacking | Visibility | Security Campus Fabric | Segmentation | L2 Flexibility
Future Proofed Designed for Evolution

Long Life Cycle | Investment Protection Strong Foundational Capabilities | HA
Driving Innovation Through Technology Investment

Simplified Provisioning
Provision Deploy devices using “best practice” configurations
using Smart CLI and Programmability models
Wired and Wireless
Host Mobility
Always connect to the same L3 gateway
Mobility
X Simple Segmentation constructs
Segmentation Security to build Secure boundaries for “users and things”
Network Wide
Intelligent
Policy Enforcement
Policy Based on your identity, not on your address
Key Concepts
What is a Fabric?
What exactly is a Fabric?
A Fabric is an Overlay
An Overlay is a logical topology used to virtually connect devices, built
on top of an arbitrary physical Underlay topology.
An Overlay network often uses alternate forwarding attributes to provide
additional services, not provided by the Underlay.
Examples of Network Overlays

• GRE or mGRE • LISP
• MPLS or VPLS • OTV
• IPSec or DMVPN • DFA
• CAPWAP • ACI
Overlay Terminology
Overlay Network Overlay Control Plane
Encapsulation
Edge Device Edge Device
Hosts
(End-Points)
Underlay Network Underlay Control Plane
Why Overlays?
Separate the Forwarding Plane from the Services Plane
Reliable Transport Forwarding Flexible Virtual Services

• Physical Devices and Paths • Mobility – Track End-points at Edges
• Keep It Simple and Manageable • Scalability – Reduce core state
• Distribute state to network edge
• Intelligent Packet Handling
• Maximize Network Availability • Flexibility & Programmability
• Reduced number of touch points
What is unique about Campus Fabric?
Key Components
1. Control-Plane based on LISP

2. Data-Plane based on VXLAN
3. Policy-Plane based on TrustSec
Key Differences
• L2 + L3 Overlay -vs- L2 or L3 Only
• Host Mobility with Anycast Gateway
• Adds VRF + SGT into Data-Plane
• Virtual Tunnel Endpoints (No Static)
• No Topology Limitations (Basic IP)
Key Components – LISP
1. Control-Plane based on LISP Host

Mobility
Routing Protocols = Big Tables & More CPU LISP DB + Cache = Small Tables & Less CPU
with Local L3 Gateway with Anycast L3 Gateway
BEFORE AFTER
IP Address = Location + Identity Separate Identity from Location
Prefix RLOC
192.58.28.128 ….....171.68.228.121
….....171.68.226.120
Prefix Next-hop 189.16.17.89
22.78.190.64 ….....171.68.226.121
189.16.17.89 ….....171.68.226.120 172.16.19.90 ….....171.68.226.120
22.78.190.64 ….....171.68.226.121 192.58.28.128 ….....171.68.228.121
172.16.19.90 ….....171.68.226.120 192.58.28.128 ….....171.68.228.121
192.58.28.128
189.16.17.89
…....171.68.228.121
…....171.68.226.120
Prefix Next-hop 189.16.17.89
22.78.190.64
….....171.68.226.120
….....171.68.226.121
22.78.190.64
172.16.19.90
192.58.28.128
189.16.17.89
….....171.68.226.121
…......171.68.226.120
….....171.68.228.121
…....171.68.226.120 Endpoint
189.16.17.89
22.78.190.64
172.16.19.90
192.58.28.128
….....171.68.226.120
….....171.68.226.121
….....171.68.226.120
…....171.68.228.121
172.16.19.90
192.58.28.128
….....171.68.226.120
….....171.68.228.121 Mapping
22.78.190.64
172.16.19.90
….....171.68.226.121
…......171.68.226.120
Database
192.58.28.128
189.16.17.89
22.78.190.64
172.16.19.90
192.58.28.128
…......171.68.228.121
….....171.68.226.120
…......171.68.226.121
….....171.68.226.120
….....171.68.228.121
Routes are
Consolidated
Prefix
189.16.17.89
Next-hop
…......171.68.226.120
to LISP DB
22.78.190.64 ….....171.68.226.121
172.16.19.90 ….....171.68.226.120
192.58.28.128 …....171.68.228.121
189.16.17.89 …....171.68.226.120 Prefix Next-hop
22.78.190.64 ….....171.68.226.121 189.16.17.89 ….....171.68.226.120
172.16.19.90 …......171.68.226.120 22.78.190.64 ….....171.68.226.121
192.58.28.128 ….....171.68.228.121 172.16.19.90 ….....171.68.226.120
189.16.17.89 …....171.68.226.120 192.58.28.128 …....171.68.228.121
22.78.190.64 ….....171.68.226.121
172.16.19.90 …......171.68.226.120
192.58.28.128 ….....171.68.228.121
189.16.17.89 ….....171.68.226.120
22.78.190.64 …......171.68.226.121
172.16.19.90 ….....171.68.226.120
192.58.28.128 ….....171.68.228.121
Prefix Next-hop
Prefix Next-hop 189.16.17.89 ….....171.68.226.120
189.16.17.89 ….....171.68.226.120 22.78.190.64 ….....171.68.226.121
22.78.190.64 ….....171.68.226.121 172.16.19.90 ….....171.68.226.120
172.16.19.90 ….....171.68.226.120 192.58.28.128 …....171.68.228.121
192.58.28.128 …....171.68.228.121
189.16.17.89 …....171.68.226.120
22.78.190.64 ….....171.68.226.121
172.16.19.90 …......171.68.226.120
192.58.28.128 ….....171.68.228.121
Topology + Endpoint Routes

189.16.17.89
22.78.190.64
172.16.19.90
192.58.28.128
…....171.68.226.120
….....171.68.226.121
…......171.68.226.120
….....171.68.228.121
Only Local Routes
189.16.17.89
22.78.190.64
….....171.68.226.120
…......171.68.226.121
Topology Routes
172.16.19.90 ….....171.68.226.120
192.58.28.128 ….....171.68.228.121
Endpoint Routes
Key Components – VXLAN

ORIGINAL
ETHERNET IP PAYLOAD
PACKET
Supports L3
Overlay
PACKET IN
ETHERNET IP UDP LISP IP PAYLOAD
LISP
Supports L2
& L3 Overlay
PACKET IN
ETHERNET IP UDP VXLAN ETHERNET IP PAYLOAD
VXLAN
Key Components – CTS

Virtual Routing & Forwarding
Scalable Group Tagging
VRF + SGT
Fabric Roles & Terminology
User / Group Control-Plane

Repository Nodes
ISE / AD Host DB
 User / Group Repository – External
ID Store device (e.g. ISE or AD)
can be leveraged to provide dynamic
User / Device to Group mapping.
 Control-Plane Nodes – Map System
Fabric Domain Fabric Border that manages the Endpoint to Gateway
(Overlay) Nodes (Edge or Border) relationship.
 Border Nodes – The L3 Gateway

device (Core), that connects External
L3 network(s) to Fabric.
 Edge Nodes – The L3 Gateway
Fabric Edge Fabric Intermediate device (Access or Distribution), that
Nodes Nodes (Underlay) connects Endpoints to Fabric.
 Intermediate Nodes – Normal L3
(IP) Forwarders in the Underlay.
Campus Fabric
New Terminology
• “Control-Plane Node” ≈ “LISP Map-Server”
• “Edge Node” ≈ “LISP XTR + Endpoints”
• “Border Node” ≈ “LISP XTR + Subnets” or “PXTR”
• “Intermediate Node” ≈ “Non-LISP IP Forwarder”
Campus Fabric
Control-Plane Nodes – A Closer Look
Control-Plane Node runs a Host Tracking Database to map location information

Based on a LISP Map Server / Resolver
• A simple Host Database, that tracks Endpoint ID to

Device mappings, along with other attributes C
Known Unknown
Networks Networks
• Host Database supports multiple types of Endpoint ID

lookup keys (IPv4, IPv6 or MAC)
• Receives map registrations from Edge and Border

Nodes with “known” IP prefixes
• Resolves lookup requests from Edge and Border

Nodes, to locate destination Endpoint IDs
Campus Fabric
Edge Nodes – A Closer Look
Edge Node provides first-hop services for Users & Devices connected to the Fabric
Based on a LISP XTR + Dynamic Endpoint Mapping
• Responsible for Identifying and Authenticating Endpoints

(e.g. Static, 802.1X, Active Directory)
Known Unknown
Networks Networks
• Register the specific Endpoint ID info (e.g. /32 or /128)

with the Control-Plane Node(s)
• Provide Anycast L3 Gateway for all connected Endpoints

(same IP address on all Edge nodes)
• Performs encapsulation / de-encapsulation of data traffic

to and from all connected Endpoints
Campus Fabric
Border Nodes – A Closer Look
Border Node is an entry & exit point for all data traffic going in & out of the Fabric
There are 2 Types of Border Node!
• Internal Border based on XTR + Subnets

• “Known” Routes use Internal Border
Known Unknown
Networks Networks
B B
• External Border based on PXTR
• Unknown Routes use External Border
Campus Fabric
Internal Border advertises Endpoints to outside, and known Subnets to inside

Based on a LISP XTR + Subnet Mapping
• Connects to any “known” IP subnets attached to the

outside network (e.g. DC, WLC, FW, etc.) Known Unknown
Networks Networks
• Exports all internal IP Pools to outside (as aggregate), B B

using a traditional IP routing protocol(s).
• Imports and registers (known) IP subnets from outside,

into the LISP Map System
• Hand-off requires mapping the context (VRF & SGT)

from one domain to another.
Campus Fabric
External Border is a “Gateway of Last Resort” for unknown destinations

Based on a LISP PXTR
• Connects to any “unknown” IP subnets (e.g. Internet)

Known Unknown
• Exports all internal IP Pools outside (as aggregate) Networks Networks
into traditional IP routing protocol(s). B B
• LISP PXTR is a “default” exit point, if no specific entry

present in Map System.
• Hand-off requires mapping the context (VRF & SGT)

from one domain to another.
Campus Fabric
New Terminology
• “Fabric Domain” ≈ “FD” ≈ “LISP Process”
• “Virtual Network” ≈ “VN” ≈ “VRF” ≈ “LISP Instance”
• “Endpoint ID Group” ≈ “EIG” ≈ “Segment” ≈ “SGT”
• “Host Pool” ≈ “Dynamic EID” ≈ “VLAN + Address”
Campus Fabric
Virtual Network– A Closer Look
Virtual Network maintains a separate Routing & Switching instance for each VN
Based on Virtual Routing & Forwarding (VRF)
• LISP uses Instance ID to maintain independent VRF

topologies (“Default” VRF is Instance ID “0”) Known Unknown
Networks Networks
• LISP adds VNID to the LISP / VXLAN encapsulation
• Endpoint prefixes (Host Pools) are advertised within

one (or more) LISP Instance IDs VN VN VN
“A” “B” “C”
• Uses normal “vrf definition” configuration, along with
RD & RT for remote advertisement (Border Node)
Campus Fabric
Endpoint ID Groups – A Closer Look
Endpoint ID Group is a logical object to “group” Users and/or Devices

Based on a Scalable Group Tag (SGT)
• CTS uses “Endpoint ID Groups” to assign a unique

Scalable Group Tag (SGT) to Host Pools Known Unknown
Networks Networks
• LISP adds SGT to the LISP / VXLAN encapsulation
• CTS SGTs are used to manage address-independent EIG EIG EIG

“Group-Based Policies” 1 4 7
EIG EIG EIG EIG EIG EIG
2 3 5 6 8 9
• Individual Fabric Nodes use SGT to enforce local
Scalable Group ACLs (SGACLs)
Campus Fabric
Host Pools – A Closer Look
Host Pool provides basic IP functions necessary for attached Endpoints

Based on a VLAN ID + Interface Address (Anycast)
• Edge Nodes use a Switch Virtual Interface (SVI), with

IP Subnet, Gateway IP, etc. per Host Pool Known Unknown
Networks Networks
• LISP uses Dynamic EID mapping to advertise each

Host Pool (per Instance ID)
Pool Pool Pool
• LISP Dynamic EID allows Host-specific (/32, /128, 1 4 7
Pool Pool Pool Pool Pool Pool
MAC) advertisement and mobility 2 3 5 6 8 9
• Host Pools can be assigned Statically (per port) or

Dynamically (using Host Authentication)
Technical Overview
How does it work?
Locator / ID VXLAN
Separation Protocol Encapsulation
Cisco
TrustSec
Locator / ID Separation Protocol
Location and Identity Separation
Traditional Behavior -
Location + ID are “Combined”
IP core
When the Device moves, it gets a
10.1.0.1
new IPv4 or IPv6 Address for its new
Device IPv4 or IPv6 Identity and Location
Address represents both
20.2.0.9
Identity and Location
Overlay Behavior -
Location & ID are “Separated”
IP core
10.1.0.1 When the Device moves, it keeps
the same IPv4 or IPv6 Address.
Device IPv4 or IPv6 It has the Same Identity
Address represents 10.1.0.1
Identity only
Location Is Here Only the Location Changes
LISP Mapping System
LISP “Mapping System” is analogous to a DNS lookup

‒ DNS resolves IP Addresses for queried Name Answers the “WHO IS” question
[ Who is lisp.cisco.com ] ?
DNS
DNS Name -to- IP
Host Server URL Resolution
[ Address is 153.16.5.29, 2610:D0:110C:1::3 ]
‒ LISP resolves Locators for queried Identities Answers the “WHERE IS” question
[ Where is 2610:D0:110C:1::3 ] ?
LISP LISP Map
LISP
Router System ID -to- Locator
Map Resolution
[ Locator is 128.107.81.169, 128.107.81.170 ]
Locator / ID Separation Protocol Map System
LISP Roles & Responsibilities EID
a.a.a.0/24
b.b.b.0/24
RLOC
w.x.y.1
x.y.w.2
c.c.c.0/24 z.q.r.5
d.d.0.0/16 z.q.r.5
EID RLOC
EID Space a.a.a.0/24
b.b.b.0/24
w.x.y.1
x.y.w.2
Map Server / Resolver c.c.c.0/24

d.d.0.0/16
z.q.r.5
z.q.r.5
EID RLOC
• EID to RLOC Mappings ITR a.a.a.0/24
b.b.b.0/24
c.c.c.0/24
w.x.y.1
x.y.w.2
z.q.r.5
d.d.0.0/16 z.q.r.5
Non-LISP
• Can be distributed across Prefix Next-hop
w.x.y.1 e.f.g.h
x.y.w.2 e.f.g.h
multiple LISP devices z.q.r.5

z.q.r.5
e.f.g.h
e.f.g.h
PXTR RLOC Space

Tunnel Router - XTR
• Edge Devices Encap / Decap
ETR
• Ingress / Egress (ITR / ETR)
Proxy Tunnel Router - PXTR EID Space
• Connects between LISP • EID = End-point Identifier

and non-LISP domains • Host Address or Subnet
• Ingress / Egress (PITR / PETR) • RLOC = Routing Locator
• Local Router Address
Map Register & Resolution
Branch
Mapping Cache Entry (on ITR) IT

10.2.0.0/16  (2.1.1.1, 2.1.2.1) R Map Server / Resolver
5.1.1.1
Map-Reply
10.2.0.0/16  (2.1.1.1, 2.1.2.1)
2.1.1.1 2.1.2.1 3.1.1.1 3.1.2.1
Database Mapping Entry (on ETR) ETR ETR ETR ETR Database Mapping Entry (on ETR)
10.2.0.0/16  (2.1.1.1, 2.1.2.1) 10.3.0.0/16  (3.1.1.1, 3.1.2.1)
10.2.0.0 /16 10.3.0.0/16
Campus DC
How does LISP operate?
3 EID-prefix: 10.2.0.0/24
Mapping Locator-set:
Entry 2.1.1.1, priority: 1, weight: 50 (D1) Path Preference
1 Controlled
DNS Entry: 2.1.2.1, priority: 1, weight: 50 (D2)
Non-LISP Non-LISP by Destination Site
D.abc.com A 10.2.0.1
10.1.0.0/24
Branch PXTR
S ITR
2 1.1.1.1
10.1.0.1  10.2.0.1 5.3.3.3
IP Network 5.1.1.1 5.2.2.2

4 Mapping
System
1.1.1.1  2.1.1.1
10.1.0.1  10.2.0.1
2.1.1.1 ETR 2.1.2.1 3.1.1.1 ETR 3.1.2.1
5
10.1.0.1  10.2.0.1
D
Campus 10.2.0.0/24 10.3.0.0/24
DC
Forwarding from outside a LISP Domain 3 EID-Prefix: 10.2.0.0/24
Mapping Locator-Set:
1 Entry 2.1.1.1, priority: 1, weight: 50 (D1)

DNS Entry:
2.1.2.1, priority: 1, weight: 50 (D2)
D.abc.com A 10.2.0.1
Non-LISP
S
2
PXTR
192.3.0.1  10.2.0.1 4.4.4.4
4 5.3.3.3
4.4.4.4  2.1.2.1 IP Network 5.1.1.1 5.2.2.2

Mapping
192.3.0.1  10.2.0.1 System
2.1.1.1 ETR 2.1.2.1 3.1.1.1 ETR 3.1.2.1
5
192.3.0.1  10.2.0.1
D
Campus 10.2.0.0/24 10.3.0.0/24
DC
Host Mobility – Dynamic EID Migration
D
2.1.1.1
DC1
10.10.10.0/24 1.1.1.1 3.1.1.1
Mapping
System
12.0.0.1 xTR 12.0.0.2
IP Network
12.1.1.1 xTR 12.1.1.2 12.2.2.1 xTR 12.2.2.2
S
10.17.1.0/24 10.18.1.0/24
Campus Bldg 1 Campus Bldg 2
10.17.1.10
D
2.1.1.1
DC1
10.10.10.0/24 1.1.1.1 3.1.1.1
Mapping
System
12.0.0.1 xTR 12.0.0.2
IP Network
12.1.1.1 xTR 12.1.1.2 12.2.2.1 xTR 12.2.2.2
S
10.17.1.0/24 10.18.1.0/24
D
2.1.1.1
DC1
10.10.10.0/24 1.1.1.1 3.1.1.1
Mapping
System
Routing Table 12.0.0.1 xTR 12.0.0.2
10.17.1.0/24 – Local
10.17.1.10/32 – Local
IP Network
12.1.1.1 xTR 12.1.1.2 12.2.2.1 xTR 12.2.2.2
S
10.17.1.0/24 10.18.1.0/24
Mapping Database
Locator / ID Separation Protocol 10.10.0.0/16 – 12.0.0.1
10.17.0.0/16 – 12.1.1.1
Host Mobility – Dynamic EID Migration 10.18.0.0/16 – 12.2.2.1
Map Register
EID: 10.17.1.10/32 10.17.1.10/32 – 12.1.1.1
RLOC: 12.1.1.1
D
2.1.1.1
DC1
10.10.10.0/24 1.1.1.1 3.1.1.1
Mapping
System
10.17.1.0/24 – Local
10.17.1.10/32 – Local
IP Network
12.1.1.1 xTR 12.1.1.2 12.2.2.1 xTR 12.2.2.2
S
10.17.1.0/24 10.18.1.0/24
Mapping Database
10.17.0.0/16 – 12.1.1.1
10.17.1.10/32 – 12.1.1.1
D
2.1.1.1
DC1
10.10.10.0/24 1.1.1.1 3.1.1.1
Mapping
System
10.17.1.0/24 – Local
10.17.1.10/32 – Local
IP Network
12.1.1.1 xTR 12.1.1.2 12.2.2.1 xTR 12.2.2.2
S
10.17.1.0/24 10.18.1.0/24
Mapping Database
10.17.0.0/16 – 12.1.1.1
10.17.1.10/32 – 12.1.1.1
D
2.1.1.1
DC1
10.10.10.0/24 1.1.1.1 3.1.1.1
Mapping
System
10.17.1.0/24 – Local Routing Table

10.17.1.10/32 – Local 10.17.1.0/24 – LISP0
10.18.0.0/24 – Local
IP Network
12.1.1.1 xTR 12.1.1.2 12.2.2.1 xTR 12.2.2.2
S 1
10.17.1.0/24 10.18.1.0/24
10.17.1.10
Mapping Database
10.17.0.0/16 – 12.1.1.1
10.17.1.10/32 – 12.1.1.1
D
2.1.1.1
DC1
10.10.10.0/24 1.1.1.1 3.1.1.1
Mapping
System
10.17.1.0/24 – Local Routing Table 2

10.17.1.10/32 – Local 10.17.1.0/24 – LISP0
10.18.0.0/24 – Local
10.17.1.10/32 - Local
IP Network
12.1.1.1 xTR 12.1.1.2 12.2.2.1 xTR 12.2.2.2
S
10.17.1.0/24 10.18.1.0/24
10.17.1.10
Mapping Database
10.17.0.0/16 – 12.1.1.1
10.17.1.10/32 – 12.2.2.1
D
2.1.1.1
DC1
10.10.10.0/24 1.1.1.1 3.1.1.1
Mapping
System
10.17.1.0/24 – Local 3 Routing Table 2

10.17.1.10/32 – Local 10.17.1.0/24 – LISP0
10.18.0.0/24 – Local
10.17.1.10/32 - Local
IP Network
12.1.1.1 xTR 12.1.1.2 12.2.2.1 xTR 12.2.2.2
S
10.17.1.0/24 10.18.1.0/24
10.17.1.10
Mapping Database
10.17.0.0/16 – 12.1.1.1
10.17.1.10/32 – 12.2.2.1
D
2.1.1.1
DC1
10.10.10.0/24 1.1.1.1 3.1.1.1
Mapping
System

5 Routing Table
10.17.1.0/24 – Local 2
4
10.17.1.0/24 – LISP0
10.18.0.0/24 – Local
10.17.1.10/32 – LISP0
10.17.1.10/32 - Local
IP Network
12.1.1.1 xTR 12.1.1.2 12.2.2.1 xTR 12.2.2.2
S
10.17.1.0/24 10.18.1.0/24
10.17.1.10
Mapping Database
10.17.0.0/16 – 12.1.1.1
10.17.1.10/32 – 12.2.2.1
D
2.1.1.1
DC1
10.10.10.0/24 1.1.1.1 3.1.1.1
Mapping
System

5 Routing Table
10.17.1.0/24 – Local 2
4
10.17.1.0/24 – LISP0
10.18.0.0/24 – Local
10.17.1.10/32 – LISP0
10.17.1.10/32 - Local
IP Network
12.1.1.1 xTR 12.1.1.2 12.2.2.1 xTR 12.2.2.2
S
10.17.1.0/24 10.18.1.0/24
10.17.1.10
Mapping Database
10.17.0.0/16 – 12.1.1.1
10.17.1.10/32 – 12.2.2.1
D
2.1.1.1
DC1
10.10.10.0/24 1.1.1.1 3.1.1.1
Mapping
System

5 Routing Table
10.17.1.0/24 – Local 2
4
10.17.1.0/24 – LISP0
10.18.0.0/24 – Local
10.17.1.10/32 – LISP0
10.17.1.10/32 - Local
IP Network
12.1.1.1 xTR 12.1.1.2 12.2.2.1 xTR 12.2.2.2
S
10.17.1.0/24 10.18.1.0/24
10.17.1.10
Locator / ID Separation Protocol (LISP)
Would you like to know more?
Suggested Reading:
BRKRST-3045 - LISP - A Next Generation Networking Architecture
BRKRST-3047 - Troubleshooting LISP
BRKCRS-3510 - LISP in Campus Networks
Other References:
Cisco LISP Site http://lisp.cisco.com
Cisco LISP Marketing Site http://www.cisco.com/go/lisp/
LISP Beta Network Site http://www.lisp4.net or http://www.lisp6.net
IETF LISP Working Group http://tools.ietf.org/wg/lisp/
Fundamentals of LISP https://www.youtube.com/watch?v=lKrV1qB8uqA
Technical Overview
How does it work?
Locator / ID VXLAN
Cisco
TrustSec
Cisco TrustSec
Traditional segmentation is extremely complex
Applications
Enforcement
access-list
access-list
102
102
deny udp 167.160.188.162 0.0.0.255 gt 4230 248.11.187.246 0.255.255.255 eq 2165
deny udp 32.124.217.1 255.255.255.255 lt 907 11.38.130.82 0.0.31.255 gt 428
IP Based Policies -
access-list
access-list
102
102
permit ip 64.98.77.248 0.0.0.127 eq 639 122.201.132.164 0.0.31.255 gt 1511
deny tcp 247.54.117.116 0.0.0.127 gt 4437 136.68.158.104 0.0.1.255 gt 1945
ACLs, Firewall Rules
access-list 102 permit icmp 136.196.101.101 0.0.0.255 lt 2361 90.186.112.213 0.0.31.255 eq 116
access-list 102 deny udp 242.4.189.142 0.0.1.255 eq 1112 19.94.101.166 0.0.0.127 eq 959
access-list 102 deny tcp 82.1.221.1 255.255.255.255 eq 2587 174.222.14.125 0.0.31.255 lt 4993
access-list 102 deny tcp 103.10.93.140 255.255.255.255 eq 970 71.103.141.91 0.0.0.127 lt 848
access-list 102 deny ip 32.15.78.227 0.0.0.127 eq 1493 72.92.200.157 0.0.0.255 gt 4878
access-list 102 permit icmp 100.211.144.227 0.0.1.255 lt 4962 94.127.214.49 0.255.255.255 eq 1216 Propagation
Carry “Segment”
access-list 102 deny icmp 88.91.79.30 0.0.0.255 gt 26 207.4.250.132 0.0.1.255 gt 1111
access-list 102 deny ip 167.17.174.35 0.0.1.255 eq 3914 140.119.154.142 255.255.255.255 eq 4175
access-list 102 permit tcp 37.85.170.24 0.0.0.127 lt 3146 77.26.232.98 0.0.0.127 gt 1462 Enterprise
access-list 102 permit tcp 155.237.22.232 0.0.0.127 gt 1843 239.16.35.19 0.0.1.255 lt 4384
Backbone context through the
network using VLAN,
Aggregation Layer IP address, VRF
Limits of Traditional
Static ACL VACL
Routing Segmentation
Access Layer Classification
Redundancy • Security Policy based Static or Dynamic
DHCP Scope on Topology (Address) VLAN assignments
Address • High cost and
VLAN complex maintenance Non-Compliant Voice Employee Supplier BYOD
Quarantine Voice Data Guest BYOD

VLAN VLAN VLAN VLAN VLAN
Cisco TrustSec
Simplified segmentation with Group Based Policy
Enforcement Shared Application

Group Based Policies Services Servers
Enforcement DC Switch
or Firewall
Propagation
Carry “Group” context
through the network Enterprise
using only SGT Backbone
ISE
Classification
Static or Dynamic Campus Switch Campus Switch DC switch receives policy
for only what is connected
SGT assignments
Employee Tag
Supplier Tag
Non-Compliant Employee Voice Voice Employee Supplier Non-Compliant Non-Compliant Tag
VLAN A VLAN B
Cisco Trust Security
Identity Services Engine enables CTS
NDAC
Network Device
Admission Control
NDAC authenticates
Network Devices for a
Scalable Group ACL Cisco ISE Scalable Group Tags
trusted CTS domain
Destinations SGACL - SGT & 3: Employee
✕✓✕✓✓✓ Name Table SGT Names
Sources
4: Contractors
SGT & SGT Names
Centrally defined ✓✓✕✓✕✕ 8: PCI_Servers
Endpoint ID Groups ✕✓✓✕✕✕ 9: App_Servers
SGACL - Name Table

Policy matrix to be
pushed down to the
network devices
ISE dynamically Rogue

authenticates endpoint Device(s) 802.1X Dynamic SGT Static SGT
Assignment Assignment
users and devices,
and assigns SGTs
Two ways to assign SGT
Dynamic Classification Static Classification
L3 Interface (SVI) to SGT L2 Port to SGT
Campus
Access Distribution Core DC Core DC Access
MAB Enterprise
Backbone
WLC Firewall Hypervisor SW
VLAN to SGT Subnet to SGT VM (Port Profile) to SGT
Ingress Classification with Egress Enforcement
Destination Classification
CRM: SGT 20
Web: SGT 30
User Authenticated = FIB Lookup =
Classified as Marketing (5) Destination MAC = SGT 20
Cat3850 Cat6800 Cat6800 Nexus 7000 Nexus 5500 Nexus 2248

CRM
Enterprise DST: 10.1.100.52
5 Backbone 5
SRC: 10.1.10.220 SGT: 20
DST: 10.1.100.52
SRC: 10.1.10.220 SGT: 5 Web
DST: 10.1.200.100
Egress SGT: 30
Enforcement
(SGACL)
WLC5508
DST  CRM Web
 SRC (20) (30)
Marketing (5) Permit Deny
BYOD (7) Deny Permit
SGT Propagation & Enforcement Options
SXP SXP
Heterogeneous WAN
L2 / L3 Networks
User Switch Switch Router Router Firewall DC Switch Server
Classification SGFW Classification
SGT over Fabric SGT over SGT over Fabric

VPN
TrustSec Capable WAN

(GETVPN, DMVPN
L2 / L3 Networks
User Switch Switch Router Router Firewall DC Switch Server
Classification SGACL SGACL SGACL SGFW SGACL Classification
Cisco Trust Security (CTS)
Suggested Reading:
BRKCRS-2891 - Enterprise Network Segmentation with Cisco TrustSec
BRKSEC-2203 - Intermediate - Enabling TrustSec Software-Defined Segmentation
BRKSEC-2044 - Building an Enterprise Access Control Architecture Using ISE and TrustSec
Other References:
Cisco TrustSec Marketing Site http://www.cisco.com/go/trustsec/
Cisco TrustSec Config Guide cisco.com/c/en/us/td/docs/switches/lan/trustsec/configuration/guide/trustsec.html
Cisco TrustSec Architecture cisco.com/docs/switches/lan/trustsec/configuration/guide/trustsec/arch_over.html
Cisco TrustSec Design Guide cisco.com/td/docs/solutions/Enterprise/Security/TrustSec_2-0/trustsec_2-0_dig.pdf
Fundamentals of TrustSec https://www.youtube.com/watch?v=78-GV7Pz18I
Technical Overview
How does it work?
Locator / ID VXLAN
Cisco
TrustSec
Data-Plane Overview
Fabric Header Encapsulation
Inner
Fabric Data-Plane provides the following:
• Underlay address advertisement & mapping
Outer
• Automatic tunnel setup (Virtual Tunnel End-Points)
• Frame encapsulation between Routing Locators
Support for LISP or VXLAN header format Decap
Outer
• Nearly the same, with different fields & payload
Inner
Inner
• LISP header carries IP payload (IP in IP)
• VXLAN header carries MAC payload (MAC in IP)
Encap
Triggered by LISP Control-Plane events
• ARP or NDP Learning on L3 Gateways
• Map-Reply or Cache on Routing Locators
Inner
LISP & VXLAN Headers
Similar Format - Different Payload
LISP Header - IP based VXLAN Header - Ethernet based
OUTER
HEADER
4789
OVERLAY
HEADER
INNER
HEADER
VXLAN Header
Next-Hop MAC Address
Src VTEP MAC Address

Dest. MAC 48
MAC-in-IP Encapsulation
Source MAC 48
VLAN Type 14 Bytes

16 IP Header
0x8100 (4 Bytes Optional) 72
Misc. Data
VLAN ID 16
Protocol 0x11 (UDP) 8
Ether Type
16 Header
0x0800 16 20 Bytes
Outer MAC Header
Underlay
Checksum
Source IP 32
Src RLOC IP Address
Outer IP Header Dest. IP 32
Source Port 16 Dst RLOC IP Address
UDP Header Dest Port 16

8 Bytes Hash of inner L2/L3/L4 headers of original frame.
UDP Length 16 Enables entropy for ECMP load balancing.
VXLAN Header
Checksum 0x0000 16 UDP 4789
Inner (Original) MAC Header

Allows 64K
VXLAN Flags possible SGTs
Inner (Original) IP Header RRRRIRRR
8
Overlay
Segment ID 16
8 Bytes
Original Payload VN ID 24
Allows 16M
Reserved 8 possible VRFs
Putting It Together
Where do things go?
Platform Support
Fabric Edge Nodes - Options
Catalyst 3K Catalyst 4K
• Catalyst 3650 • Catalyst 4500

• Catalyst 3850 • Sup8E (Uplinks)
• 1G/MGIG (Copper) • 4700 Cards
• IOS-XE 16.3.1+ • IOS-XE 3.9.1+
Platform Support
Fabric Border Nodes - Options
Catalyst 3K Catalyst 6K ASR1K & ISR4K Nexus 7K
• Catalyst 3850 • Catalyst 6800 • ASR1000-X • Nexus 7700

• 12/24 or 48XS • Sup2T or 6T • X or HX Series • Sup2E
• 1/10G (Fiber) • 6880 or 6840-X • ISR4430 / 4450 • M3 Cards
• IOS-XE 16.3.1+ • IOS 15.4.1SY+ • IOS-XE 16.4.1+ • NXOS 7.3.2+
Platform Support
Fabric Control-Plane - Options
Catalyst 3K Catalyst 6K ASR1K & ISR4K
• Catalyst 3850 • Catalyst 6800 • ASR1000-X

• 12/24 or 48XS • Sup2T or 6T • X or HX Series
• 1/10G (Fiber) • 6880 or 6840-X • ISR4430 / 4450
• IOS-XE 16.3.1+ • IOS 15.4.1SY+ • IOS-XE 16.4.1+
Campus Fabric Config Control-Plane Node
Control-Plane Nodes
5.1.1.1/32 C
IP Network
10.1.1.0/24 20.1.1.0/24
Host Pool 10 Edge Node 1 Edge Node 2 Host Pool 20
router lisp
• Organize networks into LISP Site
ipv4-interface Loopback0
• Configure the Authentication Key site San_Jose
authentication-key S3cr3t
• Add the IP prefixes to be mapped eid-prefix 10.1.1.0/24 accept-more-specifics
• accept more-specific updates (e.g. /32) eid-prefix 20.1.1.0/24 accept-more-specifics
exit
• Operate as IPv4 Map-Server !
ipv4 map-server
• Operate as IPv4 Map-Resolver ipv4 map-resolver
exit
Edge Nodes (1)
5.1.1.1/32 C
10.1.1.1/24 1.1.1.1/32
IP Network
10.1.1.0/24 20.1.1.0/24
• Organize XTRs into a Locator Set router lisp

locator-set campus_fabric
• Enable VXLAN encapsulation ipv4-interface Loopback0
encapsulation vxlan
• Add a Dynamic EID Group !
eid-table default instance-id 0
• associate with an Instance ID (VRF)
dynamic-eid Default_10_1_1_0
• Add a Dynamic EID mapping database-mapping 10.1.1.0/24 locator-set campus_fabric
exit
• advertise the specified IP/mask !
ipv4 sgt
• Add IPv4 SGT (to LISP/VXLAN) ipv4 itr map-resolver 5.1.1.1
ipv4 itr
• Operate as an IPv4 ITR and ETR ipv4 etr map-server 5.1.1.1 key S3cr3t
ipv4 etr
• Designate Map-Server and Resolver
Edge Nodes (2)
5.1.1.1/32 C
2.1.1.1/32 20.1.1.1/24
IP Network
10.1.1.0/24 20.1.1.0/24
• Process Repeated on each XTR router lisp

locator-set campus_fabric
ipv4-interface Loopback0
• Configure any Local (Static) EIDs encapsulation vxlan
!
• Or you can simply Copy + Paste for eid-table default instance-id 0
dynamic-eid Default_20_1_1_0
all XTRs using same Dynamic EID database-mapping 20.1.1.0/24 locator-set campus_fabric
exit
• Host Pools that exist on all XTRs !
ipv4 sgt
• Uses LISP Dynamic EID mobility ipv4 itr map-resolver 5.1.1.1
ipv4 itr
ipv4 etr map-server 5.1.1.1 key S3cr3t
• Refer to Host Pools for Anycast details ipv4 etr
Virtual Networks (VRFs)
C
10.1.1.0/24 20.1.1.0/24
10.1.1.0/24 IP Network 20.1.1.0/24
Edge Node 1 Edge Node 2

10.1.1.0/24 20.1.1.0/24
ip vrf RED
• Create new VRF definition ip vrf BLUE
ip vrf GREEN
• add RD/RT info as necessary !
router lisp
• Enable VXLAN encapsulation locator-set campus_fabric
encapsulation vxlan
• Create a new LISP Instance ID !
eid-table vrf RED instance-id 10
dynamic-eid RED_20_1_1_0
• Add Dynamic EID mappings database-mapping 20.1.1.0/24 locator-set campus_fabric
!
• associate with VRF & Instance ID eid-table vrf BLUE instance-id 11
dynamic-eid BLUE_20_1_1_0
• Add local prefixes to Dynamic EID database-mapping 20.1.1.0/24 locator-set campus_fabric
!
• non-overlapping can be routed natively eid-table vrf GREEN instance-id 12
dynamic-eid GREEN_20_1_1_0
• overlapping prefixes require NAT/FW * database-mapping 20.1.1.0/24 locator-set campus_fabric
exit
Border Nodes - Internal
5.1.1.1/32 C
3.1.1.1/32 172.1.1.1/24
B
IP Network
10.1.1.0/24 BGP 172.0.0.0/8
Host Pool 10 Edge Node 1 Border Node External IP
router lisp
• Operate as an IPv4 ITR and ETR locator-table default
!
• Enable Export of inside prefixes eid-table vrf USER instance-id 10
ipv4 route-import database bgp 65004 locator-set campus_fabric
• from LISP to external Protocol (e.g. BGP) ipv4 route-export site-registrations
ipv4 map-cache site-registrations
• set a map-cache entry for internal registrations ipv4 distance site-registrations 250
exit
• set the LISP AD to 250 (> Protocol routes) !
ipv4 itr
• Configure External Routing ipv4 etr
!
• Enable Import of outside prefixes router bgp 65004
!
• from external Protocol(s) into LISP address-family ipv4 vrf USER
redistribue lisp metric 10
• * Repeat per VRF (AF) aggregate-address 10.1.1.0 255.255.255.0 summary-only
exit-address-family
Border Nodes - External
5.1.1.1/32 C
3.1.1.1/32 172.1.1.1/24
B
IP Network
10.1.1.0/24 BGP 0.0.0.0/0
Host Pool 10 Edge Node 1 Border Node Internet
router lisp
• Operate as an IPv4 PITR and PETR locator-table default
!
• Same configuration as Internal Border, eid-table vrf USER instance-id 10
ipv4 route-export site-registrations
but EXPORT ONLY! ipv4 map-cache site-registrations
ipv4 distance site-registrations 250
• Used for Stub Routing and/or Internet exit
!
• Gateway of Last Resort ipv4 proxy-etr
ipv4 proxy-itr 3.1.1.1
!
router bgp 65004
!
address-family ipv4 vrf USER
redistribue lisp metric 10
aggregate-address 10.1.1.0 255.255.255.0 summary-only
exit-address-family
Campus Fabric Config Identity
Endpoint ID Groups – Dynamic SGT Services Engine
172.26.204.150
IP Network
10.1.1.0/24 20.1.1.0/24
• Enable the AAA new-model aaa new-model

!
aaa group server radius ISE
• Create a RADIUS server group, server name ISE
with one or more RADIUS server(s) !
radius server ISE
• Enable AAA dynamic-author address ipv4 172.26.204.150 auth-port 1812 acct-port 1813
key cisco
• Enable AAA authorization to use !
aaa server radius dynamic-author
CTS authorization client 172.26.204.150 server-key cisco
!
• Enable CTS Role-Based Enforcement aaa authentication dot1x default group ISE
aaa accounting dot1x default start-stop group ISE
aaa authorization network cts-list group ISE
!
cts authorization list cts-list
cts role-based enforcement
Campus Fabric Config
Endpoint ID Groups – Static SGT
20.1.1.1/24
IP Network
10.1.1.0/24 20.1.1.0/24
• Enable CTS Role-Based Enforcement !

• Define the list of VLANs to be enabled cts role-based enforcement vlan-list 1-4094
for Role-Based Enforcement !
cts role-based sgt-map vlan-list 20 sgt 20
• Create a new Static SGT-MAP for: !
! cts role-based sgt-map 20.1.1.0/24 sgt 20
• VLAN List to SGT tag !
• Or a new Static SGT-MAP for:

• Subnet/mask to SGT tag
Campus Fabric Config Identity
Host Pools – Dynamic Assignment Services Engine
172.26.204.150
20.1.1.1/24
IP Network
10.1.1.0/24 20.1.1.0/24
• Create a Host VLAN vlan 20

name Host_Pool_20
• Create a L3 VLAN Interface (SVI) !
with Subnet IP address /mask interface Vlan20
ip address 20.1.1.1 255.255.255.0
• Add LISP mobility (Dynamic EID group) lisp mobility Default_20_1_1_0
!
• Configure AAA order + priority on Port interface GigabitEthernet1/0/1
switchport
• Configure 802.1X and/or MAB on Port switchport mode access
authentication order dot1x mab
authentication priority dot1x mab
NOTE: Host will be dynamically assigned authentication port-control auto
mab
to the VLAN (e.g. 20) after Authentication dot1x pae authenticator
spanning-tree portfast
Host Pools – Static Assignment
20.1.1.1/24
IP Network
10.1.1.0/24 20.1.1.0/24
• Create a Host VLAN vlan 20

name Host_Pool_20
• Create a L3 VLAN Interface (SVI) !
interface Vlan20
with Subnet IP address /mask ip address 20.1.1.1 255.255.255.0
lisp mobility Default_20_1_1_0
• Add LISP mobility (Dynamic EID group) !
interface GigabitEthernet1/0/1
• Configure the (static) VLAN number on switchport
each Port switchport mode access
switchport access vlan 20
spanning-tree portfast
Campus Fabric - Smart CLI
Provisioning & Troubleshooting Made Simple
What is Smart CLI?

• Its a new configuration mode to simplify config
and management of Campus Fabric
• Invoked by a new Global command “fabric auto”

fabric_device(config)# fabric auto
• Provides a simple set of easy-to-understand CLI
• Auto-generates all of the equivalent (traditional)

LISP, VRF, IP, CTS, etc. CLI commands
Smart CLI – Example
Adding a new Edge Node
 Generate all LISP XTR baseline configs

 Set up Loopback0 as locator address
 Creates default neighborhood as instance ID 0
 Enables VXLAN encapsulation
 Adds SGT to VXLAN encapsulation
Edge(config)# fabric auto

Edge(config-fabric-auto)# domain default
Edge(config-fabric-auto-domain)# control-plane 2.2.2.2 auth-key key1
Edge(config-fabric-auto-domain)# border 4.4.4.4
Edge(config-fabric-auto-domain)# exit
Smart CLI – Example
Show Fabric Domain
Edge# show fabric domain

Fabric Domain : "default"
Role : Edge
Control-Plane Service: Disabled
Border Service: Disabled
Number of Control-Plane Nodes: 1

IP Address Auth-key
---------------------------------
2.2.2.2 key1
Number of Border Nodes: 1

IP Address
---------------------------------  Shows current domain (default)
4.4.4.4
 Shows current Role(s)
Number of Neighborhood(s): 4  Shows Control-Plane Node(s)
Name ID Host-pools
---------------------------------------------  Shows Border Node(s)
default
guest
0
50
2
1
 Shows Neighborhood(s)
pcie 60 1  Associated Host Pool(s)
cisco 70 *
Campus Fabric - Smart CLI
Provisioning & Troubleshooting Made Simple
More to Come! 
• Underlay Network – Configure the Interfaces
and Protocols to bring up the Underlay network
• Endpoint ID Groups – Configure the AAA and

CTS commands for Static & Dynamic ID
fabric_device(config)# fabric auto
• Group Based Policy – Configure SGT and

SGACL policies
• And More…
External Connectivity
Agenda
1 Border Functionality
• Internal Border
• External Border
• Platform Support
2 Border Design
• Collocated Border + C-Plane
• Separated Border / C-Plane
• One Box vs. Two Box
• Border Resiliency (HA)
3 Border Deployment
• Shared, WAN, DC @ Internal Border
• Service Chaining @ Internal Border
• Internet Connect @ External Border
Border Functionality
How does the Border work?
Campus Fabric
Border Nodes – Internal and External
Internal Border: External Border:

• Connects Campus Fabric to Known • Connects Campus Fabric to Un-
networks i.e. other fabric or non- Known networks.
fabric domain in same company
network . • These Un-known networks generally
is the Internet and Cloud.
• These known networks generally
are the WAN, DC, Shared Services • Responsible for only advertising
etc prefixes from the local fabric domain
to external domain.
• Responsible for advertising prefixes
from and to the local fabric domain
and external domain.
Internal Border
Campus Fabric
Border Nodes – Internal Border
Fabric Internal Border Node is based on a LISP Tunnel Router + IP
Subnet’s
All traffic entering or leaving the Fabric from and to a known destination goes
through this type of node
• Connects the DC ,WAN and any other known
network’s to the local fabric domain.
• Where two domains exchange Endpoint reachability B B

and policy (VRF & SGT) information
• Responsible for advertising prefixes from and to the

local fabric domain.
• Provides a domain exit point for all Edge Nodes
Campus Fabric
Border Nodes- Internal Border
Data center
Data Center
B B Border
WAN
WAN
Border
Border Nodes- Internal Border Co-located with Control Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24 BGP 192.1.1.0/24
Host Pool 10 Edge Node 1 Border and External Domain(

Control Plane DC,WAN)
Node
• Operate as an IPv4 ITR & ETR + IP router lisp
locator-table default
Subnets locator-set border
IPv4-interface Loopback0 priority 10 weight 10
• The EID prefixes are exported from !
Control plane node to the Internal eid-table vrf USER instance-id 10
Border node with AD of “250” ipv4 distance site-registrations 250
exit
!
ipv4 etr
ipv4 itr
!
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Node
• The Internal Border node advertises the router lisp
EID prefix into external protocol of locator-set border
choice(eBGP). IPv4-interface Loopback0 priority 10 weight 10
!
• The advertisement is summarized so router bgp 65004
!
that /32 host routes are not exposed to address-family ipv4 vrf USER
the external domain. redistribue LISP metric 10
exit-address-family
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Node
• The Internal Border also imports the router lisp
external prefixes into the Campus Fabric locator-set border
LISP domain. IPv4-interface Loopback0 priority 10 weight 10
!
eid-table vrf USER instance-id 10
ipv4 route-import database bgp 65004 locator-set rlocs
exit
!
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Node
• Add a Map Cache + Map-Request for router lisp

Dynamic EIDs locator-set border
• trigger a lookup for traffic coming from outside IPv4-interface Loopback0 priority 10 weight 10
!
ipv4 map-cache site-registration
exit
Campus Fabric
Border Nodes- Forwarding on Internal Border ( Fabric to External Domain)
3 EID-prefix: 192.1.1.0/24
Path Preference
Mapping Locator-set: Controlled
Entry 2.1.1.1, priority: 1, weight: 100 (D1) by Destination Site
192.1.1.0/24
Branch
Internal
D Border
5 2.1.1.1
10.1.1.1  192.1.1.1 5.3.3.3
IP Network 5.1.1.1 5.2.2.2

4 Mapping
System
2.1.1.1  1.1.1.1
10.1.1.1  192.1.1.1
1.1.1.1 XTR 1.1.2.1 1.1.3.1 XTR 1.1.4.1
2
10.1.1.1  192.1.1.1
1 S
DNS Entry: Campus DC
10.1.1.0/24 10.3.0.0/24
D.abc.com A 192.1.1.1
Campus Fabric
Border Nodes- Forwarding on Internal Border(External to fabric Domain)
3 EID-prefix: 10.1.1.1/32
Mapping Locator-set:
Entry 1.1.1.1, priority: 1, weight: 50 (D1) Path Preference
1 Controlled
Routing Entry: 1.1.2.1, priority: 1, weight: 50 (D2)
by Destination Site
Send traffic to exit point of
192.1.1.0/24
domain(Internal Border)
Branch
Internal
S Border
2 2.1.1.1
192.1.1.1  10.1.1.1 5.3.3.3
IP Network 5.1.1.1 5.2.2.2

4 Mapping
System
2.1.1.1  1.1.1.1
192.1.1.1  10.1.1.1
1.1.1.1 XTR 1.1.2.1 1.1.3.1 XTR 1.1.4.1
5
192.1.1.1  10.1.1.1
D
Campus 10.1.1.0/24 10.3.0.0/24
DC
External Border
Campus Fabric
Border Nodes – External Border
Fabric External Border Node is based on a LISP Proxy Tunnel
Router
All traffic leaving the Fabric to a un-known destination goes through this type of
node
• Connects the Internet, Cloud and any other Un-
known network’s to the local fabric domain.

• Responsible only for advertising prefixes from the

• Provides a default domain exit point for all Edge

Nodes
Campus Fabric
Border Nodes- External Border
Cloud
Cloud
Border
B B
Internet
Internet
Border
Border Nodes-External Border Co-located with Control Plane Node
10.1.1.1/24 1.1.1.1/32 3.1.1.1/32 192.1.1.1/24

172.1.1.1/24
IP Network
10.1.1.0/24 150.1.1.0/24
192.1.1.0/24
Host Pool 10 Edge Node 1 Border and

Internet
Control Plane
Node
router lisp
• Operate as an IPv4 PITR & PETR locator-table default
locator-set border
• The EID prefixes are exported from IPv4-interface Loopback0 priority 10 weight 10
!
Control plane node to the External eid-table vrf USER instance-id 10
Border node with AD of “250” ipv4 route-export site-registrations
• The Border node only advertises the EID exit
!
prefix into external protocol of ipv4 proxy-etr
choice(BGP) ipv4 proxy-itr 3.1.1.1
!
router bgp 65004
!
redistribue LISP metric 10
exit-address-family
10.1.1.1/24 1.1.1.1/32 3.1.1.1/32 192.1.1.1/24

172.1.1.1/24
IP Network
10.1.1.0/24 150.1.1.0/24
192.1.1.0/24

Internet
Control Plane
Node
!
exit
!
10.1.1.1/24 1.1.1.1/32 3.1.1.1/32 172.1.1.1/24
IP Network
10.1.1.0/24 150.1.1.0/24
192.1.1.0/24

Internet
Control Plane
Node
router lisp
• Fabric edge node has a default route to locator-table default
the External Border. locator-set edge
!
ipv4 use-petr 3.1.1.1
Campus Fabric 2 EID-Prefix: Not found , map-cache miss
Mapping Locator-Set: ( use-petr)
Border Nodes- Forwarding on External Border
Entry 3.1.1.1, priority: 1, weight: 100 (D1)
192.3.0.0/24
INTERNET
D
4 External
Border
10.2.0.1  193.3.0.1 3.1.1.1
3 5.3.3.3
1.1.2.1  3.1.1.1 IP Network 5.1.1.1 5.2.2.2

Mapping
10.2.0.1  193.3.0.1 System
1.1.1.1 ETR 1.1.2.1 1.1.3.1 ETR 1.1.4.1
1
10.2.0.1  193.3.0.1
S
Campus 10.2.0.0/24 10.3.0.0/24
DC
Fabric Border Platform Support and
Recommendations
Platform Support
Fabric Border Nodes - Options
Catalyst 3K Catalyst 6K ASR1K & ISR4K Nexus 7K
• Catalyst 3850 • Catalyst 6800 • ASR1000-X • Nexus 7700

• 12/24 or 48XS • Sup2T or 6T • X or HX Series • Sup2E
• 1/10G (Fiber) • 6880 or 6840-X • ISR4430 / 4450 • M3 Cards
• IOS-XE 16.3.1+ • IOS 15.4.1SY+ • IOS-XE 16.4.1+ • NXOS 7.3.2+
Platform Support
Fabric Control-Plane - Options
Catalyst 3K Catalyst 6K ASR1K & ISR4K
• Catalyst 3850 • Catalyst 6800 • ASR1000-X

• 12/24 or 48XS • Sup2T or 6T • X or HX Series
• 1/10G (Fiber) • 6880 or 6840-X • ISR4430 / 4450
• IOS-XE 16.3.1+ • IOS 15.4.1SY+ • IOS-XE 16.4.1+
Cisco ISE
N7K, C6K, C3K
Campus Fabric Components:
ASR1K, ISR4K
1. Control-Plane Nodes
a. LISP Map Server/Resolver
b. EID to RLOC Mapping
2. Fabric Border Nodes

a. Internal Border - XTR
C3K, C4K B B b. External Border - PXTR
C3K,C6K, 3. Fabric Edge Nodes

C ASR1K, a. Host Registration
ISR4K b. Host Resolution
c. Host Mobility - Dynamic EID
4. Identity Services Engine (ISE)

a. AAA/Radius
b. 802.1x/MAB
c. TrustSec (SGT)
Border Design Considerations
Where do things go?
Border with HTDB Co-Located
Campus Fabric- Border Design Options
Border Nodes- Border Co-located with Control Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24 192.1.1.0/24

External Domain
Control Plane
Node
• The Border node and Control plane node is on the same device
• The Control plane node maintains the database of every prefix/subnet in the Local Fabric
Domain.
• Simple Design and Configuration
• No additional protocols needed
• Every border(Internal and External) cannot be a control plane node.
• Control plane node scale is different on different platform’s.
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Node
• Operate as an IPv4 LISP Map-resolver router lisp
and Map-server locator-set border
• Operate as an IPv4 ITR & ETR + IP !
Subnets eid-table vrf USER instance-id 10
• The Control plane node registers all ipv4 distance site-registrations 250
local prefixes in the database !
site Campus
authentication-key cisco
• The EID prefixes are exported from eid-prefix instance-id 10 0.0.0.0/0 accept-more-specifics
Control plane node to the Internal eid-prefix instance-id 10 10.1.1.0/24 accept-more-
specifics
Border node with AD of “250” !
ipv4 map-server
ipv4 map-resolver
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Node
• The Internal Border also imports the router lisp
external prefixes into the Campus Fabric locator-table default
locator-set border
LISP domain. IPv4-interface Loopback0 priority 10 weight 10
!
ipv4 route-import database bgp 65004 locator-set rlocs
exit
!
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Node

!
exit
Border with HTDB Non Co-Located
Border Nodes- Border Non Co-located with Control Plane Node
5.1.1.1/32 C Control-Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24 OSPF 192.1.1.0/24
Host Pool 10 Edge Node 1 Border Node External Domain
• The Border node and Control plane node are different devices device
• The Control plane node maintains the database of every prefix/subnet in the Local Fabric
Domain and hence need an additional protocol(iBGP in this case) to share EID mapping
information from control plane node to border.
• Multiple Border nodes(Internal/External) can connect to single or multiple set of Control plane nodes.
• Detailed configuration is required
5.1.1.1/32 C
Control-Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network OSPF
10.1.1.0/24 192.1.1.0/24

router lisp
• Operate as an IPv4 LISP Map-resolver locator-table default
locator-set control_node
and Map-server IPv4-interface Loopback0 priority 10 weight 10
!
• The Control plane node registers all eid-table vrf USER instance-id 10
local prefixes in the database ipv4 route-export site-registrations
• The EID prefixes are exported from !
site Campus
Control plane node to its own authentication-key cisco
RIB(routing information base) with AD of eid-prefix instance-id 10 0.0.0.0/0 accept-more-specifics
eid-prefix instance-id 10 10.1.1.0/24 accept-more-
“250” specifics
!
ipv4 map-server
ipv4 map-resolver
5.1.1.1/32 C
Control-Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network OSPF
10.1.1.0/24 192.1.1.0/24

router bgp 65555
• The Control plane has a iBGP neighbor 2.1.1.1 remote-as 65555
!
connection from itself to the Border address-family vpvnv4
node and advertises the EID prefix into neighbor 2.1.1.1 activate
neighbor 2.1.1.1 send-community both
BGP !
• Border node learns the EID prefixes in redistribue LISP metric 10
the Local Fabric domain from the aggregate-address 10.1.1.0 255.255.255.0 summary-only
exit-address-family
Control Plane node.
5.1.1.1/32 C
Control-Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24
OSPF
192.1.1.0/24
• The Border receivers the EID prefix router lisp

information from the control plane node locator-set border
through the iBGP connection. IPv4-interface Loopback0 priority 10 weight 10
!
• The Border also imports the external eid-table vrf USER instance-id 10
prefixes into the Campus Fabric LISP ipv4 route-import database ospf 123 locator-set border
!
domain. (Depending on whether it is router bgp 65535
external or Internal Border). !
neighbor 5.1.1.1 remote-as 65555
!
address-family vpvnv4
neighbor 5.1.1.1 activate
!
5.1.1.1/32 C
Control-Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24
OSPF
192.1.1.0/24
• The Border advertises the EID prefix router lisp

learnt from control plane node via iBGP locator-set border
to the external domain through OSPF by IPv4-interface Loopback0 priority 10 weight 10
redistributing routes. !
ipv4 route-import database ospf 123 locator-set border
exit
!
router ospf 123 VRF USER
!
redistribue bgp metric 10 subnets
5.1.1.1/32 C
Control-Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24
OSPF
192.1.1.0/24

!
ipv4 route-import map-cache bgp 65535
exit
!
Campus Fabric- Border Resiliency Options
Border Nodes- Loop Prevention
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
B
IP Network
10.1.1.0/24
Host Pool 10 Edge Node 1 Border Node
192.1.1.0/24
10.1.1.1/24 1.1.2.1/32 3.1.1.1/32 192.1.1.5/24
External Domain
B
IP Network
10.1.1.0/24
Host Pool 10 Edge Node 1 Border Node
• eBGP is used to break loops caused by the bidirectional advertisement of routes from fabric to external
domain when using multiple Internal Borders for redundancy/resiliency. This is done via AS-Path loop
prevention.
• When using any other protocol other than eBGP appropriate loop prevention methodology needs to
be sued (distribute lists , etc).
Border Design Options
One Box vs Two Box
One Box Border Design
• One Box solution is where the Border is the boundary between the external
domain and the Local fabric domain.
• The Border device will advertise routes to and from the Local Fabric domain
to the External Domain.
One Box Border Design- Control Plane Interworking
CONTROL-PLANE
1
LISP External Domain(BGP/IGP)
B
B External
Domain
B
One Box Border Design- Data Plane Interworking
DATA-PLANE
2
VXLAN External Domain(IP/MPLS/VXLAN)
B
B External
Domain
B
One Box Border Design- Policy Plane Interworking
POLICY-PLANE
3
SGT in VXLAN External Domain(IP ACL/SGT)
B
B External
Domain
B
Two Box Border Design
• Two Box solution is a normalized hand off solution where the Border is the
edge of the fabric domain and another device represents the edge of the
external domain.
• This solution requires two devices and BGP is the used between these two
domain edges for exchanging connectivity and reachability information.
• This design model is chosen when the Border does not support the
functionality needed to run the external domain on the same device. This can
due to hardware or software support on the device.
Two Box Border Design- Control Plane Interworking
CONTROL-PLANE
1
LISP BGP External Domain(BGP/IGP)
B
B External
Domain
B
Two Box Border Design- Data Plane Interworking
DATA-PLANE
12
VXLAN VRF-LITE External Domain(IP/MPLS/VXLAN)
B
B External
Domain
B
Two Box Border Design- Policy Plane Interworking
POLICY-PLANE
13
SGT in VXLAN SGT Tagging External Domain ( IP ACL/SGT)
B
B External
Domain
B
Border Resiliency Options
Resiliency at the Border
Track or propagate events across domains
CONTROL-PLANE LISP IGP/MP-BGP/BGP-EVPN
External
Border
Router
Map Server B IP Network
Border External
Campus Fabric Router External Domain
VXLAN/+SGT IP/MPLS/VXLAN
DATA-PLANE
Failures & Changes in the External Domain
External advertisements to reflect state of the External Domain
Border
Map Server B IP Network
Border
Campus Fabric External Domain
Host reachability
Border Routing from router lost
Tables updated or degraded
to remove faulty
router
Host
advertisements
from this router
withdrawn
Failures & Changes in the Campus Fabric
Dynamic redistribution of LISP state into External Domain @ Border
CONTROL-PLANE LISP IGP/MP-BGP/BGP-EVPN
Border
B
IP Network
Border
Border connectivity
to Campus Fabric
Network degraded: Prefix
Registration advertisements Routing Tables
State Changes • Dynamic LISP from this border updated to route
Communicated to State updates withdrawn around failure
Border • Core Reachability
Tracking
Failures & Changes in the Campus Fabric
LISP Control Plane Node Separate From Border
Border
Map Server
BFD
B
Adjacency
IP Network
Border
Campus fabric Prefixes are advertised in BGP from Control Plane Node to Border
The BGP adjacencies between Control Plane node and Border are monitored with BFD
Upon failure, the adjacency is broken, prefixes removed at the Border and withdrawn
Fast convergence (BFD  180ms)
Border Deployment Models
How does things connect ?
Shared Services with Internal
Border
Campus Fabric- Border Deployment Options
Shared Services (DHCP,AAA etc) With Internal Border
• The hosts in their respective virtual networks in the local fabric domain will need to have access to
common shared services like
 Identity Service / AAA

 DNS
 DHCP
 IPAM
 Monitoring tools
 Data Collectors
 Other common infrastructure elements
• These shared services will generally reside outside of the fabric domain.
Shared Services (DHCP,AAA etc) With Internal Border
• RLOC/Underlay connectivity in Global

Routing Table
• Loopback interfaces for management in Fabric scope of management
their own VN (Default) USER #2
• Other VNs can be used for segmentation USER #1

Border
for users, devices, roles, and others
USER2
Management Access
• Scalable Group Tags (SGTs) can be used USER1
Default
for further access control within a VN RLOC/Underlay GRT
• The CORPORATE VN is being shown in

this slide deck as an example.
• Similar steps can be followed for other
VNs shown
Shared Services (DHCP,AAA etc) With Internal Border in Global Routing Table
B B APIC
EM
APIC-EM DHCP/ Identity Service

GRT DNS
Shared Services
5.1.1.1/32 C
Control-Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24 EIGRP 172.10.10.0/24
Host Pool 10 Edge Node 1 Border Node Shared Services
• The Shared Services are in the Global router eigrp 65535

routing table and form a routing !
adjacency to the Campus to the Campus redistribute lisp metric 10000 1 255 1 9100
Fabric using the Global routing table. network 192.1.1.1 0.0.0.0
distribute-list in USER
• On the Campus Fabric side we will form autonomous-system 65535
exit-address-family
a routing adjacency using the VRF table address-family ipv4 vrf default
of the EID space. network 172.1.1.1 0.0.0.0
distribute-list in default
• This will ensure that prefixes from both autonomous-system 65535
exit-address-family
the domains are reachable to each
other. !
5.1.1.1/32 C
Control-Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.2.1.1/24
IP Network
10.1.1.0/24 EIGRP 172.10.10.0/24
Host Pool 10 Edge Node 1 Border Node Shared Services
• The Shared Services are in the Global router eigrp 65535

routing table and form a routing !
network 192.2.1.1 0.0.0.0
adjacency to the Campus to the Campus exit-address-family
Fabric using the Global routing table. !
• On the Campus Fabric side we will form

a routing adjacency using the VRF table
of the EID space.
• This will ensure that prefixes from both
the domains are reachable to each
other.
Shared Services (DHCP,AAA etc) With Internal Border in a secure VRF
B B APIC
EM
VRF APIC-EM DHCP/ Identity Service

Fusion DNS
Router Shared Services
Shared Services (DHCP,AAA etc) With Internal Border in a secure VRF
5.1.1.1/32 C
Control-Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
IP Network
10.1.1.0/24 BGP BGP
172.10.10.0/24
Host Pool 10 Edge Node 1 Border Node Fusion Router Shared Services
• The Shared Services are in a secure ip vrf User

rd 1:1
VRF of their own. route-target export 1:1
route-target import 1:1
• A fusion router is used in the scenario to route-target import 3:3
leak routes from the VRF’s in Campus !
ip vrf User
fabric to the Services VRF. rd 2:2
route-target export 2:2
ip vrf Services
rd 3:3
Wan Connectivity with Internal
Border
Wan Connectivity With Internal Border
B B
IWAN 2.x/MPLS
Campus Fabric – Border Deployment Options
WAN connectivity with Internal Border
CONTROL-PLANE
1
LISP IWAN 2.x (BGP/EIGRP)
BRANCH
B
B IWAN 2.x
HOST-H2
B
HOST-H1
DATA-PLANE
12
VXLAN DMVPN
BRANCH
B
B IWAN 2.x
HOST-H2
B
POLICY-PLANE
13
SGT in VXLAN SGT in DMVPN
BRANCH
B
B IWAN 2.x
HOST-H2
B
CONTROL-PLANE
LISP MP-BGP LISP/IP
Campus Fabric
HTDB
Branch
BGP MPLS Domain
Border
BRANCH
Border Border
SXP Connection between the Border’s HOST-H2

for SGT information exchange
HOST-H1
VXLAN+SGT MPLS with SXP for SGT exchange VXLAN+SGT/IP/MPLS
DATA+POLICY PLANE
Service Chaining with Internal
Border
Service Chaining with Internal Border
Non-Cisco Firewall: Cisco Firewall :
• Firewall is connected externally to the • Firewall is connected externally to the Campus Fabric.
Campus Fabric.
• The prefixes from the local Campus Fabric domain will be
advertised to the firewall with a routing protocol of choice.
• The prefixes from the local Campus
Fabric domain will be advertised to the
• SXP connection between ISE and Firewall needed for
firewall with a routing protocol of derivation of SGTs on the Firewall.
choice.
• Firewall policy is based on SGT’s and SG ACL’s ( Group
• Firewall policy is interface/subnet/IP based Policy).
based.
• SGACL’s are enforced on the egress direction in the
firewall and they are derived info from SGT’s & ISE
connection.
• Firewall also has subnet/interface/IP based policy for

brownfield integration
B
B
B
Firewall
CONTROL-PLANE
1
LISP BGP/IGP
B
B
B
Firewall
DATA-PLANE
2
VXLAN VRF-LITE
B
B
B
Firewall
POLICY-PLANE
3
SGT in VXLAN VXLAN SGT in-line Tagging
B
B
B
Firewall
Service Chaining with Internal Border – Cisco Firewall
ISE
POLICY-PLANE
3
SGT in VXLAN VXLAN SGT in-line Tagging
Group Policy
SXP
B
B
B
Firewall
Firewall gets group

policy from ISE
Data Center Connectivity with
Internal Border
Data Center Connectivity With Internal Border – ACI Fabric
CONTROL-PLANE
LISP EVNPN-BGP
N7K ASR1K
B ✔ ✔
Border ACI Fabric

Map Server B
Border
Campus Fabric
DATA-PLANE
VXLAN+SGT VXLAN+EPG
* N7K Roadmap
MP-BGP EVPN standard control plane between the Internal Border devices and the Cisco ACI fabric spine switches:
1 A single BGP session is required to exchange reachability information for multiple user contexts (VRF-1, VRF-2, and
VRF-3 etc.) thus removing the per-VRF session requirements of the traditional integration models.
B MP-BGP EVPN
Border ACI Fabric

Map Server B
Border
Campus Fabric
OpFlex control plane between the Internal Border

and the Cisco ACI spine switches to automate the
Campus Fabric
B B Border ACI fabric-facing tenant provisioning on the Internal
Border: The network administrator simply configures
2 a new external Layer 3 outside (L3 Out) policy for a
tenant on the Cisco Application Policy Infrastructure
OpFlex
MP-BGP Controller (APIC). The controller then programs all

EVPN related information associated with that tenant, such
as VRF instance name and BGP extended
community route-target attributes, for the Cisco ACI
spine switches.
1) VRFs A,B and C are the Campus Fabric domain Internal Border – VRF stitching/mapping
VRF’s
Extranet
2) VRF’s 1 and 2 are simply extensions of ACI VN’s 1 VRF A VRF 1

VRF B Extranet VRF 2
and 2 from the external domain to the Internal VRF C VRF C
Border. Campus Fabric Data Center
VRF VRF
3) Extranets are then used to provide connectivity

between them.
VXLAN data plane between Internal Border the Cisco ACI fabric to establish
3 communication with the different domains and also to carry the information
needed(SGT/EPG) for policy enforcement.
Campus Fabric Policy Domain ACI Policy Domain

Security Groups/ End Point
SGT Groups/EPG
ISE & APIC Exchange Groups
and Member information
Cisco APIC-DC
ISE creates SGT to EPG
Cisco ISE 2.2 translation table
IP, SGT mappings Send translation table to

IP-ClassId, VNI bindings
ASR 1K/N7K
Campus Fabric
B
User Edge Border* Nexus9000 Nexus9000 Server

Spine Leaf
Classification
LISP,SGT & VXLAN BGP EVPN, EPG &VXLAN
Data Center Connectivity With Internal Border – Programmable Fabric Data Center
CONTROL-PLANE
1 LISP VXLAN-EVPN
MP-BGP EVPN
B
Border
IP/MPLS
Map Server B
Network
Border
Campus Fabric
DATA-PLANE
2
VXLAN+SGT VXLAN + (SGT)
Data Center Connectivity With Internal Border – Programmable Fabric Data Center
 Programmable Fabric data center uses VXLAN BGP-EVPN without group.

 Policy is based on IP subnets using IP ACLs
 APIC-EM / ISE* retrieves all the groups from DCNM
 Policy Authoring is done from APIC-EM for those IP subnets
 Policy is pushed by DCNM and enforced on border spine /leaf in the Data
Center
* Roadmap
Data Center Connectivity With Internal Border – Traditional Data Center
CONTROL-PLANE
1 LISP IGP/MP-BGP
Border
IP/MPLS Network
Map Server B
Border
Campus Fabric
DATA-PLANE SXP Connection between the Border’s

for SGT information exchange
2
VXLAN+SGT IP/MPLS + (SGT)
Data Center Connectivity With Internal Border – Traditional Data Center
 Traditional data center can be an vPC/FabricPath/MPLS based data center

with our without group policy.
 If Group Policy/Cisco TrustSec was implemented then Policy is based on
SGT groups using SGACL’s.
 If Group Policy/Cisco TrustSec was not implemented then Policy is based on
IP subnets using IP ACLs.
Internet Connectivity with External
Border
Internet Connectivity With External Border
CONTROL-PLANE
1 LISP BGP
Border
Map Server B
Border
Campus Fabric Internet
DATA-PLANE
2 VXLAN+SGT IP
Cloud Connectivity with External
Border
Cloud Connectivity With External Border
Cloud edge gets
ISE 3 group policy from
ISE
CONTROL-PLANE
1 LISP LISP
Group Policy
SXP
B
IP/MPLS
Border
Network CLOUD
Map Server B
Cloud Edge
CSR1Kv
Border
Campus Fabric ✔
DATA-PLANE
2 VXLAN+SGT VXLAN+SGT
* Roadmap
DC Integration
Agenda
1 Introduction
Why do I care?
2 Campus Fabric & ACI Integration

Connectivity, Benefits, Deployment Models
3 Solution Demo
How does it work?
4 Campus Fabric & non-ACI Integration

VXLAN EVPN DC, Traditional VPC / FP DC
Introduction
Why do I care?
Icon Legend For Your
Reference
Frame Encapsulation Cisco ISE
Campus Fabric Edge Node

SGT Tag
B
Campus Fabric Border Node APIC DC
Campus Fabric LISP MS/MR DCNM
C Nexus Fabric Manager

Campus Fabric Control Plane Node FABRIC
MANAGER
Endpoint in Campus Fabric Virtual Topology System
Today
Static Access Control
Need to allow Employees
talk to Webservers.
Deny access to Guests
John gets the identity

info from Amy and sets
access policies in DC
Amy manages John manages

Campus network Datacenter network
Distinct Functionality, Distinct Domains
Network Operator
Data Center A
Network Operator
Access Domain Data Center B

(Campus/Branch/WAN)
Focused on User Access Focused on Applications

Wireless Integration Virtualization: VMs, Containers
User Identity / AAA Compute Integration
QoS / Path Engineering Agile Application Deployment
Hybrid Cloud Mobility
Fate Separation, Scale, Administrative Delineation

IETF
The Good News 
http://tinyurl.com/sgt-draft
VXLAN Group
based policy packet
header, encode
SGT = 16bit ID SGT / EPG

EPG = 16bit ID
Outer MAC Header Outer MAC Header
Flags Flags
Outer IP Header Flags / Outer IP Header Flags /

DRE DRE
UDP Header UDP Header
Source Source
Group Class
VXLAN Header Tag == VXLAN Header ID ==
SGT EPG
Inner (Original) MAC Header Inner (Original) MAC Header
BD / BD /
Inner (Original) IP Header VRF == Inner (Original) IP Header VRF ==
VNID VNID
M / LB / M / LB /
Original Payload SP Original Payload SP
VXLAN and GBP extensions
Ethernet in IP with a shim for scalable segmentation and policy metadata
SGT = EPG
(Campus Fabric) (ACI)
VXLAN-GBP
VXLAN
FCS
Outer MAC Header Outer IP Header Outer UDP Header VXLAN Header Original Layer 2 Frame
GBP = Group Based Policy
More Good News 
Campus Fabric ACI Fabric
• Underlay • Underlay
• Overlay • Overlay
• Logical constructs • Logical constructs
• VNID • VNID
• SGT • EPG
• User Endpoint • App Endpoint
• Group Based Policy • Group Based Policy
Why integrate Campus Fabric and ACI?
ACI POLICY
ENDPOINT GROUPS
VM VM
USERS VM VM
WEB APP DB
CONTRACTS
 What users? (Employee / Contractors / Guests) I can help!

 What device-type? (Corporate / BYOD / IOT)
 Posture compliant? (Compliant / Non-compliant)
 Threats / Vulnerabilities? (Safe / Compromised hosts)
 Location? (Corporate / Public / Home)
Integration
Campus and ACI Fabric Policy Key Concepts, Benefits
Recap - ACI Fabric Integrated VXLAN Overlay
• Decoupled Identity, Location and Policy
ACI Spine Nodes
ACI Fabric
VTEP VXLAN IP Payload
ACI Leaf Nodes
 Forwarding within the Fabric is between VTEPs (ACI VXLAN tunnel endpoints) and leverages an
extended VXLAN header format referred to as the ACI VXLAN policy header
 Any workload any where, Consistent Latency, Mapping of tenant MAC or Ip address to location is
performed by VTEP using distributed mapping database
What are SGTs?
How do they differ from EPGs?
EPG is end point group in ACI
 SGT is a security group tag assigned to user’s
or device’s traffic in campus networks based on fabric used to group servers that
their roles require similar treatment of policy
 SGT is a 16 bit value that the Cisco ISE assigns EPG is hierarchical in nature
to the user or endpoint’s session upon login
 SGT is globally unique
How can we achieve
normalized identity
between Campus
and ACI
Campus Fabric SGTs Provisioned in ACI
ACI
ISE
ISE dynamically provisions
SGTs and IP mappings
(SXP service) into APIC-
DC
Campus Fabric Domain
EXT- EXT-
EPG1 EPG3
Security Groups External (Outside Fabric) EPGs
ACI EPGs Automatically Propagated into Campus Fabric
ACI
ISE
ISE dynamically learns EPGs

and VM Bindings from ACI
fabric – shared to SXP
VM1
Campus Fabric Domain VM25
Security Group from APIC-DC Internal (Inside Fabric) EPGs
Enabling Group-based Policies in each Domain
DB DB
SG-FW
SG-ACL
Contract
Campus / Branch Data Center

Campus Fabric Policy Domain APIC
APIC Policy Domain
Shared Policy Groups
Voice Employee Supplier BYOD

Web App DB
Voice Data
VLAN VLAN Campus Fabric ACI Fabric
Sharing Context Across the Enterprise
ISE
Campus
Fabric
Domain
Auditor Campus Fabric Border
10.1.10.220 Device ACI Spine (N9K)
(ASR 1K/N7K*)
PCI
10.1.100.52
* M3 Roadmap
ISE
5 Campus
Fabric
SRC:10.1.10.220 Domain
DST: 10.1.100.52
Auditor SGT: 5 Campus Fabric Border
(ASR 1K/N7K*)
PCI
10.1.100.52
* M3 Roadmap
ISE
Campus
Fabric 5
Domain SRC:10.1.10.220
Auditor DST: 10.1.100.52
Campus Fabric Border
10.1.10.220 SGT: 5
Device ACI Spine (N9K)
(ASR 1K/N7K*)
PCI
10.1.100.52
* M3 Roadmap
ISE SGT/EPG
Namespace Alignment
EPG #
SGT # to EPG #
Translation Table
Campus
Fabric 5
Domain SRC:10.1.10.220
Auditor DST: 10.1.100.52
Campus Fabric Border
10.1.10.220 SGT: 5
Device ACI Spine (N9K)
(ASR 1K/N7K*)
PCI
10.1.100.52
* M3 Roadmap
ISE SGT/EPG
Namespace Alignment
EPG #
SGT # to EPG #
Translation Table
SRC:10.1.10.220
DST: 10.1.100.52
EPG :#
Campus #
Fabric
Domain
(ASR 1K/N7K*)
PCI
10.1.100.52
* M3 Roadmap
ISE SGT/EPG
Namespace Alignment
EPG #
SGT # to EPG #
Translation Table
SRC:10.1.10.220
DST: 10.1.100.52
EPG :#
Campus #
Fabric
Domain
(ASR 1K/N7K*)
PCI
10.1.100.52
* M3 Roadmap
Goal: Mapping Policy contents across domains
Web
DB
C
User-App User to App Contracts
Application Prioritization
Web1 App1 DB
Qo Qo
Se S
Filt
rvi
er
ce
User-User
Access Control: SG-ACL
App to App Contracts
Campus / Branch Exchange Complete

Campus Fabric Policy Domain Policy
Data Center
ACI Policy Domain
It’s a
Journey …
Key Benefits with this integration
End to End security and segmentation
EIG EIG EIG Virtual Virtual Virtual

1 4 7 Network Network Network
EIG EIG EIG EIG EIG EIG Context
2 3 5 6 8 9
Context Context
“A” “B” “C”
Key Benefits with this integration
Consistency with Group based policies across both domains
DB
DB
SG-ACL
SG-FW
Key Benefits cont’d
 Businesses lowers Capex /

Opex with simplified
provisioning using Software
Defined Networking
 Businesses gain Agility due to
better efficiencies in managing
FCAPS
Connectivity
Deployment Models
Campus Fabric - ACI Policy Plane
Integration
How does it work?
Hardware and Software recommendations
Shipping
NOW!
ACI Fabric
ACI Software ISE APIC
Hardware
Nexus 9K* 12.1 2.1 2.1
* – Please check release notes for latest information

* – (9396PX/TX, 9372PX/TX, 93120TX, 93128TX, 9736PQ LC, 9336PQ, 93108-EX, 93180-EX
Campus Fabric SGT Info Used in ACI Policies
ISE
Controller Layer
Controller Layer
Network Layer
Network Layer
Plain ACI Spine (N9K)

Campus Ethernet
Fabric
(no CMD) ACI Leaf
ACI Border
Leaf (N9K) (N9K)
SGT Groups available in ACI Policies
ISE
Controller Layer
Controller Layer
PCI EPG
10.1.100.52
Network Layer
Network Layer

Campus Ethernet
Fabric
(no CMD) ACI Leaf
ACI Border PCI
Leaf (N9K) (N9K) 10.1.100.52
ISE
Controller Layer
ISE Retrieves:
Controller Layer
EPG Name: PCI EPG
EPG Binding = 10.1.100.52
Network Layer
Network Layer

Campus Ethernet
Fabric
(no CMD) ACI Leaf
ACI Border PCI
Leaf (N9K) (N9K) 10.1.100.52
ISE
Controller Layer
ISE Exchanges:
Controller Layer
SGT Name: Auditor
SGT Binding = 10.1.10.220
Network Layer
Network Layer

Campus Ethernet
Fabric
(no CMD) ACI Leaf
ACI Border PCI
Auditor Leaf (N9K) (N9K) 10.1.100.52
10.1.10.220
ISE
Controller Layer
ISE Exchanges:
Controller Layer
SGT Name: Auditor
EPG Name = Auditor

Groups= 10.1.10.220
Network Layer
Network Layer

Campus Ethernet
Fabric
(no CMD) ACI Leaf
ACI Border PCI
10.1.10.220
ISE
Controller Layer
ISE Exchanges:
Controller Layer
SGT Name: Auditor
EPG Name = Auditor

Groups= 10.1.10.220
Network Layer
Network Layer

5
SRC:10.1.10.220 Campus Ethernet
DST: 10.1.100.52 Fabric
SGT: 5 (no CMD) ACI Leaf
ACI Border PCI
10.1.10.220
ISE
Controller Layer
ISE Exchanges:
Controller Layer
SGT Name: Auditor
EPG Name = Auditor

Groups= 10.1.10.220
Network Layer
Network Layer

5
Campus SRC:10.1.10.220 Ethernet
Fabric DST: 10.1.100.52
SGT: 5 (no CMD) ACI Leaf
ACI Border PCI
10.1.10.220
ISE
Controller Layer
ISE Exchanges:
Controller Layer
SGT Name: Auditor
EPG Name = Auditor

Groups= 10.1.10.220
Network Layer
Network Layer

SRC:10.1.10.220
Campus DST: 10.1.100.52Ethernet
Fabric
(no CMD) ACI Leaf
ACI Border PCI
10.1.10.220
ISE
Controller Layer
ISE Exchanges:
Controller Layer
SGT Name: Auditor
EPG Name = Auditor

Groups= 10.1.10.220
Network Layer
Network Layer

SRC:10.1.10.220
Campus Ethernet DST: 10.1.100.52
Fabric
(no CMD) ACI Leaf
ACI Border PCI
10.1.10.220
ISE
Controller Layer
ISE Exchanges:
Controller Layer
SGT Name: Auditor
EPG Name = Auditor

Groups= 10.1.10.220
Network Layer
Network Layer
17000 ACI Spine (N9K)

Plain
SRC:10.1.10.220
Campus Ethernet DST: 10.1.100.52
Fabric
(no CMD) EPG
ACI Leaf
ACI Border PCI
10.1.10.220
ISE
Controller Layer
ISE Exchanges:
Controller Layer
SGT Name: Auditor
EPG Name = Auditor

Groups= 10.1.10.220
Network Layer
Network Layer
17000

SRC:10.1.10.220
DST: 10.1.100.52
Campus Ethernet EPG
Fabric
(no CMD) ACI Leaf
ACI Border PCI
10.1.10.220
How to enable this integration?
 Access both ISE and APIC-DC

 To establish communication
between them, import APIC-DC
certificate into ISE as “Trusted
Certificates”
ACI Settings in ISE
ACI Settings:
• Controller
• Credentials
• Tenant name defined in ACI
• L3 Routed Network defined
in ACI
ACI View
Campus Fabric Groups & Group Members
shared with ACI
SGTs appear as
External EPGs
Group
Members
ISE View
ACI Groups & Group Members shared with ISE
and APIC-EM EPGs appear as SGTs
Group members
learnt via SXP
Campus Fabric and ACI Integration
Frequently Asked Questions
 No, we are exchanging groups and
their membership information
Are the policy contents
being exchanged?  Policy is applied in each domain. ACI
can enforce more granular policies
 Yes, one SGT maps to one external

EPG in ACI per context
Is there a 1:1 mapping
between groups? What  250 groups supported today
is the scale?  This is not a hard limit, there is no

limit on ISE side
Campus Fabric -ACI Integration Deployment Models
Summary
Policy Plane only Integration

 Exchanges groups and their membership information
(group to IP bindings) between ISE and APIC
 SGT tag is removed at the border when sending traffic
outside Campus Fabric domain towards DC
 DC Border devices add group information based on
source / destination IP address
Considerations of Policy Plane Integration
• User scale - Mappings appear as /32

per group
Less no. of mappings = 4000/10000 (non-
EX/EX based) per Border Leaf in ACI fabric
• Support of more than single VRF in

Campus consider Data Plane
integration
User / Group Control-Plane

Repository Nodes
ISE / AD Host DB
 User / Group Repository – External
ID Store device (e.g. ISE or AD)
can be leveraged to provide dynamic
User / Device to Group mapping.
 Control-Plane Nodes – Map System
Fabric Domain Fabric Border that manages the Endpoint to Gateway
(Overlay) Nodes (Edge or Border) relationship.

device (Core), that connects External
 Edge Nodes – The L3 Gateway
Fabric Edge Intermediate Nodes device (Access or Distribution), that
Nodes (Underlay) connects Endpoints to Fabric.
 Intermediate Nodes – Normal L3
(IP) Forwarders in the Underlay.
Campus Fabric
Border Nodes – Border
Fabric Border Node is based on a LISP Tunnel Router + IP Subnet’s
All traffic entering or leaving the Fabric from and to a known destination goes
through this type of node
• Connects the DC ,WAN and any other known

network’s to the local fabric domain.

• Responsible for advertising prefixes from and to the

• Provides a domain exit point for all Edge Nodes
Connectivity High Level view
INTERNET Traditional L3
IWAN 2.x
Campus Fabric Non ACI Fabric

ACI Fabric
B
E E
VXLAN-LISP
FABRIC
MANAGER
Recap: What is an L3Out?
 L3Out is a logical construct defined to
allow L3 connectivity between the ACI
Fabric and the external network
 One or more L3Outs can be defined for
each given tenant
L3Outs Container
 L3 interfaces are used on specific ACI
Specific L3Out devices (named Border Leaf nodes) to
interconnect to the external routed network
L3 Interface on  The external routed domain is modeled

Border Leaf Node with one (or more) External EPGs
(‘Networks’)
Border Leaf
Node A security policy (contract) is required to allow
communication between External and Internal
External EPG
EPGs
Campus Fabric to ACI
Campus Fabric Border connectivity with ACI Fabric
LISP COOP
VXLAN SGT VXLAN EPG

C IP Network
B
MP-BGP – EVPN
MS/MR Trusted
B
L3Out
EVPN
SGT-EPG
Translation
E E E
Control Data Plane ACI ASR1K N77XX/M3

Plane
BGP-EVPN VXLAN OpFlex March 16.5.1 Roadmap
✔ ✔
Campus Fabric - ACI Data Plane
Integration
ASR1K
Target
Higher Scale Data Plane Solution March 16.5.1
✔
Q2-CY17

2
Policy Plane (REST API)
ISE Builds Translation Table

1. GET: VRF-ID, Class-ID
Download
2. SGT <==> VRF-ID, Class-ID
Translation
3 Table
C Routing Plane (MP-BGP EVPN)

B “Trusted Mode”
Golf L3out
MS/MR
SGT <-> EPG 1
ASR1k(config)# cts sg-epg translations
B translation
Data Plane (GBP VXLAN)

4
EPG Starts on ASR1k
E E E
Leaf: -EX only
What’s unique about this integration
Details
Key Points • BGP EVPN based Control-Plane

• Standards Based BGP EVPN
between ASR1K and ACI Spine Border ACI Spine
• Forwarding decision based on IP Network

control plane
• Single BGP EVPN session to carry
all DC VRFs instead of using VRF-
lite IP Network
• Multi-tenancy at scale
Campus ACI Fabric

Fabric
What’s unique about this integration Details
Key Points • VXLAN based Data-Plane
• VXLAN GBP between ASR1K and N9K ACI Spine Border ACI Spine
• Trust the Group policy id field in GBP VXLAN header on
ingress
IP Network
• Propagate Group policy id field in GBP VXLAN header on

egress
• ASR1K replaces the VRF Name with locally generated IP Network
VRF-ID prior to enforcement in data plane leading to a
table of 3 tuples: (SGT, Classid, VRF-ID)

interface nve1
no ip address
source-interface Loopback1
host-reachability protocol bgp
group-based policy
What’s unique about this integration Details
Key Points • Radius based Control-Plane
• Download SG-EPG translation table Border ACI Spine
• Using RADIUS environment data support, IP Network
download SG-EPG tables
• Existing trustsec based network devices will
include merge of SGs defined in ISE and IP Network
corresponding SGs assigned to EPGs learnt from
ACI / APIC controller
cts sg-epg translation
radius server ISE
address ipv4 172.26.204.150 auth-port 1812 acct-port 1813
pac key cisco Campus Fabric ACI Fabric
!
aaa server radius dynamic-author
client 172.26.204.150 server-key cisco
!
aaa authentication dot1x default group ISE
aaa accounting dot1x default start-stop group ISE
aaa authorization network cts-list group ISE
!
cts authorization list cts-list
Campus Fabric and ACI Integration
Frequently Asked Questions
 Default is 24 Hrs same as
environment-data refresh timer.
How often does ISE However, timer is tunable in ISE
refresh translation table on
ASR periodically  If you need to do manual refresh issue
“ cts refresh environment-data” cli
on Campus border
 Assuming communication between

How often does APIC APIC and ISE is active, when ever
there is group add/delete it is updated
and ISE sync instantly
What’s unique about this integration
IETF
Details https://tools.ietf.org/html/draft-smith-OpFlex-00
Key Points • OpFlex Automation-Plane

• OpFlex is a communication channel
used for configuring policies between Border ACI Spine
ACI fabric and external devices that
IP Network
support OpFlex
IP Network
opflex agent
service vxlan-evpn
nve-id 1
bdi-ip 10.20.30.40 255.255.255.0
domain DCI identity dci-[10.4.254.115]
peer 1 ip-address 10.4.11.1 tcp-port 8009
src-ip-address 10.4.10.1
Campus
Fabric ACI Fabric
Where does the
enforcement happen
ASR1K
Campus to ACI Flow March 16.5.1

Target
Q2-CY17
✔
C VXLAN GBP
B SGT-EPG
Golf L3out
MS/MR
B
SGT <-> EPG
translation
E E E
Contract Applied on Leaf APP-EPG

Lookup:s-class, d-class, policy
ASR1K
ACI to Campus Flow March 16.5.1

Target
Q2-CY17
✔
TrustSec/ISE Policy Domain ACI Policy Domain
VXLAN GBP
C
B
SGT-EPG Golf L3out
MS/MR
B
SGT <-> EPG
translation
E E E
SGACL Policy Applied
VzAny Contract APP-EPG

Employee-SGT Permit-all or filter ports
Campus Fabric -ACI Integration Deployment Models
Summary
Data Plane Integration
 Exchanges groups including VNI in policy plane between
ISE and APIC
 Translation table, SG Name to SGT table, EPG Name to
class id table, VRF Name to VNI table downloaded via
ISE using RADIUS protocol to Border devices
 Border learns remote DC prefixes using BGP EVPN
control plane
 SGT to EPG translation in data plane per VRF or VNI
when sending VxLAN traffic towards DC
 SGT / EPG information carried inline
Key Benefits
Data Plane Integration
New Capabilities:
• Take current SGT propagation methods (DMVPN, GETVPN, SXP,
IPSEC, GRE, LISP/VXLAN (campus fabric) into ACI fabric
Benefits:
• Greater scale (remove IP/Group info from leaf)
• Seamless integration
Integration
Campus and non ACI Fabric
RADAR
Integration with Public Cloud / Traditional DC – Non

context aware DC
ISE 1. ISE and DC MGMT tool / AWS /Azure establish
API communication ORCHESTRATOR LAYER
2. DC MGMT tool could learn about vlan port Open APIs
groups / networks from vcenter or upper layer
orchestration layer (openstack) and exchange DC MGMT
them to ISE
3. ISE retrieves the vlan networks and assigns
SGTs
Campus Fabric 4. ISE pushes policy on Campus Fabric Border
Traditional DC
Wired Wireless Badges
Campus Fabric Connectivity to Programmable Fabric
Campus Fabric Policy Domain VTS/NFM/DCNM/CLI
ISE VXLAN FABRIC

MANAGER
Data-Plane Border
B
Enterprise Backbone
Border
BGP-EVPN
Control-Plane
Campus Fabric VXLAN Fabric
VXLAN GBP*, LISP VXLAN, BGP-EVPN

SGT
*GBP – Group Based Policy
RADAR
Policies between Campus Fabric and Programmable

Fabric
 Programmable Fabric uses VxLAN BGP-EVPN
without groups
 Policy is based on IP subnets using ACLs
 ISE* retrieves all the groups from DC MGMT tool
 Policy is pushed by ISE and enforced on border device
at Campus
Agenda
1 Troubleshooting
Where do I begin ?
2 Monitoring
What do i gain ?
3 Putting It Together
Where do things go?
Troubleshooting
Where do I begin?
Where do I begin ?
Overlay Network Overlay Control Plane

2. Data-Plane based on VXLAN Encapsulation
3. Policy-Plane based on TrustSec Edge Device
Hosts
(End-Points)
Edge Device
Underlay Network Underlay Control Plane

Assumption: Underlay is working
Our Playground
Fabric Border Control Plane  Control-Plane(CP) Nodes – Map
Nodes Node
System that manages the Endpoint to
C Gateway (Edge or Border) relationship.

B B device (Core), that connects External
Intermediate  Fabric Edge(FE) Nodes – The L3
Nodes Gateway device (Access or
Distribution), that connects Endpoints
to Fabric.
 Intermediate Nodes – Basic L3 (IP)

Forwarders in the Underlay.
Fabric Edge
Nodes
More on our Playground
 Underlay Network
C  Routing ID (RLOC) – IP address of
the LISP router facing ISP
B B  Overlay Network
10.2.100.1
 Endpoint Identifier(EID) - IP address
of a host
10.2.100.2 10.2.100.2
 VRF - Campus
 Instance Id - 4098
10.2.120.1 10.2.120.2 10.2.120.3
 Dynamic EID – Campus_10_2_1_0
10.2.1.99 10.2.1.89
Fabric Domain
(Overlay)
Terminology
• Egress Tunnel Router (ETR): An ETR is a device that is the tunnel endpoint; it accepts an IP packet where the
destination address in the "outer" IP header is one of its own RLOCs.
• Ingress Tunnel Router (ITR): An ITR is a device that is the tunnel start point; it receives IP packets from site end-
systems on one side and sends LISP-encapsulated IP packets, across the Internet to an ETR, on the other side.
• xTR: A xTR refers to a device which functions both as an ITR and an ETR (which is typical), when the direction of
data flow is not part of the context description
• Proxy xTR (PxTR): A PxTR is used for inter-networking between LISP and Non-LISP sites.
• Security Group (SG): Cisco TrustSec uses the device and user credentials acquired during authentication for
classifying the packets by security groups (SGs) as they enter the network
• Security Group Tag (SGT): Security group tag is the tag that is added in the packet to classify the security group.
Using loopback as LISP source
Troubleshooting becomes simple when source and destination IP are predictable
interface TenGigabitEthernet1/1/1
...
ip lisp source-locator Loopback0
...
end SRC: 10.2.120.1
DST: 10.2.100.1
SRC: 10.2.120.1
DST: 10.2.100.1
Troubleshooting
Where do I begin?
Control Plane Policy Plane
Data Plane
What is new in the control plane ?
Control Plane based on LISP
Multiple Routing Single Host tracking

Tables database - Map Server
BEFORE AFTER
Here is how you begin
DHCP Packet Flow Host Registration
External Connectivity Host Resolution
East West Traffic Host Mobility
Case: 1 – DHCP packet flow
C
DHCP
Server B B
ip dhcp relay information option
ip dhcp relay information option vpn
interface vlan 3000
ip dhcp relay source-interface Loopback0
FE1
DHCP packet flow in Campus Fabric
1 The DHCP client generates a
DHCP request and broadcasts it
on the network
1 2 FE add circuit id in option 82 and

forwards the packet natively in the
underlay
2
3 DHCP Server replies with offer
4 FE install the DHCP binding and

3 forwards the reply to client
4
DHCP binding on FE
FE#show ip dhcp snooping binding

MacAddress IpAddress Lease(sec) Type VLAN Interface
------------------ --------------- ---------- ------------- ---- --------------------
00:13:a9:1f:b2:b0 10.1.2.99 691197 dhcp-snooping 3000 TenGigabitEthernet1/0/23
FE#debug ip dhcp snooping ?

H.H.H DHCP packet MAC address
agent DHCP Snooping agent
event DHCP Snooping event
packet DHCP Snooping packet
redundancy DHCP Snooping redundancy
Case: 2 - Host Registration
10.2.100.1 CP
C
B B router lisp
site site_sjc
...
eid-prefix instance-id 4098 10.2.1.0/24 accept-more-specifics
exit
10.2.120.1 4098 10.2.1.0/24
FE1
router lisp
...
10.2.1.99 eid-table Campus instance-id 4098
dynamic-eid Campus_10_2_1_0
database-mapping 10.2.1.0/24 locator-set campus_fabric
exit
10.2.100.1 CP
C
B B router lisp
site site_sjc
...
exit
10.2.120.1 4098 10.2.1.0/24
FE1
router lisp
...
exit
10.2.100.1 CP
C
B B router lisp
site site_sjc
...
exit
10.2.120.1 4098 10.2.1.0/24
FE1
router lisp
...
exit
10.2.100.1 CP
C
B B router lisp
site site_sjc
...
exit
10.2.120.1 4098 10.2.1.0/24
FE1
router lisp
...
exit
Client
Registration
1d 1f
Check Check
local registr
Enable LISP
1a LISP
debugs on FE msg to
DB CP?
Is ?
MAC
Silent Host
learnt
?
LISP process issue
on FE
1e 1g
Check
Check registr
1c 1b LISP
Client
Registration msg
DB on from
COMPLETE
If CP ? FE?
Check ARP
IPDT table
table ? has an
entry ?
Packet dropped on
FE
or on CP
Client
didnt get IP
Client
Registration
1d 1f
Check Check
local registr
Enable LISP
1a LISP
debugs on FE msg to
DB CP?
Is ?
MAC
Silent Host
learnt
?
LISP process issue
on FE
1e 1g
Check
Check registr
1c 1b LISP
Client
Registration msg
DB on from
COMPLETE
If CP ? FE?
Check ARP
IPDT table
table ? has an
entry ?
Packet dropped on
FE
or on CP
Client
didnt get IP
C
B B
Verification at the FE1

1a FE1#show mac address
3000 0013.a91f.b2b0 DYNAMIC Te1/0/23
1b FE1#show arp vrf Campus

Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.2.1.99 0 0013.a91f.b2b0 ARPA Vlan3000
1c FE1#show device-tracking database

Network Layer Address Link Layer Address Interface vlan
ARP 10.2.1.99 0013.a91f.b2b0 Te1/0/23 3000
DH4
PKT
EID Host MAC VLAN
Client
Registration
1d 1f
Check Check
local registr
Enable LISP
1a LISP
debugs on FE msg to
DB CP?
Is ?
MAC
Silent Host
learnt
?
LISP process issue
on FE
m 1e 1g
Check
Check registr
1c 1b LISP
Client
Registration msg
DB on from
COMPLETE
If CP ? FE?
Check ARP
IPDT table
table ? has an
entry ?
Packet dropped on
FE
or on CP
Client
didnt get IP
1d C
B B
Verification at the FE1
FE1#show ip lisp instance-id 4098 database

LISP ETR IPv4 Mapping Database for EID-table vrf Campus (IID 4098), LSBs: 0x1
Entries total 3, no-route 0, inactive 0
10.2.1.99/32, dynamic-eid Campus_10_2_1_0, locator-set rloc_021

Locator Pri/Wgt Source State
10.2.120.1 10/10 cfg-intf site-self, reachable
Instance
ID
EID
FE1 RLOC Dynamic
EID
C
B B
If no local database entry ?

debug lisp control-plane local-eid-database
*Jan 17 01:47:15.101: LISP-0: Local EID IID 4098 prefix 10.2.1.99/32, Setting state to
active (state: inactive, rlocs: 0/0, sources: NONE).
debug lisp control-plane dynamic-eid

*Jan 17 01:47:15.102: LISP-0: Local dynEID Campus_10_2_1_0 IID 4098 prefix 10.2.1.99/32 RLOC
10.2.120.1 pri/wei=10/10, Created (IPv4 intf RLOC Loopback0) (state: active, rlocs: 1/1, sources: dynamic).
debug lisp forwarding data-signal-discover-dyn-eid

*Jan 17 01:47:15.102: LISP-0: DynEID IID 4098 10.2.1.99 [Campus_10_2_1_0:Vlan3000] Created.
Dynamic EID
FE1 RLOC EID
Instance
ID
1e
C
B B
Verification on Control Plane
CP#show lisp site instance-id 4098

Site Name Last Up Who Last Inst EID Prefix
Register Registered ID
site_sjc never no -- 4098 10.2.1.0/24
3d23h yes# 10.2.120.1 4098 10.2.1.99/32
FE1 RLOC Instance EID

ID
Client
Registration
1d 1f
Check Check
local registr
Enable LISP
1a LISP
debugs on FE msg to
DB CP?
Is ?
MAC
Silent Host
learnt
?
LISP process issue
on FE
1e 1g
Check
Check registr
1c 1b LISP
Client
Registration msg
DB on from
COMPLETE
If CP ? FE?
Check ARP
IPDT table
table ? has an
entry ?
Packet dropped on
FE
or on CP
Client
didnt get IP
Registration Message flow
C Client send ARP, DHCP or DATA
1
pkt
1 2 FE saves the host info in local

database. Send the registration
message to CP (Map–server)
2
3 3 CP receives the registration

message saves the host tracking
database and send the reply
1f
C
Check if FE has sent the registration B B
message ?
debug lisp control map-request
*Jan 17 01:56:01.045: LISP: Send map request for EID prefix IID 4098 10.2.1.99/32
debug lisp forwarding data-signal-map-request

*Jan 17 01:56:02.204: LISP-0: EID-AF IPv4, Sending map-request from 10.2.1.99 to 10.2.1.99 for EID
10.2.1.99/32, ITR-RLOCs 1, nonce 0x0B5B0D11-0x5110DF55 (encap src 10.2.120.1, dst 10.2.100.1).
FE1 RLOC Control

Plane
1g
C
B B
Verification for registration message

debug lisp control-plane map-server-registration
*Jan 17 01:57:27.716: LISP-0: MS EID IID 4098 prefix 10.2.1.99/32 site site_sjc, Forwarding map request to
ETR RLOC 10.2.120.1 C
FE1 RLOC B B
debug lisp forwarding eligibility-process-switching

*Jan 17 01:56:02.209: LISP: Processing received Map-Reply(2) message on
TenGigabitEthernet1/0/1 from 10.2.100.1:4342 to 10.2.120.1:4342
B B
Control
Plane
Case: 3 - Host Resolution
CP
C
B B router lisp
site site_sjc
10.2.100.1 ...
exit
10.2.120.1 10.2.120.3 4098 10.2.1.0/24
FE1 FE3 router lisp

...
10.2.1.99 10.2.1.89 eid-table Campus instance-id 4098
exit
Host
Resolution
2c
Check
on CP Case 2
?
2a
Check
Map
Complete cache 2e
on
FE1 ? Check
LISP process issue
on
on CP
FE1 ?
2d
2b Check
LISP process issue
on
on CP
Check FE3 ?
LISP process issue debug
on FE ON LISP process issue
FE1 ? on CP
Host
Resolution
2c
Check
on CP Case 2
?
2a
Check
Map
Complete cache 2e
on
FE1 ? Check
LISP process issue
on
on CP
FE1 ?
2d
2b Check
LISP process issue
on
on CP
Check FE3 ?
FE1 ? on CP
2a C
B B
Map cache entry on FE1

FE1#show ip lisp map-cache instance-id 4098
LISP IPv4 Mapping Cache for EID-table vrf Campus (IID 4098), 5 entries
10.2.1.89/32, uptime: 00:05:16, expires: 23:57:59, via map-reply, complete

Locator Uptime State Pri/Wgt
10.2.120.3 00:04:23 up 10/10
FE3 RLOC
Host 2 EID
Map Request Message flow
C 1 A client wants to establish
communication to a Host2
2 No local map-cache entry Host2
on FE1. Map-Request is sent to
1 the CP(Map-Resolver)
3 CP(Map Server) forwards the original
2 Map-Request to the FE3(ETR) that
last registered the EID subnet
3\ 3 4 FE3(ETR) sends to the FE1(ITR) a

Map-Reply containing the
requested mapping information
4
5 FE1(ITR) installs the mapping
information in its local map-cache
5
2b C
Verify map-request messages sent to the fabric B B
control-plane ?
*Jan 18 16:12:57.741: LISP: Send map request for EID prefix IID 4098 10.2.1.89/32
debug lisp forwarding data-signal-map-request

*Jan 18 16:12:57.610: LISPdata-signal: sending signal for 10.2.1.99 ->10.2.1.89 on in
IPv4:Campus
Host1
debug lisp forwarding eligibility-process-switching EID
*Jan 18 16:12:57.741: LISP-0: EID-AF IPv4, Sending map-request from 10.2.1.89 to 10.2.1.89
for EID 10.2.1.89 /32, ITR-RLOCs 1, nonce 0x0579975B-0x0823B8E4 (encap src 10.2.120.1, dst
10.2.100.1).
Host2
EID
Host
Resolution
2c
Check
on CP Case 2
?
2a
Check
Map
Complete cache 2e
on
FE1 ? Check
LISP process issue
on
on CP
FE1 ?
2d
2b Check
LISP process issue
on
on CP
Check FE3 ?
FE1 ? on CP
2c C
B B
Verification on Control Plane ?

site_sjc never no -- 4098 10.2.1.0/24
3d23h yes# 10.2.120.1 4098 10.2.1.99/32
3d23h yes# 10.2.120.3 4098 10.2.1.89/32
debug lisp control map-server-map-request

*Jan 18 16:15:27.529: LISP: Received map request for IID 4098 10.2.1.89/32, source_eid IID
4098 10.2.1.99, ITR-RLOCs: 10.2.120.1, records 1, nonce 0x0579975B-0x0823B8E4
*Jan 18 16:15:27.529: LISP-0: MS EID IID 4098 prefix 10.2.1.89/32 site site_sjc,
Forwarding map request to ETR RLOC 10.2.120.3.
FE3 RLOC
FE1 RLOC
2d
Verify map-request forwarded to the fabric B
C
edge?
Jan 18 16:12:58.531: LISP: Received map request for IID 4098 10.2.1.89/32, source_eid IID
4098 10.2.1.99, ITR-RLOCs: 10.2.120.1, records 1, nonce 0x0579975B-0x0823B8E4
Jan 18 16:12:58.531: LISP-0: Sending map-reply from 10.2.120.3 to 10.2.120.1.
FE3 RLOC FE1 RLOC
Host
Resolution
2c
Check
on CP Case 2
?
2a
Check
Map
Complete cache 2e
on
FE1 ? Check
LISP process issue
on
on CP
FE1 ?
2d
2b Check
LISP process issue
on
on CP
Check FE3 ?
FE1 ? on CP
2e C
B B
Verify map-reply recevied from FE 3?

*Jan 18 16:12:57.748: LISP: Processing Map-Reply mapping record for IID 4098
10.2.1.89/32, ttl 1440, action none, authoritative, 1 locator 10.2.120.3 pri/wei=10/10 LpR
It is the same sequence if border is requesting
C
B B
Map Cache
10.2.1.99/32,
Locator 10.2.120.1
Local Database
10.2.1.99/32,
Locator 10.2.120.1
Recap

Host entry missing in
Control Plane node Case 2: Host Registration

Map cache entry
missing Case 3: Host Resolution
Case: 4 - External Connectivity
CP
C
router lisp
site site_sjc
...
B B eid-prefix instance-id 4098 10.2.1.0/24 accept-more-specifics
exit
10.2.100.1
BDR
10.2.100.2 router lisp
encapsulation vxlan
!
eid-table default instance-id 4098
map-cache 10.2.1.0/24 map-request exit
10.2.120.3
router lisp
FE3 ...
eid-table Campus instance-id 4098
10.2.1.89 database-mapping 10.2.1.0/24 locator-set campus_fabric
exit
Client
Traffic
Outside
3d
BDR
has
Look at routing
route
3a to
config for external
routes
dst IP
Is
Case Client
?
in
2 CP ?
3e
Is src
and
3b 3c dst in
Check if VRF
Leaking is working
Check BDR same
map has VRF?
cache entry
entry for
On client
FE3 ? ip ?
Either packet
Dropped in FE or
CP
Case
3
Client
Traffic
Outside
3d
BDR
has
Look at routing
route
3a to
config for external
routes
dst IP
Is
Case Client
?
in
2 CP ?
3e
Is src
and
3b 3c dst in
Check if VRF
Leaking is working
Check BDR same
map has VRF?
cache entry
entry for
On client
FE3 ? ip ?
Either packet
Dropped in FE or
CP
Case
3
3a
C
Verification on Control Plane

site_sjc never no -- 4098 10.2.1.0/24
3d23h yes# 10.2.120.3 4098 10.2.1.89/32
B B
3b
Verification at the FE
32.0.0.0/4, uptime: 00:01:30, expires: 00:00:21, via map-reply, forward-native

Encapsulating to proxy ETR
B B
Encapsulation prefix
DST IP not in EID spaces
DST: 40
.1.1.40
0 0 1 0 1 0 0 0
Map-Server sends the shortest

prefix that satisfies the DST ip
32.0.0.0/4
0 0 1 0 0 0 0 0
128 64 32 16 8 4 2 1
Encapsulation prefix condt..
10.2.1.0/24 is the EID spaces and
hosts up till 10.2.1.99 have joined
DST:10.2.1. 200
1 1 0 0 1 0 0 0
Map-Server sends the shortest

prefix that satisfies the DST ip
10.2.1.128/25
1 0 0 0 0 0 0 0
128 64 32 16 8 4 2 1
3c B
Verification at the Border

BDR#show ip lisp map-cache instance-id 4098

10.2.120.3 00:04:23 up 10/10
B B
Client
Traffic
Outside
3d
BDR
has
Look at routing
route
3a to
config for external
routes
dst IP
Is
Case Client
?
in
2 CP ?
3e
Is src
and
3b 3c dst in
Check if VRF
Leaking is working
Check BDR same
map has VRF?
cache entry
entry for
On client
FE3 ? ip ?
Either packet
Dropped in FE or
CP
Case
3
Types of Border Node
• Border Node is an entry & exit point for all data traffic going in & out of the Fabric
There are 2 Types of Border Node!
• Internal Border based on XTR + Subnets

• “Known” Routes use Internal Border
Known Unknown
Networks Networks
B B
• External Border based on PXTR
• “Unknown” Routes use External Border
In case of Internal Border
Verify the routes that are being imported
Internal-DBR#show ip lisp route-import map-cache instance 10

LISP IPv4 imported routes for EID-table vrf PACAF (IID 10)
Config: 1, Entries: 7 (limit 1000)
Prefix Uptime Source RLOC-set Cache/DB State
10.1.18.0/24 21:59:17 bgp 65002 installed
10.1.100.1/32 21:59:17 bgp 65002 installed
100.1.1.0/24 21:59:17 bgp 65002 installed
101.1.1.0/24 21:59:17 bgp 65002 installed
192.168.111.0/24 21:59:17 bgp 65002 installed
192.168.206.0/24 21:59:17 bgp 65002 installed
Case: 5 - East West Traffic
CP
C
router lisp
site site_sjc
B B ...
exit
router lisp
...
exit
FE1 FE3
router lisp
...
Host1 Host2 database-mapping 10.2.1.0/24 locator-set campus_fabric
exit
Client
Traffic
Inside
4b 4c
Check Check
Get the RLOC ip for
LISP LISP
SRC and DST
map Case map
cache cache
on 3 on
FE1 ? FE3 ?
4a
Check
Case if both
IP
2 are in
Find out where the
CP ?
packet is getting
Dropped
Client
Traffic
Inside
4b 4c
Check Check
Get the RLOC ip for
LISP LISP
SRC and DST
map Case map
cache cache
on 3 on
FE1 ? FE3 ?
4a
Check
Case if both
IP
2 are in
Find out where the
CP ?
packet is getting
Dropped
4a
C
Verification on Control Plane ?

site_sjc never no -- 4098 10.2.1.0/24
2d05h yes# 10.2.120.1 4098 10.2.1.99/32
2d02h yes# 10.2.120.2 4098 10.2.1.89/32
4d02h yes# 10.2.120.2 4098 10.2.1.88/32
If any of Host IP are missing.
Run Host Registration

flow (Case 2).
Client
Traffic
Inside
4b 4c
Check Check
Get the RLOC ip for
LISP LISP
SRC and DST
map Case map
cache cache
on 3 on
FE1 ? FE3 ?
4a
Check
Case if both
IP
2 are in
Find out where the
CP ?
packet is getting
Dropped
Verification at the FEs
4b FE1#show ip lisp instance-id 4098 database 10.2.120.1
10.2.1.99/32, locator-set rloc_021a8c01-5c45-4529-addd-b0d626971a5f

10.2.120.3 00:00:06 up 10/10
4c FE3#show ip lisp instance-id 4098 database 10.2.120.3


10.2.120.1 00:00:06 up 10/10
Case: 6 - Host Mobility
CP
C
B B
FE1 FE2 FE3
Host1 Host2
Map Request Message flow 1 Host1 moves from FE1 to FE2
C 2 FE2 saves the host info in local
database. Send the registration
message to control plane
3 The Map-Server adds to the
1 database the entry for the
specific EID, associated to the
RLOCs
2
4 The Map-Server sends a Map-
Notify message to the last FE1
3 that registered the 10.2.1.99/32
prefix
4 5 FE1 receives the Map-Notify
message from the CP and adds
route associated to the 10.2.1.99
5 EID to away table
Verification at the FEs
FE1#show ip lisp away instance-id 4098
LISP Away Table for router lisp 0 (Campus) IID 4098
Entries: 1
Prefix Producer
Host EID
10.2.1.99/32 local EID
FE2#show ip lisp instance-id 4098 database

FE2

10.2.120.1 00:00:06 up 10/10
FE1
1 The LISP process on FE1
receiving the first data packet
Map Request Message flow creates a control plane message
SMR and sends it to the remote
FE3(ITR) that generated the packet
C
2 Send a new Map-Request for the
desired destination (10.17.1.99) to
the Map-Server
1 3 Map-Request is forwarded by
the Map-Server to the FE2 that
registered last the /32 EID
address
2 4 FE2 replies with updated
mapping information to the
3 remote FE3
5 FE3 updates the information in
its map-cache, adding the
4 specific /32 EID address
5 associated to the xTRs deployed
in the East site (10.2.120.1 and
5
10.2.120.2)
Locator/ID Separation Protocol (LISP) Internet
Groper – “lig”
FE1#lig 18.18.18.18 instance-id 4098
Mapping information for EID 18.18.18.18 from 172.16.1.2 with RTT 7 msecs
10.2.120.4 00:00:00 up 10/10
FE1#lig self instance-id 4098

10.2.1.40/32, uptime: 00:00:00, expires: 23:59:59, via map-reply, self, complete
10.2.120.2 00:00:00 up, self 10/10
FE1#lig 17.17.17.17 instance-id 4098

16.0.0.0/4, uptime: 00:00:00, expires: 00:14:59, via map-reply, forward-native
Encapsulating to proxy ETR
Troubleshooting
Where do I begin?
Data Plane
Key Components – VXLAN
VXLAN based Data-Plane
ORIGINAL
ETHERNET IP PAYLOAD
PACKET
Supports L3
Overlay
PACKET IN
ETHERNET IP UDP LISP IP PAYLOAD
LISP
Supports L2
& L3 Overlay
PACKET IN
VXLAN
LISP & VXLAN Headers
Similar Format - Different Payload
LISP Header - IP based VXLAN Header - Ethernet based
OUTER
HEADER
4789
OVERLAY
HEADER
INNER
HEADER
VXLAN Header
Next-Hop MAC Address
Src VTEP MAC Address

Dest. MAC 48
MAC-in-IP Encapsulation
Source MAC 48
VLAN Type 14 Bytes

16 IP Header
0x8100 (4 Bytes Optional) 72
Misc. Data
VLAN ID 16
Protocol 0x11 (UDP) 8
Ether Type
16 Header
0x0800 16 20 Bytes
Outer MAC Header
Underlay
Checksum
Source IP 32
Src RLOC IP Address
Outer IP Header Dest. IP 32
Source Port 16 Dst RLOC IP Address
UDP Header Dest Port 16

8 Bytes Hash of inner L2/L3/L4 headers of original frame.
UDP Length 16 Enables entropy for ECMP load balancing.
VXLAN Header
Checksum 0x0000 16 UDP 4789
Inner (Original) MAC Header

Allows 64K
VXLAN Flags possible SGTs
Inner (Original) IP Header RRRRIRRR
8
Overlay
Segment ID 16
8 Bytes
Original Payload VN ID 24
Allows 16M
Reserved 8 possible VRFs
Packet Flow in Fabric
Encapsulation Decapsulation
IP Network
Edge Node 1 Edge Node 2
VXLAN VXLAN
VN ID SGTag VN ID SGTag
• Verify VN ID matches with

instance ID
• Verify SGT policy
What to look for in packet capture?
Frame 1: 192 bytes on wire (1536 bits), 192 bytes captured (1536 bits)
Ethernet II, Src: CiscoInc_c5:db:47 (88:90:8d:c5:db:47), Dst: CiscoInc_5b:58:fb (0c:f5:a4:5b:58:fb)
Internet Protocol Version 4, Src: 10.2.120.1, Dst: 10.2.120.3
User Datagram Protocol, Src Port: 65354 (65354), Dst Port: 4789 (4789)
Source Port: 65354 OUTER
Destination Port: 4789 HEADER
Length: 158
Checksum: 0x0000 (none)
[Stream index: 0]
Virtual eXtensible Local Area Network

Flags: 0x0800, VXLAN Network ID (VNI) OVERLAY
Group Policy ID: 0 HEADER
VXLAN Network Identifier (VNI): 4098
Reserved: 0
Ethernet II, Src: CiscoInc_c5:00:00 (88:90:8d:c5:00:00), Dst: ba:25:cd:f4:ad:38 (ba:25:cd:f4:ad:38)

Destination: ba:25:cd:f4:ad:38 (ba:25:cd:f4:ad:38) INNER
Source: CiscoInc_c5:00:00 (88:90:8d:c5:00:00) HEADER
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.2.1.89, Dst: 10.2.1.99
Internet Control Message Protocol
Packet in Wireshark
Underlay MTU
FE1#ping 10.2.120.3 source 10.2.120.1 size 1500 df-bit
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.2.120.3, timeout is 2 seconds:
Packet sent with a source address of 10.2.120.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 3/3/5 ms
FE1# C
FE1#ping 10.2.120.3 source 10.2.120.1 size 1501 df-bit
Type escape sequence to abort. B B
.....
Success rate is 0 percent (0/5)
FE1#
Configure jumbo MTU on the devices

participating in underlay connectivity
Default route in Underlay
If you have this config
router lisp
...
eid-table default instance-id <>
DON’T
...
LISP will consume all packets hitting default entry (0.0.0.0/0)

It is recommended that you create a Default vrf and configure
router lisp
...
eid-table vrf Default instance-id <>
DO
...
Overlay EID Loopback
CP
C
router lisp
site site_sjc
...
B B eid-prefix instance-id 4098 20.20.20.20/32
eid-prefix instance-id 4098 21.21.21.21/32
exit
router lisp
...
interface Loopback20
ip vrf forwarding Campus
ip address 20.20.20.20 255.255.255.255
FE1 FE3
router lisp
...
Host1 Host2
interface Loopback21
ip address 21.21.21.21 255.255.255.255
Fabric Edge Loopback Ping Test
FE1#ping vrf Campus 20.20.20.20 source 21.21.21.21 size 1500 df-bit
Type escape sequence to abort.
..!!!
Success rate is 60 percent (3/5), round-trip min/avg/max = 3/3/5 ms
FE1# C
B B
Initial packets get dropped until Host Resolution is
complete
Embedded Packet Capture
FE#monitor capture lispcap interface te 1/0/1 both match any
limit file location flash:lispcap
FE#show monitor capture file flash:lispcap

Starting the packet display ........ Press Ctrl + Shift + 6 to exit
1 0.000000000 172.16.1.2 -> 10.2.110.2 UDP 124 Source port: 65357 Destination port: vxlan
3 0.114937000 172.16.1.1 -> 224.0.0.10 EIGRP 74 Hello
9 2.202622000 10.2.200.2 -> 172.16.1.2 ICMP 70 Destination unreachable (Port unreachable)
Troubleshooting
Where do I begin?
Data Plane
Cisco TrustSec
Simplified segmentation with Group Based Policy
Enforcement Shared Application

Group Based Policies Services Servers
Enforcement DC Switch
or Firewall
Propagation
Carry “Group” context
through the network Enterprise
using only SGT Backbone
ISE
Classification
Static or Dynamic Campus Switch Campus Switch DC switch receives policy
for only what is connected
SGT assignments
Employee Tag
Supplier Tag
Non-Compliant Employee Voice Voice Employee Supplier Non-Compliant Non-Compliant Tag
VLAN A VLAN B
Two ways to assign SGT
Dynamic Classification Static Classification
L3 Interface (SVI) to SGT L2 Port to SGT
Campus
Access Distribution Core DC Core DC Access
MAB Enterprise
Backbone
WLC Firewall Hypervisor SW
VLAN to SGT Subnet to SGT VM (Port Profile) to SGT
Verification on FEs
FE1#show authentication sessions mac 0050.5694.d054 details
Interface: GigabitEthernet1/0/2
IIF-ID: 0x100CBC000000088
MAC Address: 0050.5694.d054
IPv6 Address: Unknown
IPv4 Address: 10.2.1.99
User-Name: joe
Status: Authorized
Domain: DATA
Oper host mode: multi-auth
Oper control dir: both
Session timeout: N/A
Restart timeout: N/A
Session Uptime: 28127s
Host EID
Common Session ID: 0A04010300000FB00003640C
Acct Session ID: 0x00000FA5
Handle: 0x98000003
Current Policy: POLICY_Gi1/0/2 VLAN
Local Policies:
Service Template: DEFAULT_LINKSEC_POLICY_SHOULD_SECURE (priority 150)
Server Policies:
Vlan Group: Vlan: 3000
SGT Value: 5 SGT Tag
Method status list:
Method State
dot1x Authc Success Auth type
Ingress Classification with Egress Enforcement
Destination Classification
CRM: SGT 20
Web: SGT 30
User Authenticated = FIB Lookup =
Classified as Marketing (5) Destination MAC = SGT 20
Cat3850 Cat6800 Cat6800 Nexus 7000 Nexus 5500 Nexus 2248

CRM
Enterprise DST: 10.1.100.52
5 Backbone 5
SRC: 10.2.1.99 SGT: 20
DST: 10.2.1.52
SRC: 10.1.10.220 SGT: 5 Web
DST: 10.1.200.100
Egress SGT: 30
Enforcement
(SGACL)
WLC5508
DST  CRM Web
 SRC (20) (30)
Marketing (5) Permit Deny
BYOD (7) Deny Permit
Verification on FEs
FE1#show cts role-based sgt-map 10.2.1.99/32
Active IPv4-SGT Bindings Information
IP Address SGT Source
============================================
10.2.1.99 5 L3IF
FE2#show cts role-based sgt-map 10.1.200.100/32

Active IPv4-SGT Bindings Information
IP Address SGT Source
============================================
10.1.200.100 30 L3IF
FE2#show cts role-based permissions

IPv4 Role-based permissions from group 5 to group 30 (configured):
denyip
RBACL Monitor All for Dynamic Policies : FALSE
RBACL Monitor All for Configured Policies : FALSE
Cisco Trust Security (CTS)
• Suggested Reading:
• BRKCRS-2891 - Enterprise Network Segmentation with Cisco TrustSec
• BRKSEC-2203 - Intermediate - Enabling TrustSec Software-Defined Segmentation
• BRKSEC-2044 - Building an Enterprise Access Control Architecture Using ISE and TrustSec
• Other References:
• Radius and AAA troubleshooting
cisco.com/c/en/us/td/docs/storage/san_switches/mds9000/sw/rel_3_x/troubleshooting/guide/trblgd/ts_aaa.pdf
• Cisco TrustSec Troubleshooting https://communities.cisco.com/docs/DOC-69479#jive_content_id_Debugging
• Cisco TrustSec Config Guide cisco.com/c/en/us/td/docs/switches/lan/trustsec/configuration/guide/trustsec.html
• CTS Architecture Overview cisco.com/docs/switches/lan/trustsec/configuration/guide/trustsec/arch_over.html
• CTS 2.0 Design Guide cisco.com/td/docs/solutions/Enterprise/Security/TrustSec_2-0/trustsec_2-0_dig.pdf
• Fundamentals of TrustSec https://www.youtube.com/watch?v=78-GV7Pz18I
Monitoring
What do I gain ?
Data Plane
Control Plane Monitoring
• CLI
C
•Switch#show
YANG Modelslisp
Router-lisp ID: 0
B B Locator table: default
EID instance count: 3
Monitor via CLI Router# show ip lisp statistics
LISP Statistics - last cleared: never
Control Packets:
Map-Requests in/out: 76/35
Encapsulated Map-Requests in/out: 76/35
• Show lisp RLOC-probe Map-Requests in/out: 0/0
Map-Reply records in/out: 35/76
• Show ip lisp statistics Authoritative records in/out: 0/76
• Show ip lisp database Non-authoritative records in: 35
Negative records in: 35
• Show ip lisp map-cache RLOC-probe records in/out: 0/0
Map-Registers out: 626
• Show ip lisp route-import Errors:
Map-Request format errors: 0
Map-Reply format errors: 0
Map-Reply spoof alerts: 0
Mapping record TTL alerts: 0
Cache Related:
Cache entries created/deleted: 72/69
Number of EID-prefixes in map-cache: 3
Number of negative entries in map-cache: 3
Total number of RLOCs in map-cache: 0
Average RLOCs per EID-prefix: 0
Forwarding:
Number of data signals processed: 35 (+ dropped
0)
Number of reachability reports: 0 (+ dropped 0)
Operation Data Models (ODM)
Interface Model
definition
Interface Model Instances in XML
interface: list, key = name
name: string
speed: string
duplex: string
YANG ODM  Data Models that can be consumed
EID space 1
EID Space Monitoring EID space 2
EID space 3
C 350
300
B B
250
200
Users
150
100
50
0
1 2 3 4 5
Months
Monitoring
What do I gain ?
Data Plane
Flexible Netflow—Input VRF
C
flow record Campus
match ipv4 source address
B match ipv4 destination address
B
match interface input
match routing vrf input
collect timestamp absolute first
collect timestamp absolute last
collect counter packets long
!
flow monitor Campus_mon
record Campus
!
interface Vlan3000
ip address 172.16.2.2 255.255.255.252
ip flow monitor Campus_mon
Monitoring
What do I gain ?
Data Plane
SGACL Counter
FE#show cts role-based counters
Role-based IPv4 counters
From To SW-Denied HW-Denied SW-Permitted HW-Permitted
* * 0 0 4279066 432961
11
6 11 0 0 0 0
8 11 0 435 0 0
4 12 0 0 0 0
6 12 0 0 0 0
Suppliers App Servers
Putting It Together
Where do things go?
Campus Fabric Control-Plane Node
B
10.2.100.1/32 C
10.2.1.99/24 10.2.120.1/32 10.2.120.3/32 10.2.1.89/24
IP Network
10.2.1.0/24 10.2.1.0/24
Host Pool Edge Node 1 Edge Node 3 Host Pool
Control Plane Data Plane Policy Plane

1. Start with Control-Plane Node 1. Enable Packet Capture 1. Check Auth Information
2. Look at Local and Map-Cache DB a. Verify VN-ID and SGT a. Verify Vlan and SGT for
the client
a. at Edge and Border 2. VRF Netflow
2. Verify the IP and SGT binding
3. Verify the Site and EID space
3. Verify the Policy
4. Breakdown into the Cases
4. Policy insight from SGACL
5. Use Data Models for simplified
counter
provisioning
Locator / ID Separation Protocol (LISP)
Suggested Reading:
BRKRST-3045 - LISP - A Next Generation Networking Architecture
BRKRST-3047 - Troubleshooting LISP
BRKCRS-3510 - LISP in Campus Networks
Other References:
Cisco LISP Site http://lisp.cisco.com
Cisco LISP Marketing Site http://www.cisco.com/go/lisp/
LISP Beta Network Site http://www.lisp4.net or http://www.lisp6.net
IETF LISP Working Group http://tools.ietf.org/wg/lisp/
Fundamentals of LISP https://www.youtube.com/watch?v=lKrV1qB8uqA
Agenda
1 Key Benefits
Why do I care?
2 Campus Fabric Overview

What is a Fabric?
3 Getting Started
What are the Platform/Network considerations?
4 Network Deployment Models

Layer-2 Access
5 Wireless
How does Wireless work over Campus Fabric?
6 Takeaway
How do I get started?
Getting Started
Network Considerations
Network Considerations - MTU
MTU 1500 + Encapsulation

MTU 1500
• MTU and Overlay
• VXLAN adds 50 (54) Bytes to the Overlay Network
Original Ethernet Frame
• Avoid Fragmentation by adjusting
the network MTU
Underlay Network
• Ensure Jumbo Frame support on
switches in the underlay network
L3
Underlay Networks L2
• Campus fabric runs over arbitrary topologies: 3-Tier Hierarchical

• Traditional 3-tier hierarchical network
• Collapsed core/aggregation designs
• Routed access L2
• U-topology
Collapsed Core
• Ensure that all switches have IP reachability to
infrastructure elements
L3
• Ideal design is routed access – allows fabric to extend
Routed Access
to very edge of campus network
Strong recommendation to follow campus CVDs with
routed access L2
U-Topology
Overlay Network
• Assumption is underlay network provides routing and IP connectivity
• Campus fabric configuration defines:

• Overlay IP space
• Segmentation context – VRF and SGT
• Mobility (map database updates)
IP Addressing for Overlay and Underlay
• Know your IP addressing and IP
scale requirements
• Best to use single Aggregate for all 10.10.10.254/32 10.10.10.253/32
Underlay Links and Loopbacks 10.10.10.0/30
10.10.10.4/30
• IPv4 only (today) Overlay Network
• Fabric uses Loopback as Source-

Interface for Encapsulation
Underlay Network
10.10.10.252/32
Virtual Networks
• RLOC/Underlay connectivity in Global
Routing Table
• Loopback interfaces for management in
their own VN (Default) Fabric scope of management
USERS #2
• Other VNs can be used for segmentation
Border
for users, devices, roles, and others USERS #1
• Scalable Group Tags (SGTs) can be used Management Access

USERS*
USERS
for further access control within a VN Default
RLOC/Underlay GRT
• The CORPORATE VN is being shown in
this slide deck as an example.
• Similar steps can be followed for other
VNs shown
Getting Started
Services Location Considerations
Location of Shared Services Infrastructure
• Campus fabric leverages traditional infrastructure services
• IP reachability from underlay/overlay to DNS, DHCP, etc. required
• Services may be hosted inside or outside the campus fabric
• Other infrastructure services include AAA, LDAP/AD, syslog server, Netflow
collector, 3rd-party monitoring systems
DHCP NTP
Server Server
Location of Shared Services Infrastructure
• Could be in campus distribution block or campus core for small commercial or
enterprise deployments
• Larger deployments have infrastructure services hosted in Data Center
• Hybrid model also possible (mix of distribution/core/Data Center)
Infrastructure
Services Infrastructure
at Core Services
in Data Center
Infrastructure
Services at
Distribution
Small Commercial / Enterprise

Deployment
Large Enterprise
Deployment
Know What is Connecting to the Existing Network
• Deploy ISE
• Turn on device sensor on switches
• Turn on profiling on ISE
• This provides visibility into what types of endpoints are connecting into the
network
• Also provides data on from which part of the network are they connecting from
• This data will be useful in determining Segmentation policy in Campus Fabric
Deployments
Deployments
• Campus Networks
• Branch Networks
Campus Network
DDI
MPLS MPLS I-NET
Branch IWAN DC IWAN Internet
DC Internet
WAN Block
Block Block
Services Block
Super
Core
Core Core
Aggregation Aggregation Aggregation

Layer Layer Layer
Branch Network
MPLS I-NET
DDI
Branch IWAN
Collapsed
Core
Access
Layer
Approaches to Migration
• Parallel Install
• Migrating One Switch at a time
Parallel Install Option
Conditions and Advantages
• May work in Branch deployments
• Sufficient cable runs exist in the current networking plan
• Sufficient power and outlets exist in the current power plan
• Existing brownfield network has legacy hardware
• Upgrade most of the wired network
• Option of redesigning IP networks from scratch instead of continuing the complexities of
legacy network
• Advantage lies in testing users on entire new network prior to full migration of entire site
• During migration, users with problems but immediate access needs can be moved back to
old network allowing them to continue their work, while troubleshooting can be performed
on the SDA network
Migrate One Switch At A Time Option
Conditions and Advantages
• Works in both Campus and Branch deployments
• Needs an extra couple fiber runs to the distribution switch
• Sufficient power and couple outlets needed in the current power plan
• Existing brownfield network has legacy hardware
• Upgrade some of the wired network
• Switch by Switch upgrade of certain layers of the network is possible
• Legacy IP design has to be continued for reducing downtime
• During migration, users with problems but immediate access needs can be moved back to
old network allowing them to continue their work, while troubleshooting can be performed
on the SDA network
Parallel Install Option for Campus Networks
DDI
MPLS MPLS I-NET
Parallel Network Option for Branch Networks
MPLS I-NET
DDI
Branch IWAN
Hardware Refresh – Hardware Reconfigure
• Two scenarios for migration to Campus Fabric

• Hardware Refresh: Existing network consists of switches that need hardware
upgrade since they do not support Campus Fabric
• Example: 3750X, 2960X, 4500E SUP7-E in the access
• Hardware Reconfigure: Existing network consists of switches that are
compatible with Campus Fabric and just need software upgrade and
reconfiguration
• Example: 3850, 4500E SUP-8E in the access
Access Network Designs
Access Networks Designs
• Multi-layer L2 Access – Will also address hardware refresh scenario
Layer-2 Access Network
DDI
MPLS MPLS I-NET
DC Internet
WAN Block
Block Block
Services Block
Super
Core
Core Core
Aggregation Aggregation Aggregation

Layer Layer Layer
Connecting the Fabric External Border
• Strong desire not to touch the Core layer in the existing network
• Current Core platform does not support Fabric functionality
• Add a Border platform switch and connect it to the Core layer
• Choose a platform that will be re-purposed to a dedicated Control Plane Node (if
needed)
• Current Core platform supports Fabric External Border functionality

• Convert one of the Core switches as External Border
• In this example, we will add a Fabric External Border switch and connect it to the
Core layer
Connecting the first Fabric Edge
• Depends on across which layer in the network the VLANs are being spanned –
• Aggregation
• Core
• Or sometimes even SuperCore
• The Fabric Edge switch connects to where the VLANs are being aggregated
• Example – If VLANs are NOT being spanned across Core layer, connect first
Fabric Edge switch at Aggregation; if the VLANs ARE being spanned across
Aggregation layer, connect the first Fabric Edge switch at Core, and so on.
• In this example, we will assume that VLANs are being spanned across Access
layer, so Fabric Edge switch is attached to the aggregation switch.
Simplified View
192.168.200.254/32 192.168.200.1/32
C
IP Network
Edge Node Border/Control External Network
Plane Node
• Access switch as the Fabric Edge node

• Intermediate network reduced to IP Network
• Fabric External Border node is the Router connecting to rest of the network
• Control Plane node forms the Control Plane of the Campus Fabric.
Getting Started Steps
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
• Connect a switch to the Core layer that will act as the External Border
• Host the Control Plane function on the External Border for simplicity
• Add a switch in the access layer that will act as the Fabric Edge
• Integrate the switch in the existing network in Routed Access design.
• IS-IS is the recommended option for Fabric networks, but any IGP could do.
• APIC-EM PnP can be used for Day Zero operations to integrate the switch.
Layer-2 Access Network – Simplified View
DDI
MPLS MPLS I-NET
Prepping the Switch
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
• Do not forget to set following on the Edge node and other nodes in the underlay:
• Set MTU to 9100 on the switch and the existing network.
• Configure ‘ip routing’
• Set ‘username’ and ’password’ for device access
• Configure VTY and console lines for device access
• Configure NTP
• Configure SNMP, syslog
• Configure Loopback0 (/32) for RLOC, another interface for Management and underlay
IP addresses
Fabric Configuration on Edge node
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
router lisp
encapsulation vxlan
locator-set rloc_SJC18
exit
!
disable-ttl-propagate
ipv4 sgt
ipv4 use-petr 192.168.200.1
ipv4 itr map-resolver 192.168.200.1
ipv4 itr
ipv4 etr map-server 192.168.200.1 key cisco
ipv4 etr
exit
Border and Control Plane Configuration
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
router lisp
encapsulation vxlan
locator-set border
IPv4-interface Loopback0 priority 10 weight 10 router lisp
exit site site_uci
! authentication-key cisco
disable-ttl-propagate exit
ipv4 map-server ipv4 map-server
ipv4 map-resolver ipv4 map-resolver
ipv4 sgt exit
ipv4 proxy-etr
ipv4 proxy-itr 192.168.200.1
ipv4 itr map-resolver 192.168.200.1
ipv4 etr map-server 192.168.200.1 key cisco
ipv4 etr
exit
VRF Configuration on Edge and Border
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
ip vrf CORPORATE
rd 1:1
Configure L2 VLAN and SVI at Edge Node
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
vlan 3
name Corporate_Users
!
ip dhcp snooping
ip dhcp snooping vlan 3 interface Vlan3
! ip vrf forwarding CORPORATE
device-tracking tracking ip dhcp relay source-interface Loopback0
ip address 10.2.3.254 255.255.255.0
ip helper-address global 10.1.5.252
no ip redirects
ip local-proxy-arp
ip route-cache same-interface
logging event link-status
load-interval 30
lisp mobility CORPORATE_10_2_3_0
shutdown
Adding EID space on Edge node
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
router lisp
locator-set rloc_SJC18_01
eid-table vrf CORPORATE instance-id 10
dynamic-eid CORPORATE_10_2_3_0
database-mapping 10.2.3.0/24 locator-set rloc_SJC18
exit
Adding EID space on Border/Control Plane node
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
router lisp
map-cache 10.2.3.0/24 map-request
exit
!
site site_uci
authentication-key cisco
exit
Exporting Fabric Prefixes to External Network
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
• Only export Fabric prefixes (overlay) to the External network

• No need to import External prefixes into Fabric since Border acts as default to
unknown destinations
• External network needs a route to direct traffic back to the Fabric prefixes.
• Recommended choice of exchanging routing information is BGP
Advertising Fabric Prefixes to External Network - OSPF
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
router lisp • Export LISP into RIB (Routing Info Base)

! • Enable Map-Server lookup for fabric prefixes
ipv4 route-export site-registrations • Redistribute LISP prefixes into OSPF
exit
• Use route-filter in the global instance to filter
! incoming fabric prefixes routes
router ospfv3 123 • This will prevent underlay from learning fabric
!
address-family ipv4 unicast vrf CORPORATE prefixes
summary-prefix 10.2.3.0/24
redistribute lisp metric 10
exit-address-family
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
External routing protocol – in this example OSPFv3 interface Vlan4090

ip vrf forwarding CORPORATE
ip address 192.168.1.253 255.255.255.0
ip ospf network point-to-point
ip ospf mtu-ignore
ipv6 enable
ospfv3 123 ipv4 area 0
end
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
OSPFv3 in Global Routing Table on Fusion Router interface GigabitEthernet0/0/4.4090

encapsulation dot1Q 4090
ip address 192.168.1.254 255.255.255.0
ip ospf network point-to-point
Advertises Fabric prefixes to rest of the network ip ospf mtu-ignore
ipv6 enable
ospfv3 123 ipv4 area 0
end
!
router ospfv3 123
!
address-family ipv4 unicast
exit-address-family
Advertising Fabric Prefixes to External Network - BGP
192.168.200.254/32 192.168.200.1/32
C
IP Network
Plane Node
router lisp
!
exit
!
router bgp 65001
address-family ipv4 vrf CORPORATE
exit
Why BGP?
• BGP has built-in loop prevention features like AS_PATH to break loops
• Simple to keep routes distributed between Global Routing and Virtual Networks
• If IGP is used then route-maps, distribute-lists, IP ACLs need to be maintained
• Failure to maintain the above might cause routing loops in the network
Layer-2 Connection from Existing Network
192.168.200.254/32 192.168.200.1/32
Edge Node Distribution
Switch
Layer-2 C
connection
between existing
VLAN and VLAN IP Network
in Fabric
Border/Control External Network
Plane Node
• Connect the Edge node and existing Distribution switch on a Trunk Port
• Allow only VLAN003 for now
192.168.200.254/32 192.168.200.1/32
Switch
Layer-2 C
connection
between existing
in Fabric
X
Border/Control
SVI Plane Node
External Network
VLAN003
• Shut down the SVI of VLAN003 on Aggregation switches in existing network.
192.168.200.254/32 192.168.200.1/32
Switch
Layer-2 C
connection
between existing
in Fabric
Border/Control
SVI Plane Node
External Network
VLAN003
• No shutdown on the SVI VLAN3 on Fabric Edge switch.
192.168.200.254/32 192.168.200.1/32
Switch
Layer-2 C
connection
between existing
in Fabric
Border/Control
SVI Plane Node
External Network
VLAN003
L2 Network
• VLAN003 gets integrated into the fabric.

• All ingress traffic from endpoints in VLAN003 now enters the fabric via the Edge
node and exits via the Border node.
192.168.200.254/32 192.168.200.1/32
Switch
Layer-2 C
connection
between existing
in Fabric
Border/Control
SVI Plane Node
External Network
VLAN X
L2 Network
• Perform similar configuration of other VLANs, and SVIs on the Fabric Edge node
• Shutdown the SVI of the other VLANs in existing Distribution switches
• No shutdown the respective SVI on Fabric Edge to funnel all VLAN traffic to it
192.168.200.2/32 192.168.200.1/32
Switch
C
IP Network
Plane Node
• Add a new Fabric Edge switch in the access layer

• Connect it to the Distribution layer with Routed Access with its own Loopback0
• Copy the Fabric Edge configuration from previous Fabric Edge including the
VLAN X/SVI X configuration as is, and paste onto the new Fabric Edge switch
192.168.200.2/32 192.168.200.1/32
Switch
C
IP Network
Plane Node
X
• Configure the access ports in their VLANs similar to the legacy switch
• Move all the physical connections from legacy switch to new Fabric Edge
• Decommission the legacy switch from existing network
Add Second External Border/Control Plane node
192.168.200.2/32 192.168.200.1/32
C
IP Network
Plane Node
192.168.200.3/32
Border/Control
Plane Node
• Add or upgrade a second switch or a router as the Border/Control Plane node for
redundancy.
• Modify the configurations on all the Fabric Edge nodes to add the second Border/Control
Plane node.
Migration @ Work – Simplified View
DDI
MPLS MPLS I-NET
Add Internal Border nodes as necessary
192.168.200.2/32 192.168.200.22/32
IP Network
Edge Node Internal Border/s WAN Branch
192.168.200.23/32
Internal Border/s Datacenter WAN
• Add or upgrade Internal Border nodes in the Fabric.
Campus Fabric
Border Nodes
Internal Border: External Border:

• Connects Campus Fabric to Known • Connects Campus Fabric to Un-
networks i.e. other fabric or non- Known networks.
fabric domain in same company
network . • These Un-known networks generally
is the Internet and Cloud.
• These known networks generally
are the WAN, DC, Shared Services • Responsible for only advertising
etc prefixes from the local fabric domain
to external domain.
• Responsible for advertising prefixes
from and to the local fabric domain
and external domain.
Why Internal Border?
• These prevent hair pinning at the External Border node for traffic destined for
known internal destinations like remote branches or datacenter.
• Flexibility in designing different platforms for Border functionality different than
External Border
• Can have any number of Internal borders than External borders (depends on
network design)
Routing on the Internal Borders
192.168.200.2/32 192.168.200.22/32
IP Network
192.168.200.23/32
• Routing needs to be configured on the Internal

Borders to
• Advertise Fabric overlay prefixes outside to the rest of the
network Internal Border/s Datacenter WAN
• Known network prefixes to be redistributed into the fabric
• Use route-filter in the global instance to filter incoming
fabric prefixes routes
• This will prevent underlay from learning fabric prefixes or
VRFs from learning other VRF’s routes
Internal Border Routing – Importing from OSPF in LISP
192.168.200.2/32 192.168.200.22/32
IP Network
Import known external prefixes into Fabric
Increase the administrative distance 250 – so

router chooses OSPFv3 over LISP
router lisp
locator-set int_border
!
ipv4 route-import database ospfv3 123 locator-set int_border
exit
Internal Border Routing – Importing from EIGRP in LISP
192.168.200.2/32 192.168.200.22/32
IP Network
router lisp
!
ipv4 route-import database eigrp 65535 locator-set int_border
exit
Internal Border Routing – Advertise from LISP into OSPF
192.168.200.2/32 192.168.200.22/32
IP Network
router lisp
!
ipv4 map-cache site-registration Summarize exported LISP prefixes
exit
!
router ospfv3 123
! Use distribute-list to filter incoming routes
address-family ipv4 unicast vrf CORPORATE
summary-prefix 10.2.3.0/24
distribute-list 2 in
exit-address-family
Internal Border Routing – Advertise from LISP into BGP
192.168.200.2/32 192.168.200.22/32
IP Network
router lisp
!
exit
!
router bgp 65003
redistribute LISP metric 10
exit
Shared Resources
192.168.200.2/32 192.168.200.22/32
DDI
IP Network
Edge Node Internal Border/s ISE/AD
router lisp
encapsulation vxlan
exit !
ipv4 route-import database eigrp 65535 locator-set border
ipv4 map-cache site-registration exit
Shared Resources
192.168.200.2/32 192.168.200.22/32
DDI
IP Network
router eigrp 65535

!
redistribute lisp metric 10000 1 255 1 9100
network 192.168.2.253 0.0.0.0
autonomous-system 65535
exit-address-family
!
Shared Resources
192.168.200.2/32 192.168.200.22/32
DDI
IP Network
router eigrp 65535

!
network 192.168.2.254 0.0.0.0
exit-address-family
Distribute Control Plane Node from External Border
192.168.200.1/32
C
Control
192.168.200.2/32 Plane
192.168.200.25/32
B
IP Network
Edge Node External
Border/s
C
Control
Plane
router lisp
encapsulation vxlan
192.168.200.3/32 locator-table default
locator-set msmr
exit
!
exit
192.168.200.1/32
C
Control
192.168.200.2/32 Plane
192.168.200.25/32
B
IP Network
Edge Node External
Border/s
C
Control
Plane
site site_uci
description map-server configured from apic-em
192.168.200.3/32 authentication-key uci
exit
!
ipv4 map-server
ipv4 map-resolver
exit
192.168.200.1/32
C
Control
192.168.200.2/32 Plane
192.168.200.25/32
B
IP Network
Edge Node External
Border/s
C
Control
Plane router bgp 65002
bgp log-neighbor-changes
192.168.200.3/32 neighbor 192.168.200.25 update-source lo0
!
address-family vpnv4
• Set up iBGP connection between exit-address-family
the Control Plane node and !
External Border aggregate-address 10.2.3.0 255.255.255.0 summary—only
exit-address-family
192.168.200.1/32
C
Control
192.168.200.2/32 Plane
192.168.200.25/32
B
IP Network
Edge Node External
Border/s
C
Control
Plane router lisp
encapsulation vxlan
locator-set border
192.168.200.3/32 IP-v4-interface Loopback 0 priority 10 weight 10
exit
!
ipv4 route-import map-cache bgp 65002
exit
192.168.200.1/32
C
Control
192.168.200.2/32 Plane
192.168.200.25/32
B
IP Network
Edge Node External
Border/s
C
Control
Plane router lisp
ipv4 proxy-etr
ipv4 proxy-itr 192.168.200.25
192.168.200.3/32 ipv4 itr map-resolver 192.168.200.3
ipv4 itr-map-resolver 192.168.200.1
ipv4 map-server 192.168.200.3 key cisco
ipv4 map-server 192.168.200.1 key cisco
ipv4 etr
exit
192.168.200.1/32
C
Control
192.168.200.2/32 Plane
192.168.200.25/32
B
IP Network
Edge Node External
Border/s
C
Control
Plane router bgp 65002
bgp log-neighbor-changes
192.168.200.3/32 neighbor 192.168.200.1 update-source Loopback0
neighbor 192.168.200.3 update-source Loopback0
!
address-family vpnv4
• Set up iBGP connection between neighbor 192.168.200.1 activate
the External Border and Control neighbor 192.168.200.1 send-community both
Plane nodes neighbor 192.168.200.1 send-community both
exit-address-family
192.168.200.1/32
C
Control
192.168.200.2/32 Plane
192.168.200.25/32
B
IP Network
Edge Node External
Border/s
C
Control
Plane
192.168.200.3/32
• Redistribute BGP into IGP at the external router to advertise fabric prefixes to external
network – as mentioned previously (slides 58-61)
Migration @ Work – Simplified View
DDI
MPLS MPLS I-NET
Internal Borders
External
Borders
Control Control
Plane Node Plane Node
Fabric Edge
Nodes
Replace Legacy Access Switches in the Network
• Use the same procedure outlined in the last three slides (67-68) to add Fabric-
enabled Edge switches
• While replacing legacy switches in the network
• After all the legacy switches in that Distribution block are replaced with Fabric-
enabled Edge switches,
• Remove the Fabric Edge connected to the Distribution switch,
• Use it to migrate the second Distribution block,
• Following the same procedure as outlined previously (61-66).
Migration @ work
DDI
MPLS MPLS I-NET
Internal Borders
External
Borders
Campus Fabric
Wireless
Wireless Deployment models
• Cisco Unified Wireless Network (Centralized Wireless)
• Flex Connect
• Converged Access
Where do I connect WLCs and APs
• WLC connect outside the fabric to Internal Border
• APs can connect to in the overlay EID space in fabric
• Leverage stretched wired subnets to create one VLAN across fabric for all APs
Centralized Wireless and Campus Fabric
192.168.200.2/32 192.168.200.22/32
C
Management IP
192.168.1.253/24
Campus Fabric
Edge Node Internal Border/s
10.1.0.0/20
192.168.1.0/24
• WLCs connect behind Internal Border in the Underlay – still external to Fabric
• Internal Border advertises WLC Management subnet to the Fabric
• Internal Border advertises Fabric prefixes to the WLC Management network
192.168.200.2/32 192.168.200.22/32
C
Management IP
192.168.1.253/24
Campus Fabric
10.2.7.254.1/21
Wireless Clients Subnet
• Wireless SSIDs are mapped to VLAN/Subnet at WLC in the form of dynamic interfaces
• Internal Border advertises Wireless client subnets to the Fabric
192.168.200.2/32 192.168.200.22/32
AP VLAN C
10.1.15.254/20
Campus Fabric
10.1.0.1/20 192.168.200.30/32
AP VLAN
10.1.15.254/20
• Access Points are in overlay space on Fabric Edge switches

Edge Node
10.1.0.2/20 • One subnet for APs across the entire Fabric in Campus
• Simplified IP design for the network
• APs get registered in the Host Tracking Database (HTDB)
running on Control node
192.168.200.2/32 192.168.200.22/32
Management IP
192.168.1.253/24
Campus Fabric
• CAPWAP is built from the AP to the WLC

• When this traffic hits the Fabric Edge switch, it encapsulates CAPWAP in VXLAN and
forwards it to Internal Border
• The outer VXLAN header is removed by the Internal Border, and underlying CAPWAP
packet is forwarded to the WLC
Impact of Multiple Encapsulations to Frame size
ETHERNET 802.11 IP PAYLOAD
ETHERNET IP UDP CAPWAP ETHERNET 802.11 IP PAYLOAD
ETHERNET IP UDP VXLAN ETHERNET IP UDP CAPWAP ETHERNET 802.11 IP PAYLOAD
Centralized Wireless and Campus Fabric: AP Join
192.168.200.2/32 192.168.200.22/32
Management IP
192.168.1.253/24
Campus Fabric
• WLC discovery by AP happens the same as of today.

• Layer-3 CAPWAP, Locally configured Controller IP Address, DHCP Server discovery via
Option 43, DNS Discovery
• AP sends a frame padded to 1485 bytes with DF=1
• Edge encapsulates frame in VXLAN that takes it above 1500 bytes
Centralized Wireless and Campus Fabric: AP Join
192.168.200.2/32 192.168.200.22/32
Management IP
192.168.1.253/24
Campus Fabric
• Fabric Edge drops the packet and sends an ICMP error back to AP
• AP drops frame size to 576 bytes and Joins WLC successfully
• AP tries to find the optimum frame size by stepping up to 1000 bytes, 1300 bytes and 1485
bytes again
• Increase MTU to 9100 of existing network interfaces in the underlay to avoid fragmentation
challenges
192.168.200.22/32 Client VLAN
AP VLAN 10.2.7.254.1/21
10.1.15.254/20
Campus Fabric
Internal Border/s
10.1.0.1/20
10.2.0.1/21
• Clients are authenticated and on-boarded by WLC

• Wireless clients are external to fabric in this case
192.168.200.22/32 Client VLAN
AP VLAN 10.2.7.254.1/21
10.1.15.254/20
Campus Fabric
Wired VLAN Internal Border/s
10.1.0.1/20 10.1.31.254/20
10.2.0.1/21 10.1.16.1/20
• Communication from a wired host in Fabric to Wireless Client outside fabric will occur
through Internal Border – JUST LIKE TODAY!!
• For the fabric, it is a fabric host communicating to a known destination external to the fabric
• Over-The-Top (OTT) Wireless

• Consider increasing MTU on transit switches to prevent fragmentation issues
• Least impact to wireless since fabric is just a transport
• Supports all the APs that are supported by the WLC release software
• Leverage common subnet for AP across campus
• No changes to wireless roaming performance
• All the other features of Wireless such as AVC, Location services, QoS, Bonjour,
mDNS, RRM and others will work EXACTLY like they work today
• Managed by Cisco Prime Infrastructure
Take Away
Session Summary

What to do next?
1. Update your Hardware and Software!

• Catalyst 3650 or 3850 - New IOS-XE 16.3+
• Catalyst 4500 w/ Sup8E - New IOS-XE 3.9+
• Catalyst 6807, 6880 or 6840 - New IOS 15.4SY+
• Nexus 7700 w/ M3 Cards - New NX-OS 7.3.2+
• ASR1000-X or ISR4400 - New IOS-XE 16.4+
2. Try out “Campus Fabric” in your Lab!

• You only need 2 or 3 (+) switches to test this solution
• At least 1 Control-Plane + Border and 1 Fabric Edge
IP Network
3. Trial Deployments (Remember: its an Overlay)
• You can install new C-Plane, Border and Edge Nodes
without modifying your existing (Underlay) network
Campus Fabric CVD on Cisco.com
http://www.cisco.com/c/dam/en/us/td/docs/solutions/CVD/Oct2016/CVD-CampusFabricDesign-2016OCT.pdf
Coming Soon…
Secure, Policy-based Complete Faster Service

Automation Visibility and Assurance Enablement
Policy-based Automated Monitor the entire Wired, Quickly enable services

Network Provisioning across Wireless and WAN network using open APIs across a
ALL network domains. as a Single Entity. Services Ecosystem.
Campus Fabric
Related Sessions
We recommend the following sessions:

1. BRKCRS-1800: DNA Campus Fabric – An Introduction
• 21/02/17 (Tuesday) @ 11:15 – 1.5 hours
2. BRKCRS-3800: DNA Campus Fabric – A Look Under the Hood

• 22/02/17 (Wednesday) @ 09:00– 2 hours
3. BRKCRS-2801: DNA Campus Fabric - How to Integrate with Your Existing Network
• 22/02/17 (Wednesday) @ 11:30 – 1.5 hours
4. BRKCRS-2802: DNA Campus Fabric – Monitoring & Troubleshooting

• 22/02/17 (Wednesday) @ 14:30 – 1.5 hours
5. BRKCRS-2803: DNA Campus Fabric – Connecting Outside the Fabric

• 22/02/17 (Wednesday) @ 16:30 – 1.5 hours
6. BRKACI-2400: DNA Campus Fabric – Integration with Data Center Architectures

• 23/02/17 (Thursday) @ 14:30 – 1.5 hours
7. BRKEWN-2300: Virtualize Your Wired and Wireless Network (w/ Campus Fabric)
• 24/02/17 (Friday) @ 09:00 – 2 hours
Questions? Lunch and Learn!
During Lunch time on Tuesday,
Wednesday and Thursday, you
can join Cisco “subject matter experts”
and your peers in casual conversation
about topics of interest to you.
More Questions?
Join our Lunch & Learn Table

• LALCRS-2001
• DNA Campus Fabric
List of topics:
In the Catering Area (Hall 6.2) http://www.ciscolive.com/emea/activities/lunch-learn/
Complete Your Online Session Evaluation
• Please complete your
Online Session Evaluations
after each session
• Complete 4 Session Evaluations
+ Overall Conference Evaluation
(available from Thursday) to
receive your Cisco Live T-shirt
• All surveys can be completed via
the Cisco Live Mobile App or the Don’t forget: Cisco Live sessions will be available
for viewing on-demand after the event at
Communication Stations CiscoLive.com/Online
Thank You

Teccrs 3800

Uploaded by

Copyright:

Available Formats

Teccrs 3800

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Teccrs 3800

Uploaded by

Copyright:

Available Formats

Campus Fabric

Is your Campus network facing some, or all, of these challenges?

• Host Mobility (w/o stretching VLANs)

We recommend the following sessions:

2. BRKCRS-3800: DNA Campus Fabric – A Look Under the Hood

4. BRKCRS-2802: DNA Campus Fabric – Monitoring & Troubleshooting

5. BRKCRS-2803: DNA Campus Fabric – Connecting Outside the Fabric

6. BRKACI-2400: DNA Campus Fabric – Integration with Data Center Architectures

Shawn Sanjay Satish Karthik Vimarsh Kedar

11:00 – 11:15 15 min BREAK TIME 

11:15 – 12:45 90 min External Connectivity

13:15 – 14:15 60 min LUNCH TIME 

16:15 – 16:30 15 min BREAK TIME 

16:30 – 18:00 90 min Design and Migration

Cloud Service Management

Open APIs | Developers Environment Insights &

Open & Programmable | Standards-Based

Programmable Custom ASICs Converged Software Services

Optimized for Campus Virtualization

Future Proofed Designed for Evolution

Driving Innovation Through Technology Investment

Examples of Network Overlays

Overlay Network Overlay Control Plane

Edge Device Edge Device

Underlay Network Underlay Control Plane

Separate the Forwarding Plane from the Services Plane

Reliable Transport Forwarding Flexible Virtual Services

1. Control-Plane based on LISP

1. Control-Plane based on LISP Host

Topology + Endpoint Routes

1. Control-Plane based on LISP

1. Control-Plane based on LISP

ETHERNET IP UDP VXLAN ETHERNET IP PAYLOAD

User / Group Control-Plane

 Border Nodes – The L3 Gateway

• “Control-Plane Node” ≈ “LISP Map-Server”

• “Edge Node” ≈ “LISP XTR + Endpoints”

• “Border Node” ≈ “LISP XTR + Subnets” or “PXTR”

• “Intermediate Node” ≈ “Non-LISP IP Forwarder”

Control-Plane Node runs a Host Tracking Database to map location information

• A simple Host Database, that tracks Endpoint ID to

• Host Database supports multiple types of Endpoint ID

• Receives map registrations from Edge and Border

• Resolves lookup requests from Edge and Border

• Responsible for Identifying and Authenticating Endpoints

• Register the specific Endpoint ID info (e.g. /32 or /128)

• Provide Anycast L3 Gateway for all connected Endpoints

• Performs encapsulation / de-encapsulation of data traffic

• Internal Border based on XTR + Subnets

Internal Border advertises Endpoints to outside, and known Subnets to inside

• Connects to any “known” IP subnets attached to the

• Exports all internal IP Pools to outside (as aggregate), B B

• Imports and registers (known) IP subnets from outside,

• Hand-off requires mapping the context (VRF & SGT)

External Border is a “Gateway of Last Resort” for unknown destinations

• Connects to any “unknown” IP subnets (e.g. Internet)

into traditional IP routing protocol(s). B B

• LISP PXTR is a “default” exit point, if no specific entry

• Hand-off requires mapping the context (VRF & SGT)