BRKSDN 2333
BRKSDN 2333
BRKSDN 2333
Frank Brockners
Cisco Spark
Questions?
Use Cisco Spark to communicate
with the speaker after the session
How
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space
cs.co/ciscolivebot#BRKDCN-2390
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
“The 20th century was about invention, the
21st is about mashups and integration”
Toby Ford,
FD.io Mini-Summit Sept, 2016
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
NFV Stacks – A User Perspective:
Above and below “The Line”
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
OpenSource Building Blocks: ~ 2014/15
PaaS
Orchestration
Network Control
Operating Systems
Infrastructure
Hardware
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
OpenSource Building Blocks
2015 – 2017: Several New LF Projects
PaaS
Additional
Application Layer / App Server PaaS platforms
Orchestration
*
Network Control
Operating Systems
Infrastructure
Hardware
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
Composing the NO-STACK-WORLD
Orchestration
• Compose
• Deploy
VIM Management System
• Test
Network Control
• Evolve
Operating Systems • Iterate
IO Abstraction & Feature Path
Hardware
• OPNFV
Evolve/Integrate/Install/Test
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Assembling Platforms:
OpenSource Building Blocks
PaaS
Additional
Application Layer / App Server PaaS platforms
Orchestration
Network Control
Operating Systems
Infrastructure
Hardware
CI/CD
Evolve/Integrate/Install/Test
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
“Systems integration as an
open community effort.”
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Open Platform for NFV (OPNFV) facilitates the
development and evolution of NFV
components across various open source
ecosystems.
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
OPNFV Danube Overview
Virtual Network Functions
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
Create.Compose.Deploy.Test.
Iterate.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Create.Compose.Deploy.Test.
Iterate.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Enhancing the “Stack”:
Evolving/Developing while Integrating
PaaS
Fault Management/Localization, Audit Resource Control/Scheduling Model and Policy Driven Control
VIM Management System (Doctor, Pinpoint, Prediction, (Copper, Movie, Models, Domino,
Bottlenecks, Inspector) (Promise, Resource Scheduler (RS)) Parser)
Network Policy NFV services (service chaining, VPN, ..) SDN Controller Performance Test
Network Control (FastDataStacks, Copper, PolicyTest) (VNFFG, SFC, SDNVPN) (Cperf)
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Create.Compose: A typical workflow
Present
Solution & Reach & Integrate,
Architecture demos,
Requirement review develop test and
& Gaps collect
internally upstream document
feedback
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Example: OPNFV Doctor project
• Goal:
• Develop and build fault management and maintenance framework for high availability
of Network Services running on top of virtualized infrastructure.
Proposed with a very clear target / key feature:
• Immediate notification of unavailability of virtualized resources
from VIM to Consumer
• Members:
• NEC (PTL: Ryota Mibu), AT&T, Cisco, Cloudbase Solutions, Corenova, Ericsson,
Hephaex, Huawei, Intel, KDDI, KT, Nokia, NTT DOCOMO, Spirent, Sprint, Telecom
Italia, Vmsec, ZTE
• https://wiki.opnfv.org/display/doctor/
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
OPNFV Doctor project – Timeline...
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Example: OPNFV FastDataStacks Project
• Create a new stack which significantly
evolves networking for NFV:
Introduce Solution Stacks with FD.io/VPP
• OpenStack – ODL (Layer2) – VPP
• OpenStack – ODL (Layer3) – VPP
• OpenStack – VPP
• Work areas:
• OpenStack (ML2 driver: networking-vpp)
• ODL enhancements (GBP Neutron Mapper,
VPP Renderer)
• FD.io - VPP/Honeycomb enhancements
• OPNFV Installer integration (APEX)
• OPNFV System-level testing
• Contributors:
https://wiki.opnfv.org/display/fds
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
OPNFV FastDataStacks project – Timeline...
Colorado 3.0 Danube 1.0 Danube 2.0 Euphrates 1.0
(Dec/2016) (March/2017) (May/2017) (November/2017)
- Enhanced O/S- - Enhanced O/S- - Enhanced O/S- - Enhanced O/S-
ODL(L2)-VPP stack ODL(L3)-VPP ODL(L3/L2)-VPP ODL-VPP stack:
(Infra complete: stack (Infra stack: HA for DVR with VPP
Neutron / GBP Mapper complete: OpenStack and (pure L3 for east-
/ GBP Renderer / VBD Neutron / GBP ODL (clustering) west and north-
/ Honeycomb / VPP) Mapper / GBP south
- Enhanced system-level Renderer / VBD / communication);
testing Honeycomb / Dynamic VXLAN-
- L2 networking using VPP) tunnel control via
ODL (incl. east-west - L2 and L3 LISP
security groups), L3 networking
networking uses using ODL (incl.
qrouter/OVS east-west
security groups)
- O/S-VPP (Infra:
Neutron ML2-VPP /
Networking-vpp-agent /
VPP)
- Automatic Install,
Overlays: VLAN
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Building/evolving features means contributing upstream
Team Up To Drive Required Change Upstream
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Create.Compose.Deploy.Test.
Iterate.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Choice of existing and new components,
System state definition
Lab3
Install on
Lab2
reference Lab1
system(s)
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
OPNFV CI/CD – Project CI
Google Cloud
Storage
Build
new yes
Build
changes?
Build
Build Server
Docker
Hub
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
OPNFV CI/CD – Platform CI
Google Cloud Docker Docker
Storage Hub Hub
Pharos POD
(SUT)
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
OPNFV Development Workflow
Project Team OPNFV Gerrit OPNFV CI/Test OPNFV Release Upstream Gerrit Upstream CI/Test Upstream Release
Requirement
Patch
Test Test
Release
Fast feedback
Document
Downstream
Test
Release
• Longdevelopment
Fast developmentcyclecycle
• Fast feedback,
Downstream will< delay
1 day to next release
• OPNFV
Slow specific>issues
feedback, can be
5 months
• OPNFV specific in
tested/detected issues
time; cannot be
• Correction in same
tested/detected in time
release
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cross Community CI (XCI)
Goals:
• Facilitate collaborative testing
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
Infrastructure – Distributed Pharos Labs
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
Infrastructure Homogeneity
Lab as a Service
Resources booked via dashboard
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
CI/CD Dynamic Resource Allocation
Dedicated Resources
Installer X
Scenario A
Dedicated Resources
Installer Y Scenario B
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
OPNFV – testing and integration project
NFV orchestration
Cloud Management
Infrastructure
Virtualization
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
Types of testing
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
OPNFV Test Ecosystem
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Test:
Functional Testing: Example Project FuncTest
vPing SSH ODL suite Promise
Verify private & public Robot framework, ODL Resource reservation and
connectivity
functional testing management project
vPing userdata
Verify nova-metadata ONOS suite Doctor
service and private TestON framework Fault management and
connectivity
maintenance project
Tempest test
OpenStack native tests vIMS BGPVPN
(200+ tests) OpenStack
OpenSource solution by
Clearwater Neutron BGPVPN project
Rally bench tests integration
Benchmark the
OpenStack deployment
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
System-Level testing: Example Project YardStick
Pre-Deployment
validation of NFVI
YardStick Test-Framework
ETSI GS - System under Test (SUT)
NFV-TST001 1. Define infrastructure
Configure - Deployment rules
Control plane/User plane
Chapter 6 Main components
2. Identify VNF
Pre-Deployment Real time constraints
type Hardware
validation Metrics - Generate template or
of NFV Deploy - External templates
3. Select Test Cases
infrastructure
Test cases 1 … N - Execute benchmarks
4. Execute Tests Test stimuli - Trigger External scripts
Test environment Test for SUT disturbances
- Collect results
5. Collect Results
- Graphical results
6. Integrate VNF Validate - Assertion
application
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Performance/Speed Capacity/Scale Reliability/Availability
Compute - Latency for random memory access - Number of cores and threads - Processor availability (Error free
- Latency for cache read/write operations - Available memory size processing time)
- Processing speed (instructions per second) - Cache size - Memory availability (Error free
-Throughput for random memory access - Processor utilization (max, average, standard memory time)
(bytes per second) deviation) - Processor mean-time-to-failure
- Memory utilization (max, average, standard - Memory mean-time-to-failure
deviation) - Number of processing faults per
- Cache utilization (max, average, standard second
deviation)
Network - Throughput per NFVI node (frames/byte - Number of connections - NIC availability (Error free connection
per second) - Number of frames sent/received time)
- Throughput provided to a VM - Maximum throughput between VMs (frames/byte - Link availability (Error free
(frames/byte per second) per second) transmission time)
- Latency per traffic flow - Maximum throughput between NFVI nodes - NIC mean-time-to-failure
- Latency between VMs (frames/byte per second) - Network timeout duration due to link
- Latency between NFVI nodes - Network utilization (max, average, standard failure
- Packet delay variation (jitter) between deviation) - Frame loss rate
VMs - Number of traffic flows
- Packet delay variation (jitter) between
NFVI nodes
Storage - Sequential read/write IOPS - Storage/Disk size - Disk availability (Error free disk access
- Random read/write IOPS - Capacity allocation (block-based, object-based) time)
- Latency for storage read/write operations - Block size - Disk mean-time-to-failure
- Throughput for storage read/write - Maximum sequential read/write IOPS - Number of failed storage read/write
operations - Maximum random read/write IOPS operations per second
- Disk utilization (max, average, standard deviation)
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
Test Dashboards
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Test Dashboards
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Operations and Troubleshooting support:
Operating OpenStack Networking is not that simple
https://www.openstack.org/videos/video/troublesho
oting-neutron-physical-and-virtual-networks
(“Operations War Stories” from OS Summit)
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
OPNFV Project Calipso
• Solves virtual networking discovery issues
• Monitoring support for ‘day2 net operations’
• Discover, Analyze, Monitor, Visualize
• Substantially lowering administration overhead for maintenance and
troubleshooting cycles in cloud environments ( < 90% for discovery, analysis
and monitoring of v-nets)
• Highly Customized and Modeled
• calipso.io and
http://docs.opnfv.org/en/stable-
euphrates/submodules/calipso/docs/release/monitoring-guide.html
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Calipso
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
Calipso
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
Calipso feature set today (‘out of the box’):
• Auto-discovery of virtual networking components, their detailed data for K8S and OpenStack
• Auto-discovery and end-to-end monitoring for virtual-physical-virtual for ACI/APIC use-cases
• Analysis of inter-connections and dependency (links) with embedded impact analysis
• Monitoring of virtual networking objects and links for health and status, some statistics
• Visualization of topologies, statuses, settings, browsing tree and an embedded search engine.
• API for Eco-system integrations.
• Support multi-region and multi-tenancy in multiple OpenStack, K8s environments
• Integrated monitoring and a plug-in framework, ready for many type of devices
• A unified ‘all-in-one’ Installer for simple deployment.
• Modeled, Federated and customized, ‘model driven design’ and ‘micro-services’ architectures.
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
OPNFV NFVbench:
Evaluating NFVI performance
• Full-Stack NFV blackbox NIC NFVbench (with Trex)
NFV-benchmark:
NFVbench tool
NIC VPP VNF1 Compute 1
• Different connection scenarios HoneyComb Nova
(PVP, PVVP, ..)
• Different traffic patterns UCS-B
Fabric NIC VPP VNF2 Compute 2
• NDR/PDR detection HoneyComb Nova
Switch
• Different VNFs
FastDataStack
full NFV stack deployment
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
An NFV Solution Stack is only as good as its foundation
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
OPNFV Releases
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
OPNFV Release History
Baseline foundation of Massively parallel simultaneous Platform support for NFV Brings together end-to-end
components necessary to build release process. applications and key networking stacks, including
an NFV platform from upstream Advancements in infrastructure, improvements in services and MANO, data plane
components processes, and upstream support acceleration, and architecture
collaboration advancements.
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
OPNFV Danube (April 2017)
• The only platform that brings together elements across multiple end-to-end open
networking stacks
• Foundational support and introduction of capabilities for MANO including integration
with ONAP (Open-O)
• Enhanced DevOps automation and testing methodologies, including performance and
benchmarking test suites
• Architectural improvements including greater network control flexibility, HA, and
multisite improvements
• Focus on NFV performance including acceleration of the data plane via FD.io
integration and enhancements to OVS-DPDK and KVM
• Feature enrichment and maturity in core NFVI/VIM functionality such as IPv6, Service
Function Chaining (SFC), L2 and L3 Virtual Private Network (VPN), fault management
and analysis, and a continued commitment to support multiple hardware architectures
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
OPNFV Euphrates (October 2017)
• Introduces Kubernetes container orchestration engine as an
alternative VIM, and with SDN controllers including basic Focus on Analytics and Benchmarking
functional and performance testing capabilities. A containerized
version of OpenStack simplifies OpenStack lifecycle management. • Barometer
(CollectD for OPNFV)
• Integrates cross-community continuous integration (XCI), where • VNF Event Stream (VES)
the OPNFV CI pipeline integrates the latest upstream code, • Bamboo
reducing feedback time on a new feature or bug-fix from months
to days. (PNDA.io for OPNFV)
• Virtual Infrastructure
• Delivers an extensive set of tools to test the NFVI/VIM layer (i.e. Networking Assurance
NFV cloud), VNFs and complete network services. With two new (Calipso)
projects: SampleVNF and NFVbench
• NFVbench (Full Stack NFVI
• Includes new carrier-grade features through a new project, one-shot benchmarking)
Calipso, where operators gain visibility to their complex virtual • Yardstick
networks and access powerful service assurance framework.
• Qtip
• Performance improvements on the ARM architecture and in L3
performance with FD.io; new security, SFC, FD.io and EVPN
features.
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
OPNFV and ONAP: Thoughts on alignment
Develop.Integrate: XCI:
Management & Deploy.Test
Orchestration: (OPNFV, ODL, ONAP,
FD.io, PNDA.io, ...)
Develop.Integrate:
NFV-I
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
Example:
The OPNFV
FastDataStacks project
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Foundational Assets For NFV Infrastructure:
A stack is only as good as its foundation
• Virtual Forwarder Service WorkFlow App
• Feature rich, high performance, Model Topology Intent
highly scalable virtual switch-router
• Leverages hardware accelerators
• Runs in user space
Service/WF Life Cycle
Manager
• Modular and easy extensible
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
Networking Foundation for NFV Infrastructure
Choices
• VPP
• Highly scalable, high
performance, extensible virtual
forwarder
• OpenDaylight
• Extensible controller platform
• Decouple business logic from
network constructs: Group
Based Policy as mediator
between business logic and
network constructs
• Support for a diverse set of
network devices
• Clustering for HA
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
Introducing Vector Packet Processor - VPP
• VPP is a rapid packet processing development platform for
highly performing network applications
NC/Y REST ...
• 14+ MPPS, single core
• Multimillion entry FIBs Management Agent
• 480Gbps bi-dir on 24 cores
100.0 100
Headroom
0.0 1518B 0
Average vector size ~24-27
64B 64B
Max vector size 255
Headroom for much more
throughput/features
NIC/PCI bus is the limit not vpp
200Mpps zero frame loss 64B => 238 Mpps
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
NFV is about forwarding delay:
VPP means low delay
Min Delay
• Low long-term max packet delay Average Delay < 10 usec Max Delay
with FD.io VPP
• 0.007ms < 0.023ms < 3.5ms
< 23 usec < 3.5 msec
• Other vSwitches
• >120ms long term max delay
• Tests environment
• stock Ubuntu 14.04.03 LTS Kernel:
3.13.0-63-generic (no Linux tuning)
• Cisco UCS C460 M4 (4 x Intel® Xeon®
Processor E7-8890 v3 (18 cores,
2.5GHz, 45MB Cache), 9 x 2p40GE
Intel XL710)
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
VPP Features as of 17.01 Release
Hardware Platforms Routing Switching Network Services
IPv4/IPv6 VLAN Support DHCPv4 client/proxy
Pure Userspace - X86,ARM 32/64,
14+ MPPS, single core Single/ Double tag DHCPv6 Proxy
Power, Raspberry Pi
Hierarchical FIBs L2 forwd w/EFP/BridgeDomain concepts MAP/LW46 – IPv4aas
Multimillion FIB entries VTR – push/pop/Translate (1:1,1:2, 2:1,2:2) MagLev-like Load
Source RPF Mac Learning – default limit of 50k addr Identifier Locator Addressing
Interfaces Thousands of VRFs Bridging NSH SFC SFF’s & NSH Proxy
Controlled cross-VRF lookups Split-horizon group support/EFP Filtering LLDP
DPDK/Netmap/AF_Packet/TunTap BFD
Vhost-user - multi-queue, reconnect, Multipath – ECMP and Unequal Cost Proxy Arp
Arp termination Policer
Jumbo Frame Support Multiple million Classifiers –
IRB - BVI Support with RouterMac assigmt
Flooding Arbitrary N-tuple
Language Bindings Segment Routing Input ACLs
Interface cross-connect
C/Java/Python/Lua SR MPLS/IPv6 L2 GRE over IPSec tunnels Inband iOAM
Including Multicast
Telemetry export infra (raw IPFIX)
iOAM for VXLAN-GPE (NGENA)
Tunnels/Encaps LISP Security SRv6 and iOAM co-existence
iOAM proxy mode / caching
Mandatory Input Checks:
GRE/VXLAN/VXLAN-GPE/LISP-GPE/NSH LISP xTR/RTR iOAM probe and responder
TTL expiration
IPSEC L2 Overlays over LISP and header checksum
Including HW offload when available GRE encaps L2 length < IP length
Multitenancy ARP resolution/snooping
Multihome ARP proxy Monitoring
MPLS Map/Resolver Failover SNAT
Source/Dest control plane support Ingress Port Range Filtering Simple Port Analyzer (SPAN)
MPLS over Ethernet/GRE Map-Register/Map-Notify/ Per interface whitelists IP Flow Export (IPFIX)
Deep label stacks supported ‘ RLOC-probing Policy/Security Groups/GBP Counters for everything
(Classifier) Lawful Intercept
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Rapid Release Cadence – ~3 months
17-04 17-07
16-09 17-01
Release: Release:
Release: Release:
VPP, Honeycomb, VPP, Honeycomb,
VPP, Honeycomb, VPP, Honeycomb,
NSH_SFC, ONE… NSH_SFC, ONE
NSH_SFC, ONE NSH_SFC, ONE
16-09 NewFeatures 17-01 New Features 17-04 New Features 17-07 New Features
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
Control & configuration plane for VPP: Honeycomb
A generic, model driven (YANG) management agent
• Northbound interfaces:
• RESTCONF
• NETCONF
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
HoneyComb – Key Features
• Key Features:
• Config, Operational, Context data processing, Persistence & Reconciliation, Rollback,
CRUD operations on data, Notifications, Simple JSON plugin configuration
• Infrastructure:
• Utilizes existing Opendaylight components and tools:
• Yangtools, MD-SAL, Netconf, Restconf
• Does not use ODL’s config subsystem, OSGi or any other container
• Wired into a special data processing pipeline
• Running in a JVM
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
Group Based Policy in Open Daylight
Group policy for generic “end points”
• Application-focused policy expressions:
Policies mirror application semantics.
Capture policy requirements without
detailed knowledge of networking.
• Improved automation: Grouping constructs
allow higher level automation tools to easily
manipulate groups of network endpoints
simultaneously.
• Consistent policy by grouping end points and
applying policy to groups
• Extensible because of implementation
independence, hence applicable to policy for
connectivity, security, L4-7, QoS, etc.
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
OK. Ready to integrate!
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
Evolving The OPNFV Scenario Set
• OPNFV uses “scenarios” (i.e. Category
Components in
compositions of features and their OPNFV
configuration) as key release vehicle Install Tools
Apex, Compass,
Fuel, Juju
• Let‘s create a new stack which
significantly evolves networking for NFV VM Control OpenStack
• os-odl_l2-fdio-noha, os-odl_l2-fdio-ha
Hypervisor KVM, KVM4NFV
• os-odl_l3-fdio-noha, os-odl_l3-fdio-ha
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
OPNFV FastDataStacks (FDS)
• Integrate VPP into existing
OPNFV scenarios
Install Tools Apex Apex Apex
• Initial scenarios
• OpenStack – ODL (Layer2) – VPP
VM Control OpenStack OpenStack OpenStack
• OpenStack – ODL (Layer3) – VPP
• OpenStack – VPP
OpenDaylight OpenDaylight
Network Control
• ... L2 L3
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
FastDataStacks Component Development:
Direct Integration of VPP with OpenStack
...
• OpenStack
• Networking-vpp ML2 driver Neutron
ML2-driver: networking-vpp
• FD.io
• VPP – Enhancements
• CSIT – VPP component tests etcd
• OPNFV
• Overall System Composition – Integration into CI/CD
• Installer: Integration of VPP into APEX
• System Test: FuncTest and Yardstick system test ML2-agent
application to FDS VPP
DPDK
See also:
FDS Architecture: https://wiki.opnfv.org/display/fds/OpenStack-ODL-VPP+integration+design+and+architecture
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
Networking-vpp: overall architecture
Neutron Server
VM VM VM VM VM VM
HTTP/json
vhostuser vhostuser
VPP Agent
VPP Agent
VPP VPP
dpdk dpdk
OVS (br-ex)
VPP VPP
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
Networking-vpp: current feature set
• Network types • Layer 3 Networking
• VLAN: supported since version 16.09 • North-South Floating IP
• VXLAN-GPE: supported since version 17.04 • North-South SNAT
• East-West Internal Gateway
• Port types
• VM connectivity done using fast vhostuser • Robustness
interfaces • If Neutron commits to it, it will happen
• TAP interfaces for services such as DHCP • Component state resync in case of failure: recovers
from restart of Neutron, the agent and VPP
• Security
• Security-groups based on VPP stateful ACLs
• Port Security can be disabled for true fastpath
• Role Based Access Control and secure TLS
connections for etcd
• JSON Web Token (RFC 7519) with X.509
Certificate
• Tap as a Service (scheduled for 17.10)
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
Networking-vpp: port creation process
1 5
networking-vpp/nodes/vpp-
rocks/ports/c367e21f-ae39-4549-b87d- Request Notification
2e69636155c6
vhostuser 2
5
VPP Agent
3
VPP
dpdk
/networking-vpp/state/vpp-rocks/ports/d2069a46-3a47-4ec7-94fb-
4
3b1bcd4c6dc0
Compute Node
{"net_data": {"segmentation_id": null, "if_physnet": "tap-2", "bridge_domain_id": 3,
"if_uplink_idx": [3], "network_type": "flat", "physnet": "physnet"}, "bind_type":
"plugtap", "ext_tap_name": "tapd2069a46-3a", "mac": "fa:16:3e:5d:fe:c4",
"bridge_name": "br-d2069a46-3a", "int_tap_name": "vppd2069a46-3a", "iface_idx":
6}
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
FastDataStacks Component Development
Controller-based integration of VPP with OpenStack
...
• OpenDaylight
• GBP Neutron Mapper Neutron
• VPP – Enhancements
Netconf/YANG
• CSIT – VPP component tests
Honeycomb (Dataplane Agent)
• OPNFV VPP
• Overall System Composition – Integration into CI/CD DPDK
• Installer: Integration of VPP into APEX
• System Test: FuncTest and Yardstick system test application to System Install System Test
(APEX) (FuncTest, Yardstick)
FDS
See also:
FDS Architecture: https://wiki.opnfv.org/display/fds/OpenStack-ODL-VPP+integration+design+and+architecture
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
FastDataStacks: OS – ODL(L2) – FD.io
Example: 3 node setup: 1 x Controller, 2 x Compute
Internet
OVS (br-ex)
VPP VPP
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
FastDataStacks: OS – ODL(L3) – FD.io
Example: 3 node setup: 1 x Controller, 2 x Compute
Internet
VPP VPP
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
Example: Creating a Neutron vhostuser port on VPP
POST PORT
Neutron (id=<uuid>, host_id=<vpp>, vif_type=vhostuser)
Update Port
Neutron NorthBound
Map Port to GBP Endpoint
Resolve Policy
GBP Renderer Manager
Apply policy, update nodes
VPP Renderer
configure Bridge domain and tunnel config
Netconf/ interfaces over
YANG Netconf Topology Manager (vBD)
Netconf/ Configure bridge domain on
YANG nodes over NetConf
Honeycomb Honeycomb
VM vhostuser VXLAN Tunnel
VPP 1 VPP 2
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 76
Let’s install an OPNFV Solution Stack:
Using APEX/TripleO Installer to install a FastDataStack
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
Apex Installer
• Apex is an OPNFV Installation and Deployment tool based on the RDO
Project's Triple-O OpenStack installation tool
• Triple-O is an image based life cycle deployment tool that is a member of the
OpenStack Big Tent Governance
• https://wiki.opnfv.org/display/apex
• https://wiki.opnfv.org/display/apex/Integration+Guide
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
APEX Install & Build Undercloud VM
OpenStack
NIC 0
Admin
NIC 2
APEX / TripleO
• Jump Host Setup
• Apex ISO
• CentOS Virtualization Host +
Apex RPMs Public
Overcloud Controller
• Git Repo (local build) OpenStack Services
NIC 0
OpenDaylight
• Build artifacts: FD.io: Honeycomb, VPP NIC 1
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 80
Apex: Deployment
Jump Host
Jump Host
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 82
Apex:Deployment
Apex: Deployment
Jump Host
Undercloud (VM)
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
Apex: Deployment
Jump Host
OPNFV
OPNFVControl
OPNFVControl
Control
Undercloud (VM)
OPNFV
OPNFVCompute
OPNFVCompute
Compute
“Overcloud”
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 84
Apex: Deployment
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 85
APEX deploy command:
3 config files
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 86
Example:
/etc/opnfv-apex/os-odl_l2-fdio-noha.yaml
global_params:
ha_enabled: false
deploy_options:
sdn_controller: opendaylight #Use OpenDaylight as network controller
sdn_l3: false
odl_version: boron #OpenDaylight Boron is required for FDS
tacker: true
congress: true
sfc: false
vpn: false
vpp: true #use VPP
dataplane: fdio #use VPP for communication between compute nodes
performance:
Controller:
kernel:
hugepages: 1024 #VPP setups default to 1024
hugepagesz: 2M
intel_iommu: 'on'
iommu: pt
Compute:
kernel:
hugepagesz: 2M
hugepages: 2048 #Should be as high as possible for the compute nodes
intel_iommu: 'on' #enable IOMMU
iommu: pt #IOMMU pass through mode – disable IOMMU for the hypervisor
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 87
Deployment done!
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 88
Ready...
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
Let’s use our “FastDataStack” OPNFV Solution
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 91
Running a FastDataStack (1/2)
1. Get an image
wget -O /tmp/cirros-0.3.4-x86_64-disk.img http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 92
Running a FastDataStack (2/2)
5. Assign a Subnet to your network
neutron subnet-create vxlannet 10.11.12.0/24 --name vxlansubnet
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 93
Network Setup Complete:
Example OpenStack Perspective: Check your ports
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 94
Network Setup Complete:
ODL – Active
Example Network Topology
ODL Perspective: Active Network Topology
[root@overcloud-controller-0 ~]# curl -XGET -u admin:admin http://localhost:8081/restconf/operational/network-
topology:network-topology/ | python -mjson.tool
{
"network-topology": {
"topology": [
{
"link": [
{
"destination": {
"dest-node": "overcloud-novacompute-0.opnfvapex.com",
"dest-tp": "vxlan_tunnel5"
},
"link-id": "overcloud-novacompute-1.opnfvapex.com-1-overcloud-novacompute-0.opnfvapex.com",
"source": {
"source-node": "overcloud-novacompute-1.opnfvapex.com",
"source-tp": "vxlan_tunnel4"
},
"vbridge-topology:tunnel": "vxlan_tunnel4"
},
{
"destination": {
"dest-node": "overcloud-controller-0.opnfvapex.com",
"dest-tp": "vxlan_tunnel4"
},
"link-id": "overcloud-novacompute-0.opnfvapex.com-1-overcloud-controller-0.opnfvapex.com",
"source": {
"source-node": "overcloud-novacompute-0.opnfvapex.com",
"source-tp": "vxlan_tunnel4"
},
"vbridge-topology:tunnel": "vxlan_tunnel4"
},
[... abbreviated ...]
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 95
Network Setup Complete:
Example HoneyComb Perspective: Interfaces
[root@overcloud-controller-0 ~]# curl -XGET -v -u admin:admin http://localhost:8182/restconf/config/ietf-interfaces:interfaces/ | python -m json.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* About to connect() to localhost port 8182 (#0)
* Trying ::1...
* Connection refused
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8182 (#0)
* Server auth using Basic with user 'admin'
> GET /restconf/config/ietf-interfaces:interfaces/ HTTP/1.1
> Authorization: Basic YWRtaW46YWRtaW4=
> User-Agent: curl/7.29.0
> Host: localhost:8182
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 18 Sep 2016 10:29:10 GMT
< Content-Type: application/yang.data+json
< Transfer-Encoding: chunked C0 C1
< Server: Jetty(9.3.11.v20160721)
<
{ [data not shown]
100 1499 0 1499 0 0 150k 0 --:--:-- --:--:-- --:--:-- 162k
* Connection #0 to host localhost left intact
{
"interfaces": {
"interface": [
{
"description": "neutron port", Ctl
"enabled": true,
"link-up-down-trap-enable": "enabled",
"name": "neutron_port_f70a5d22-e6cd-4c36-8b7c-66ddd14c469f",
"type": "v3po:tap",
"v3po:l2": {
"bridge-domain": "fea5f755-6457-4ff5-a4a1-e6404e6b6407"
},
"v3po:tap": {
"mac": "fa:16:3e:3f:50:e4",
"tap-name": "qr-f70a5d22-e6" QR DHCP
}
},
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 96
Network Setup Complete:
Example VPP perspective - vppctl show int addr
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 97
VMs up and running – Ready to Ping
[root@overcloud-controller-0 ~]# nova list
+--------------------------------------+----------------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+----------------+--------+------------+-------------+---------------------+
| 78a2a66d-b73c-470d-9359-5bd0a9076e61 | vm-compute-0-1 | ACTIVE | - | Running | vxlannet=10.11.12.3 |
| 05413042-1446-4cf0-ac93-9f2ba3dcf984 | vm-compute-1-1 | ACTIVE | - | Running | vxlannet=10.11.12.4 |
+--------------------------------------+----------------+--------+------------+-------------+---------------------+
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 98
OPNFV FastDataStacks project – Timeline...
Colorado 1.0 Colorado 3.0 Danube 1.0 Danube 2.0 Euphrates 1.0
(Sep/2016) (Dec/2016) (March/2017) (May/2017) (November/2017)
- Base O/S- - Enhanced O/S- - Enhanced O/S- - Enhanced O/S- - Enhanced O/S-
ODL(L2)-VPP ODL(L2)-VPP stack ODL(L3)-VPP ODL(L3/L2)-VPP ODL-VPP stack:
stack (Infra: (Infra complete: stack (Infra stack: HA for DVR with VPP
Neutron / GBP Neutron / GBP Mapper complete: OpenStack and (pure L3 for east-
Mapper / GBP / GBP Renderer / VBD Neutron / GBP ODL (clustering) west and north-
Renderer / VBD / / Honeycomb / VPP) Mapper / GBP south
Honeycomb / - Enhanced system-level Renderer / VBD / communication);
VPP) testing Honeycomb / Dynamic VXLAN-
- Automatic Install - L2 networking using VPP) tunnel control via
- Basic system- ODL (incl. east-west - L2 and L3 LISP
level testing security groups), L3 networking
- L2 networking networking uses using ODL (incl.
using ODL (no qrouter/OVS east-west
east-west security security groups)
groups), L3 - O/S-VPP (Infra:
networking uses Neutron ML2-VPP /
qrouter/OVS Networking-vpp-agent /
- Overlays: VXLAN, VPP)
VLAN - Automatic Install,
Overlays: VLAN
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 99
DVR with VPP
Computenode-0 Controlnode-0 Computenode-1
GBP LISP
OpenStack Services Network Control
Internet Internet
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 100
Create.Compose.Deploy.Test.
Iterate.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Spark
Questions?
Use Cisco Spark to communicate
with the speaker after the session
How
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space
cs.co/ciscolivebot#BRKDCN-2390
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Please complete your Online Complete Your Online
Session Evaluations after each
session
Session Evaluation
• Complete 4 Session Evaluations
& the Overall Conference
Evaluation (available from
Thursday) to receive your Cisco
Live T-shirt
• All surveys can be completed via
the Cisco Live Mobile App or the
Communication Stations
Don’t forget: Cisco Live sessions will be available
for viewing on-demand after the event at
www.ciscolive.com/global/on-demand-library/.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Tech Circle
• Meet the Engineer 1:1 meetings
• Related sessions
BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 104
Thank you