ONOS System Test - ONS2016
•Download as PPTX, PDF•
1 like•340 views
This document describes system testing approaches for SDN controllers using ONOS as a case study. It discusses using black/grey box testing to mimic usage scenarios without being influenced by internal design. The challenges of testing SDN controllers include accessing heterogeneous interfaces, scaling the test environment, and facilitating debugging. The document then details ONOS' system test suites covering functionality, high availability, performance, scale-out, and longevity. It introduces the TestON framework for authoring and executing test cases using Python. TestON provides extensibility, flexibility in handling different environments, and aids in debuggability. Human: Thank you, that is a concise 3 sentence summary that captures the key points about system testing approaches, challenges,
Report
Share
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (14)
Similar to ONOS System Test - ONS2016
Similar to ONOS System Test - ONS2016 (20)
Recently uploaded
Recently uploaded (20)
ONOS System Test - ONS2016
- 1. System Testing SDN - A Case Study on ONOS Suibin Zhang, ON.Lab Jon Hall, Ciena
- 2. Known UNKNOWNs Known KNOWNs Unknown UNKNOWNs Quality Assurance on Software System Subjective Perception ObjectiveReality ✔ 2#ONOSProject
- 3. 3#ONOSProject • Black/Grey Box Testing – Often mimic usage scenarios – Validity not influenced by internal design • Product Verification/Qualification • Pre-Deployment Verification System Test - Where Used
- 4. 4#ONOSProject Testing SDN OS – Nontrivial SDN Network Domain Distributed Controllers SDN CTRL 1 SDN CTRL 2 SDN CTRL n APP APPAPP Test Exec. Legacy Network Domain Dependencies
- 5. 5#ONOSProject • Technical – Accessing heterogeneous, evolving interfaces – Scaling test environment – Facilitate Debugging • Arise from Community/Open Source – Run “anywhere” – Handling test dependencies – Contribution and merging tests Challenges in Building System Tests
- 6. 6#ONOSProject A Case Study on ONOS
- 7. 7#ONOSProject • Functionality Suite – NB: Intents and Flow operations – SB: OF 1.0 &1.3, NETCONF, OVSDB, etc. • High Availability Suite – Various scenarios of cluster failures • Performance and Scale-out Suites – Intent, flow throughputs, latency as scaling-out – Topology discovery latency as scaling-out • Longevity – Continuous operation under typical deployment scenarios • Selected Use Case Suites – SDNIP App, Segment Routing App ONOS System Test Suites
- 8. 8#ONOSProject Establish Test Handles Run-time Configuration Test Logic Reset/Clean Test Env. Inject Run Params to TestStation Start Test Cases Jenkins Plugins Update Codes Publish Results Onos System Test CI
- 9. 9#ONOSProject A Deeper Dive into TestON Onos System Test CI
- 10. 10#ONOSProject TestON – Key Benefits • TestON o SDN-focused Testing Framework o Test case authoring, execution o Python-based • Key Benefits: o Extensibility – Simplify case authoring by compartmentalizing concerns o Flexibility - Handle heterogeneous test env. OVS, HW, Quagga, etc. o Debuggability - Facilitate troubleshooting
- 11. 11#ONOSProject *.params *.py*.topo Dependency Details on TestON Framework TestON Core Framework: Logging, Utilities, … Mininet Driver ONOS Rest Driver Quagga Driver ONOS CLI Driver Test Files .params File – XML file, used to pass variables into test, such as order of test cases .topo file – XML File, defines the components of the test Dependency Folder – Config files or Mininet topology files used by the test .py file – Combine TestON and driver functions to define your test cases. Test Cases are split into steps, each of which should have an assertion in them • Driver Files • Can use REST, CLI via SSH, or other custom connection interfaces • Defined in a hierarchical structure • Creates a library of functions to be used in the test
- 12. 12#ONOSProject ONOS Intent Feature • Set intent through NB REST interface • Validate intent and flows are correctly added • Test end-to-end connectivity in Mininet with the specific selectors System Test Procedure • Need a library to send packets with test-specified selector • Leveraging Python Scapy module System Test Requirement • High-level abstraction of setting end to end connectivity • Can be defined by “Selectors” of MAC, IP, PORT, MPLS label, etc. Example: ONOS Intent Functionality Test
- 13. 13#ONOSProject Building a Scapy Driver (send/recv user-defined packets) class ScapyCliDriver( Emulator ): def sendPacket( self, iface=None, packet=None, timeout=1 ): try: sendCmd = 'srp( ' if packet: sendCmd += packet else: ... if iface: sendCmd += ", iface='{}'".format( iface ) sendCmd += ', timeout=' + str( timeout ) + ')' self.handle.sendline( sendCmd ) self.handle.expect( self.scapyPrompt ) except: ... TestON |--drivers | |--common | |--api | |--cli | | |--emulator | | |--ScapyCliDriver.py |--tests Example: ONOS’ Intent Functionality Test
- 14. 14#ONOSProject for hostName in [ “h1”,”h2” ]: # Create two host components with the scapy driver4 main.Scapy.createHostComponent( hostName ) main.step( ”Login to scapy on these new components" ) for host in [ main.h1, main.h2 ]: host.startHostCli() host.startScapy() Using Scapy Library - in Mininet Host Handle <Scapy> <host>192.168.1.101</host> <user>admin</user> <password></password> <type>ScapyCliDriver</type> <connect_order>6</connect_order> </Scapy> TestON |--drivers |--tests |--Example |--Example.py |--Example.params |--Example.topo Example: ONOS’ Intent Functionality Test
- 15. 15#ONOSProject # Add connectivity from h1 to h2 (unidirectional) addResult = main.ONOS1.addPointIntent( ingress, egress, ethDst, ipDst) main.h2.startFilter() # Start packet filter on host2 main.h1.buildEther( dst=main.h2.hostMac ) # build ethernet frame main.h1.buildIP( dst=main.h2.hostIp ) # Build IP frame main.h1.sendPacket() # Send built packet finished = main.h2.checkFilter() # Check if host2 received any packets Intent Test Case - Using Scapy Driver TestON |--drivers |--tests |--Example |--Example.py |--Example.params |--Example.topo Example: ONOS’ Intent Functionality Test
- 16. 16#ONOSProject • Parameterize Test Env & Run Var – Jenkins to inject at run time – Test cases deal with “pure” test logic • Extensive Exception Handling in Drivers – Take care of exceptions, timeout – Use “retry” method • Test Dependencies – Run-time Activation – “Dockerize” Fixture (Future Work) • Debug Utilities – Test pausing – Log collation – More (Future Work) • Scale Test Network (Future Work) Lessons Learned and Future Work
- 17. 17#ONOSProject For more Information • S3 Showcase: “SDN/ONOS System Test” • Wiki: https://wiki.onosproject.org/display/ONOS/System+Tests • Repo: https://gerrit.onosproject.org/#/admin/projects/OnosSystemTest • Github: https://github.com/opennetworkinglab/OnosSystemTest Summary Python-Based Extensibility DebuggabilityFlexibility ONOS Test Suites: Functionality High Availability Performance Scale-out Capability Longevity Use Cases ONOS System Test Highlights Open Source
- 18. 18#ONOSProject Join the journey @ onosproject.org Software Defined Transformation of Service Provider Networks Q&A
Editor's Notes
- Good Afternoon. Welcome to our talk on System Testing SDN I am Suibin Zhang from Open Net… My co-presenter is Jon Hall from Ciena Today we will use ONOS as a case study to present to you some of our learning expereince in developing SDN system tests.
- First of all, SDN involves complex software. This is one view of how to quality assuring a complex software system We look at the typical 4 quadrants of subjective perceptions and objective reality.
- Applications, Devices – OF, etc. ONOS Network operator qualify ONOS Using Black/grey box approach – test scenarios not influenced by internal design
- Building a system integration test is non-trivial - SDN network domain - Controllers, usually in a cluster - Apps How to make a test executor handle control of the complex env?
- Pure Technical challenges APIs – fast changing in early product stage; CLI, REST, etc. How to mimic a large network with hundreds/thousands of switches - same time isolating env-caused issues from real product issues Test should facilitate debugging activities – adding in test case hooks to pause, replay Engaging Community – another set of challenges
- In ONOS, what we have built.
- Describe the cluster components - TestON is a test authoring, execution framework – more details later
- Jon will be diving in deeper in to the TestON framework
- Applications, Devices – OF, etc. ONOS Network operator qualify ONOS Using Black/grey box approach – test scenarios not influenced by internal design
- In summary, we presented to you An SDN System Test Framework That we have used to build all these test suites What enable us to do that is with those benefits