research-article

NetBOA: Self-Driving Network Benchmarking

Authors:

Johannes Zerwas,

Patrick Kalmbach,

Laurenz Henkel,

Gábor Rétvári,

Wolfgang Kellerer,

Stefan SchmidAuthors Info & Claims

NetAI'19: Proceedings of the 2019 Workshop on Network Meets AI & ML

Pages 8 - 14

https://doi.org/10.1145/3341216.3342207

Published: 14 August 2019 Publication History

Abstract

Communication networks have not only become a critical infrastructure of our digital society, but are also increasingly complex and hence error-prone. This has recently motivated the study of more automated and "self-driving" networks: networks which measure, analyze, and control themselves in an adaptive manner, reacting to changes in the environment. In particular, such networks hence require a mechanism to recognize potential performance issues.

This paper presents NetBOA, an adaptive and "data-driven" approach to measure network performance, allowing the network to identify bottlenecks and to perform automated what-if analysis, exploring improved network configurations. As a case study, we demonstrate how the NetBOA approach can be used to bench-mark a popular software switch, Open vSwitch. We report on our implementation and evaluation, and show that NetBOA can find performance issues efficiently, compared to a non-data-driven approach. Our results hence indicate that NetBOA may also be useful to identify algorithmic complexity attacks.

Supplementary Material

MP4 File (p8-zerwas.mp4)

Download
645.16 MB

References

[1]

Omid Alipourfard, Hongqiang Harry Liu, Jianshu Chen, Shivaram Venkataraman, Minlan Yu, and Ming Zhang. 2017. CherryPick: Adaptively Unearthing the Best Cloud Configurations for Big Data Analytics. Proc. USENIX NSDI (2017), 469--482.

Digital Library

[2]

Simon Bauer, Daniel Raumer, Paul Emmerich, and Georg Carle. 2018. Behind the scenes: what device benchmarks can tell us. (2018).

[3]

Ryan Beckett, Aarti Gupta, Ratul Mahajan, and David Walker. 2017. A General Approach to Network Configuration Verification. In Proc. ACM SIGCOMM. Los Angeles, CA, USA, 155--168.

Digital Library

[4]

Andreas Blenk, Patrick Kalmbach, Stefan Schmid, and Wolfgang Kellerer. 2017. o'zapft is: Tap Your Network Algorithm's Big Data!. In Proc. ACM SIGCOMM Big-DAMA. Los Angeles, CA, USA.

Digital Library

[5]

S. Bradner and J. McQuaid. 1999. Benchmarking Methodology for Network Interconnect Devices. RFC 2544. (March 1999).

Digital Library

[6]

Levente Csikor, Christian Rothenberg, Dimitrios P. Pezaros, Stefan Schmid, László Toka, and Gábor Rétvári. 2018. Policy Injection: A Cloud Dataplane DoS Attack. In Proc. ACM SIGCOMM Conference on Posters and Demos. Budapest, Hungary, 147--149.

Digital Library

[7]

Paul Emmerich, Daniel Raumer, Florian Wohlfart, and Georg Carle. 2014. Performance characteristics of virtual switching. In Proc. IEEE CloudNet. IEEE, Luxembourg, Luxembourg, 120--125.

[8]

Vlad M. Cora Eric Brochu and Nando de Freitas. 2010. "A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning". arXiv preprint (2010). https://arxiv.org/pdf/1012.2599.pdf

[9]

Vivian Fang, Tamás Lévai, Sangjin Han, Sylvia Ratnasamy, Barath Raghavan, and Justine Sherry. 2018. Evaluating Software Switches: Hard or Hopeless? Technical Report. 8 pages.

[10]

Nick Feamster and Jennifer Rexford. 2017. Why (and how) networks should run themselves. arXiv preprint arXiv:1710.11583 (2017).

[11]

Matthias Feurer, Jost Tobias Springenberg, and Frank Hutter. 2015. Initializing Bayesian Hyperparameter Optimization via Meta-learning. In Proc. AAAI (AAAI'15). AAAI Press, Austin, Texas, USA, 1128--1135.

Digital Library

[12]

Chin-Jung Hsu, Vivek Nair, Tim Menzies, and Vincent W. Freeh. 2018. Scout: An Experienced Guide to Find the Best Cloud Configuration. arXiv:1803.01296 {cs} (March 2018). http://arxiv.org/abs/1803.01296

[13]

Danny Yuxing Huang, Kenneth Yocum, and Alex C Snoeren. 2013. High-fidelity switch models for software-defined network emulation. In Proc. of the second ACM SIGCOMM workshop on HotSDN. 43--48.

Digital Library

[14]

Rishabh Iyer, Luis Pedrosa, Arseniy Zaostrovnykh, Solal Pirelli, Katerina Argyraki, and George Candea. 2019. Performance Contracts for Software Network Functions. In Proc. USENIX NSDI. Boston, MA, USA, 517--530.

Digital Library

[15]

Michael Jarschel, Simon Oechsner, Daniel Schlosser, Rastin Pries, Sebastian Goll, and Phuoc Tran-Gia. 2011. Modeling and performance evaluation of an OpenFlow architecture. In Proc. ITC. San Francisco, CA, USA, 1--7.

Digital Library

[16]

Patrick Kalmbach, Johannes Zerwas, Peter Babarczi, Andreas Blenk, Wolfgang Kellerer, and Stefan Schmid. 2018. Empowering Self-Driving Networks. In Proc. ACM SIGCOMM 2018 Workshop on Self-Driving Networks (SDN). Budapest, Hungary.

Digital Library

[17]

Wolfgang Kellerer, Patrick Kalmbach, Andreas Blenk, Arsany Basta, Martin Reisslein, and Stefan Schmid. 2019. Adaptable and Data-Driven Softwarized Networks: Review, Opportunities, and Challenges. Proc. IEEE 107, 4 (April 2019), 711--731.

[18]

Maciej Kuźniar, Peter Perešíni, and Dejan Kostić. 2015. What you need to know about SDN flow tables. In Passive and Active Measurement. Springer, 347--359.

[19]

Caroline Lemieux, Rohan Padhye, Koushik Sen, and Dawn Song. 2018. Perf-Fuzz: automatically generating pathological inputs. In Proc. ACM SIGSOFT. ACM, Amsterdam, Netherlands, 254--265.

Digital Library

[20]

T. Lévai, G. Pongrácz, P. Megyesi, P. Vörös, S. Laki, F. Németh, and G. Rétvári. 2018. The Price for Programmability in the Software Data Plane: The Vendor Perspective. IEEE Jrnl. on Sel. Ar. in Com. 36, 12 (Dec. 2018), 2621--2630.

[21]

Jed Liu, William Hallahan, Cole Schlesinger, Milad Sharif, Jeongkeun Lee, Robert Soulé, Han Wang, Călin Caşcaval, Nick McKeown, and Nate Foster. 2018. P4V: Practical Verification for Programmable Data Planes. In Proc. ACM SIGCOMM. Budapest, Hungary, 490--503.

Digital Library

[22]

Chenyang Lv, Shouling Ji, Yuwei Li, Junfeng Zhou, Jianhai Chen, Pan Zhou, and Jing Chen. 2018. SmartSeed: Smart Seed Generation for Efficient Fuzzing. arXiv:1807.02606 {cs} (July 2018).

[23]

Mark McLeod, Michael A. Osborne, and Stephen J. Roberts. 2018. Optimization, fast and slow: optimally switching between local and Bayesian optimization. arXiv:1805.08610 {cs, stat} (May 2018).

[24]

Wei Meng, Chenxiong Qian, Shuang Hao, Kevin Borgolte, Giovanni Vigna, Christopher Kruegel, and Wenke Lee. 2018. Rampart: Protecting Web Applications from CPU-Exhaustion Denial-of-Service Attacks. In USENIX Security. Baltimore, MD, USA, 393--410.

Digital Library

[25]

Luis Pedrosa, Rishabh Iyer, Arseniy Zaostrovnykh, Jonas Fietz, and Katerina Argyraki. 2018. Automated Synthesis of Adversarial Workloads for Network Functions. In Proc. ACM SIGCOMM. Budapest, Hungary, 372--385.

Digital Library

[26]

Theofilos Petsios, Jason Zhao, Angelos D. Keromytis, and Suman Jana. 2017. SlowFuzz: Automated Domain-Independent Detection of Algorithmic Complexity Vulnerabilities. In Proc. ACM SIGSAC CCS. Dallas, Texas, USA, 2155--2168.

Digital Library

[27]

Ben Pfaff, Justin Pettit, Teemu Koponen, Ethan Jackson, Andy Zhou, Jarno Rajahalme, Jesse Gross, Alex Wang, Joe Stringer, Pravin Shelar, Keith Amidon, and Martin Casado. 2015. The Design and Implementation of Open vSwitch. In Proc. USENIX NSDI. Oakland, CA, USA, 117--130.

Digital Library

[28]

Daniel Raumer, Sebastian Gallenmüller, Florian Wohlfart, Paul Emmerich, Patrick Werneck, and Georg Carle. 2016. Revisiting benchmarking methodology for interconnect devices. In Proc. ACM ANRW. Berlin, Germany, 55--61.

Digital Library

[29]

Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. 2012. Practical Bayesian Optimization of Machine Learning Algorithms. arXiv:1206.2944 {cs, stat} (June 2012). http://arxiv.org/abs/1206.2944 arXiv: 1206.2944.

[30]

Jiayi Wei, Jia Chen, Yu Feng, Kostas Ferles, and Isil Dillig. 2018. Singularity: Pattern Fuzzing for Worst Case Complexity. In Proc. ACM ESEC/FSE. ACM, Lake Buena Vista, FL, USA, 213--223.

Digital Library

Cited By

Schmidt SZerwas JKellerer W(2023)AdFAT: Adversarial Flow Arrival Time Generation for Demand-Oblivious Data Center Networks2023 19th International Conference on Network and Service Management (CNSM)10.23919/CNSM59352.2023.10327896(1-5)Online publication date: 30-Oct-2023
https://doi.org/10.23919/CNSM59352.2023.10327896
Benson TYoneki ENardi L(2023)Illuminating the hidden challenges of data-driven CDNsProceedings of the 3rd Workshop on Machine Learning and Systems10.1145/3578356.3592574(94-103)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3578356.3592574
Yu SZeng QChen JChen YMambretti J(2023)AIDTN: Towards a Real-Time AI Optimized DTN System With NVMeoFIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.3260806(1-12)Online publication date: 2023
https://doi.org/10.1109/TPDS.2023.3260806
Show More Cited By

Index Terms

NetBOA: Self-Driving Network Benchmarking
1. Computing methodologies
  1. Machine learning
2. Networks
  1. Network performance evaluation
    1. Network experimentation
  2. Network properties
    1. Network reliability
    2. Network security

Recommendations

Empowering Self-Driving Networks
SelfDN 2018: Proceedings of the Afternoon Workshop on Self-Driving Networks

As emerging network technologies and softwareization render networks more flexible, the question arises of how to exploit these flexibilities for optimization. Given the complexity of the involved network protocols and the context in which networks are ...
Refining Network Intents for Self-Driving Networks
SelfDN 2018: Proceedings of the Afternoon Workshop on Self-Driving Networks

Recent advances in artificial intelligence (AI) offer an opportunity for the adoption of self-driving networks. However, network operators or home-network users still do not have the right tools to exploit these new advancements in AI, since they have ...
Refining network intents for self-driving networks

Recent advances in artificial intelligence (AI) offer an opportunity for the adoption of self-driving networks. However, network operators or home-network users still do not have the right tools to exploit these new advancements in AI, since they have ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

NetAI'19: Proceedings of the 2019 Workshop on Network Meets AI & ML

August 2019

96 pages

ISBN:9781450368728

DOI:10.1145/3341216

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Horizon 2020
Deutsche Forschungsgemeinschaft

Conference

SIGCOMM '19

Sponsor:

SIGCOMM

SIGCOMM '19: ACM SIGCOMM 2019 Conference

August 23, 2019

Beijing, China

Acceptance Rates

NetAI'19 Paper Acceptance Rate 13 of 38 submissions, 34%;

Overall Acceptance Rate 13 of 38 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
602
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)2

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Schmidt SZerwas JKellerer W(2023)AdFAT: Adversarial Flow Arrival Time Generation for Demand-Oblivious Data Center Networks2023 19th International Conference on Network and Service Management (CNSM)10.23919/CNSM59352.2023.10327896(1-5)Online publication date: 30-Oct-2023
https://doi.org/10.23919/CNSM59352.2023.10327896
Benson TYoneki ENardi L(2023)Illuminating the hidden challenges of data-driven CDNsProceedings of the 3rd Workshop on Machine Learning and Systems10.1145/3578356.3592574(94-103)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3578356.3592574
Yu SZeng QChen JChen YMambretti J(2023)AIDTN: Towards a Real-Time AI Optimized DTN System With NVMeoFIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.3260806(1-12)Online publication date: 2023
https://doi.org/10.1109/TPDS.2023.3260806
Zhang HWei Y(2023)INVA: An Intelligent Network Virtualization Architecture for Big Data Platform2023 9th International Conference on Big Data Computing and Communications (BigCom)10.1109/BIGCOM61073.2023.00011(16-23)Online publication date: 4-Aug-2023
https://doi.org/10.1109/BIGCOM61073.2023.00011
Alcock PSimms BFantom WRotsos CRace N(2022)Improving Intent Correctness with Automated Testing2022 IEEE 8th International Conference on Network Softwarization (NetSoft)10.1109/NetSoft54395.2022.9844054(61-66)Online publication date: 27-Jun-2022
https://doi.org/10.1109/NetSoft54395.2022.9844054
Riboni AGhioldi NCandelieri ABorrotti M(2022)Bayesian optimization and deep learning for steering wheel angle predictionScientific Reports10.1038/s41598-022-12509-612:1Online publication date: 24-May-2022
https://doi.org/10.1038/s41598-022-12509-6
Luizelli MCanofre RLorenzon ARossi FCordeiro WCaicedo O(2021)In-Network Neural Networks: Challenges and Opportunities for InnovationIEEE Network10.1109/MNET.101.210009835:6(68-74)Online publication date: Nov-2021
https://doi.org/10.1109/MNET.101.2100098
Dzeparoska KBeigi-Mohammadi NTizghadam ALeon-Garcia A(2021)Towards a Self-Driving Management System for the Automated Realization of IntentsIEEE Access10.1109/ACCESS.2021.31299909(159882-159907)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3129990
Amin RRojas EAqdus ARamzan SCasillas-Perez DArco J(2021)A Survey on Machine Learning Techniques for Routing Optimization in SDNIEEE Access10.1109/ACCESS.2021.30990929(104582-104611)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3099092
Lavinia YDurairajan RRejaie RWillinger W(2020)Challenges in Using ML for Networking ResearchProceedings of the Workshop on Network Meets AI & ML10.1145/3405671.3405812(21-27)Online publication date: 10-Aug-2020
https://dl.acm.org/doi/10.1145/3405671.3405812
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents