Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article
Open access

Comparative Synthesis: Learning Near-Optimal Network Designs by Query

Published: 11 January 2023 Publication History

Abstract

When managing wide-area networks, network architects must decide how to balance multiple conflicting metrics, and ensure fair allocations to competing traffic while prioritizing critical traffic. The state of practice poses challenges since architects must precisely encode their intent into formal optimization models using abstract notions such as utility functions, and ad-hoc manually tuned knobs. In this paper, we present the first effort to synthesize optimal network designs with indeterminate objectives using an interactive program-synthesis-based approach. We make three contributions. First, we present comparative synthesis, an interactive synthesis framework which produces near-optimal programs (network designs) through two kinds of queries (Validate and Compare), without an objective explicitly given. Second, we develop the first learning algorithm for comparative synthesis in which a voting-guided learner picks the most informative query in each iteration. We present theoretical analysis of the convergence rate of the algorithm. Third, we implemented Net10Q, a system based on our approach, and demonstrate its effectiveness on four real-world network case studies using black-box oracles and simulation experiments, as well as a pilot user study comprising network researchers and practitioners. Both theoretical and experimental results show the promise of our approach.

Supplementary Material

Auxiliary Archive (popl23main-p33-p-archive.zip)
This appendix includes additional experimental results.

References

[1]
Glenn Ammons, Rastislav Bodík, and James R. Larus. 2002. Mining Specifications. In Proceedings of the 29th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’02). Association for Computing Machinery, New York, NY, USA. 4–16. isbn:1581134509 https://doi.org/10.1145/503272.503275
[2]
Dana Angluin. 2004. Queries revisited. Theoretical Computer Science, 313, 2 (2004), 175 – 194. issn:0304-3975 https://doi.org/10.1016/j.tcs.2003.11.004 Algorithmic Learning Theory
[3]
Ryan Beckett, Aarti Gupta, Ratul Mahajan, and David Walker. 2019. Abstract Interpretation of Distributed Network Control Planes. Proc. ACM Program. Lang., 4, POPL (2019), Article 42, Dec., 27 pages. https://doi.org/10.1145/3371110
[4]
Rüdiger Birkner, Dana Drachsler-Cohen, Laurent Vanbever, and Martin Vechev. 2020. Config2Spec: Mining Network Specifications from Network Configurations. In Proceedings of 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’20).
[5]
Jeremy Bogle, Nikhil Bhatia, Manya Ghobadi, Ishai Menache, Nikolaj Bjørner, Asaf Valadarsky, and Michael Schapira. 2019. TEAVAR: Striking the Right Utilization-Availability Balance in WAN Traffic Engineering. In Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM ’19). Association for Computing Machinery, New York, NY, USA. 29–43. isbn:9781450359566 https://doi.org/10.1145/3341302.3342069
[6]
James Bornholt, Emina Torlak, Dan Grossman, and Luis Ceze. 2016. Optimizing Synthesis with Metasketches. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’16). Association for Computing Machinery, New York, NY, USA. 775–788. isbn:9781450335492 https://doi.org/10.1145/2837614.2837666
[7]
Stephen Boyd and Lieven Vandenberghe. 2004. Convex Optimization. Cambridge University Press, USA. isbn:0521833787
[8]
Gregory J. Chaitin. 1975. A Theory of Program Size Formally Identical to Information Theory. J. ACM, 22, 3 (1975), July, 329–340. issn:0004-5411 https://doi.org/10.1145/321892.321894
[9]
Yiyang Chang, Sanjay Rao, and Mohit Tawarmalani. 2017. Robust Validation of Network Designs under Uncertain Demands and Failures. In 14^th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 347–362.
[10]
Swarat Chaudhuri, Martin Clochard, and Armando Solar-Lezama. 2014. Bridging Boolean and Quantitative Synthesis Using Smoothed Proof Search. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’14). Association for Computing Machinery, New York, NY, USA. 207–220. isbn:9781450325448 https://doi.org/10.1145/2535838.2535859
[11]
Emilie Danna, Subhasree Mandal, and Arjun Singh. 2012. A practical algorithm for balancing the max-min fairness and throughput objectives in traffic engineering. In 2012 Proceedings IEEE INFOCOM. 846–854. https://doi.org/10.1109/INFCOM.2012.6195833
[12]
Leonardo Mendonça de Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In TACAS’08 (LNCS, Vol. 4963). Springer, 337–340. isbn:978-3-540-78799-0 https://doi.org/10.1007/978-3-540-78800-3_24
[13]
Thomas G. Dietterich. 2000. Ensemble Methods in Machine Learning. Lecture Notes in Computer Science, 1–15. isbn:9783540450146 issn:0302-9743 https://doi.org/10.1007/3-540-45014-9_1
[14]
Dana Drachsler-Cohen, Sharon Shoham, and Eran Yahav. 2017. Synthesis with Abstract Examples. In Computer Aided Verification, Rupak Majumdar and Viktor Kunčak (Eds.). Springer International Publishing, Cham. 254–278. isbn:978-3-319-63387-9
[15]
Ian Drosos, Titus Barik, Philip J. Guo, Robert DeLine, and Sumit Gulwani. 2020. Wrex: A Unified Programming-by-Example Interaction for Synthesizing Readable Code for Data Scientists. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). Association for Computing Machinery, New York, NY, USA. 1–12. isbn:9781450367080 https://doi.org/10.1145/3313831.3376442
[16]
Ahmed El-Hassany, Petar Tsankov, Laurent Vanbever, and Martin Vechev. 2017. Network-Wide Configuration Synthesis. In Computer Aided Verification, Rupak Majumdar and Viktor Kunčak (Eds.). Springer International Publishing, Cham. 261–281. isbn:978-3-319-63390-9
[17]
Ahmed El-Hassany, Petar Tsankov, Laurent Vanbever, and Martin Vechev. 2018. NetComplete: Practical Network-Wide Configuration Synthesis with Autocompletion. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). USENIX Association, Renton, WA. 579–594. isbn:978-1-931971-43-0 https://www.usenix.org/conference/nsdi18/presentation/el-hassany
[18]
B. Fortz and M. Thorup. 2000. Internet traffic engineering by optimizing OSPF weights. In INFOCOM 2000. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE. 519–528.
[19]
Xiangyu Gao, Taegyun Kim, Aatish Kishan Varma, Anirudh Sivaraman, and Srinivas Narayana. 2019. Autogenerating Fast Packet-Processing Code Using Program Synthesis. In Proceedings of the 18th ACM Workshop on Hot Topics in Networks (HotNets ’19). Association for Computing Machinery, New York, NY, USA. 150–160. isbn:9781450370202 https://doi.org/10.1145/3365609.3365858
[20]
Pranav Garg, Christof Löding, P. Madhusudan, and Daniel Neider. 2014. ICE: A Robust Framework for Learning Invariants. In CAV’14. 69–87. https://doi.org/10.1007/978-3-319-08867-9_5
[21]
Walter Gautschi. 1997. Numerical analysis: an introduction. Birkhäuser, 215.
[22]
A. Ghosh, Sangtae Ha, E. Crabbe, and J. Rexford. 2013. Scalable Multi-Class Traffic Management in Data Center Backbone Networks. IEEE Journal on Selected Areas in Communications, 31 (2013), 2673–2684.
[23]
Sumit Gulwani, Kunal Pathak, Arjun Radhakrishna, Ashish Tiwari, and Abhishek Udupa. 2019. Quantitative Programming by Examples. arxiv:1909.05964.
[24]
LLC Gurobi Optimization. 2020. Gurobi Optimizer Reference Manual. http://www.gurobi.com
[25]
Nikola Gvozdiev, Stefano Vissicchio, Brad Karp, and Mark Handley. 2018. On Low-Latency-Capable Topologies, and Their Impact on the Design of Intra-Domain Routing. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication (SIGCOMM ’18). Association for Computing Machinery, New York, NY, USA. 88–102. isbn:9781450355674 https://doi.org/10.1145/3230543.3230575
[26]
Chi-Yao Hong, Srikanth Kandula, Ratul Mahajan, Ming Zhang, Vijay Gill, Mohan Nanduri, and Roger Wattenhofer. 2013. Achieving High Utilization with Software-driven WAN. In Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM (SIGCOMM ’13). ACM, New York, NY, USA. 15–26. isbn:978-1-4503-2056-6 https://doi.org/10.1145/2486001.2486012
[27]
Qinheping Hu and Loris D’Antoni. 2018. Syntax-Guided Synthesis with Quantitative Syntactic Objectives. In Computer Aided Verification, Hana Chockler and Georg Weissenbacher (Eds.). Springer International Publishing, Cham. 386–403. isbn:978-3-319-96145-3
[28]
Sushant Jain, Alok Kumar, Subhasree Mandal, Joon Ong, Leon Poutievski, Arjun Singh, Subbaiah Venkata, Jim Wanderer, Junlan Zhou, Min Zhu, and et al. 2013. B4: Experience with a Globally-Deployed Software Defined Wan. SIGCOMM Comput. Commun. Rev., 43, 4 (2013), Aug., 3–14. issn:0146-4833 https://doi.org/10.1145/2534169.2486019
[29]
Susmit Jha, Sumit Gulwani, Sanjit A. Seshia, and Ashish Tiwari. 2010. Oracle-Guided Component-Based Program Synthesis. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1 (ICSE ’10). Association for Computing Machinery, New York, NY, USA. 215–224. isbn:9781605587196 https://doi.org/10.1145/1806799.1806833
[30]
Susmit Jha and Sanjit A. Seshia. 2017. A theory of formal synthesis via inductive learning. Acta Informatica, 54, 7 (2017), Feb, 693–726. issn:1432-0525 https://doi.org/10.1007/s00236-017-0294-5
[31]
Ruyi Ji, Jingjing Liang, Yingfei Xiong, Lu Zhang, and Zhenjiang Hu. 2020. Question Selection for Interactive Program Synthesis. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2020). Association for Computing Machinery, New York, NY, USA. 1143–1158. isbn:9781450376136 https://doi.org/10.1145/3385412.3386025
[32]
Chuan Jiang, Sanjay Rao, and Mohit Tawarmalani. 2020. PCF: Provably Resilient Flexible Routing. In Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM ’20). Association for Computing Machinery, New York, NY, USA. 139–153. isbn:9781450379557 https://doi.org/10.1145/3387514.3405858
[33]
S. Knight, H. X. Nguyen, N. Falkner, R. Bowden, and M. Roughan. 2011. The Internet Topology Zoo. IEEE Journal on Selected Areas in Communications, 29, 9 (2011), October, 1765–1775. issn:1558-0008 https://doi.org/10.1109/JSAC.2011.111002
[34]
Alok Kumar, Sushant Jain, Uday Naik, Anand Raghuraman, Nikhil Kasinadhuni, Enrique Cauich Zermeno, C. Stephen Gunn, Jing Ai, Björn Carlin, Mihai Amarandei-Stavila, and et al. 2015. BwE: Flexible, Hierarchical Bandwidth Allocation for WAN Distributed Computing. SIGCOMM Comput. Commun. Rev., 45, 4 (2015), Aug., 1–14. issn:0146-4833 https://doi.org/10.1145/2829988.2787478
[35]
Praveen Kumar, Yang Yuan, Chris Yu, Nate Foster, Robert Kleinberg, Petr Lapukhov, Chiun Lin Lim, and Robert Soulé. 2018. Semi-Oblivious Traffic Engineering: The Road Not Taken. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). USENIX Association, Renton, WA. 157–170. isbn:978-1-939133-01-4 https://www.usenix.org/conference/nsdi18/presentation/kumar
[36]
Yi Li, Aws Albarghouthi, Zachary Kincaid, Arie Gurfinkel, and Marsha Chechik. 2014. Symbolic Optimization with SMT Solvers. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’14). Association for Computing Machinery, New York, NY, USA. 607–618. isbn:9781450325448 https://doi.org/10.1145/2535838.2535857
[37]
Hongqiang Harry Liu, Srikanth Kandula, Ratul Mahajan, Ming Zhang, and David Gelernter. 2014. Traffic Engineering with Forward Fault Correction. In Proceedings of the 2014 ACM Conference on SIGCOMM (SIGCOMM ’14). Association for Computing Machinery, New York, NY, USA. 527–538. isbn:9781450328364 https://doi.org/10.1145/2619239.2626314
[38]
Mikaël Mayer, Gustavo Soares, Maxim Grechkin, Vu Le, Mark Marron, Oleksandr Polozov, Rishabh Singh, Benjamin Zorn, and Sumit Gulwani. 2015. User Interaction Models for Disambiguation in Programming by Example. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST ’15). Association for Computing Machinery, New York, NY, USA. 291–301. isbn:9781450337793 https://doi.org/10.1145/2807442.2807459
[39]
Jedidiah McClurg, Hossein Hojjat, Nate Foster, and Pavol Černý. 2016. Event-Driven Network Programming. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’16). Association for Computing Machinery, New York, NY, USA. 369–385. isbn:9781450342612 https://doi.org/10.1145/2908080.2908097
[40]
Jedidiah McClurg, Hossein Hojjat, Pavol Černý, and Nate Foster. 2015. Efficient Synthesis of Network Updates. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’15). Association for Computing Machinery, New York, NY, USA. 196–207. isbn:9781450334686 https://doi.org/10.1145/2737924.2737980
[41]
Kaisa Miettinen, Francisco Ruiz, and Andrzej P. Wierzbicki. 2008. Introduction to Multiobjective Optimization: Interactive Approaches. In Multiobjective Optimization: Interactive and Evolutionary Approaches, Jürgen Branke, Kalyanmoy Deb, Kaisa Miettinen, and Roman Sł owiński (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 27–57. isbn:978-3-540-88908-3 https://doi.org/10.1007/978-3-540-88908-3_2
[42]
Hila Peleg, Sharon Shoham, and Eran Yahav. 2018. Programming Not Only by Example. In Proceedings of the 40th International Conference on Software Engineering (ICSE ’18). Association for Computing Machinery, New York, NY, USA. 1114–1124. isbn:9781450356381 https://doi.org/10.1145/3180155.3180189
[43]
Leonid Ryzhyk, Nikolaj Bjørner, Marco Canini, Jean-Baptiste Jeannin, Cole Schlesinger, Douglas B. Terry, and George Varghese. 2017. Correct by Construction Networks Using Stepwise Refinement. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). USENIX Association, Boston, MA. 683–698. isbn:978-1-931971-37-9 https://www.usenix.org/conference/nsdi17/technical-sessions/presentation/ryzhyk
[44]
Shambwaditya Saha, Santhosh Prabhu, and P. Madhusudan. 2015. NetGen: Synthesizing Data-plane Configurations for Network Policies. In Proceedings of the 1st ACM SIGCOMM Symposium on Software Defined Networking Research (SOSR ’15). ACM, New York, NY, USA. Article 17, 6 pages. isbn:978-1-4503-3451-8 https://doi.org/10.1145/2774993.2775006
[45]
Eric Schkufza, Rahul Sharma, and Alex Aiken. 2013. Stochastic Superoptimization. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS ’13). ACM, New York, NY, USA. 305–316. isbn:978-1-4503-1870-9 https://doi.org/10.1145/2451116.2451150
[46]
Eric Schkufza, Rahul Sharma, and Alex Aiken. 2014. Stochastic Optimization of Floating-point Programs with Tunable Precision. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’14). ACM, New York, NY, USA. 53–64. isbn:978-1-4503-2784-8 https://doi.org/10.1145/2594291.2594302
[47]
Burr Settles. 2012. Active Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 6, 1 (2012), 1–114. https://doi.org/10.2200/S00429ED1V01Y201207AIM018 arxiv:https://doi.org/10.2200/S00429ED1V01Y201207AIM018.
[48]
H. S. Seung, M. Opper, and H. Sompolinsky. 1992. Query by Committee. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory (COLT ’92). Association for Computing Machinery, New York, NY, USA. 287–294. isbn:089791497X https://doi.org/10.1145/130385.130417
[49]
Lei Shi, Yahui Li, Boon Thau Loo, and Rajeev Alur. 2021. Network Traffic Classification by Program Synthesis. In Tools and Algorithms for the Construction and Analysis of Systems, Jan Friso Groote and Kim Guldstrand Larsen (Eds.). Springer International Publishing, Cham. 430–448. isbn:978-3-030-72016-2
[50]
Anirudh Sivaraman, Alvin Cheung, Mihai Budiu, Changhoon Kim, Mohammad Alizadeh, Hari Balakrishnan, George Varghese, Nick McKeown, and Steve Licking. 2016. Packet Transactions: High-Level Programming for Line-Rate Switches. In Proceedings of the 2016 ACM SIGCOMM Conference (SIGCOMM ’16). Association for Computing Machinery, New York, NY, USA. 15–28. isbn:9781450341936 https://doi.org/10.1145/2934872.2934900
[51]
Armando Solar-Lezama. 2016. The Sketch Programmers Manual. Version 1.7.2
[52]
Armando Solar-Lezama, Liviu Tancau, Rastislav Bodik, Sanjit Seshia, and Vijay Saraswat. 2006. Combinatorial sketching for finite programs. In ASPLOS’06. ACM, 404–415.
[53]
Robert Soulé, Shrutarshi Basu, Parisa Jalili Marandi, Fernando Pedone, Robert Kleinberg, Emin Gun Sirer, and Nate Foster. 2014. Merlin: A Language for Provisioning Network Resources. In Proceedings of the 10th ACM International on Conference on Emerging Networking Experiments and Technologies (CoNEXT ’14). ACM, New York, NY, USA. 213–226. isbn:978-1-4503-3279-8 https://doi.org/10.1145/2674005.2674989
[54]
Rayadurgam Srikant. 2004. The Mathematics of Internet Congestion Control (Systems and Control: Foundations and Applications). SpringerVerlag. isbn:0817632271 https://doi.org/10.1007/978-0-8176-8216-3
[55]
Samuel Steffen, Timon Gehr, Petar Tsankov, Laurent Vanbever, and Martin Vechev. 2020. Probabilistic Verification of Network Configurations. In Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM ’20). Association for Computing Machinery, New York, NY, USA. 750–764. isbn:9781450379557 https://doi.org/10.1145/3387514.3405900
[56]
Kausik Subramanian, Anubhavnidhi Abhashkumar, Loris D’Antoni, and Aditya Akella. 2020. Detecting Network Load Violations for Distributed Control Planes. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2020). Association for Computing Machinery, New York, NY, USA. 974–988. isbn:9781450376136 https://doi.org/10.1145/3385412.3385976
[57]
Kausik Subramanian, Loris D’Antoni, and Aditya Akella. 2017. Genesis: Synthesizing Forwarding Tables in Multi-tenant Networks. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL 2017). ACM, New York, NY, USA. 572–585. isbn:978-1-4503-4660-3 https://doi.org/10.1145/3009837.3009845
[58]
Pavol Černý and Thomas A. Henzinger. 2011. From Boolean to Quantitative Synthesis. In Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT ’11). Association for Computing Machinery, New York, NY, USA. 149–154. isbn:9781450307147 https://doi.org/10.1145/2038642.2038666
[59]
Yi Wang, Ioannis Avramopoulos, and Jennifer Rexford. 2009. Design for configurability: rethinking interdomain routing policies from the ground up. IEEE Journal on Selected Areas in Communications, 27, 3 (2009), 336–348. https://doi.org/10.1109/JSAC.2009.090409
[60]
Yanjun Wang, Chuan Jiang, Xiaokang Qiu, and Sanjay G. Rao. 2019. Learning Network Design Objectives Using A Program Synthesis Approach. In Proceedings of the 18th ACM Workshop on Hot Topics in Networks (HotNets ’19). ACM, New York, NY, USA. 69–76. isbn:978-1-4503-7020-2 https://doi.org/10.1145/3365609.3365861
[61]
Ye Wang, Hao Wang, Ajay Mahimkar, Richard Alimi, Yin Zhang, Lili Qiu, and Yang Richard Yang. 2010. R3: Resilient Routing Reconfiguration. In Proceedings of the ACM SIGCOMM 2010 Conference (SIGCOMM ’10). Association for Computing Machinery, New York, NY, USA. 291–302. isbn:9781450302012 https://doi.org/10.1145/1851182.1851218
[62]
Yifei Yuan, Dong Lin, Rajeev Alur, and Boon Thau Loo. 2015. Scenario-based Programming for SDN Policies. In Proceedings of the 11th ACM Conference on Emerging Networking Experiments and Technologies (CoNEXT ’15). ACM, New York, NY, USA. Article 34, 13 pages. isbn:978-1-4503-3412-9 https://doi.org/10.1145/2716281.2836119

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages
Proceedings of the ACM on Programming Languages  Volume 7, Issue POPL
January 2023
2196 pages
EISSN:2475-1421
DOI:10.1145/3554308
  • Editor:
Issue’s Table of Contents
This work is licensed under a Creative Commons Attribution 4.0 International License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 January 2023
Published in PACMPL Volume 7, Issue POPL

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. Optimization
  2. Program Synthesis
  3. Query
  4. Traffic Engineering
  5. User Interaction

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 382
    Total Downloads
  • Downloads (Last 12 months)263
  • Downloads (Last 6 weeks)13
Reflects downloads up to 01 Sep 2024

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Full Access

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media