Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3387514.3405879acmconferencesArticle/Chapter ViewAbstractPublication PagescommConference Proceedingsconference-collections
research-article

Lyra: A Cross-Platform Language and Compiler for Data Plane Programming on Heterogeneous ASICs

Published: 30 July 2020 Publication History

Abstract

Programmable data plane has been moving towards deployments in data centers as mainstream vendors of switching ASICs enable programmability in their newly launched products, such as Broadcom's Trident-4, Intel/Barefoot's Tofino, and Cisco's Silicon One. However, current data plane programs are written in low-level, chip-specific languages (e.g., P4 and NPL) and thus tightly coupled to the chip-specific architecture. As a result, it is arduous and error-prone to develop, maintain, and composite data plane programs in production networks. This paper presents Lyra, the first cross-platform, high-level language & compiler system that aids the programmers in programming data planes efficiently. Lyra offers a one-big-pipeline abstraction that allows programmers to use simple statements to express their intent, without laboriously taking care of the details in hardware; Lyra also proposes a set of synthesis and optimization techniques to automatically compile this "big-pipeline" program into multiple pieces of runnable chip-specific code that can be launched directly on the individual programmable switches of the target network. We built and evaluated Lyra. Lyra not only generates runnable real-world programs (in both P4 and NPL), but also uses up to 87.5% fewer hardware resources and up to 78% fewer lines of code than human-written programs.

Supplementary Material

MP4 File (3387514.3405879.mp4)
Programmable data plane has been moving towards deployments in data centers as mainstream vendors of switching ASICs enable programmability in their newly launched products, such as Broadcom?s Trident-4 and Barefoot?s Tofino. However, current data planes are written in low-level, chip languages (e.g., P4 and NPL) and thus tightly coupled to the chip-specific architecture. It is arduous and error-prone to develop and composite data plane programs in production networks. We present Lyra, the first cross-platform, high-level language compiler that aids the programmer in programming data planes efficiently. Lyra offers a one-big-pipeline abstraction that allows programmers to easily express their intent, without taking care of the hardware details; Lyra also proposes a set of synthesis and optimization techniques to compile this ?big-pipeline? program into chip-specific code. We built and evaluated Lyra. Data plane programs generated by Lyra use up to 87.5% fewer hardware resources than human-written programs.

References

[1]
2019. Broadcom's new Trident 4 and Jericho 2 switch devices offer programmability at scale. https://www.broadcom.com/blog/trident4-and-jericho2-offer-programmability-at-scale.
[2]
2019. In-band Network Telemetry (INT) Dataplane Specification. https://github.com/p4lang/p4-applications/blob/master/docs/INT.pdf.
[3]
2019. NPL 1.3 Specification. https://github.com/nplang/NPL-Spec.
[4]
2019. ONE Silicon, ONE Experience, MULTIPLE Roles. https://blogs.cisco.com/sp/one-silicon-one-experience-multiple-roles.
[5]
2019. p4c, a reference compiler for P4 programming language. https://github.com/p4lang/p4c.
[6]
2020. Barefoot Tofino. https://www.barefootnetworks.com/products/brief-tofino.
[7]
2020. Barefoot Tofino's 32Q-model and 64Q-model. https://www.arista.com/en/products/7170-series.
[8]
2020. Netcope P4 - Flexible FPGA Programming. https://www.netcope.com/en/products/netcopep4.
[9]
2020. Netronome P4. https://www.netronome.com/technology/p4/.
[10]
2020. P4Runtime. https://p4.org/p4-runtime/.
[11]
2020. Pensando Expands What SmartNIC Offloads Can Do. https://pivotnine.com/2020/05/18/pensando-expands-what-smartnic-offloads-can-do/.
[12]
Mohammad Alizadeh, Tom Edsall, Sarang Dharmapurikar, Ramanan Vaidyanathan, Kevin Chu, Andy Fingerhut, Vinh The Lam, Francis Matus, Rong Pan, Navindra Yadav, and George Varghese. 2014. CONGA: distributed congestion-aware load balancing for datacenters. In ACM SIGCOMM (SIGCOMM).
[13]
Carolyn Jane Anderson, Nate Foster, Arjun Guha, Jean-Baptiste Jeannin, Dexter Kozen, Cole Schlesinger, and David Walker. 2014. NetKAT: Semantic foundations for networks. In 41st Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL).
[14]
Mina Tahmasbi Arashloo, Yaron Koral, Michael Greenberg, Jennifer Rexford, and David Walker. 2016. SNAP: Stateful network-wide abstractions for packet processing. In ACM SIGCOMM (SIGCOMM).
[15]
Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, and David Walker. 2014. P4: programming protocol-independent packet processors. Computer Communication Review 44, 3 (2014), 87--95.
[16]
Pat Bosshart, Glen Gibb, Hun-Seok Kim, George Varghese, Nick McKeown, Martin Izzard, Fernando A. Mujica, and Mark Horowitz. 2013. Forwarding metamorphosis: fast programmable match-action processing in hardware for SDN. In ACM SIGCOMM (SIGCOMM).
[17]
Pat Bosshart, Glen Gibb, Hun-Seok Kim, George Varghese, Nick McKeown, Martin Izzard, Fernando Mujica, and Mark Horowitz. 2013. Forwarding metamorphosis: Fast programmable match-action processing in hardware for SDN. ACM SIGCOMM Computer Communication Review 43, 4 (2013).
[18]
Leonardo Mendonça de Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In 14th Tools and Algorithms for the Construction and Analysis of Systems (TACAS).
[19]
Nate Foster, Rob Harrison, Michael J. Freedman, Christopher Monsanto, Jennifer Rexford, Alec Story, and David Walker. 2011. Frenetic: a network programming language. In 16th ACM SIGPLAN international conference on Functional Programming (ICFP).
[20]
Xiangyu Gao, Taegyun Kim, Aatish Kishan Varma, Anirudh Sivaraman, and Srinivas Narayana. 2019. Autogenerating fast packet-processing code using program synthesis. In 18th ACM Workshop on Hot Topics in Networks (HotNets).
[21]
Xiangyu Gao, Taegyun Kim, Michael D. Wong, Divya Raghunathan, Aatish Kishan Varma, Pravein Govindan Kannan, Anirudh Sivaraman, Srinivas Narayana, and Aarti Gupta. 2020. Switch code generation using program synthesis. In ACM SIGCOMM (SIGCOMM).
[22]
Arpit Gupta, Rob Harrison, Marco Canini, Nick Feamster, Jennifer Rexford, and Walter Willinger. 2018. Sonata: query-driven streaming network telemetry. In ACM SIGCOMM (SIGCOMM).
[23]
David Hancock and Jacobus E. van der Merwe. 2016. HyPer4: Using P4 to virtualize the programmable data plane. In 12th International Conference on emerging Networking EXperiments and Technologies (CoNEXT).
[24]
Xin Jin, Xiaozhou Li, Haoyu Zhang, Nate Foster, Jeongkeun Lee, Robert Soulé, Changhoon Kim, and Ion Stoica. 2018. NetChain: Scale-free sub-RTT coordination. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI).
[25]
Xin Jin, Xiaozhou Li, Haoyu Zhang, Robert Soulé, Jeongkeun Lee, Nate Foster, Changhoon Kim, and Ion Stoica. 2017. NetCache: Balancing key-value stores with fast in-network caching. In 26th Symposium on Operating Systems Principles (SOSP).
[26]
Lavanya Jose, Lisa Yan, George Varghese, and Nick McKeown. 2015. Compiling packet programs to reconfigurable switches. In 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI).
[27]
Naga Praveen Katta, Mukesh Hira, Changhoon Kim, Anirudh Sivaraman, and Jennifer Rexford. 2016. HULA: Scalable load balancing using programmable data planes. In Symposium on SDN Research (SOSR).
[28]
Jialin Li, Ellis Michael, Naveen Kr. Sharma, Adriana Szekeres, and Dan R. K. Ports. 2016. Just say NO to Paxos overhead: Replacing consensus with network ordering. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI).
[29]
Yuliang Li, Rui Miao, Hongqiang Harry Liu, Yan Zhuang, Fei Feng, Lingbo Tang, Zheng Cao, Ming Zhang, Frank Kelly, Mohammad Alizadeh, and Minlan Yu. 2019. HPCC: High precision congestion control. In ACM SIGCOMM (SIGCOMM).
[30]
Hongqiang Harry Liu, Yibo Zhu, Jitu Padhye, Jiaxin Cao, Sri Tallapragada, Nuno P. Lopes, Andrey Rybalchenko, Guohan Lu, and Lihua Yuan. 2017. CrystalNet: Faithfully emulating large production networks. In 26th Symposium on Operating Systems Principles (SOSP).
[31]
Jedidiah McClurg, Hossein Hojjat, Nate Foster, and Pavol Cerný. 2016. Event-driven network programming. In 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI).
[32]
Rui Miao, Hongyi Zeng, Changhoon Kim, Jeongkeun Lee, and Minlan Yu. 2017. SilkRoad: Making stateful layer-4 load balancing fast and cheap using switching ASICs. In ACM SIGCOMM (SIGCOMM).
[33]
Christopher Monsanto, Nate Foster, Rob Harrison, and David Walker. 2012. A compiler and run-time system for network programming languages. In 39th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL).
[34]
Christopher Monsanto, Joshua Reich, Nate Foster, Jennifer Rexford, and David Walker. 2013. Composing software defined networks. In 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI).
[35]
Srinivas Narayana, Anirudh Sivaraman, Vikram Nathan, Prateesh Goyal, Venkat Arun, Mohammad Alizadeh, Vimalkumar Jeyakumar, and Changhoon Kim. 2017. Language-directed hardware design for network performance monitoring. In ACM SIGCOMM (SIGCOMM).
[36]
Arjun Singh, Joon Ong, Amit Agarwal, Glen Anderson, Ashby Armistead, Roy Bannon, Seb Boving, Gaurav Desai, Bob Felderman, Paulie Germano, Anand Kanagala, Jeff Provost, Jason Simmons, Eiichi Tanda, Jim Wanderer, Urs Hölzle, Stephen Stuart, and Amin Vahdat. 2015. Jupiter rising: A decade of Clos topologies and centralized control in Google's datacenter network. In ACM SIGCOMM (SIGCOMM).
[37]
Anirudh Sivaraman, Alvin Cheung, Mihai Budiu, Changhoon Kim, Mohammad Alizadeh, Hari Balakrishnan, George Varghese, Nick McKeown, and Steve Licking. 2016. Packet transactions: High-level programming for line-rate switches. In ACM SIGCOMM (SIGCOMM).
[38]
Anirudh Sivaraman, Thomas Mason, Aurojit Panda, Ravi Netravali, and Sai Anirudh Kondaveeti. 2020. Network architecture in the age of programmability. Computer Communication Review 50, 1 (2020).
[39]
Hardik Soni, Myriana Rifai, Praveen Kumar, Ryan Doenges, and Nate Foster. 2020. Composing dataplane programs with μP4. In ACM SIGCOMM (SIGCOMM).
[40]
Dingming Wu, Ang Chen, T. S. Eugene Ng, Guohui Wang, and Haiyong Wang. 2019. Accelerated service chaining on a singles witch ASIC. In 18th ACM Workshop on Hot Topics in Networks (HotNets).
[41]
Yang Richard Yang, Kai Gao, Kerim Gokarslan, Dong Guo, and Christopher Leet. 2019. Magellan: Toward high-level programming and analysis of SDN using flow algebra. In ACM SIGCOMM Workshopon Networking and Programming Languages (NetPL).
[42]
Cheng Zhang, Jun Bi, Yu Zhou, Adbul Basit Dogar, and Jianping Wu. 2017. HyperV: A high performance hypervisor for virtualization of the programmable data plane. In 26th International Conference on Computer Communication and Networks (ICCCN).
[43]
Cheng Zhang, Jun Bi, Yu Zhou, and Jianping Wu. 2019. HyperVDP: High-Performance virtualization of the programmable data plane. IEEE J. Sel. Areas Commun. 37, 3 (2019), 556--569.
[44]
Peng Zheng, Theophilus Benson, and Chengchen Hu. 2018. P4Visor: Lightweight virtualization and composition primitives for building and testing modular programs. In 14th International Conference on emerging Networking EXperiments and Technologies (CoNEXT).

Cited By

View all
  • (2024)Shape-shifting Elephants: Multi-modal Transport for Integrated Research InfrastructureProceedings of the 23rd ACM Workshop on Hot Topics in Networks10.1145/3696348.3696855(308-317)Online publication date: 18-Nov-2024
  • (2024)P4CGO: Control Plane Guided P4 Program OptimizationProceedings of the 2024 SIGCOMM Workshop on Formal Methods Aided Network Operation10.1145/3672199.3673892(1-7)Online publication date: 4-Aug-2024
  • (2024)P4-Ace: Resource-Efficient Optimization and Verification for Programmable SwitchesProceedings of the 2024 SIGCOMM Workshop on Formal Methods Aided Network Operation10.1145/3672199.3673884(8-13)Online publication date: 4-Aug-2024
  • Show More Cited By

Index Terms

  1. Lyra: A Cross-Platform Language and Compiler for Data Plane Programming on Heterogeneous ASICs

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        SIGCOMM '20: Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication
        July 2020
        814 pages
        ISBN:9781450379557
        DOI:10.1145/3387514
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 30 July 2020

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. Compiler
        2. P4 Synthesis
        3. Programmable Networks
        4. Programmable switching ASIC
        5. Programming Language

        Qualifiers

        • Research-article
        • Research
        • Refereed limited

        Funding Sources

        • NSF

        Conference

        SIGCOMM '20
        Sponsor:

        Acceptance Rates

        Overall Acceptance Rate 462 of 3,389 submissions, 14%

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)344
        • Downloads (Last 6 weeks)23
        Reflects downloads up to 12 Nov 2024

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Shape-shifting Elephants: Multi-modal Transport for Integrated Research InfrastructureProceedings of the 23rd ACM Workshop on Hot Topics in Networks10.1145/3696348.3696855(308-317)Online publication date: 18-Nov-2024
        • (2024)P4CGO: Control Plane Guided P4 Program OptimizationProceedings of the 2024 SIGCOMM Workshop on Formal Methods Aided Network Operation10.1145/3672199.3673892(1-7)Online publication date: 4-Aug-2024
        • (2024)P4-Ace: Resource-Efficient Optimization and Verification for Programmable SwitchesProceedings of the 2024 SIGCOMM Workshop on Formal Methods Aided Network Operation10.1145/3672199.3673884(8-13)Online publication date: 4-Aug-2024
        • (2024)NetBlocks: Staging Layouts for High-Performance Custom Host Network StacksProceedings of the ACM on Programming Languages10.1145/36563968:PLDI(467-491)Online publication date: 20-Jun-2024
        • (2024)Eagle: Toward Scalable and Near-Optimal Network-Wide Sketch Deployment in Network MeasurementProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672244(291-310)Online publication date: 4-Aug-2024
        • (2024)OptimusPrime: Unleash Dataplane Programmability through a Transformable ArchitectureProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672214(904-920)Online publication date: 4-Aug-2024
        • (2024)Charting the Complexity Landscape of Compiling Packet Programs to Reconfigurable SwitchesIEEE/ACM Transactions on Networking10.1109/TNET.2024.342433732:5(4519-4534)Online publication date: Oct-2024
        • (2024)Resource-Efficient and Timely Packet Header Vector (PHV) Encoding on Programmable SwitchesIEEE/ACM Transactions on Networking10.1109/TNET.2024.341353032:5(4191-4206)Online publication date: Oct-2024
        • (2024)Toward Resource-Efficient and High- Performance Program Deployment in Programmable NetworksIEEE/ACM Transactions on Networking10.1109/TNET.2024.341338832:5(4270-4285)Online publication date: Oct-2024
        • (2024)Hermes: Low-Overhead Inter-Switch Coordination in Network-Wide Data Plane Program DeploymentIEEE/ACM Transactions on Networking10.1109/TNET.2024.336132432:4(2842-2857)Online publication date: Aug-2024
        • Show More Cited By

        View Options

        Get Access

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media