research-article

Sailfish: accelerating cloud-scale multi-tenant multi-service gateways with programmable switches

Authors:

Shunmin ZhuAuthors Info & Claims

SIGCOMM '21: Proceedings of the 2021 ACM SIGCOMM 2021 Conference

Pages 194 - 206

https://doi.org/10.1145/3452296.3472889

Published: 09 August 2021 Publication History

Abstract

The cloud gateway is essential in the public cloud as the central hub of cloud traffic. We show that horizontal scaling of software gateways, once sustainable for years, is no longer future-proof facing the massive scale and rapid growth of today's cloud. The root cause is the stagnant performance of the CPU core, which is prone to be overloaded by heavy hitters as traffic growth goes far beyond Moore's law. To address this, we propose \emph{Sailfish}, a cloud-scale multi-tenant multi-service gateway accelerated by programmable switches. The new challenge is that large forwarding tables due to multi-tenancy cannot be fit into the limited on-chip memories. To this end, we devise a multi-pronged approach with (1) hardware/software co-design for table sharing, (2) horizontal table splitting among gateway clusters, (3) pipeline-aware table compression for a single node. Compared with the x86 gateway of a similar price, Sailfish reduces latency by 95% (2μs), improves throughput by more than 20x in bps (3.2Tbps) and 71x in pps (1.8Gpps) with packet length < 256B. Sailfish has been deployed in Alibaba Cloud for more than two years. It is the first P4-based cloud gateway in the industry, of which a single cluster carries dozens of Tbps traffic, withstanding peak-hour traffic in large online shopping festivals.

Supplementary Material

liu-public-review (8-public-review.pdf)

Sailfish: Accelerating Cloud-Scale Multi-Tenant Multi-Service Gateways with Programmable Switches: Public Review

Download
67.09 KB

MP4 File (video-presentation.mp4)

Conference Presentation Video

Download
184.55 MB

MP4 File (video-long.mp4)

Long Version Video

Download
336.81 MB

References

[1]

2017. 12.8 Tb/s StrataXGS Tomahawk 3 Ethernet Switch Series. https://www.broadcom.com/products/ethernet-connectivity/switching/strataxgs/bcm56980-series. (2017).

[2]

2020. Cisco Silicon One. https://www.cisco.com/c/en/us/solutions/silicon-one.html. (2020).

[3]

2020. ECMP Flow-Based Forwarding. https://www.juniper.net/documentation/en_US/junos/topics/topic-map/security-ecmp-flow-based-forwarding.html. (2020).

[4]

2020. High-Capacity StrataXGS Trident4 Ethernet Switch Series. https://www.broadcom.com/products/ethernet-connectivity/switching/strataxgs/bcm56880-series. (2020).

[5]

2021. Intel Xeon Processor E7 Family. https://www.intel.com/content/www/us/en/products/processors/xeon/e7-processors.html. (2021).

[6]

2021. Practice and thinking of migrating entire Alibaba services to the cloud (in Chinese). https://developer.aliyun.com/article/765369. (2021).

[7]

2021. Tofino: P4-programmable Ethernet switch ASIC that delivers better performance at lower power. https://www.intel.com/content/www/us/en/products/network-io/programmable-ethernet-switch/tofino-series/tofino.html. (2021).

[8]

Mina Tahmasbi Arashloo, Pavel Shirshov, Rohan Gandhi, Guohan Lu, Lihua Yuan, and Jennifer Rexford. 2018. A scalable VPN gateway for multi-tenant cloud services. ACM SIGCOMM Computer Communication Review 48, 1 (2018), 49--55.

Digital Library

[9]

Tom Barbette, Cyril Soldani, and Laurent Mathy. 2015. Fast userspace packet processing. In 2015 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS). IEEE, 5--16.

[10]

Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, et al. 2014. P4: Programming protocol-independent packet processors. ACM SIGCOMM Computer Communication Review 44, 3 (2014), 87--95.

Digital Library

[11]

Pat Bosshart, Glen Gibb, Hun-Seok Kim, George Varghese, Nick McKeown, Martin Izzard, Fernando Mujica, and Mark Horowitz. 2013. Forwarding metamorphosis: Fast programmable match-action processing in hardware for SDN. ACM SIGCOMM Computer Communication Review 43, 4 (2013), 99--110.

Digital Library

[12]

K Costello and M Rimol. 2021. Gartner Forecasts Worldwide Public Cloud End-User Spending to Grow 23% in 2021. Gartner. Available online: https://www.gartner.com/en/newsroom/press-releases/2021-04-21-gartner-forecasts-worldwide-public-cloud-end-user-spending-to-grow-23-percent-in-2021 (2021).

[13]

Tom Coughlin. 2020. Impact of COVID-19 on the consumer electronics market. IEEE Consumer Electronics Magazine 10, 1 (2020), 58--59.

[14]

Jakub Czyz, Mark Allman, Jing Zhang, Scott Iekel-Johnson, Eric Osterweil, and Michael Bailey. 2014. Measuring ipv6 adoption. In Proceedings of the 2014 ACM Conference on SIGCOMM. 87--98.

Digital Library

[15]

Sudipto Das, Divyakant Agrawal, and Amr El Abbadi. 2013. Elastras: An elastic, scalable, and self-managing transactional database for the cloud. ACM Transactions on Database Systems (TODS) 38, 1 (2013), 1--45.

Digital Library

[16]

Daniel E Eisenbud, Cheng Yi, Carlo Contavalli, Cody Smith, Roman Kononov, Eric Mann-Hielscher, Ardas Cilingiroglu, Bin Cheyney, Wentao Shang, and Jinnah Dylan Hosein. 2016. Maglev: A fast and reliable software network load balancer. In 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16). 523--535.

Digital Library

[17]

Chongrong Fang, Haoyu Liu, Mao Miao, Jie Ye, Lei Wang, Wansheng Zhang, Daxiang Kang, Biao Lyv, Peng Cheng, and Jiming Chen. 2020. VTrace: Automatic Diagnostic System for Persistent Packet Loss in Cloud-Scale Overlay Network. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 31--43.

Digital Library

[18]

Michael Ferdman, Almutaz Adileh, Onur Kocberber, Stavros Volos, Mohammad Alisafaee, Djordje Jevdjic, Cansu Kaynak, Adrian Daniel Popescu, Anastasia Ailamaki, and Babak Falsafi. 2012. Clearing the clouds: a study of emerging scale-out workloads on modern hardware. Acm sigplan notices 47, 4 (2012), 37--48.

[19]

Daniel Firestone, Andrew Putnam, Sambhrama Mundkur, Derek Chiou, Alireza Dabagh, Mike Andrewartha, Hari Angepat, Vivek Bhanu, Adrian Caulfield, Eric Chung, et al. 2018. Azure accelerated networking: Smartnics in the public cloud. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). 51--66.

Digital Library

[20]

Rohan Gandhi, Hongqiang Harry Liu, Y Charlie Hu, Guohan Lu, Jitendra Padhye, Lihua Yuan, and Ming Zhang. 2014. Duet: Cloud scale load balancing with hardware and software. ACM SIGCOMM Computer Communication Review 44, 4 (2014), 27--38.

Digital Library

[21]

Gary Garrison, Sanghyun Kim, and Robin L Wakefield. 2012. Success factors for deploying cloud computing. Commun. ACM 55, 9 (2012), 62--68.

Digital Library

[22]

Stephen D Goglin and Linden Cornett. 2009. Flexible and extensible receive side scaling. (Sept. 1 2009). US Patent 7,584,286.

[23]

Christian Hopps et al. 2000. Analysis of an equal-cost multi-path algorithm. Technical Report. RFC 2992, November.

[24]

Daehyeok Kim, Zaoxing Liu, Yibo Zhu, Changhoon Kim, Jeongkeun Lee, Vyas Sekar, and Srinivasan Seshan. 2020. Tea: Enabling state-intensive network functions on programmable switches. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 90--106.

Digital Library

[25]

Teemu Koponen, Keith Amidon, Peter Balland, Martín Casado, Anupam Chanda, Bryan Fulton, Igor Ganichev, Jesse Gross, Paul Ingram, Ethan Jackson, et al. 2014. Network virtualization in multi-tenant datacenters. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14). 203--216.

Digital Library

[26]

Ang Li, Xiaowei Yang, Srikanth Kandula, and Ming Zhang. 2010. CloudCmp: comparing public cloud providers. In Proceedings of the 10th ACM SIGCOMM conference on Internet measurement. 1--14.

Digital Library

[27]

Mallik Mahalingam, Dinesh G Dutt, Kenneth Duda, Puneet Agarwal, Lawrence Kreeger, T Sridhar, Mike Bursell, and Chris Wright. 2014. Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks. RFC 7348 (2014), 1--22.

[28]

Ilias Marinos, Robert NM Watson, and Mark Handley. 2014. Network stack specialization for performance. ACM SIGCOMM Computer Communication Review 44, 4 (2014), 175--186.

Digital Library

[29]

Christopher McCarthy, Kevin Sullivan, and Rejith Krishnan. 2013. Systems and methods for private cloud computing. (July 23 2013). US Patent 8,495,611.

[30]

Rui Miao, Hongyi Zeng, Changhoon Kim, Jeongkeun Lee, and Minlan Yu. 2017. Silkroad: Making stateful layer-4 load balancing fast and cheap using switching asics. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. 15--28.

Digital Library

[31]

Parveen Patel, Deepak Bansal, Lihua Yuan, Ashwin Murthy, Albert Greenberg, David A Maltz, Randy Kern, Hemant Kumar, Marios Zikos, Hongyu Wu, et al. 2013. Ananta: Cloud scale load balancing. ACM SIGCOMM Computer Communication Review 43, 4 (2013), 207--218.

Digital Library

[32]

Siyi Qiao, Chengchen Hu, Gordon Brebner, Jianhua Zou, and Xiaohong Guan. 2020. Adaptable Switch: A Heterogeneous Switch Architecture for Network-Centric Computing. IEEE Communications Magazine 58, 12 (2020), 64--69.

[33]

Martin Raab and Angelika Steger. 1998. "Balls into bins"---A simple and tight analysis. In International Workshop on Randomization and Approximation Techniques in Computer Science. Springer, 159--170.

[34]

Lawrence G Roberts. 2000. Beyond Moore's law: Internet growth trends. Computer 33, 1 (2000), 117--119.

Digital Library

[35]

Theodoros Rokkas, Ioannis Neokosmidis, and Ioannis Tomkos. 2018. Cost and Power Consumption Comparison of 400 Gbps Intra-Datacenter Transceiver Modules. In 2018 20th International Conference on Transparent Optical Networks (ICTON). IEEE, 1--4.

[36]

Zhiming Shen, Sethuraman Subbiah, Xiaohui Gu, and John Wilkes. 2011. Cloudscale: elastic resource scaling for multi-tenant cloud systems. In Proceedings of the 2nd ACM Symposium on Cloud Computing. 1--14.

Digital Library

[37]

Vibhaalakshmi Sivaraman, Srinivas Narayana, Ori Rottenstreich, Shan Muthukrishnan, and Jennifer Rexford. 2017. Heavy-hitter detection entirely in the data plane. In Proceedings of the Symposium on SDN Research. 164--176.

Digital Library

[38]

Jeongseok Son, Yongqiang Xiong, Kun Tan, Paul Wang, Ze Gan, and Sue Moon. 2017. Protego: Cloud-scale multitenant ipsec gateway. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). 473--485.

[39]

Neil C Thompson and Svenja Spanuth. 2021. The decline of computers as a general purpose technology. Commun. ACM 64, 3 (2021), 64--72.

Digital Library

[40]

Henry Wang. 2019. Algorithmic longest prefix matching in programmable switch. (Dec. 17 2019). US Patent 10,511,532.

[41]

Timothy Wood, Prashant J Shenoy, Alexandre Gerber, Jacobus E van der Merwe, and Kadangode K Ramakrishnan. 2009. The Case for Enterprise-Ready Virtual Private Clouds. In HotCloud.

[42]

Beibei Wu, Yang Xu, Hongbin Lu, and Bin Liu. 2005. A practical packet reordering mechanism with flow granularity for parallelism exploiting in network processors. In 19th IEEE International Parallel and Distributed Processing Symposium. IEEE, 8--pp.

[43]

Nofel Yaseen, Behnaz Arzani, Ryan Beckett, Selim Ciraci, and Vincent Liu. 2020. Aragog: Scalable Runtime Verification of Shardable Networked Systems. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). 701--718.

[44]

Jiao Zhang, Shubo Wen, Jinsheng Zhang, Hua Chai, Tian Pan, Tao Huang, Linquan Zhang, Yunjie Liu, and F Richard Yu. 2020. Fast Switch-Based Load Balancer Considering Application Server States. IEEE/ACM Transactions on Networking 28, 3 (2020), 1391--1404.

Digital Library

[45]

Menghao Zhang, Jun Bi, Kai Gao, Yi Qiao, Guanyu Li, Xiao Kong, Zhaogeng Li, and Hongxin Hu. 2019. Tripod: Towards a scalable, efficient and resilient cloud gateway. IEEE Journal on Selected Areas in Communications 37, 3 (2019), 570--585.

[46]

Peng Zheng, Arvind Narayanan, and Zhi-Li Zhang. 2019. A closer look at NFV execution models. In Proceedings of the 3rd Asia-Pacific Workshop on Networking 2019. 85--91.

Digital Library

Cited By

Ran DChen XSong L(2024)ComPipe: A Novel Flow Placement and Measurement Algorithm for Programmable Composite PipelinesElectronics10.3390/electronics1306102213:6(1022)Online publication date: 8-Mar-2024
https://doi.org/10.3390/electronics13061022
Lu JZhao HQi YFeng YGuo XLi SSong EWei XLyu BWen RZhu S(2024)QuarkTable: Building Compact Forwarding Tables for Programmable Switches on Public CloudsProceedings of the 8th Asia-Pacific Workshop on Networking10.1145/3663408.3663415(45-51)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.1145/3663408.3663415
Yang YHe LZhou JShi XCao JLiu YSekar VYu MSeneviratne AVeitch D(2024)P4runpro: Enabling Runtime Programmability for RMT Programmable SwitchesProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672230(921-937)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672230
Show More Cited By

Index Terms

Sailfish: accelerating cloud-scale multi-tenant multi-service gateways with programmable switches
1. Networks

Recommendations

Building secure, resilient and compliant solutions on the IBM cloud for financial services
CASCON '21: Proceedings of the 31st Annual International Conference on Computer Science and Software Engineering

The IBM Cloud for Financial services is an industry specialized public cloud, grounded in enterprise grade security and open technology. It provides the security, compliance and resiliency required by financial institutions to confidently operate ...
Elastic virtual private cloud
Q2SWinet '14: Proceedings of the 10th ACM symposium on QoS and security for wireless and mobile networks

Several Virtual Private Networks are based on IPsec. However, IPsec has not been designed with elasticity in mind, which makes clusters of IPsec security gateways hard to manage for providing high Service Level Agreement (SLA). Thus, these SG clusters ...
Sustainable Network Resource Management System for Virtual Private Clouds
CLOUDCOM '10: Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science

To satisfy the requirement of secure isolation of Infrastructure–as-a-Service (IaaS) for enterprise customers, virtual private clouds, which are separated from others by using virtualization technologies, are deployed. However, isolation with ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCOMM '21: Proceedings of the 2021 ACM SIGCOMM 2021 Conference

August 2021

868 pages

ISBN:9781450383837

DOI:10.1145/3452296

General Chairs:
Fernando Kuipers
Delft University of Technology
,
Matthew Caesar
University of Illinois at Urbana-Champaign

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCOMM '21

Sponsor:

SIGCOMM

SIGCOMM '21: ACM SIGCOMM 2021 Conference

August 23 - 27, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 462 of 3,389 submissions, 14%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
2,813
Total Downloads

Downloads (Last 12 months)287
Downloads (Last 6 weeks)32

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ran DChen XSong L(2024)ComPipe: A Novel Flow Placement and Measurement Algorithm for Programmable Composite PipelinesElectronics10.3390/electronics1306102213:6(1022)Online publication date: 8-Mar-2024
https://doi.org/10.3390/electronics13061022
Lu JZhao HQi YFeng YGuo XLi SSong EWei XLyu BWen RZhu S(2024)QuarkTable: Building Compact Forwarding Tables for Programmable Switches on Public CloudsProceedings of the 8th Asia-Pacific Workshop on Networking10.1145/3663408.3663415(45-51)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.1145/3663408.3663415
Yang YHe LZhou JShi XCao JLiu YSekar VYu MSeneviratne AVeitch D(2024)P4runpro: Enabling Runtime Programmability for RMT Programmable SwitchesProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672230(921-937)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672230
Song ESong YLu CPan TZhang SLu JZhao JWang XWu XGao MLi ZFang ZLyu BZhang PWen RYi LZong ZZhu SSekar VYu MSeneviratne AVeitch D(2024)Canal Mesh: A Cloud-Scale Sidecar-Free Multi-Tenant Service Mesh ArchitectureProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672221(860-875)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672221
Li KXu SShao ZZheng RLiao XJin H(2024)ScalaBFS2: A High-performance BFS Accelerator on an HBM-enhanced FPGA ChipACM Transactions on Reconfigurable Technology and Systems10.1145/365003717:2(1-39)Online publication date: 30-Apr-2024
https://dl.acm.org/doi/10.1145/3650037
Lu JPan THe SMiao MZhou GQi YZhang SSong ESun XZhao HLyu BZhu S(2024)CloudSentry: Two-Stage Heavy Hitter Detection for Cloud-Scale Gateway Overload ProtectionIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.330185235:4(616-633)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TPDS.2023.3301852
Zhang MLi GGuo CBao HXu MHu HLi F(2024)IMap: Toward a Fast, Scalable and Reconfigurable In-Network Scanner With Programmable SwitchesIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.332766519(601-615)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIFS.2023.3327665
Chen XSun XZhang WYao XWang ZLiu HHuang QPan GLiu XZhou HWu C(2024)Accelerating Sketch-based End-Host Traffic Measurement with Automatic DPU OffloadingIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621293(171-180)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621293
Liu HSun XChen XHuang QZhang DZhou HWu CLiu XKhan M(2024) : Low-latency and reliable event collection in network measurement Journal of Network and Computer Applications10.1016/j.jnca.2024.103904228(103904)Online publication date: Aug-2024
https://doi.org/10.1016/j.jnca.2024.103904
Xiong BWu JHuang QZhao JTang QZhang JYang KLi K(2024)Elastically accelerating lookup on virtual SDN flow tables for software-defined cloud gatewaysComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2023.110092238:COnline publication date: 14-Mar-2024
https://dl.acm.org/doi/10.1016/j.comnet.2023.110092
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents