research-article

Sailfish: accelerating cloud-scale multi-tenant multi-service gateways with programmable switches

Authors:

Shunmin ZhuAuthors Info & Claims

SIGCOMM '21: Proceedings of the 2021 ACM SIGCOMM 2021 Conference

Pages 194 - 206

https://doi.org/10.1145/3452296.3472889

Published: 09 August 2021 Publication History

Abstract

The cloud gateway is essential in the public cloud as the central hub of cloud traffic. We show that horizontal scaling of software gateways, once sustainable for years, is no longer future-proof facing the massive scale and rapid growth of today's cloud. The root cause is the stagnant performance of the CPU core, which is prone to be overloaded by heavy hitters as traffic growth goes far beyond Moore's law. To address this, we propose \emph{Sailfish}, a cloud-scale multi-tenant multi-service gateway accelerated by programmable switches. The new challenge is that large forwarding tables due to multi-tenancy cannot be fit into the limited on-chip memories. To this end, we devise a multi-pronged approach with (1) hardware/software co-design for table sharing, (2) horizontal table splitting among gateway clusters, (3) pipeline-aware table compression for a single node. Compared with the x86 gateway of a similar price, Sailfish reduces latency by 95% (2μs), improves throughput by more than 20x in bps (3.2Tbps) and 71x in pps (1.8Gpps) with packet length < 256B. Sailfish has been deployed in Alibaba Cloud for more than two years. It is the first P4-based cloud gateway in the industry, of which a single cluster carries dozens of Tbps traffic, withstanding peak-hour traffic in large online shopping festivals.

Supplementary Material

liu-public-review (8-public-review.pdf)

Sailfish: Accelerating Cloud-Scale Multi-Tenant Multi-Service Gateways with Programmable Switches: Public Review

Download
67.09 KB

MP4 File (video-presentation.mp4)

Conference Presentation Video

Download
184.55 MB

MP4 File (video-long.mp4)

Long Version Video

Download
336.81 MB

References

[1]

2017. 12.8 Tb/s StrataXGS Tomahawk 3 Ethernet Switch Series. https://www.broadcom.com/products/ethernet-connectivity/switching/strataxgs/bcm56980-series. (2017).

[2]

2020. Cisco Silicon One. https://www.cisco.com/c/en/us/solutions/silicon-one.html. (2020).

[3]

2020. ECMP Flow-Based Forwarding. https://www.juniper.net/documentation/en_US/junos/topics/topic-map/security-ecmp-flow-based-forwarding.html. (2020).

[4]

2020. High-Capacity StrataXGS Trident4 Ethernet Switch Series. https://www.broadcom.com/products/ethernet-connectivity/switching/strataxgs/bcm56880-series. (2020).

[5]

2021. Intel Xeon Processor E7 Family. https://www.intel.com/content/www/us/en/products/processors/xeon/e7-processors.html. (2021).

[6]

2021. Practice and thinking of migrating entire Alibaba services to the cloud (in Chinese). https://developer.aliyun.com/article/765369. (2021).

[7]

2021. Tofino: P4-programmable Ethernet switch ASIC that delivers better performance at lower power. https://www.intel.com/content/www/us/en/products/network-io/programmable-ethernet-switch/tofino-series/tofino.html. (2021).

[8]

Mina Tahmasbi Arashloo, Pavel Shirshov, Rohan Gandhi, Guohan Lu, Lihua Yuan, and Jennifer Rexford. 2018. A scalable VPN gateway for multi-tenant cloud services. ACM SIGCOMM Computer Communication Review 48, 1 (2018), 49--55.

Digital Library

[9]

Tom Barbette, Cyril Soldani, and Laurent Mathy. 2015. Fast userspace packet processing. In 2015 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS). IEEE, 5--16.

[10]

Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, et al. 2014. P4: Programming protocol-independent packet processors. ACM SIGCOMM Computer Communication Review 44, 3 (2014), 87--95.

Digital Library

[11]

Pat Bosshart, Glen Gibb, Hun-Seok Kim, George Varghese, Nick McKeown, Martin Izzard, Fernando Mujica, and Mark Horowitz. 2013. Forwarding metamorphosis: Fast programmable match-action processing in hardware for SDN. ACM SIGCOMM Computer Communication Review 43, 4 (2013), 99--110.

Digital Library

[12]

K Costello and M Rimol. 2021. Gartner Forecasts Worldwide Public Cloud End-User Spending to Grow 23% in 2021. Gartner. Available online: https://www.gartner.com/en/newsroom/press-releases/2021-04-21-gartner-forecasts-worldwide-public-cloud-end-user-spending-to-grow-23-percent-in-2021 (2021).

[13]

Tom Coughlin. 2020. Impact of COVID-19 on the consumer electronics market. IEEE Consumer Electronics Magazine 10, 1 (2020), 58--59.

[14]

Jakub Czyz, Mark Allman, Jing Zhang, Scott Iekel-Johnson, Eric Osterweil, and Michael Bailey. 2014. Measuring ipv6 adoption. In Proceedings of the 2014 ACM Conference on SIGCOMM. 87--98.

Digital Library

[15]

Sudipto Das, Divyakant Agrawal, and Amr El Abbadi. 2013. Elastras: An elastic, scalable, and self-managing transactional database for the cloud. ACM Transactions on Database Systems (TODS) 38, 1 (2013), 1--45.

Digital Library

[16]

Daniel E Eisenbud, Cheng Yi, Carlo Contavalli, Cody Smith, Roman Kononov, Eric Mann-Hielscher, Ardas Cilingiroglu, Bin Cheyney, Wentao Shang, and Jinnah Dylan Hosein. 2016. Maglev: A fast and reliable software network load balancer. In 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16). 523--535.

Digital Library

[17]

Chongrong Fang, Haoyu Liu, Mao Miao, Jie Ye, Lei Wang, Wansheng Zhang, Daxiang Kang, Biao Lyv, Peng Cheng, and Jiming Chen. 2020. VTrace: Automatic Diagnostic System for Persistent Packet Loss in Cloud-Scale Overlay Network. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 31--43.

Digital Library

[18]

Michael Ferdman, Almutaz Adileh, Onur Kocberber, Stavros Volos, Mohammad Alisafaee, Djordje Jevdjic, Cansu Kaynak, Adrian Daniel Popescu, Anastasia Ailamaki, and Babak Falsafi. 2012. Clearing the clouds: a study of emerging scale-out workloads on modern hardware. Acm sigplan notices 47, 4 (2012), 37--48.

[19]

Daniel Firestone, Andrew Putnam, Sambhrama Mundkur, Derek Chiou, Alireza Dabagh, Mike Andrewartha, Hari Angepat, Vivek Bhanu, Adrian Caulfield, Eric Chung, et al. 2018. Azure accelerated networking: Smartnics in the public cloud. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). 51--66.

Digital Library

[20]

Rohan Gandhi, Hongqiang Harry Liu, Y Charlie Hu, Guohan Lu, Jitendra Padhye, Lihua Yuan, and Ming Zhang. 2014. Duet: Cloud scale load balancing with hardware and software. ACM SIGCOMM Computer Communication Review 44, 4 (2014), 27--38.

Digital Library

[21]

Gary Garrison, Sanghyun Kim, and Robin L Wakefield. 2012. Success factors for deploying cloud computing. Commun. ACM 55, 9 (2012), 62--68.

Digital Library

[22]

Stephen D Goglin and Linden Cornett. 2009. Flexible and extensible receive side scaling. (Sept. 1 2009). US Patent 7,584,286.

[23]

Christian Hopps et al. 2000. Analysis of an equal-cost multi-path algorithm. Technical Report. RFC 2992, November.

[24]

Daehyeok Kim, Zaoxing Liu, Yibo Zhu, Changhoon Kim, Jeongkeun Lee, Vyas Sekar, and Srinivasan Seshan. 2020. Tea: Enabling state-intensive network functions on programmable switches. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 90--106.

Digital Library

[25]

Teemu Koponen, Keith Amidon, Peter Balland, Martín Casado, Anupam Chanda, Bryan Fulton, Igor Ganichev, Jesse Gross, Paul Ingram, Ethan Jackson, et al. 2014. Network virtualization in multi-tenant datacenters. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14). 203--216.

Digital Library

[26]

Ang Li, Xiaowei Yang, Srikanth Kandula, and Ming Zhang. 2010. CloudCmp: comparing public cloud providers. In Proceedings of the 10th ACM SIGCOMM conference on Internet measurement. 1--14.

Digital Library

[27]

Mallik Mahalingam, Dinesh G Dutt, Kenneth Duda, Puneet Agarwal, Lawrence Kreeger, T Sridhar, Mike Bursell, and Chris Wright. 2014. Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks. RFC 7348 (2014), 1--22.

[28]

Ilias Marinos, Robert NM Watson, and Mark Handley. 2014. Network stack specialization for performance. ACM SIGCOMM Computer Communication Review 44, 4 (2014), 175--186.

Digital Library

[29]

Christopher McCarthy, Kevin Sullivan, and Rejith Krishnan. 2013. Systems and methods for private cloud computing. (July 23 2013). US Patent 8,495,611.

[30]

Rui Miao, Hongyi Zeng, Changhoon Kim, Jeongkeun Lee, and Minlan Yu. 2017. Silkroad: Making stateful layer-4 load balancing fast and cheap using switching asics. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. 15--28.

Digital Library

[31]

Parveen Patel, Deepak Bansal, Lihua Yuan, Ashwin Murthy, Albert Greenberg, David A Maltz, Randy Kern, Hemant Kumar, Marios Zikos, Hongyu Wu, et al. 2013. Ananta: Cloud scale load balancing. ACM SIGCOMM Computer Communication Review 43, 4 (2013), 207--218.

Digital Library

[32]

Siyi Qiao, Chengchen Hu, Gordon Brebner, Jianhua Zou, and Xiaohong Guan. 2020. Adaptable Switch: A Heterogeneous Switch Architecture for Network-Centric Computing. IEEE Communications Magazine 58, 12 (2020), 64--69.

[33]

Martin Raab and Angelika Steger. 1998. "Balls into bins"---A simple and tight analysis. In International Workshop on Randomization and Approximation Techniques in Computer Science. Springer, 159--170.

[34]

Lawrence G Roberts. 2000. Beyond Moore's law: Internet growth trends. Computer 33, 1 (2000), 117--119.

Digital Library

[35]

Theodoros Rokkas, Ioannis Neokosmidis, and Ioannis Tomkos. 2018. Cost and Power Consumption Comparison of 400 Gbps Intra-Datacenter Transceiver Modules. In 2018 20th International Conference on Transparent Optical Networks (ICTON). IEEE, 1--4.

[36]

Zhiming Shen, Sethuraman Subbiah, Xiaohui Gu, and John Wilkes. 2011. Cloudscale: elastic resource scaling for multi-tenant cloud systems. In Proceedings of the 2nd ACM Symposium on Cloud Computing. 1--14.

Digital Library

[37]

Vibhaalakshmi Sivaraman, Srinivas Narayana, Ori Rottenstreich, Shan Muthukrishnan, and Jennifer Rexford. 2017. Heavy-hitter detection entirely in the data plane. In Proceedings of the Symposium on SDN Research. 164--176.

Digital Library

[38]

Jeongseok Son, Yongqiang Xiong, Kun Tan, Paul Wang, Ze Gan, and Sue Moon. 2017. Protego: Cloud-scale multitenant ipsec gateway. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). 473--485.

[39]

Neil C Thompson and Svenja Spanuth. 2021. The decline of computers as a general purpose technology. Commun. ACM 64, 3 (2021), 64--72.

Digital Library

[40]

Henry Wang. 2019. Algorithmic longest prefix matching in programmable switch. (Dec. 17 2019). US Patent 10,511,532.

[41]

Timothy Wood, Prashant J Shenoy, Alexandre Gerber, Jacobus E van der Merwe, and Kadangode K Ramakrishnan. 2009. The Case for Enterprise-Ready Virtual Private Clouds. In HotCloud.

[42]

Beibei Wu, Yang Xu, Hongbin Lu, and Bin Liu. 2005. A practical packet reordering mechanism with flow granularity for parallelism exploiting in network processors. In 19th IEEE International Parallel and Distributed Processing Symposium. IEEE, 8--pp.

[43]

Nofel Yaseen, Behnaz Arzani, Ryan Beckett, Selim Ciraci, and Vincent Liu. 2020. Aragog: Scalable Runtime Verification of Shardable Networked Systems. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). 701--718.

[44]

Jiao Zhang, Shubo Wen, Jinsheng Zhang, Hua Chai, Tian Pan, Tao Huang, Linquan Zhang, Yunjie Liu, and F Richard Yu. 2020. Fast Switch-Based Load Balancer Considering Application Server States. IEEE/ACM Transactions on Networking 28, 3 (2020), 1391--1404.

Digital Library

[45]

Menghao Zhang, Jun Bi, Kai Gao, Yi Qiao, Guanyu Li, Xiao Kong, Zhaogeng Li, and Hongxin Hu. 2019. Tripod: Towards a scalable, efficient and resilient cloud gateway. IEEE Journal on Selected Areas in Communications 37, 3 (2019), 570--585.

[46]

Peng Zheng, Arvind Narayanan, and Zhi-Li Zhang. 2019. A closer look at NFV execution models. In Proceedings of the 3rd Asia-Pacific Workshop on Networking 2019. 85--91.

Digital Library

Cited By

Xie GLi QCui CLi RMa LQi ZJiang Y(2025)Intelligent In-Network Attack Detection on Programmable Switches With Soterv2IEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2024.340295522:1(440-456)Online publication date: Jan-2025
https://doi.org/10.1109/TDSC.2024.3402955
Wang ZHu HKong LKang XMa TXiang QLi JLu YSong ZYang PWu JYang YMa TLiu ZZeng XCai DChen GBagchi SZhang Y(2024)Diagnosing application-network anomalies for millions of IPs in production cloudsProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692046(885-899)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.5555/3691992.3692046
Yu LZhang XZhang HSonchack JPorts DLiu VGavrilovska ATerry D(2024)BeaverProceedings of the 18th USENIX Conference on Operating Systems Design and Implementation10.5555/3691938.3691951(233-249)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.5555/3691938.3691951
Show More Cited By

Index Terms

Sailfish: accelerating cloud-scale multi-tenant multi-service gateways with programmable switches
1. Networks

Recommendations

Building secure, resilient and compliant solutions on the IBM cloud for financial services
CASCON '21: Proceedings of the 31st Annual International Conference on Computer Science and Software Engineering

The IBM Cloud for Financial services is an industry specialized public cloud, grounded in enterprise grade security and open technology. It provides the security, compliance and resiliency required by financial institutions to confidently operate ...
Poster: Jakiro: Spitting RDMA and TCP over Virtual Private Cloud
CoNEXT '24: Proceedings of the 20th International Conference on emerging Networking EXperiments and Technologies

A Virtual Private Cloud (VPC) that enables both RDMA and TCP provides advantages for both tenants and cloud providers. In this study, we introduce Jakiro, an innovative framework of vNIC design that supports both RDMA and TCP over a unified VPC. Jakiro ...
Elastic virtual private cloud
Q2SWinet '14: Proceedings of the 10th ACM symposium on QoS and security for wireless and mobile networks

Several Virtual Private Networks are based on IPsec. However, IPsec has not been designed with elasticity in mind, which makes clusters of IPsec security gateways hard to manage for providing high Service Level Agreement (SLA). Thus, these SG clusters ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCOMM '21: Proceedings of the 2021 ACM SIGCOMM 2021 Conference

August 2021

868 pages

ISBN:9781450383837

DOI:10.1145/3452296

General Chairs:
Fernando Kuipers
Delft University of Technology
,
Matthew Caesar
University of Illinois at Urbana-Champaign

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCOMM '21

Sponsor:

SIGCOMM

SIGCOMM '21: ACM SIGCOMM 2021 Conference

August 23 - 27, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 462 of 3,389 submissions, 14%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

59
Total Citations
View Citations
2,934
Total Downloads

Downloads (Last 12 months)261
Downloads (Last 6 weeks)26

Reflects downloads up to 01 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xie GLi QCui CLi RMa LQi ZJiang Y(2025)Intelligent In-Network Attack Detection on Programmable Switches With Soterv2IEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2024.340295522:1(440-456)Online publication date: Jan-2025
https://doi.org/10.1109/TDSC.2024.3402955
Wang ZHu HKong LKang XMa TXiang QLi JLu YSong ZYang PWu JYang YMa TLiu ZZeng XCai DChen GBagchi SZhang Y(2024)Diagnosing application-network anomalies for millions of IPs in production cloudsProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692046(885-899)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.5555/3691992.3692046
Yu LZhang XZhang HSonchack JPorts DLiu VGavrilovska ATerry D(2024)BeaverProceedings of the 18th USENIX Conference on Operating Systems Design and Implementation10.5555/3691938.3691951(233-249)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.5555/3691938.3691951
Lyu BSong EPan TLu JZhang SSun XGao LWang CXiao HPan YChen XDuan YWang WLong JWang YZhou KZong ZLi XLi GZhang PCheng PChen JZhu SVanbever LZhang I(2024)POSEIDONProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691885(1083-1099)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.5555/3691825.3691885
Pan TLiu KWei XQiao YHu JLi ZLiang JCheng TSu WLu JHong YWang ZXu ZDai CWang PJia XLu JSong EZeng JLyu BZhai EZhang JHuang TCai DZhu SVanbever LZhang I(2024)LuoShenProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691874(877-892)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.5555/3691825.3691874
Gao JCao JLi YLiu MTang MCai DZhai EVanbever LZhang I(2024)SiriusProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691852(477-490)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.5555/3691825.3691852
Ran DChen XSong L(2024)ComPipe: A Novel Flow Placement and Measurement Algorithm for Programmable Composite PipelinesElectronics10.3390/electronics1306102213:6(1022)Online publication date: 8-Mar-2024
https://doi.org/10.3390/electronics13061022
Zou YPan TLu LLi ZYao KMu YWan YHuang TLiu Y(2024)HyperSFC: State-Intensive Service Function Chaining on Hyper-Converged Edge Infrastructure2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619908(1-9)Online publication date: 3-Jun-2024
https://doi.org/10.23919/IFIPNetworking62109.2024.10619908
Mittal SV. HHeetkumar PTammana PGopalakrishnan VWang JAyyub Qazi ITyson G(2024)iGuard: Efficient Isolation Forest Design for Malicious Traffic Detection in Programmable SwitchesProceedings of the 20th International Conference on emerging Networking EXperiments and Technologies10.1145/3680121.3697807(55-64)Online publication date: 9-Dec-2024
https://dl.acm.org/doi/10.1145/3680121.3697807
Chu YLiu ZJiang RGan ZZhai JLu GNiu ZXiong Y(2024)FHA: Flow-level High Availability on Programmable Network Hardware for Cloud ProviderProceedings of the 15th ACM SIGOPS Asia-Pacific Workshop on Systems10.1145/3678015.3680483(74-80)Online publication date: 4-Sep-2024
https://dl.acm.org/doi/10.1145/3678015.3680483
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten