Flame: A Centralized Cache Controller for Serverless Computing
Authors: 
Yanan Yang, Laiping Zhao, Yiming Li, Shihao Wu, Yuechan Hao, Yuchi Ma, Keqiu LiAuthors Info & Claims
Yanan Yang, Laiping Zhao, Yiming Li, Shihao Wu, Yuechan Hao, Yuchi Ma, Keqiu LiAuthors Info & ClaimsPages 153 - 168
Abstract
Caching function is a promising way to mitigate coldstart overhead in serverless computing. However, as caching also increases the resource cost significantly, how to make caching decisions is still challenging. We find that the prior "local cache control" designs are insufficient to achieve high cache efficiency due to the workload skewness across servers.
In this paper, inspired by the idea of software defined network management, we propose Flame, an efficient cache system to manage cached functions with a "centralized cache control" design. By decoupling the cache control plane from local servers and setting up a separate centralized controller, Flame is able to make caching decisions considering a global view of cluster status, enabling the optimized cache-hit ratio and resource efficiency. We evaluate Flame with real-world workloads and the evaluation results show that it can reduce the cache resource usage by 36% on average while improving the coldstart ratio by nearly 7x than the state-of-the-art method.
References
[1]
Alexandra Agache, Marc Brooker, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. 2020. Firecracker: Lightweight Virtualization for Serverless Applications. In 17th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2020, Santa Clara, CA, USA, February 25--27, 2020, Ranjita Bhagwan and George Porter (Eds.). USENIX Association, 419--434. https://www.usenix.org/conference/nsdi20/presentation/agache
[2]
Istemi Ekin Akkus, Ruichuan Chen, Ivica Rimac, Manuel Stein, Klaus Satzke, Andre Beck, Paarijaat Aditya, and Volker Hilt. 2018. SAND: Towards High-Performance Serverless Computing. In 2018 USENIX Annual Technical Conference, USENIX ATC 2018, Boston, MA, USA, July 11--13, 2018, Haryadi S. Gunawi and Benjamin Reed (Eds.). USENIX Association, 923--935. https://www.usenix.org/conference/atc18/presentation/akkus
[3]
Lixiang Ao, George Porter, and Geoffrey M. Voelker. 2022. FaaSnap: FaaS made fast using snapshot-based VMs. In EuroSys '22: Seventeenth European Conference on Computer Systems, Rennes, France, April 5 - 8, 2022, Yérom-David Bromberg, Anne-Marie Kermarrec, and Christos Kozyrakis (Eds.). ACM, 730--746.
[4]
Cliff Click and John Rose. 2002. Fast subtype checking in the HotSpot JVM. In Proceedings of the 2002 Joint ACM-ISCOPE Conference on Java Grande 2002, Seattle, Washington, USA, November 3--5, 2002, José E. Moreira, Geoffrey C. Fox, and Vladimir Getov (Eds.). ACM, 96--107.
[5]
Graham Cormode, Vladislav Shkapenyuk, Divesh Srivastava, and Bojian Xu. 2009. Forward Decay: A Practical Time Decay Model for Streaming Systems. In 2009 IEEE 25th International Conference on Data Engineering. 138--149.
[6]
Dong Du, Tianyi Yu, Yubin Xia, Binyu Zang, Guanglu Yan, Chenggang Qin, Qixuan Wu, and Haibo Chen. 2020. Catalyzer: Sub-millisecond Startup for Serverless Computing with Initialization-less Booting. In ASPLOS '20: Architectural Support for Programming Languages and Operating Systems, Lausanne, Switzerland, March 16--20, 2020, James R. Larus, Luis Ceze, and Karin Strauss (Eds.). ACM, 467--481.
[7]
Jim Esch. 2015. Software-Defined Networking: A Comprehensive Survey. Proc. IEEE 103, 1 (2015), 10--13.
[8]
Alexander Fuerst and Prateek Sharma. 2021. FaasCache: keeping serverless computing alive with greedy-dual caching. In ASPLOS '21: 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Virtual Event, USA, April 19--23, 2021, Tim Sherwood, Emery D. Berger, and Christos Kozyrakis (Eds.). ACM, 386--400.
[9]
Alexander Fuerst and Prateek Sharma. 2022. Locality-aware Load-Balancing For Serverless Clusters. In HPDC '22: The 31st International Symposium on High-Performance Parallel and Distributed Computing, Minneapolis, MN, USA, 27 June 2022 - 1 July 2022, Jon B. Weissman, Abhishek Chandra, Ada Gavrilovska, and Devesh Tiwari (Eds.). ACM, 227--239.
[10]
Google Cloud Function. [n.d.]. https://cloud.google.com/functions/. Referenced 2022.
[11]
Jashwant Raj Gunasekaran, Prashanth Thinakaran, Nachiappan Chidambaram Nachiappan, Mahmut Taylan Kandemir, and Chita R. Das. 2020. Fifer: Tackling Resource Underutilization in the Serverless Era. In Middleware '20: 21st International Middleware Conference, Delft, The Netherlands, December 7--11, 2020, Dilma Da Silva and Rüdiger Kapitza (Eds.). ACM, 280--295.
[12]
gVisor: Application Kernel for Containers. [n. d.]. https://gvisor.dev/. Referenced 2022.
[13]
Jeongchul Kim and Kyungyong Lee. 2019. FunctionBench: A Suite of Workloads for Serverless Cloud Function Service. In 12th IEEE International Conference on Cloud Computing, CLOUD 2019, Milan, Italy, July 8--13, 2019, Elisa Bertino, Carl K. Chang, Peter Chen, Ernesto Damiani, Michael Goul, and Katsunori Oyama (Eds.). IEEE, 502--504.
[14]
Swaroop Kotni, Ajay Nayak, Vinod Ganapathy, and Arkaprava Basu. 2021. Faast-lane: Accelerating Function-as-a-Service Workflows. In 2021 USENIX Annual Technical Conference, USENIX ATC 2021, July 14--16, 2021, Irina Calciu and Geoff Kuenning (Eds.). USENIX Association, 805--820. https://www.usenix.org/conference/atc21/presentation/kotni
[15]
Swaroop Kotni, Ajay Nayak, Vinod Ganapathy, and Arkaprava Basu. 2021. Faast-lane: Accelerating Function-as-a-Service Workflows. In 2021 USENIX Annual Technical Conference, USENIX ATC 2021, July 14--16, 2021, Irina Calciu and Geoff Kuenning (Eds.). USENIX Association, 805--820. https://www.usenix.org/conference/atc21/presentation/kotni
[16]
Kubernetes. [n. d.]. http://k8s.io. Referenced 2022.
[17]
AWS Lambda. [n.d.]. https://aws.amazon.com/lambda/?nc1=h_ls. Referenced 2022.
[18]
Zijun Li, Linsong Guo, Quan Chen, Jiagan Cheng, Chuhao Xu, Deze Zeng, Zhuo Song, Tao Ma, Yong Yang, Chao Li, and Minyi Guo. 2022. Help Rather Than Recycle: Alleviating Cold Startup in Serverless Computing Through Inter-Function Container Sharing. In 2022 USENIX Annual Technical Conference, USENIX ATC 2022, Carlsbad, CA, USA, July 11--13, 2022, Jiri Schindler and Noa Zilberman (Eds.). USENIX Association, 69--84. https://www.usenix.org/conference/atc22/presentation/li-zijun-help
[19]
Anil Madhavapeddy, Richard Mortier, Charalampos Rotsos, David J. Scott, Balraj Singh, Thomas Gazagnaire, Steven Smith, Steven Hand, and Jon Crowcroft. 2013. Unikernels: library operating systems for the cloud. In Architectural Support for Programming Languages and Operating Systems, ASPLOS 2013, Houston, TX, USA, March 16--20, 2013, Vivek Sarkar and Rastislav Bodik (Eds.). ACM, 461--472.
[20]
Anup Mohan, Harshad S. Sane, Kshitij Doshi, Saikrishna Edupuganti, Naren Nayak, and Vadim Sukhomlinov. 2019. Agile Cold Starts for Scalable Serverless. In 11th USENIX Workshop on Hot Topics in Cloud Computing, HotCloud 2019, Renton, WA, USA, July 8, 2019, Christina Delimitrou and Dan R. K. Ports (Eds.). USENIX Association. https://www.usenix.org/conference/hotcloud19/presentation/mohan
[21]
Edward Oakes, Leon Yang, Dennis Zhou, Kevin Houck, Tyler Harter, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2018. SOCK: Rapid Task Provisioning with Serverless-Optimized Containers. In 2018 USENIX Annual Technical Conference, USENIX ATC 2018, Boston, MA, USA, July 11--13, 2018, Haryadi S. Gunawi and Benjamin Reed (Eds.). USENIX Association, 57--70. https://www.usenix.org/conference/atc18/presentation/oakes
[22]
OpenFaas. [n. d.]. https://docs.openfaas.com/. Referenced 2022.
[23]
Docker pause. [n. d.]. https://docs.docker.com/engine/reference/commandline/container_pause/. Referenced 2020.
[24]
Apache OpenWhisk: Open Source Serverless Cloud Platform. [n. d.]. https://openwhisk.apache.org/. Referenced 2022.
[25]
AWS Lambda Price. [n. d.]. https://aws.amazon.com/cn/lambda/pricing/. Referenced 2022.
[26]
Rohan Basu Roy, Tirthak Patel, and Devesh Tiwari. 2022. IceBreaker: warming serverless functions better with heterogeneity. In ASPLOS '22: 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Lausanne, Switzerland, 28 February 2022 - 4 March 2022, Babak Falsafi, Michael Ferdman, Shan Lu, and Thomas F. Wenisch (Eds.). ACM, 753--767.
[27]
Mohammad Shahrad, Rodrigo Fonseca, Iñigo Goiri, Gohar Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider. In 2020 USENIX Annual Technical Conference, USENIX ATC 2020, July 15--17, 2020, Ada Gavrilovska and Erez Zadok (Eds.). USENIX Association, 205--218. https://www.usenix.org/conference/atc20/presentation/shahrad
[28]
Dmitrii Ustiugov, Plamen Petrov, Marios Kogias, Edouard Bugnion, and Boris Grot. 2021. Benchmarking, analysis, and optimization of serverless function snapshots. In ASPLOS '21: 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Virtual Event, USA, April 19--23, 2021, Tim Sherwood, Emery D. Berger, and Christos Kozyrakis (Eds.). ACM, 559--572.
[29]
Keeping Functions Warm. [n. d.]. https://docs.aws.amazon.com/lambda/latest/dg/lambda-concurrency.html. Referenced 2022.
[30]
Xingda Wei, Fangming Lu, Tianxia Wang, Jinyu Gu, Yuhan Yang, Rong Chen, and Haibo Chen. 2022. No Provisioned Concurrency: Fast RDMA-codesigned Remote Fork for Serverless Computing. CoRR abs/2203.10225 (2022). arXiv:2203.10225 https://arxiv.org/abs:2203.10225
[31]
Jinfeng Wen, Zhenpeng Chen, Ding Li, Junkai Chen, Yi Liu, Haoyu Wang, Xin Jin, and Xuanzhe Liu. 2022. LambdaLite: Application-Level Optimization for Cold Start Latency in Serverless Computing. CoRR abs/2207.08175 (2022). arXiv:2207.08175
[32]
Wenfeng Xia, Peng Zhao, Yonggang Wen, and Haiyong Xie. 2017. A Survey on Data Center Networking (DCN): Infrastructure and Operations. IEEE Commun. Surv. Tutorials 19, 1 (2017), 640--656.
[33]
Mahendra Pratap Yadav, Nisha Pal, and Dharmendra Kumar Yadav. 2021. A formal approach for Docker container deployment. Concurr. Comput. Pract. Exp. 33, 20 (2021).
[34]
Yanan Yang, Laiping Zhao, Yiming Li, Huanyu Zhang, Jie Li, Mingyang Zhao, Xingzhen Chen, and Keqiu Li. 2022. INFless: a native serverless system for low-latency, high-throughput inference. In ASPLOS '22: 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Lausanne, Switzerland, 28 February 2022 - 4 March 2022, Babak Falsafi, Michael Ferdman, Shan Lu, and Thomas F. Wenisch (Eds.). ACM, 768--781.
[35]
Tianyi Yu, Qingyuan Liu, Dong Du, Yubin Xia, Binyu Zang, Ziqian Lu, Pingchao Yang, Chenggang Qin, and Haibo Chen. 2020. Characterizing serverless platforms with serverlessbench. In SoCC '20: ACM Symposium on Cloud Computing, Virtual Event, USA, October 19--21, 2020, Rodrigo Fonseca, Christina Delimitrou, and Beng Chin Ooi (Eds.). ACM, 30--44.
[36]
Laiping Zhao, Yanan Yang, Yiming Li, Xian Zhou, and Keqiu Li. 2021. Understanding, predicting and scheduling serverless workloads under partial interference. In SC '21: The International Conference for High Performance Computing, Networking, Storage and Analysis, St. Louis, Missouri, USA, November 14 - 19, 2021, Bronis R. de Supinski, Mary W. Hall, and Todd Gamblin (Eds.). ACM, 22:1--22:15.
Index Terms
- Flame: A Centralized Cache Controller for Serverless Computing
Recommendations
Hermes: Efficient Cache Management for Container-based Serverless Computing
Internetware '20: Proceedings of the 12th Asia-Pacific Symposium on InternetwareServerless computing systems are shifting towards shorter function durations and larger degrees of parallelism to eliminate intolerable latency. For container-based serverless computing, the state-of-the-art efforts fail to ensure low latency because ...
S-Cache: Function Caching for Serverless Edge Computing
EdgeSys '23: Proceedings of the 6th International Workshop on Edge Systems, Analytics and NetworkingServerless edge computing uses an event-driven model in which Internet-of-Things (IoT) services are run in short-lived, stateless containers only when invoked, leading to significant reduction of resource utilization. However, a cold-start of a ...
FaasCache: keeping serverless computing alive with greedy-dual caching
ASPLOS '21: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating SystemsFunctions as a Service (also called serverless computing) promises to revolutionize how applications use cloud resources. However, functions suffer from cold-start problems due to the overhead of initializing their code and data dependencies before they ...
Comments
Information & Contributors
Information
Published In
March 2023
430 pages
ISBN:9798400703942
DOI:10.1145/3623278
- Chair:
- Tor Aamodt,
- Program Chair:
- Michael M Swift,
- Program Co-chair:
- Natalie Enright Jerger
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 February 2024
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Key Research and Development Program of China
- Shandong Provincial Natural Science Foundation
- National Natural Science Foundation of China
- open project of Zhejiang Lab
Conference
ASPLOS '23: 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 4
March 25 - 29, 2023
BC, Vancouver, Canada
Acceptance Rates
Overall Acceptance Rate 535 of 2,713 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 486Total Downloads
- Downloads (Last 12 months)486
- Downloads (Last 6 weeks)79
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in