research-article

Photons: lambdas on a diet

Authors:

Vojislav Dukic,

Gustavo AlonsoAuthors Info & Claims

SoCC '20: Proceedings of the 11th ACM Symposium on Cloud Computing

Pages 45 - 59

https://doi.org/10.1145/3419111.3421297

Published: 12 October 2020 Publication History

Abstract

Serverless computing allows users to create short, stateless functions and invoke hundreds of them concurrently to tackle massively parallel workloads. We observe that even though most of the footprint of a serverless function is fixed across its invocations --- language runtime, libraries, and other application state --- today's serverless platforms do not exploit this redundancy. Such an inefficiency has cascading negative impacts: longer startup times, lower throughput, higher latency, and higher cost. To mitigate these problems, we have built Photons, a framework leveraging workload parallelism to co-locate multiple instances of the same function within the same runtime. Concurrent invocations can then share the runtime and application state transparently, without compromising execution safety. Photons reduce function's memory consumption by 25% to 98% per invocation, with no performance degradation compared to today's serverless platforms. We also show that our approach can reduce the overall memory utilization by 30%, and the total number of cold starts by 52%.

Supplementary Material

MP4 File (p45-dukic-presentation.mp4)

Download
186.81 MB

References

[1]

2009 (Accessed May 25, 2020). Understanding Container Reuse in AWS Lambda. https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/.

[2]

(Accessed May 25, 2020). Apache OpenWhisk. http://openwhisk.apache.org/.

[3]

(Accessed May 25, 2020). Apply machine learning models in Azure Functions with Python and TensorFlow. https://docs.microsoft.com/en-us/azure/azure-functions/functions-machine-learning-tensorflow.

[4]

(Accessed May 25, 2020). Architecture of a Serverless Machine Learning Model. https://cloud.google.com/solutions/architecture-of-a-serverless-ml-model.

[5]

(Accessed May 25, 2020). AWS Lambda pricing. https://aws.amazon.com/lambda/pricing/.

[6]

(Accessed May 25, 2020). AWS Lambda web tutorial. https://aws.amazon.com/getting-started/hands-on/build-serverless-web-app-lambda-apigateway-s3-dynamodb-cognito/.

[7]

(Accessed May 25, 2020). Cloud Computing without Containers. https://blog.cloudflare.com/cloud-computing-without-containers/.

[8]

(Accessed May 25, 2020). FFMPEG for Java. https://github.com/bramp/ffmpeg-cli-wrapper.

[9]

(Accessed May 25, 2020). gVisor. https://gvisor.dev/.

[10]

(Accessed May 25, 2020). How to Deploy Deep Learning Models with AWS Lambda and Tensorflow. https://aws.amazon.com/blogs/machine-learning/how-to-deploy-deep-learning-models-with-aws-lambda-and-tensorflow/.

[11]

(Accessed May 25, 2020). How to serve deep learning models using TensorFlow 2.0 with Cloud Functions. https://cloud.google.com/blog/products/ai-machine-learning/how-to-serve-deep-learning-models-using-tensorflow-2-0-with-cloud-functions.

[12]

(Accessed May 25, 2020). JEP 346: Promptly Return Unused Committed Memory from G1. http://openjdk.java.net/jeps/346.

[13]

(Accessed May 25, 2020). Kata Containers. https://katacontainers.io/.

[14]

(Accessed May 25, 2020). Knative serverless framework. https://knative.dev/.

[15]

(Accessed May 25, 2020). KNIX Microfunctions. https://github.com/knix-microfunctions/knix/.

[16]

(Accessed May 25, 2020). OpenFaaS serverless framework. https://www.openfaas.com/.

[17]

(Accessed May 25, 2020). Overview of memory management. https://developer.android.com/topic/performance/memory-overview.

[18]

(Accessed May 25, 2020). Security Overview of AWSLambda. https://d1.awsstatic.com/whitepapers/Overview-AWS-Lambda-Security.pdf.

[19]

(Accessed May 25, 2020). Serverless Networking. https://github.com/serverlessunicorn/ServerlessNetworkingClients.

[20]

(Accessed May 25, 2020). Understanding serverless cold start. https://azure.microsoft.com/en-us/blog/understanding-serverless-cold-start/.

[21]

(Accessed May 25, 2020). Use AWS DeepLens to give Amazon Alexa the power to detect objects via Alexa skills. https://aws.amazon.com/blogs/machine-learning/use-aws-deeplens-to-give-amazon-alexa-the-power-to-detect-objects-via-alexa-skills/.

[22]

Alexandru Agache, Marc Brooker, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. 2020. Firecracker: Lightweight Virtualization for Serverless Applications. In USENIX NSDI.

[23]

Istemi Ekin Akkus, Ruichuan Chen, Ivica Rimac, Manuel Stein, Klaus Satzke, Andre Beck, Paarijaat Aditya, and Volker Hilt. 2018. SAND: Towards High-performance Serverless Computing. In USENIX ATC.

Digital Library

[24]

Gene M. Amdahl. 1967. Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities. In AFIPS.

[25]

Lixiang Ao, Liz Izhikevich, Geoffrey M Voelker, and George Porter. 2018. Sprocket: A serverless video processing framework. In ACM SoCC.

Digital Library

[26]

Ioana Baldini, Paul Castro, Kerry Chang, Perry Cheng, Stephen Fink, Vatche Ishakian, Nick Mitchell, Vinod Muthusamy, Rodric Rabbah, Aleksander Slominski, and Philippe Suter. 2017. Serverless Computing: Current Trends and Open Problems.

[27]

Sol Boucher, Anuj Kalia, David G Andersen, and Michael Kaminsky. 2018. Putting the "Micro" back in microservice. In USENIX ATC.

[28]

Stefan Brenner and Rüdiger Kapitza. 2019. Trust more, serverless. In SYSTOR.

[29]

Rodrigo Bruno, Paulo Ferreira, Ruslan Synytsky, Tetiana Fydorenchyk, Jia Rao, Hang Huang, and Song Wu. 2018. Dynamic Vertical Memory Scalability for OpenJDK Cloud Applications. In ISMM.

[30]

James Cadden, Thomas Unger, Yara Awad, Han Dong, Orran Krieger, and Jonathan Appavoo. 2020. SEUSS: skip redundant paths to make serverless fast. In EuroSys.

[31]

Joao Carreira, Pedro Fonseca, Alexey Tumanov, Andrew Zhang, and Randy Katz. 2018. A Case for Serverless Machine Learning. In NeurIPS.

[32]

Shigeru Chiba and Muga Nishizawa. 2003. An easy-to-use toolkit for efficient Java bytecode translators. In International Conference on Generative Programming and Component Engineering.

[33]

Eyal de Lara, Carolina S Gomes, Steve Langridge, S Hossein Mortazavi, and Meysam Roodi. 2016. Hierarchical serverless computing for the mobile edge. In IEEE/ACM Symposium on Edge Computing (SEC).

[34]

Dong Du, Tianyi Yu, Yubin Xia, Binyu Zang, Guanglu Yan, Chenggang Qin, Qixuan Wu, and Haibo Chen. 2020. Catalyzer: Sub-millisecond Startup for Serverless Computing with Initialization-less Booting. In ASPLOS.

Digital Library

[35]

Sadjad Fouladi, Riad S. Wahby, Brennan Shacklett, Karthikeyan Vasuki Balasubramaniam, William Zeng, Rahul Bhalerao, Anirudh Sivaraman, George Porter, and Keith Winstein. 2017. Encoding, Fast and Slow: Low-latency Video Processing Using Thousands of Tiny Threads. In USENIX NSDI.

[36]

Yu Gan, Yanqi Zhang, Dailun Cheng, Ankitha Shetty, Priyal Rathi, Nayan Katarki, Ariana Bruno, Justin Hu, Brian Ritchken, Brendon Jackson, et al. 2019. An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In ASPLOS.

[37]

J. R. Gunasekaran, P. Thinakaran, M. T. Kandemir, B. Urgaonkar, G. Kesidis, and C. Das. 2019. Spock: Exploiting Serverless Functions for SLO and Cost Aware Resource Procurement in Public Cloud. In IEEE CLOUD.

[38]

Tyler Harter, Brandon Salmon, Rose Liu, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2016. Slacker: Fast Distribution with Lazy Docker Containers. In FAST.

[39]

Joseph M Hellerstein, Jose Faleiro, Joseph E Gonzalez, Johann Schleier-Smith, Vikram Sreekanti, Alexey Tumanov, and Chenggang Wu. 2018. Serverless computing: One step forward, two steps back. arXiv preprint arXiv:1812.03651 (2018).

[40]

Vatche Ishakian, Vinod Muthusamy, and Aleksander Slominski. 2018. Serving deep learning models in a serverless platform. In IEEE International Conference on Cloud Engineering (IC2E).

[41]

Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, and Benjamin Recht. 2017. Occupy the cloud: Distributed computing for the 99%. In ACM SoCC.

[42]

Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti, Chia-che Tsai, Anurag Khandelwal, Qifan Pu, Vaishaal Shankar, Joao Carreira, Karl Krauth, Neeraja Jayant Yadwadkar, Joseph E. Gonzalez, Raluca Ada Popa, Ion Stoica, and David A. Patterson. 2019. Cloud Programming Simplified: A Berkeley View on Serverless Computing. CoRR (2019). arXiv:1902.03383

[43]

Ana Klimovic, Yawen Wang, Patrick Stuedi, Animesh Trivedi, Jonas Pfefferle, and Christos Kozyrakis. 2018. Pocket: Elastic ephemeral storage for serverless analytics. In USENIX OSDI.

[44]

Ricardo Koller and Dan Williams. 2017. Will Serverless End the Dominance of Linux in the Cloud?. In HotOS.

[45]

James Larisch, James Mickens, and Eddie Kohler. 2018. Alto: lightweight vms using virtualization-aware managed runtimes. In MPLR.

[46]

Anil Madhavapeddy, Richard Mortier, Charalampos Rotsos, David Scott, Balraj Singh, Thomas Gazagnaire, Steven Smith, Steven Hand, and Jon Crowcroft. 2013. Unikernels: Library operating systems for the cloud. In ACM SIGARCH Computer Architecture News.

Digital Library

[47]

Filipe Manco, Costin Lupu, Florian Schmidt, Jose Mendes, Simon Kuenzer, Sumit Sati, Kenichi Yasukata, Costin Raiciu, and Felipe Huici. 2017. My VM is Lighter (and Safer) Than Your Container. In SOSP.

[48]

Dirk Merkel. 2014. Docker: lightweight linux containers for consistent development and deployment. In Linux journal.

[49]

Anup Mohan, Harshad Sane, Kshitij Doshi, Saikrishna Edupuganti, Naren Nayak, and Vadim Sukhomlinov. 2019. Agile Cold Starts for Scalable Serverless. In HotCloud.

[50]

Ingo Müller, Renato Marroquín, and Gustavo Alonso. 2020. Lambada: Interactive Data Analytics on Cold Data Using Serverless Cloud Infrastructure. In ACM SIGMOD.

[51]

Edward Oakes, Leon Yang, Dennis Zhou, Kevin Houck, Tyler Harter, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2018. SOCK: Rapid Task Provisioning with Serverless-optimized Containers. In USENIX ATC.

[52]

Alfonso Pérez, Germán Moltó, Miguel Caballer, and Amanda Calatrava. 2018. Serverless computing for container-based architectures. In Future Generation Computer Systems.

[53]

Matthew Perron, Raul Castro Fernandez, David DeWitt, and Samuel Madden. 2019. Starling: A Scalable Query Engine on Cloud Function Services. arXiv preprint arXiv:1911.11727 (2019).

[54]

Mohammad Shahrad, Rodrigo Fonseca, Íñigo Goiri, Gohar Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider. In USENIX ATC.

[55]

Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking behind the curtains of serverless platforms. In USENIX ATC.

[56]

Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking Behind the Curtains of Serverless Platforms. USENIX ATC.

[57]

Yiming Zhang, Jon Crowcroft, Dongsheng Li, Chengfen Zhang, Huiba Li, Yaozheng Wang, Kai Yu, Yongqiang Xiong, and Guihai Chen. 2018. KylinX: a dynamic library operating system for simplified and efficient cloud virtualization. In USENIX ATC.

Cited By

LI ZZHAO YCHEN QGUO M(2024)Serverless computing based on dynamic-addressable sessionSCIENTIA SINICA Informationis10.1360/SSI-2023-015554:3(582)Online publication date: 11-Mar-2024
https://doi.org/10.1360/SSI-2023-0155
Shojaee Rad ZGhobaei-Arani M(2024)Data pipeline approaches in serverless computing: a taxonomy, review, and research trendsJournal of Big Data10.1186/s40537-024-00939-011:1Online publication date: 11-Jun-2024
https://doi.org/10.1186/s40537-024-00939-0
Zhao ZWu MChen HZang B(2024)Characterization and Reclamation of Frozen Garbage in Managed FaaS WorkloadsProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3629579(281-297)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3629579
Show More Cited By

Index Terms

Photons: lambdas on a diet
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
      1. Cloud computing
2. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Software infrastructure
        Virtual machines

Recommendations

Nightcore: efficient and scalable serverless computing for latency-sensitive, interactive microservices
ASPLOS '21: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

The microservice architecture is a popular software engineering approach for building flexible, large-scale online services. Serverless functions, or function as a service (FaaS), provide a simple programming model of stateless functions which are a ...
Performance modeling and analysis of virtualized multi-tier applications under dynamic workloads

Virtual machine technology facilitates implementation of modern Internet services, especially multi-tier applications. Server virtualization aims to reduce the cost of service provisioning and improve fault tolerance, portability and security of ...
Supporting Multi-Provider Serverless Computing on the Edge
ICPP Workshops '18: Workshop Proceedings of the 47th International Conference on Parallel Processing

Serverless computing has recently emerged as a new execution model for cloud computing, in which service providers offer compute runtimes, also known as Function-as-a-Service (FaaS) platforms, allowing users to develop, execute and manage application ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SoCC '20: Proceedings of the 11th ACM Symposium on Cloud Computing

October 2020

535 pages

ISBN:9781450381376

DOI:10.1145/3419111

General Chair:
Rodrigo Fonseca
Microsoft and Brown University
,
Program Chairs:
Christina Delimitrou
Cornell University
,
Beng Chin Ooi
National University of Singapore

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SoCC '20

Sponsor:

SoCC '20: ACM Symposium on Cloud Computing

October 19 - 21, 2020

Virtual Event, USA

Acceptance Rates

SoCC '20 Paper Acceptance Rate 35 of 143 submissions, 24%;

Overall Acceptance Rate 169 of 722 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
777
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)16

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

LI ZZHAO YCHEN QGUO M(2024)Serverless computing based on dynamic-addressable sessionSCIENTIA SINICA Informationis10.1360/SSI-2023-015554:3(582)Online publication date: 11-Mar-2024
https://doi.org/10.1360/SSI-2023-0155
Shojaee Rad ZGhobaei-Arani M(2024)Data pipeline approaches in serverless computing: a taxonomy, review, and research trendsJournal of Big Data10.1186/s40537-024-00939-011:1Online publication date: 11-Jun-2024
https://doi.org/10.1186/s40537-024-00939-0
Zhao ZWu MChen HZang B(2024)Characterization and Reclamation of Frozen Garbage in Managed FaaS WorkloadsProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3629579(281-297)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3629579
Kohli SKharbanda SBruno RCarreira JFonseca P(2024)Pronghorn: Effective Checkpoint Orchestration for Serverless Hot-StartsProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3629556(298-316)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3629556
Hui XXu YGuo ZShen XMencagli GDazzi PLowenthal DBadia R(2024)ESG: Pipeline-Conscious Efficient Scheduling of DNN Workflows on Serverless Platforms with Shareable GPUsProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658657(42-55)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658657
Basu Roy RPatel TGarg RTiwari DTsafrir DMUSUVATHI MGupta RAbu-Ghazaleh N(2024)CodeCrunch: Improving Serverless Performance via Function Compression and Cost-Aware Warmup Location OptimizationProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 110.1145/3617232.3624866(85-101)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3617232.3624866
Cheng DYan KCai XGong YHu C(2024)SLO-Aware Function Placement for Serverless Workflows With Layer-Wise Memory SharingIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.339185835:6(1074-1091)Online publication date: Jun-2024
https://doi.org/10.1109/TPDS.2024.3391858
Li YGu LQu ZTian LZeng D(2024)On Efficient Zygote Container Planning and Task Scheduling for Edge Native Application AccelerationIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621106(2259-2268)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621106
Oakley JFerhatosmanoglu H(2024)FSD-Inference: Fully Serverless Distributed Inference with Scalable Cloud Communication2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00168(2109-2122)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00168
Yu LFu LChenhao S(2024)Serverless Cold Start Performance Optimization Based on Multi-Request Processing and Adaptive Hierarchical ScalingIEEE Access10.1109/ACCESS.2024.346238912(136248-136262)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3462389
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents