Article

Open access

Nightcore: efficient and scalable serverless computing for latency-sensitive, interactive microservices

Authors:

Emmett WitchelAuthors Info & Claims

ASPLOS '21: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

Pages 152 - 166

https://doi.org/10.1145/3445814.3446701

Published: 17 April 2021 Publication History

Abstract

The microservice architecture is a popular software engineering approach for building flexible, large-scale online services. Serverless functions, or function as a service (FaaS), provide a simple programming model of stateless functions which are a natural substrate for implementing the stateless RPC handlers of microservices, as an alternative to containerized RPC servers. However, current serverless platforms have millisecond-scale runtime overheads, making them unable to meet the strict sub-millisecond latency targets required by existing interactive microservices.

We present Nightcore, a serverless function runtime with microsecond-scale overheads that provides container-based isolation between functions. Nightcore’s design carefully considers various factors having microsecond-scale overheads, including scheduling of function requests, communication primitives, threading models for I/O, and concurrent function executions. Nightcore currently supports serverless functions written in C/C++, Go, Node.js, and Python. Our evaluation shows that when running latency-sensitive interactive microservices, Nightcore achieves 1.36×–2.93× higher throughput and up to 69% reduction in tail latency.

References

[1]

[n.d.]. 4 Microservices Examples : Amazon, Netflix, Uber, and Etsy. https: //blog.dreamfactory.com/microservices-examples/ [Accessed Jan, 2021 ].

[2]

[n.d.]. Accessing Amazon CloudWatch logs for AWS Lambda. https://docs. aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs. html [Accessed Dec, 2020 ].

[3]

[n.d.]. Addressing Cascading Failures. https://landing.google.com/sre/srebook/chapters/addressing-cascading-failures/ [Accessed Jan, 2021 ].

[4]

[n.d.]. Adopting Microservices at Netflix: Lessons for Architectural Design. https://www.nginx.com/blog/microservices-at-netflix-architecturalbest-practices/ [Accessed Jan, 2021 ].

[5]

[n.d.]. Airbnb's 10 Takeaways from Moving to Microservices. https: //thenewstack.io/airbnbs-10-takeaways-moving-microservices/ [Accessed Jan, 2021 ].

[6]

[n.d.]. Amazon DynamoDB | NoSQL Key-Value Database | Amazon Web Services. https://aws.amazon.com/dynamodb/ [Accessed Jan, 2021 ].

[7]

[n.d.]. Amazon ElastiCache-In-memory data store and cache. https://aws. amazon.com/elasticache/ [Accessed Jan, 2021 ].

[8]

[n.d.]. Announcing WebSocket APIs in Amazon API Gateway. https://aws.amazon.com/blogs/compute/announcing-websocket-apis-inamazon-api-gateway/ [Accessed Dec, 2020 ].

[9]

[n.d.]. Apache Thrift-Home. https://thrift.apache.org/ [Accessed Jan, 2021 ].

[10]

[n.d.]. Architecture: Scalable commerce workloads using microservices. https://cloud.google.com/solutions/architecture/scaling-commerceworkloads-architecture [Accessed Jan, 2021 ].

[11]

[n.d.]. asyncio-Asynchronous I /O. https://docs.python. org/3.8/library/ asyncio.html [Accessed Jan, 2021 ].

[12]

[n.d.]. AWS Fargate-Run containers without having to manage servers or clusters. https://aws.amazon.com/fargate/ [Accessed Jan, 2021 ].

[13]

[n.d.]. AWS Lambda execution context-AWS Lambda. https://docs.aws. amazon.com/lambda/latest/dg/runtimes-context. html [Accessed Jan, 2021 ].

[14]

[n.d.]. AWS Lambda FAQs. https://aws.amazon.com/lambda/faqs/ [Accessed Jan, 2021 ].

[15]

[n.d.]. AWS Lambda-Serverless Compute-Amazon Web Servicesy. https: //aws.amazon.com/lambda/ [Accessed Jan, 2021 ].

[16]

[n.d.]. BCC-Tools for BPF-based Linux IO analysis, networking, monitoring, and more. https://github.com/iovisor/bcc [Accessed Jan, 2021 ].

[17]

[n.d.]. Best practices for working with AWS Lambda functions. https://docs. aws.amazon.com/lambda/latest/dg/best-practices. html [Accessed Dec, 2020 ].

[18]

[n.d.]. Building serverless microservices in Azure-sample architecture. https://azure.microsoft.com/is-is/blog/building-serverless-microservicesin-azure-sample-architecture/ [Accessed Jan, 2021 ].

[19]

[n.d.]. Cloud Object Storage | Store and Retrieve Data Anywhere | Amazon Simple Storage Service (S3). https://aws.amazon.com/s3/ [Accessed Jan, 2021 ].

[20]

[n.d.]. Coursera Case Study. https://aws.amazon.com/solutions/case-studies/ coursera-ecs/ [Accessed Jan, 2021 ].

[21]

[n.d.]. delimitrou/DeathStarBench: Open-source benchmark suite for cloud microservices. https://github.com/delimitrou/DeathStarBench [Accessed Jan, 2021 ].

[22]

[n.d.]. Enough with the microservices. https://adamdrake.com/enough-withthe-microservices. html [Accessed Jan, 2021 ].

[23]

[n.d.]. Event-based Concurrency (Advanced). http://pages.cs.wisc.edu/~remzi/ OSTEP/threads-events. pdf [Accessed Jan, 2021 ].

[24]

[n.d.]. eventfd(2 )-Linux manual page. https://man7.org/linux/man-pages / man2/eventfd.2.html [Accessed Jan, 2021 ].

[25]

[n.d.]. ifrecracker/network-performance.md at master · firecrackermicrovm/firecracker. https://github.com/firecracker-microvm/firecracker/ blob/master/docs/network-performance. md [Accessed Jan, 2021 ].

[26]

[n.d.]. giltene/wrk2: A constant throughput, correct latency recording variant of wrk. https://github.com/giltene/wrk2 [Accessed Jan, 2021 ].

[27]

[n.d.]. Go, don't collect my garbage. https://blog.cloudflare.com/go-dontcollect-my-garbage/ [Accessed Jan, 2021 ].

[28]

[n.d.]. Go memory ballast: How I learnt to stop worrying and love the heap. https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i-learntto-stop-worrying-and-love-the-heap-26c2462549a2/ [Accessed Jan, 2021 ].

[29]

[n.d.]. GoogleCloudPlatform/microservices-demo. https://github.com/ GoogleCloudPlatform/microservices-demo [Accessed Jan, 2021 ].

[30]

[n.d.]. gRPC-A high-performance, open source universal RPC framework. https://grpc.io/ [Accessed Jan, 2021 ].

[31]

[n.d.]. IPC settings | Docker run reference. https://docs.docker.com/engine/ reference/run/#ipc-settings---ipc [Accessed Jan, 2021 ].

[32]

[n.d.]. libuv | Cross-platform asynchronous I/O. https://libuv.org/ [Accessed Jan, 2021 ].

[33]

[n.d.]. Lyft Case Study. https://aws.amazon.com/solutions/case-studies/lyft/ [Accessed Jan, 2021 ].

[34]

[n.d.]. Manage your function app. https://docs.microsoft.com/en-us/azure/ azure-functions/ functions-how-to-use-azure-function-app-settings [ Accessed Jan, 2021 ].

[35]

[n.d.]. Microservice Trade-Ofs. https://martinfowler.com/articles/microservicetrade-ofs. html [Accessed Jan, 2021 ].

[36]

[n.d.]. Microservices-Wikipedia. https://en.wikipedia.org/wiki/Microservices [Accessed Jan, 2021 ].

[37]

[n.d.]. OpenFaaS | Serverless Functions, Made Simple. https://www.openfaas. com/ [Accessed Jan, 2021 ].

[38]

[n.d.]. Performance Under Load. https://medium.com/@NetflixTechBlog/ performance-under-load-3e6fa9a60581 [Accessed Jan, 2021 ].

[39]

[n.d.]. plugin-The Go Programming Language. https://golang.org/pkg/plugin/ [Accessed Jan, 2021 ].

[40]

[n.d.]. Provisioned Concurrency for Lambda Functions. https://aws.amazon. com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/ [Accessed Jan, 2021 ].

[41]

[n.d.]. Remind Case Study. https://aws.amazon.com/solutions/case-studies/ remind/ [Accessed Jan, 2021 ].

[42]

[n.d.]. Rewriting Uber Engineering: The Opportunities Microservices Provide. https://eng.uber.com/building-tincup-microservice-implementation/ [Accessed Jan, 2021 ].

[43]

[n.d.]. Serverless and Microservices: a match made in heaven? https://pauldjohnston.medium.com/serverless-and-microservices-a-matchmade-in-heaven-9964f329a3bc [Accessed Dec, 2020 ].

[44]

[n.d.]. Serverless Microservices-Microservices on AWS. https: //docs.aws.amazon.com/whitepapers/latest/microservices-on-aws/serverlessmicroservices.html [Accessed Jan, 2021 ].

[45]

[n.d.]. Serverless Microservices reference architecture. https: //docs.microsoft.com/en-us/samples/azure-samples/ serverless-microservicesreference-architecture/serverless-microservices-reference-architecture/ [Accessed Dec, 2020 ].

[46]

[n.d.]. shm_overview(7)-Linux manual page. https://man7.org/linux/manpages/man7/shm_overview.7.html [Accessed Jan, 2021 ].

[47]

[n.d.]. Splitting Up a Codebase into Microservices and Artifacts. https://engineering.linkedin.com/blog/2016/02/q-a-with-jim-brikman--splitting-up-a-codebase-into-microservices [ Accessed Jan, 2021 ].

[48]

[n.d.]. "Stop Rate Limiting! Capacity Management Done Right" by Jon Moore. https://www.youtube.com/watch?v=m64SWl9bfvk [Accessed Jan, 2021 ].

[49]

[n.d.]. Thoughts on (micro)services. https://luminousmen.com/post/thoughtson-microservices [Accessed Jan, 2021 ].

[50]

[n.d.]. Uncovering the magic: How serverless platforms really work! https://medium.com/openwhisk/uncovering-the-magic-how-serverlessplatforms-really-work-3cb127b05f71 [Accessed Jan, 2021 ].

[51]

[n.d.]. Watchdog-OpenFaaS. https://docs.openfaas.com/architecture/ watchdog/ [Accessed Jan, 2021 ].

[52]

[n.d.]. What are Microservices? | AWS. https://aws.amazon.com/microservices/ [Accessed Jan, 2021 ].

[53]

[n.d.]. What is a serverless microservice? | Serverless microservices explained. https://www.cloudflare.com/learning/serverless/glossary/serverlessmicroservice/ [Accessed Dec, 2020 ].

[54]

[n.d.]. Why should you use microservices and containers? https: //developer.ibm.com/technologies/microservices/articles/why-should-weuse-microservices-and-containers/ [Accessed Jan, 2021 ].

[55]

[n.d.]. Why so slow?-Binaris Blog. https://blog.binaris.com/why-so-slow/ [Accessed Jan, 2021 ].

[56]

[n.d.]. Worker threads | Node.js v14.8.0 Documentation. https://nodejs.org/api/ worker_threads. html [Accessed Jan, 2021 ].

[57]

Istemi Ekin Akkus, Ruichuan Chen, Ivica Rimac, Manuel Stein, Klaus Satzke, Andre Beck, Paarijaat Aditya, and Volker Hilt. 2018. SAND: Towards HighPerformance Serverless Computing. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). USENIX Association, Boston, MA, 923-935. https://www. usenix.org/conference/atc18/presentation/akkus

[58]

Mohammad Alizadeh, Albert Greenberg, David A. Maltz, Jitendra Padhye, Parveen Patel, Balaji Prabhakar, Sudipta Sengupta, and Murari Sridharan. 2010. Data Center TCP (DCTCP). In Proceedings of the ACM SIGCOMM 2010 Conference (New Delhi, India) (SIGCOMM '10). Association for Computing Machinery, New York, NY, USA, 63-74. https://doi.org/10.1145/1851182.1851192

Digital Library

[59]

Lixiang Ao, Liz Izhikevich, Geofrey M. Voelker, and George Porter. 2018. Sprocket: A Serverless Video Processing Framework. In Proceedings of the ACM Symposium on Cloud Computing (Carlsbad, CA, USA) ( SoCC '18). Association for Computing Machinery, New York, NY, USA, 263-274. https: //doi.org/10.1145/3267809.3267815

Digital Library

[60]

Luiz Barroso, Mike Marty, David Patterson, and Parthasarathy Ranganathan. 2017. Attack of the Killer Microseconds. Commun. ACM 60, 4 (March 2017 ), 48-54. https://doi.org/10.1145/3015146

Digital Library

[61]

Adam Belay, George Prekas, Ana Klimovic, Samuel Grossman, Christos Kozyrakis, and Edouard Bugnion. 2014. IX: A Protected Dataplane Operating System for High Throughput and Low Latency. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14). USENIX Association, Broomfield, CO, 49-65. https://www.usenix.org/conference/osdi14/technicalsessions/presentation/belay

Digital Library

[62]

Sol Boucher, Anuj Kalia, David G. Andersen, and Michael Kaminsky. 2018. Putting the ?Micro? Back in Microservice. In Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference (Boston, MA, USA) ( USENIX ATC '18). USENIX Association, USA, 645-650.

[63]

Sol Boucher, Anuj Kalia, David G. Andersen, and Michael Kaminsky. 2020. Lightweight Preemptible Functions. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 465-477. https://www.usenix.org/ conference/atc20/presentation/boucher

[64]

James Cadden, Thomas Unger, Yara Awad, Han Dong, Orran Krieger, and Jonathan Appavoo. 2020. SEUSS: Skip Redundant Paths to Make Serverless Fast. In Proceedings of the Fifteenth European Conference on Computer Systems (Heraklion, Greece) ( EuroSys '20). Association for Computing Machinery, New York, NY, USA, Article 32, 15 pages. https://doi.org/10.1145/3342195.3392698

Digital Library

[65]

Joao Carreira, Pedro Fonseca, Alexey Tumanov, Andrew Zhang, and Randy Katz. 2019. Cirrus: A Serverless Framework for End-to-End ML Workflows. In Proceedings of the ACM Symposium on Cloud Computing (Santa Cruz, CA, USA) ( SoCC '19). Association for Computing Machinery, New York, NY, USA, 13-24. https://doi.org/10.1145/3357223.3362711

Digital Library

[66]

Jefrey Dean and Luiz André Barroso. 2013. The Tail at Scale. Commun. ACM 56, 2 (Feb. 2013 ), 74-80. https://doi.org/10.1145/2408776.2408794

Digital Library

[67]

Dong Du, Tianyi Yu, Yubin Xia, Binyu Zang, Guanglu Yan, Chenggang Qin, Qixuan Wu, and Haibo Chen. 2020. Catalyzer: Sub-Millisecond Startup for Serverless Computing with Initialization-Less Booting. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (Lausanne, Switzerland) (ASPLOS '20). Association for Computing Machinery, New York, NY, USA, 467-481. https://doi.org/10.1145/3373376.3378512

Digital Library

[68]

Sadjad Fouladi, Francisco Romero, Dan Iter, Qian Li, Shuvo Chatterjee, Christos Kozyrakis, Matei Zaharia, and Keith Winstein. 2019. From Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Functional Containers. In 2019 USENIX Annual Technical Conference (USENIX ATC 19). USENIX Association, Renton, WA, 475-488. https://www.usenix.org/conference/atc19/presentation/ fouladi

Digital Library

[69]

Sadjad Fouladi, Riad S. Wahby, Brennan Shacklett, Karthikeyan Vasuki Balasubramaniam, William Zeng, Rahul Bhalerao, Anirudh Sivaraman, George Porter, and Keith Winstein. 2017. Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). USENIX Association, Boston, MA, 363-376. https://www.usenix.org/conference/nsdi17/technicalsessions/presentation/fouladi

[70]

Yu Gan, Yanqi Zhang, Dailun Cheng, Ankitha Shetty, Priyal Rathi, Nayan Katarki, Ariana Bruno, Justin Hu, Brian Ritchken, Brendon Jackson, Kelvin Hu, Meghna Pancholi, Yuan He, Brett Clancy, Chris Colen, Fukang Wen, Catherine Leung, Siyuan Wang, Leon Zaruvinsky, Mateo Espinosa, Rick Lin, Zhongling Liu, Jake Padilla, and Christina Delimitrou. 2019. An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud & Edge Systems. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (Providence, RI, USA) ( ASPLOS '19). Association for Computing Machinery, New York, NY, USA, 3-18. https://doi.org/10.1145/3297858.3304013

Digital Library

[71]

Pedro García-López, Aleksander Slominski, Simon Shillaker, Michael Behrendt, and Barnard Metzler. 2020. Serverless End Game: Disaggregation enabling Transparency. arXiv preprint arXiv: 2006. 01251 ( 2020 ).

[72]

Sangjin Han, Scott Marshall, Byung-Gon Chun, and Sylvia Ratnasamy. 2012. MegaPipe: A New Programming Interface for Scalable Network I/O. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (Hollywood, CA, USA) ( OSDI'12). USENIX Association, USA, 135-148.

[73]

C?lin Iorgulescu, Reza Azimi, Youngjin Kwon, Sameh Elnikety, Manoj Syamala, Vivek Narasayya, Herodotos Herodotou, Paulo Tomita, Alex Chen, Jack Zhang, and Junhua Wang. 2018. PerfIso: Performance Isolation for Commercial LatencySensitive Services. In Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference (Boston, MA, USA) ( USENIX ATC '18). USENIX Association, USA, 519-531.

[74]

Eun Young Jeong, Shinae Woo, Muhammad Jamshed, Haewon Jeong, Sunghwan Ihm, Dongsu Han, and KyoungSoo Park. 2014. MTCP: A Highly Scalable UserLevel TCP Stack for Multicore Systems. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (Seattle, WA) ( NSDI'14). USENIX Association, USA, 489-502.

[75]

Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, and Benjamin Recht. 2017. Occupy the Cloud: Distributed Computing for the 99%. In Proceedings of the 2017 Symposium on Cloud Computing (Santa Clara, California) (SoCC '17). Association for Computing Machinery, New York, NY, USA, 445-451. https: //doi.org/10.1145/3127479.3128601

Digital Library

[76]

Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti, Chia-Che Tsai, Anurag Khandelwal, Qifan Pu, Vaishaal Shankar, Joao Menezes Carreira, Karl Krauth, Neeraja Yadwadkar, Joseph Gonzalez, Raluca Ada Popa, Ion Stoica, and David A. Patterson. 2019. Cloud Programming Simplified: A Berkeley View on Serverless Computing. Technical Report UCB/EECS-2019-3. EECS Department, University of California, Berkeley. http://www2.eecs.berkeley.edu/Pubs/TechRpts/2019/ EECS-2019-3.html

[77]

Kostis Kafes, Timothy Chong, Jack Tigar Humphries, Adam Belay, David Mazières, and Christos Kozyrakis. 2019. Shinjuku: Preemptive Scheduling for µSecond-Scale Tail Latency. In Proceedings of the 16th USENIX Conference on Networked Systems Design and Implementation (Boston, MA, USA) ( NSDI'19). USENIX Association, USA, 345-359.

[78]

Anuj Kalia, Michael Kaminsky, and David G. Andersen. 2019. Datacenter RPCs Can Be General and Fast. In Proceedings of the 16th USENIX Conference on Networked Systems Design and Implementation (Boston, MA, USA) ( NSDI'19). USENIX Association, USA, 1-16.

[79]

Marios Kogias, George Prekas, Adrien Ghosn, Jonas Fietz, and Edouard Bugnion. 2019. R2P2: Making RPCs first-class datacenter citizens. In 2019 USENIX Annual Technical Conference (USENIX ATC 19). USENIX Association, Renton, WA, 863-880. https://www.usenix.org/conference/atc19/presentation/kogias-r2p2

[80]

Maxwell Krohn, Eddie Kohler, and M. Frans Kaashoek. 2007. Events Can Make Sense. In 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference (Santa Clara, CA) ( ATC'07). USENIX Association, USA, Article 7, 14 pages.

[81]

Youngjin Kwon, Henrique Fingler, Tyler Hunt, Simon Peter, Emmett Witchel, and Thomas Anderson. 2017. Strata: A Cross Media File System. In Proceedings of the 26th Symposium on Operating Systems Principles (Shanghai, China) (SOSP '17). Association for Computing Machinery, New York, NY, USA, 460-477. https: //doi.org/10.1145/3132747.3132770

Digital Library

[82]

Butler W. Lampson. 1983. Hints for Computer System Design. In Proceedings of the Ninth ACM Symposium on Operating Systems Principles (Bretton Woods, New Hampshire, USA) ( SOSP '83). Association for Computing Machinery, New York, NY, USA, 33-48. https://doi.org/10.1145/800217.806614

Digital Library

[83]

N. Lazarev, N. Adit, S. Xiang, Z. Zhang, and C. Delimitrou. 2020. Dagger: Towards Eficient RPCs in Cloud Microservices With Near-Memory Reconfigurable NICs. IEEE Computer Architecture Letters 19, 2 ( 2020 ), 134-138. https://doi.org/10. 1109/LCA. 2020.3020064

[84]

Collin Lee and John Ousterhout. 2019. Granular Computing. In Proceedings of the Workshop on Hot Topics in Operating Systems (Bertinoro, Italy) ( HotOS '19). Association for Computing Machinery, New York, NY, USA, 149-154. https: //doi.org/10.1145/3317550.3321447

Digital Library

[85]

Ming Liu, Simon Peter, Arvind Krishnamurthy, and Phitchaya Mangpo Phothilimthana. 2019. E3: Energy-Eficient Microservices on SmartNICAccelerated Servers. In 2019 USENIX Annual Technical Conference (USENIX ATC 19). USENIX Association, Renton, WA, 363-378. https://www.usenix.org/ conference/atc19/presentation/liu-ming

[86]

Youyou Lu, Jiwu Shu, Youmin Chen, and Tao Li. 2017. Octopus: an RDMAenabled Distributed Persistent Memory File System. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). USENIX Association, Santa Clara, CA, 773-785. https://www.usenix.org/conference/atc17/technical-sessions/ presentation/lu

[87]

Michael Marty, Marc de Kruijf, Jacob Adriaens, Christopher Alfeld, Sean Bauer, Carlo Contavalli, Michael Dalton, Nandita Dukkipati, William C. Evans, Steve Gribble, Nicholas Kidd, Roman Kononov, Gautam Kumar, Carl Mauer, Emily Musick, Lena Olson, Erik Rubow, Michael Ryan, Kevin Springborn, Paul Turner, Valas Valancius, Xi Wang, and Amin Vahdat. 2019. Snap: A Microkernel Approach to Host Networking. In Proceedings of the 27th ACM Symposium on Operating Systems Principles (Huntsville, Ontario, Canada) ( SOSP '19). Association for Computing Machinery, New York, NY, USA, 399-413. https://doi.org/10.1145/3341301.3359657

Digital Library

[88]

Ben Maurer. 2015. Fail at Scale: Reliability in the Face of Rapid Change. Queue 13, 8 (Sept. 2015 ), 30-46. https://doi.org/10.1145/2838344.2839461

Digital Library

[89]

Anup Mohan, Harshad Sane, Kshitij Doshi, Saikrishna Edupuganti, Naren Nayak, and Vadim Sukhomlinov. 2019. Agile Cold Starts for Scalable Serverless. In 11th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 19 ). USENIX Association, Renton, WA. https://www.usenix.org/conference/hotcloud19/ presentation/mohan

[90]

Edward Oakes, Leon Yang, Dennis Zhou, Kevin Houck, Tyler Harter, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau. 2018. SOCK: Rapid Task Provisioning with Serverless-Optimized Containers. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). USENIX Association, Boston, MA, 57-70. https://www.usenix.org/conference/atc18/presentation/oakes

[91]

Amy Ousterhout, Joshua Fried, Jonathan Behrens, Adam Belay, and Hari Balakrishnan. 2019. Shenango: Achieving High CPU Eficiency for LatencySensitive Datacenter Workloads. In Proceedings of the 16th USENIX Conference on Networked Systems Design and Implementation (Boston, MA, USA) ( NSDI'19). USENIX Association, USA, 361-377.

[92]

Simon Peter, Jialin Li, Irene Zhang, Dan R. K. Ports, Doug Woos, Arvind Krishnamurthy, Thomas Anderson, and Timothy Roscoe. 2014. Arrakis: The Operating System is the Control Plane. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14). USENIX Association, Broomfield, CO, 1-16. https://www.usenix.org/conference/osdi14/technicalsessions/presentation/peter

Digital Library

[93]

Arash Pourhabibi, Siddharth Gupta, Hussein Kassir, Mark Sutherland, Zilu Tian, Mario Paulo Drumond, Babak Falsafi, and Christoph Koch. 2020. Optimus Prime: Accelerating Data Transformation in Servers. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (Lausanne, Switzerland) (ASPLOS '20). Association for Computing Machinery, New York, NY, USA, 1203-1216. https://doi.org/10. 1145/3373376.3378501

Digital Library

[94]

George Prekas, Marios Kogias, and Edouard Bugnion. 2017. ZygOS: Achieving Low Tail Latency for Microsecond-Scale Networked Tasks. In Proceedings of the 26th Symposium on Operating Systems Principles (Shanghai, China) (SOSP '17). Association for Computing Machinery, New York, NY, USA, 325-341. https: //doi.org/10.1145/3132747.3132780

Digital Library

[95]

Qifan Pu, Shivaram Venkataraman, and Ion Stoica. 2019. Shufling, Fast and Slow: Scalable Analytics on Serverless Infrastructure. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19). USENIX Association, Boston, MA, 193-206. https://www.usenix.org/conference/nsdi19/presentation/ pu

[96]

Henry Qin, Qian Li, Jacqueline Speiser, Peter Kraft, and John Ousterhout. 2018. Arachne: Core-Aware Thread Management. In Proceedings of the 13th USENIX Conference on Operating Systems Design and Implementation (Carlsbad, CA, USA) ( OSDI'18). USENIX Association, USA, 145-160.

[97]

Zhiming Shen, Zhen Sun, Gur-Eyal Sela, Eugene Bagdasaryan, Christina Delimitrou, Robbert Van Renesse, and Hakim Weatherspoon. 2019. X-Containers: Breaking Down Barriers to Improve Performance and Isolation of Cloud-Native Containers. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (Providence, RI, USA) ( ASPLOS '19). Association for Computing Machinery, New York, NY, USA, 121-135. https://doi.org/10.1145/3297858.3304016

Digital Library

[98]

Simon Shillaker and Peter Pietzuch. 2020. Faasm: Lightweight Isolation for Eficient Stateful Serverless Computing. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 419-433. https://www.usenix. org/conference/atc20/presentation/shillaker

[99]

A. Sriraman and T. F. Wenisch. 2018. Suite: A Benchmark Suite for Microservices. In 2018 IEEE International Symposium on Workload Characterization (IISWC). 1-12.

[100]

Akshitha Sriraman and Thomas F. Wenisch. 2018. µTune: Auto-Tuned Threading for OLDI Microservices. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). USENIX Association, Carlsbad, CA, 177-194. https://www.usenix.org/conference/osdi18/presentation/sriraman

[101]

M. Sutherland, S. Gupta, B. Falsafi, V. Marathe, D. Pnevmatikatos, and A. Daglis. 2020. The NEBULA RPC-Optimized Architecture. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). 199-212.

[102]

T. Ueda, T. Nakaike, and M. Ohara. 2016. Workload characterization for microservices. In 2016 IEEE International Symposium on Workload Characterization (IISWC). 1-10.

[103]

Rob von Behren, Jeremy Condit, Feng Zhou, George C. Necula, and Eric Brewer. 2003. Capriccio: Scalable Threads for Internet Services. In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles (Bolton Landing, NY, USA) ( SOSP '03). Association for Computing Machinery, New York, NY, USA, 268-281. https://doi.org/10.1145/945445.945471

Digital Library

[104]

Matt Welsh and David Culler. 2003. Adaptive Overload Control for Busy Internet Servers. In Proceedings of the 4th Conference on USENIX Symposium on Internet Technologies and Systems-Volume 4 (Seattle, WA) ( USITS'03). USENIX Association, USA, 4.

Digital Library

[105]

Matt Welsh, David Culler, and Eric Brewer. 2001. SEDA: An Architecture for Well-Conditioned, Scalable Internet Services. SIGOPS Oper. Syst. Rev. 35, 5 (Oct. 2001 ), 230-243. https://doi.org/10.1145/502059.502057

Digital Library

[106]

Christo Wilson, Hitesh Ballani, Thomas Karagiannis, and Ant Rowtron. 2011. Better Never than Late: Meeting Deadlines in Datacenter Networks. In Proceedings of the ACM SIGCOMM 2011 Conference (Toronto, Ontario, Canada) ( SIGCOMM '11). Association for Computing Machinery, New York, NY, USA, 50-61. https://doi.org/10.1145/2018436.2018443

Digital Library

[107]

Mingyu Wu, Ziming Zhao, Yanfei Yang, Haoyu Li, Haibo Chen, Binyu Zang, Haibing Guan, Sanhong Li, Chuansheng Lu, and Tongbao Zhang. 2020. Platinum: A CPU-Eficient Concurrent Garbage Collector for Tail-Reduction of Interactive Services. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 159-172. https://www.usenix.org/conference/atc20/presentation/ wu-mingyu

[108]

Jian Xu and Steven Swanson. 2016. NOVA: A Log-structured File System for Hybrid Volatile/Non-volatile Main Memories. In 14th USENIX Conference on File and Storage Technologies (FAST 16). USENIX Association, Santa Clara, CA, 323-338. https://www.usenix.org/conference/fast16/technical-sessions/ presentation/xu

[109]

Xiao Zhang, Eric Tune, Robert Hagmann, Rohit Jnagal, Vrigo Gokhale, and John Wilkes. 2013. CPI2: CPU Performance Isolation for Shared Compute Clusters. In Proceedings of the 8th ACM European Conference on Computer Systems (Prague, Czech Republic) (EuroSys '13). Association for Computing Machinery, New York, NY, USA, 379-391. https://doi.org/10.1145/2465351.2465388

Digital Library

[110]

Y. Zhang, D. Meisner, J. Mars, and L. Tang. 2016. Treadmill: Attributing the Source of Tail Latency through Precise Load Testing and Statistical Inference. In 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA). 456-468.

[111]

Hao Zhou, Ming Chen, Qian Lin, Yong Wang, Xiaobin She, Sifan Liu, Rui Gu, Beng Chin Ooi, and Junfeng Yang. 2018. Overload Control for Scaling WeChat Microservices. In Proceedings of the ACM Symposium on Cloud Computing (Carlsbad, CA, USA) ( SoCC '18). Association for Computing Machinery, New York, NY, USA, 149-161. https://doi.org/10.1145/3267809.3267823

Digital Library

[112]

Xiang Zhou, Xin Peng, Tao Xie, Jun Sun, Chenjie Xu, Chao Ji, and Wenyun Zhao. 2018. Benchmarking Microservice Systems for Software Engineering Research. In Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings (Gothenburg, Sweden) (ICSE '18). Association for Computing Machinery, New York, NY, USA, 323-324. https: //doi.org/10.1145/3183440.3194991

Digital Library

[113]

Danyang Zhuo, Kaiyuan Zhang, Yibo Zhu, Hongqiang Harry Liu, Matthew Rockett, Arvind Krishnamurthy, and Thomas Anderson. 2019. Slim: OS Kernel Support for a Low-Overhead Container Overlay Network. In Proceedings of the 16th USENIX Conference on Networked Systems Design and Implementation (Boston, MA, USA) ( NSDI'19). USENIX Association, USA, 331-344.

Cited By

Bakir FWang SEkaireb TPearson JKrintz CWolski R(2024)Ambience: an operating system for IoT microservicesJournal of Edge Computing10.55056/jec.786Online publication date: 5-Sep-2024
https://doi.org/10.55056/jec.786
Swethasri Kavuri (2024)Integrating Kubernetes Autoscaling for Cost Efficiency in Cloud ServicesInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology10.32628/CSEIT24105103810:5(480-502)Online publication date: 6-Oct-2024
https://doi.org/10.32628/CSEIT241051038
Kaur JChana IBala A(2024)An Autoscalable Approach to Optimize Energy Consumption using Smart Meters data in Serverless ComputingScience and Technology for Energy Transition10.2516/stet/2024078Online publication date: 3-Sep-2024
https://doi.org/10.2516/stet/2024078
Show More Cited By

Index Terms

Nightcore: efficient and scalable serverless computing for latency-sensitive, interactive microservices
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
      1. Cloud computing
2. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles
        Cloud computing
      2. Software architectures
        n-tier architectures

Recommendations

Supporting Multi-Provider Serverless Computing on the Edge
ICPP Workshops '18: Workshop Proceedings of the 47th International Conference on Parallel Processing

Serverless computing has recently emerged as a new execution model for cloud computing, in which service providers offer compute runtimes, also known as Function-as-a-Service (FaaS) platforms, allowing users to develop, execute and manage application ...
Serverless Computing Revisited: Evolution, State-of-the-Art, and Performance Challenges
ICPE '23 Companion: Companion of the 2023 ACM/SPEC International Conference on Performance Engineering

Market analysts are agreed that serverless computing has strong market potential, with projected compound annual growth rates varying between 21% and 28% through 2028 and a projected market value of 36.8 billion by that time. Although serverless ...
Migrating from microservices to serverless: an IoT platform case study
WoSC '22: Proceedings of the Eighth International Workshop on Serverless Computing

Microservice architecture is the common choice for developing cloud applications these days since each individual microservice can be independently modified, replaced, and scaled. As a result, application development and operating cloud infrastructure ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASPLOS '21: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

April 2021

1090 pages

ISBN:9781450383172

DOI:10.1145/3445814

General Chair:
Tim Sherwood
University of California at Santa Barbara, USA
,
Program Chairs:
Emery Berger
University of Massachusetts at Amherst, USA
,
Christos Kozyrakis
Stanford University, USA

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 April 2021

Check for updates

Badges

Author Tags

Qualifiers

Article

Funding Sources

Conference

ASPLOS '21

Sponsor:

SIGPLAN

ASPLOS '21: 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

April 19 - 23, 2021

Virtual, USA

Acceptance Rates

Overall Acceptance Rate 535 of 2,713 submissions, 20%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

86
Total Citations
View Citations
5,986
Total Downloads

Downloads (Last 12 months)1,679
Downloads (Last 6 weeks)196

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bakir FWang SEkaireb TPearson JKrintz CWolski R(2024)Ambience: an operating system for IoT microservicesJournal of Edge Computing10.55056/jec.786Online publication date: 5-Sep-2024
https://doi.org/10.55056/jec.786
Swethasri Kavuri (2024)Integrating Kubernetes Autoscaling for Cost Efficiency in Cloud ServicesInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology10.32628/CSEIT24105103810:5(480-502)Online publication date: 6-Oct-2024
https://doi.org/10.32628/CSEIT241051038
Kaur JChana IBala A(2024)An Autoscalable Approach to Optimize Energy Consumption using Smart Meters data in Serverless ComputingScience and Technology for Energy Transition10.2516/stet/2024078Online publication date: 3-Sep-2024
https://doi.org/10.2516/stet/2024078
LI ZZHAO YCHEN QGUO M(2024)Serverless computing based on dynamic-addressable sessionSCIENTIA SINICA Informationis10.1360/SSI-2023-015554:3(582)Online publication date: 11-Mar-2024
https://doi.org/10.1360/SSI-2023-0155
Tagliabue JCaraza-Harter TGreco C(2024)Bauplan: Zero-copy, Scale-up FaaS for Data PipelinesProceedings of the 10th International Workshop on Serverless Computing10.1145/3702634.3702955(31-36)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3702634.3702955
Copik MCalotoiu ARethy GBöhringer RBruno RHoefler T(2024)Process-as-a-Service: Unifying Elastic and Stateful Clouds with Serverless ProcessesProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698567(223-242)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698567
Seshagiri VGupta AJabrayilov VWildani AKaffes K(2024)Rethinking the Networking Stack for Serverless Environments: A Sidecar ApproachProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698561(213-222)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698561
Parola FQi SNarappa ARamakrishnan KRisso F(2024)SURE: Secure Unikernels Make Serverless Computing Rapid and EfficientProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698558(668-688)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698558
Lertpongrujikorn PNguyen HSalehi M(2024)Streamlining Cloud-Native Application Development and Deployment with Robust EncapsulationProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698552(847-865)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698552
Zhang XHe QFan HWu S(2024)Faascale: Scaling MicroVM Vertically for Serverless Computing with Memory ElasticityProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698512(196-212)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698512
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents