Abstract
Serverless computing recently emerged as a new run-time paradigm to disentangle the client from the burden of provisioning physical computing resources, leaving such difficulty on the service provider’s side. However, an unsolved problem in such an environment is how to cope with the challenges of executing several co-running applications while fulfilling the requested Quality of Service (QoS) level requested by all application owners. In practice, developing an efficient mechanism to reach the requested performance level (such as p-99 latency and throughput) is limited to the awareness (resource availability, performance interference among consolidation workloads, etc.) of the controller about the dynamics of the underlying platforms. In this paper, we develop an adaptive feedback controller for coping with the buffer instability of serverless platforms when several collocated applications are run in a shared environment. The goal is to support a low-latency execution by managing the arrival event rate of each application when shared resource contention causes a significant throughput degradation among workloads with different priorities. The key component of the proposed architecture is a continues management of server-side internal buffers for each application to provide a low-latency feedback control mechanism based on the requested QoS level of each application (e.g., buffer information) and the worker nodes throughput. The empirical results confirm the response stability for high priority workloads when a dynamic condition is caused by low priority applications. We evaluate the performance of the proposed solution with respect to the response time and the QoS violation rate for high priority applications in a serverless platform with four worker nodes set up in our in-house virtualized cluster. We compare the proposed architecture against the default resource management policy in Apache OpenWhisk which is extensively used in commercial serverless platforms. The results show that our approach achieves a very low overhead (less than 0.7%) while it can improve the p-99 latency of high priority applications by 64%, on average, in the presence of dynamic high traffic conditions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Menascé, D.A., Almeida, V.A.F., Riedi, R., Ribeiro, F., et al.: Hierarchical and multiscale approach to analyze e-business workloads. Perform. Eval. 54, 33–57 (2003)
Poccia, D.: AWS Lambda in Action: event-driven serverless applications. Simon and Schuster (2016)
Sbarski, P., Kroonenburg, S.: Serverless architectures on AWS: with examples using Aws Lambda. Simon and Schuster (2017)
Kim, Y.K., HoseinyFarahabady, M.R., Lee, Y.C., Zomaya, A.Y.: Automated fine-grained CPU cap control in serverless computing platform. IEEE Trans. Parallel Distrib. Syst. 31(10), 2289–2301 (2020)
Schad, J., Dittrich, J., et al.: Runtime measurements in the cloud: observing, analyzing, and reducing variance. Proc. VLDB Endow. 3, 460–471 (2010)
Wang, H., et al.: A-DRM: architecture-aware distributed RA of Virt. Clusters. In: ACM SIGPLAN/SIGOPS on Virtual Execution Environments, pp. 93–106 (2015)
Shuai, Y., Petrovic, G., Herfet, T.: OLAC: an open-loop controller for low-latency adaptive video streaming. In: 2015 IEEE International Conference on Communications (ICC), pp. 6874–6879 (2015)
Taheri, J., Zomaya, A.Y., Kassler, A.: A black-box throughput predictor for VMs in cloud environments. In: European Conference on Service-Oriented and Cloud Computing, pp. 18–33. Springer (2016). https://doi.org/10.1007/978-3-319-44482-6_2
Al-Dulaimy, A., Taheri, J., Kassler, A., HoseinyFarahabady, M.R., Deng, S., Zomaya, A.: MULTISCALER: a multi-loop auto-scaling approach for cloud-based applications. IEEE Trans. Cloud Comput. (2020)
NumFOCUS. Dask: Advanced Parallelism for Analytics, Enabling Performance. https://dask.org/ (2021)
Apache Org. OpenWhisk: Open Source Serverless Cloud Platform. https://openwhisk.incubator.apache.org (2021)
Kim, Y.K., HoseinyFarahabady, M.R., Lee, Y.C., Zomaya, A.Y., Jurdak, R.: Dynamic control of CPU usage in a lambda platform. In: 2018 IEEE International Conference on Cluster Computing (CLUSTER), pp. 234–244 (2018)
HoseinyFarahabady, M.R., Zomaya, A.Y., Tari, Z.: MPC for managing QoS enforcements & microarchitecture-level interferences in a lambda platform. IEEE Trans. Parall. Distrib. Syst. 29(7), 1442–1455 (2018)
Hoseinyfarahabady, M.R., Tari, Z., Zomaya, A.Y.: Disk throughput controller for cloud data-centers. In: International Conference on Parallel and Distributed Computing, Applications and Technologies, pp. 404–409 (2019)
HoseinyFarahabady, M.R., Taheri, J., Tari, Z., Zomaya, A.Y.: A dynamic resource controller for a lambda architecture. In: 2017 46th International Conference on Parallel Processing (ICPP), pp. 332–341 (2017)
Rawlings, J., Mayne, D.Q., Diehl, M.M.: Model predictive control: theory, computation, and design. Nob Hill Publishing, Madison, Wisconsin (2017)
Box, G., et al.: Time Series: Forecasting & Control. Wiley (2008)
Allen: Probability, Statistics, Queueing Theory. Academic Press, Cambridge (1990)
Ferdman, M., Adileh, A., et al.: Clearing the clouds: a study of emerging scale-out workloads on modern hardware. In: Architectural Support for Programming Languages & Operating Systems, ASPLOS, pp. 37–48. ACM (2012)
Acknowledgment
Prof. Albert Y. Zomaya acknowledges the support of Australian Research Council Discovery scheme (DP190103710). Prof. Javid Taheri would like to acknowledge the support of the Knowledge Foundation of Sweden through the AIDA project. Prof. Zahir Tari would like to acknowledge the support of the Australian Research Council (grant DP200100005). Dr. MohammadReza HoseinyFarahabady acknowledge the continued support and patronage of The Center for Distributed and High Performance Computing in The University of Sydney, NSW, Australia for giving access to advanced high-performance computing platforms and industry’s leading cloud facilities, machine learning (ML) and analytic infrastructure, the digital IT services and other necessary tools.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
HoseinyFarahabady, M.R., Taheri, J., Zomaya, A.Y., Tari, Z. (2022). Low Latency Execution Guarantee Under Uncertainty in Serverless Platforms. In: Shen, H., et al. Parallel and Distributed Computing, Applications and Technologies. PDCAT 2021. Lecture Notes in Computer Science(), vol 13148. Springer, Cham. https://doi.org/10.1007/978-3-030-96772-7_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-96772-7_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-96771-0
Online ISBN: 978-3-030-96772-7
eBook Packages: Computer ScienceComputer Science (R0)