research-article

Automated Demand-driven Resource Scaling in Relational Database-as-a-Service

Authors:

Vivek R. Narasayya,

Arnd Christian KönigAuthors Info & Claims

SIGMOD '16: Proceedings of the 2016 International Conference on Management of Data

Pages 1923 - 1934

https://doi.org/10.1145/2882903.2903733

Published: 26 June 2016 Publication History

Abstract

Relational Database-as-a-Service (DaaS) platforms today support the abstraction of a resource container that guarantees a fixed amount of resources. Tenants are responsible for selecting a container size suitable for their workloads, which they can change to leverage the cloud's elasticity. However, automating this task is daunting for most tenants since estimating resource demands for arbitrary SQL workloads in an RDBMS is complex and challenging. In addition, workloads and resource requirements can vary significantly within minutes to hours, and container sizes vary by orders of magnitude both in the amount of resources as well as monetary cost. We present a solution to enable a DaaS to auto-scale container sizes on behalf of its tenants. Approaches to auto-scale stateless services, such as web servers, that rely on historical resource utilization as the primary signal, often perform poorly for stateful database servers which are significantly more complex. Our solution derives a set of robust signals from database engine telemetry and combines them to significantly improve accuracy of demand estimation for database workloads resulting in more accurate scaling decisions. Our solution raises the abstraction by allowing tenants to reason about monetary budget and query latency rather than resources. We prototyped our approach in Microsoft Azure SQL Database and ran extensive experiments using workloads with realistic time-varying resource demand patterns obtained from production traces. Compared to an approach that uses only resource utilization to estimate demand, our approach results in 1.5x to 3x lower monetary costs while achieving comparable query latencies.

References

[1]

M. Ahmad, S. Duan, A. Aboulnaga, and S. Babu. Predicting completion times of batch query workloads using interaction-aware models and simulation. In EDBT, pages 449--460, 2011.

Digital Library

[2]

M. Akdere, U. Çetintemel, M. Riondato, E. Upfal, and S. B. Zdonik. Learning-based query performance modeling and prediction. In ICDE, pages 390--401, 2012.

Digital Library

[3]

C. Curino, E. P. C. Jones, S. Madden, and H. Balakrishnan. Workload-aware database monitoring and consolidation. In SIGMOD, pages 313--324, 2011.

Digital Library

[4]

S. Das, D. Agrawal, and A. El Abbadi. Elastras: An elastic, scalable, and self-managing transactional database for the cloud. ACM Trans. Database Syst., 38(1):5, 2013.

Digital Library

[5]

J. Duggan, U. Cetintemel, O. Papaemmanouil, and E. Upfal. Performance prediction for concurrent database workloads. In SIGMOD, pages 337--348, 2011.

Digital Library

[6]

A. Elmore, S. Das, A. Pucher, D. Agrawal, A. E. Abbadi, and X. Yan. Characterizing Tenant Behavior for Placement and Crisis Mitigation in Multitenant DBMSs. In SIGMOD, pages 517--528, 2013.

Digital Library

[7]

A. Ganapathi, H. Kuno, U. Dayal, J. Wiener, A. Fox, M. Jordan, and D. Patterson. Predicting multiple metrics for queries: Better decisions enabled by machine learning. In ICDE, pages 592--603, 2009.

Digital Library

[8]

A. Gandhi, M. Harchol-Balter, R. Raghunathan, and M. A. Kozuch. Autoscale: Dynamic, robust capacity management for multi-tier data centers. ACM Trans. Comput. Syst., 30(4):14:1--14:26, 2012.

Digital Library

[9]

Z. Gong, X. Gu, and J. Wilkes. Press: Predictive elastic resource scaling for cloud systems. In CNSM, pages 9--16, 2010.

[10]

D. Gupta, L. Cherkasova, R. Gardner, and A. Vahdat. Enforcing Performance Isolation Across Virtual Machines in Xen. In Middleware, pages 342--362, 2006.

Digital Library

[11]

N. Huber, F. Brosig, and S. Kounev. Model-based self-adaptive resource allocation in virtualized environments. In SEAMS, pages 90--99, 2011.

Digital Library

[12]

J. Li, A. C. König, V. R. Narasayya, and S. Chaudhuri. Robust estimation of resource consumption for SQL queries using statistical techniques. PVLDB, 5(11):1555--1566, 2012.

Digital Library

[13]

H. C. Lim, S. Babu, and J. S. Chase. Automated control for elastic storage. In ICAC, pages 1--10, 2010.

Digital Library

[14]

B. Mozafari, C. Curino, A. Jindal, and S. Madden. Performance and Resource Modeling in Highly-Concurrent OLTP Workloads. In SIGMOD, 2013.

Digital Library

[15]

V. Narasayya, S. Das, M. Syamala, B. Chandramouli, and S. Chaudhuri. SQLVM: Performance Isolation in Multi-Tenant Relational Database-as-a-Service. In CIDR, 2013.

Digital Library

[16]

H. Nguyen, Z. Shen, X. Gu, S. Subbiah, and J. Wilkes. Agile: Elastic distributed resource scaling for infrastructure-as-a-service. In ICAC, pages 69--82, 2013.

[17]

P. Padala, K.-Y. Hou, K. G. Shin, X. Zhu, M. Uysal, Z. Wang, S. Singhal, and A. Merchant. Automated control of multiple virtualized resources. In EuroSys, pages 13--26, 2009.

Digital Library

[18]

P. J. Rousseeuw and A. M. Leroy. Robust Regression and Outlier Detection. Wiley, 1987.

[19]

Z. Shen, S. Subbiah, X. Gu, and J. Wilkes. Cloudscale: Elastic resource scaling for multi-tenant cloud systems. In SOCC, pages 5:1--5:14, 2011.

Digital Library

[20]

P. Shivam, A. Demberel, P. Gunda, D. Irwin, L. Grit, A. Yumerefendi, S. Babu, and J. Chase. Automated and on-demand provisioning of virtual machines for database applications. In SIGMOD, pages 1079--1081, 2007.

Digital Library

[21]

A. A. Soror, U. F. Minhas, A. Aboulnaga, K. Salem, P. Kokosielis, and S. Kamath. Automatic virtual machine configuration for database workloads. In SIGMOD, pages 953--966, 2008.

Digital Library

[22]

C. Spearman. The proof and measurement of association between two things. American J. of Psychology, 15:72--101, 1904.

[23]

A. J. Storm, C. Garcia-Arellano, S. S. Lightstone, Y. Diao, and M. Surendra. Adaptive self-tuning memory in db2. In VLDB, pages 1081--1092, 2006.

Digital Library

[24]

A. S. Tanenbaum and D. J. Wetherall. Computer Networks. Pearson, 5th edition, 2011.

Digital Library

[25]

H. Theil. A rank-invariant method of linear and polynomial regression analysis. Nederl. Akad. Wetensch., 53, 1950.

[26]

B. Trushkowsky, P. Bodık, A. Fox, M. J. Franklin, M. I. Jordan, and D. A. Patterson. The SCADS director: Scaling a distributed storage system under stringent performance requirements. In FAST, pages 163--176, 2011.

Digital Library

[27]

C. A. Waldspurger. Memory resource management in vmware esx server. SIGOPS Oper. Syst. Rev., 36(SI):181--194, 2002.

Digital Library

Cited By

Zou DLu WZhu ZLu XZhou JWang XLiu KWang KSun RWang H(2024)OptScaler: A Collaborative Framework for Robust Autoscaling in the CloudProceedings of the VLDB Endowment10.14778/3685800.368582917:12(4090-4103)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.14778/3685800.3685829
Glaze NMcNeely TZhu YGleeson MSerr HBhopi RKrishnan S(2024)Lorentz: Learned SKU Recommendation Using Profile DataProceedings of the ACM on Management of Data10.1145/36549522:3(1-25)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654952
Shah KGandhi AGandhi KBhise M(2024)Workload Prediction for Edge ComputingProceedings of the 25th International Conference on Distributed Computing and Networking10.1145/3631461.3632522(286-291)Online publication date: 4-Jan-2024
https://dl.acm.org/doi/10.1145/3631461.3632522
Show More Cited By

Index Terms

Automated Demand-driven Resource Scaling in Relational Database-as-a-Service
1. Information systems
  1. Data management systems
    1. Database administration
      1. Autonomous database administration
2. Networks
  1. Network services
    1. Cloud computing

Recommendations

Resource provisioning for containerized applications
Abstract
Elasticity is an important feature of cloud computing, which allocates/de-allocates adequate computing resources automatically and provisions and de-provisions computing resources timely when the workload fluctuates. It can help in better resource ...
Delta Scaling: How Resources Scalability/Termination Can Be Taken Place Economically?
SERVICES '15: Proceedings of the 2015 IEEE World Congress on Services

Cloud Computing promises to completely revolutionize the capacity management of resources. The elasticity and the economy of scale are the intrinsic elements that dierentiate it from traditional computing paradigm. A good capacity planning method is a ...
Cloud services brokering for elastic workloads
SYSTOR '15: Proceedings of the 8th ACM International Systems and Storage Conference

With the growing trend of using public clouds for significant portion of enterprise workloads and the multitude of available cloud providers, arises a real need for brokerage of cloud services. The IBM Cloud Service Broker adds value to cloud services ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '16: Proceedings of the 2016 International Conference on Management of Data

June 2016

2300 pages

ISBN:9781450335317

DOI:10.1145/2882903

General Chairs:
Fatma Özcan
IBM Research, USA
,
Georgia Koutrika
HP Labs, USA
,
Program Chair:
Sam Madden
Massachusetts Institute of Technology, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGMOD/PODS'16

Sponsor:

SIGMOD

SIGMOD/PODS'16: International Conference on Management of Data

June 26 - July 1, 2016

California, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
766
Total Downloads

Downloads (Last 12 months)74
Downloads (Last 6 weeks)9

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zou DLu WZhu ZLu XZhou JWang XLiu KWang KSun RWang H(2024)OptScaler: A Collaborative Framework for Robust Autoscaling in the CloudProceedings of the VLDB Endowment10.14778/3685800.368582917:12(4090-4103)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.14778/3685800.3685829
Glaze NMcNeely TZhu YGleeson MSerr HBhopi RKrishnan S(2024)Lorentz: Learned SKU Recommendation Using Profile DataProceedings of the ACM on Management of Data10.1145/36549522:3(1-25)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654952
Shah KGandhi AGandhi KBhise M(2024)Workload Prediction for Edge ComputingProceedings of the 25th International Conference on Distributed Computing and Networking10.1145/3631461.3632522(286-291)Online publication date: 4-Jan-2024
https://dl.acm.org/doi/10.1145/3631461.3632522
Pavlenko ACahoon JZhu YKroth BNelson MCarter ALiao DWright TCamacho-Rodríguez JSaur KBarcelo PSanchez-Pi NMeliou ASudarshan S(2024)Vertically Autoscaling Monolithic Applications with CaaSPER: Scalable Container-as-a-Service Performance Enhanced Resizing Algorithm for the CloudCompanion of the 2024 International Conference on Management of Data10.1145/3626246.3653378(241-254)Online publication date: 9-Jun-2024
https://dl.acm.org/doi/10.1145/3626246.3653378
Maxenti SD’Oro SBonati LPolese MCapone AMelodia T(2024)ScalO-RAN: Energy-aware Network Intelligence Scaling in Open RANIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621159(891-900)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621159
Hang HTang XSun JBao LLo DWang H(2024)Robust Auto-Scaling with Probabilistic Workload Forecasting for Cloud Databases2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00308(4016-4029)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00308
Povzner AMahajan PGustafson JRao JJuma IMin FSridharan SBhatia NAttaluri GChandra AKozlovski SSivaram RBradstreet LBarrett BShah DJacot DArthur DDagostino RMcCabe CObili MPrakasam KSancio JSingh VNikhil AGupta K(2023)Kora: A Cloud-Native Event Streaming Platform for KafkaProceedings of the VLDB Endowment10.14778/3611540.361156716:12(3822-3834)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.14778/3611540.3611567
Poppe OCastro PLang WLeeka J(2023)Proactive Resource Allocation Policy for Microsoft Azure Cognitive SearchACM SIGMOD Record10.1145/3631504.363151652:3(41-48)Online publication date: 2-Nov-2023
https://dl.acm.org/doi/10.1145/3631504.3631516
El Danaoui MYin SHameurlain AMorvan F(2023)A Cost-Effective Query Optimizer for Multi-tenant Parallel DBMSsNew Trends in Database and Information Systems10.1007/978-3-031-42941-5_3(25-34)Online publication date: 31-Aug-2023
https://doi.org/10.1007/978-3-031-42941-5_3
Feng XGuo CJiao TSong J(2022)A maturity model for AI-empowered cloud-native databases: from the perspective of resource managementJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-022-00318-111:1Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1186/s13677-022-00318-1
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents