Benchmarking-as-a-service for cloud-hosted DBMS
Pages 12 - 13
Abstract
Database Management Systems (DBMS) operated on cloud resources are the storage backend for a multitude of data-intensive applications. In the process of finding the optimal cloud and DBMS, benchmarking is the common approach to evaluate the non-functional features. However, benchmarking cloud-hosted DBMS is a complex, error-prone and time-consuming task. The Benchmarking-as-a-Service platform benchANT demonstrates how these challenges are resolved by an end-to-end automated benchmarking process.
References
[1]
Daniel Abadi, Rakesh Agrawal, Anastasia Ailamaki, Balazinska, and et al. 2016. The Beckman Report on Database Research. Commun. ACM 59, 2 (Jan. 2016), 92--99.
[2]
Daniel Abadi, Anastasia Ailamaki, David Andersen, and et al. 2020. The Seattle Report on Database Research. SIGMOD Ree. 48, 4 (Feb. 2020), 44--53.
[3]
Lexi Brent and Alan Fekete. 2019. A Versatile Framework for Painless Benchmarking of Database Management Systems. In Databases Theory and Applications. Springer International Publishing, Cham, 45--56.
[4]
Ali Davoudian, Liu Chen, and Mengchi Liu. 2018. A Survey on NoSQL Stores. ACM Comput. Surv. 51, 2, Article 40 (April 2018), 43 pages.
[5]
Jörg Domaschka, Simon Volpert, and Daniel Seybold. 2020. Hathi: An MCDM-based Approach to Capacity Planning for Cloud-hosted DBMS. In IEEE/ACM 13th UCC. 143--154.
[6]
George Kousiouris and Dimosthenis Kyriazis. 2021. Enabling Containerized, Parametric and Distributed Database Deployment and Benchmarking as a Service. In Companion of the ACM/SPEC ICPE (Virtual Event, France) (ICPE '21). ACM, 77--80.
[7]
A. V. Papadopoulos, L. Versluis, A. Bauer, N. Herbst, J. Von Kistowski, A. Ali-eldin, C. Abad, J. N. Amaral, P. Tůma, and A. Iosup. 2019. Methodological Principles for Reproducible Performance Evaluation in Cloud Computing. IEEE Transactions on Software Engineering (2019), 1--1.
[8]
Andrew Pavlo and Matthew Aslett. 2016. What's really new with NewSQL? ACM Sigmod Record 45, 2 (2016), 45--55.
[9]
Mark Raasveldt, Pedro Holanda, Tim Gubner, and Hannes Mühleisen. 2018. Fair Benchmarking Considered Difficult: Common Pitfalls In Database Performance Testing. In Proceedings of the Workshop on DBTest (Houston, TX, USA). ACM, New York, NY, USA, Article 2, 6 pages.
[10]
Vincent Reniers, Dimitri Van Landuyt, Ansar Rafique, and Wouter Joosen. 2017. On the State of NoSQL Benchmarks. In Proceedings of the 8th ACM/SPEC ICPE (L'Aquila, Italy). ACM, 107--112.
[11]
Sherif Sakr. 2014. Cloud-hosted databases: technologies, challenges and opportunities. Cluster Computing 17, 2 (2014), 487--502.
[12]
Daniel Seybold. 2021. An automation-based approach for reproducible evaluations of distributed DBMS on elastic infrastructures. Ph.D. Dissertation.
[13]
Daniel Seybold and Jörg Domaschka. 2017. Is Distributed Database Evaluation Cloud-Ready?. In ADBIS - New Trends in Databases and Information Systems. Springer International Publishing, Cham, 100--108.
[14]
Daniel Seybold, Moritz Keppler, Daniel Gründler, and Jörg Domaschka. 2019. Mowgli: Finding Your Way in the DBMS Jungle. In Proceedings of the 2019 ACM/SPEC ICPE. ACM, New York, NY, USA, 321--332.
[15]
Daniel Seybold, Simon Volpert, Stefan Wesner, André Bauer, Nikolas Herbst, and Jörg Domaschka. 2019. Kaa: Evaluating Elasticity of Cloud-Hosted DBMS. In 2019 IEEE CloudCom. 54--61.
[16]
Daniel Seybold, Stefan Wesner, and Jörg Domaschka. 2020. King Louie: Reproducible Availability Benchmarking of Cloud-hosted DBMS. In 35th ACM/SIGAPP SAC. 144--153.
[17]
Michael Stonebraker, Andrew Pavlo, Rebecca Taft, and Michael L Brodie. 2014. Enterprise database applications and the cloud: A difficult road ahead. In 2014 IEEE International Conference on Cloud Engineering. IEEE, 1--6.
Index Terms
- Benchmarking-as-a-service for cloud-hosted DBMS
Recommendations
SPEC Cloud™ IaaS 2016 Benchmark
ICPE '17: Proceedings of the 8th ACM/SPEC on International Conference on Performance EngineeringThe SPEC Cloud (TM) IaaS2016 benchmark is the Standard Performance Evaluation Corporation's (SPEC) first benchmark suite to measure cloud performance. The benchmark suite's use is targeted at cloud providers, cloud consumers, hardware vendors, ...
Cloud Databases for Internet-of-Things Data
ITHINGS '14: Proceedings of the 2014 IEEE International Conference on Internet of Things(iThings), and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom)The Internet of Things (IoT) is posing new challenges and opportunities for data management and analysis techniques. One of the major problems is how to handle an increasing amount of data, with a variety of data types and data sources, in order to meet ...
Data mining algorithms as a service in the cloud exploiting relational database systems
SIGMOD '13: Proceedings of the 2013 ACM SIGMOD International Conference on Management of DataWe present a novel cloud system based on DBMS technology, where data mining algorithms are offered as a service. A local DBMS connects to the cloud and the cloud system returns computed data mining models as small relational tables that are archived and ...
Comments
Information & Contributors
Information
Published In
December 2021
23 pages
ISBN:9781450391542
DOI:10.1145/3491086
- General Chairs:
- Kaiwen Zhang,
- Abdelouahed Gherbi,
- Program Chairs:
- Nalini Venkatasubramanian,
- Luís Veiga
Copyright © 2021 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
In-Cooperation
- USENIX Assoc: USENIX Assoc
- IFIP
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 December 2021
Check for updates
Author Tags
Qualifiers
- Demonstration
Funding Sources
- Federal Ministry for Economic Affairs and Energy of Germany
Conference
Middleware '21
Sponsor:
Acceptance Rates
Overall Acceptance Rate 203 of 948 submissions, 21%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 109Total Downloads
- Downloads (Last 12 months)15
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Feb 2025
Other Metrics
Citations
Cited By
View allView Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in