While standardized and widely used benchmarks address either operational or real-time Business Intelligence (BI) workloads, the lack of a hybrid benchmark led us to the definition of a new, complex, mixed workload benchmark, called mixed workload CH-benCHmark. This benchmark bridges the gap between the established single-workload suites of TPC-C for OLTP and TPC-H for OLAP, and executes a complex mixed workload: a transactional workload based on the order entry processing of TPC-C and a corresponding TPC-H-equivalent OLAP query suite run in parallel on the same tables in a single database system. As it is derived from these two most widely used TPC benchmarks, the CH-benCHmark produces results highly relevant to both hybrid and classic single-workload systems.

References

[1]

A. Bog, K. Sachs, and A. Zeier. Benchmarking Database Design for Mixed OLTP and OLAP Workloads. In ICPE'11, 2011.

Digital Library

Google Scholar

[2]

A. Bog, J. Schaffiner, and J. Krüger. A Composite Benchmark for Online Transaction Processing and Operational Reporting. In IEEE Symposium on Advanced Management of Information for Globalized Enterprises (AMIGE), 2008.

Crossref

Google Scholar

[3]

D. Feinberg and M. A. Beyer. Magic Quadrant for Data Warehouse Database Management Systems. http://www.gartner.com/technology/media-products/reprints/teradata/vol3/article1/article1.html, 2011.

Google Scholar

[4]

F. Funke, A. Kemper, and T. Neumann. Benchmarking Hybrid OLTP&OLAP Database Systems. In Database Systems for Business, Technology and Web (BTW), 2011.

Google Scholar

[5]

M. Poess and C. Floyd. New TPC Benchmarks for Decision Support and Web Commerce. SIGMOD Record, 29(4):64--71, 2000.

Digital Library

Google Scholar

[6]

Transaction Processing Performance Council. TPC Benchmark C (Standard Specification), 2010.

Google Scholar

[7]

Transaction Processing Performance Council. TPC Benchmark H (Standard Specification), 2010.

Google Scholar

[8]

VoltDB. TPC-C-like Benchmark Comparison-Benchmark Description. http://community.voltdb.com/node/134.

Google Scholar

Cited By

View all

Laigner RZhang ZLiu YGomes LZhou Y(2025)Online Marketplace: A Benchmark for Data Management in MicroservicesProceedings of the ACM on Management of Data10.1145/37096533:1(1-26)Online publication date: 11-Feb-2025
https://dl.acm.org/doi/10.1145/3709653
Zhang CLi GLv T(2024)HyBench: A New Benchmark for HTAP DatabasesProceedings of the VLDB Endowment10.14778/3641204.364120617:5(939-951)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.14778/3641204.3641206
Benson LBinnig CBodensohn JLorenzi FLuo JPorobic DRabl TSanghi ASears RTözün PZiegler T(2024)Surprise Benchmarking: The Why, What, and HowProceedings of the Tenth International Workshop on Testing Database Systems10.1145/3662165.3662763(1-8)Online publication date: 9-Jun-2024
https://dl.acm.org/doi/10.1145/3662165.3662763
Show More Cited By

Index Terms

The mixed workload CH-benCHmark

Recommendations

Subsetting the SPEC CPU2006 benchmark suite

On August 24, 2006, the Standard Performance Evaluation Corporation (SPEC) announced CPU2006 -- the next generation of industry-standardized CPU-intensive benchmark suite. The SPEC CPU benchmark suite has become the most frequently used suite for ...
A generic construct based workload model for business intelligence benchmark

Benchmarks are vital tools in the performance measurement and evaluation of computer hardware and software systems. Standard benchmarks such as the TREC, TPC, SPEC, SAP, Oracle, Microsoft, IBM, Wisconsin, AS^3AP, OO1, OO7, XOO7 benchmarks have been used ...
A Benchmark Characterization of the EEMBC Benchmark Suite

Benchmark consumers expect benchmark suites to be complete, accurate, and consistent, and benchmark scores serve as relative measures of performance. However, it is important to understand how benchmarks stress the processors that they aim to test. This ...

Comments

Information & Contributors

Information

Published In

DBTest '11: Proceedings of the Fourth International Workshop on Testing Database Systems

June 2011

51 pages

ISBN:9781450306553

DOI:10.1145/1988842

Conference Chairs:
Goetz Graefe,
Kenneth Salem

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SIGMOD/PODS '11

Sponsor:

SIGMOD

SIGMOD/PODS '11: International Conference on Management of Data

June 13, 2011

Athens, Greece

Acceptance Rates

Overall Acceptance Rate 31 of 56 submissions, 55%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

99
Total Citations
View Citations
1,641
Total Downloads

Downloads (Last 12 months)172
Downloads (Last 6 weeks)11

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Laigner RZhang ZLiu YGomes LZhou Y(2025)Online Marketplace: A Benchmark for Data Management in MicroservicesProceedings of the ACM on Management of Data10.1145/37096533:1(1-26)Online publication date: 11-Feb-2025
https://dl.acm.org/doi/10.1145/3709653
Zhang CLi GLv T(2024)HyBench: A New Benchmark for HTAP DatabasesProceedings of the VLDB Endowment10.14778/3641204.364120617:5(939-951)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.14778/3641204.3641206
Benson LBinnig CBodensohn JLorenzi FLuo JPorobic DRabl TSanghi ASears RTözün PZiegler T(2024)Surprise Benchmarking: The Why, What, and HowProceedings of the Tenth International Workshop on Testing Database Systems10.1145/3662165.3662763(1-8)Online publication date: 9-Jun-2024
https://dl.acm.org/doi/10.1145/3662165.3662763
Tamimi SBernhardt AStock FPetrov IKoch A(2024)DANSEN: Database Acceleration on Native Computational Storage by Exploiting NDPACM Transactions on Reconfigurable Technology and Systems10.1145/365562518:1(1-33)Online publication date: 17-Dec-2024
https://dl.acm.org/doi/10.1145/3655625
Zhang CLi GZhang JZhang XFeng J(2024)HTAP Databases: A SurveyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.338969336:11(6410-6429)Online publication date: Nov-2024
https://doi.org/10.1109/TKDE.2024.3389693
Weng SWang QQu LZhang RCai PQian WZhou A(2024)Lauca: A Workload Duplicator for Benchmarking Transactional Database PerformanceIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.336011636:7(3180-3194)Online publication date: Jul-2024
https://doi.org/10.1109/TKDE.2024.3360116
Kang GGao WWang LLuo CYe HHe QDai SZhan J(2024)Could bibliometrics reveal top science and technology achievements and researchers? The case for evaluatology-based science and technology evaluationBenchCouncil Transactions on Benchmarks, Standards and Evaluations10.1016/j.tbench.2024.1001824:3(100182)Online publication date: Sep-2024
https://doi.org/10.1016/j.tbench.2024.100182
Song HZhou WCui HPeng XLi F(2024)A survey on hybrid transactional and analytical processingThe VLDB Journal10.1007/s00778-024-00858-933:5(1485-1515)Online publication date: 4-Jun-2024
https://doi.org/10.1007/s00778-024-00858-9
Cheng AWaudby JFirth HCrooks NStoica I(2023)Mammoths are Slow: The Overlooked Transactions of Graph DataProceedings of the VLDB Endowment10.14778/3636218.363624117:4(904-911)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.14778/3636218.3636241
Zhou XChen CLi KHe BLu MLiu QHuang WLi GZheng ZChen Y(2023)FEBench: A Benchmark for Real-Time Relational Data Feature ExtractionProceedings of the VLDB Endowment10.14778/3611540.361155016:12(3597-3609)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.14778/3611540.3611550
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Subsetting the SPEC CPU2006 benchmark suite

A generic construct based workload model for business intelligence benchmark

A Benchmark Characterization of the EEMBC Benchmark Suite

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations