research-article

Scaling Multicore Databases via Constrained Parallel Execution

Authors:

Jinyang LiAuthors Info & Claims

SIGMOD '16: Proceedings of the 2016 International Conference on Management of Data

Pages 1643 - 1658

https://doi.org/10.1145/2882903.2882934

Published: 26 June 2016 Publication History

Abstract

Multicore in-memory databases often rely on traditional con- currency control schemes such as two-phase-locking (2PL) or optimistic concurrency control (OCC). Unfortunately, when the workload exhibits a non-trivial amount of contention, both 2PL and OCC sacrifice much parallel execution op- portunity. In this paper, we describe a new concurrency control scheme, interleaving constrained concurrency con- trol (IC3), which provides serializability while allowing for parallel execution of certain conflicting transactions. IC3 combines the static analysis of the transaction workload with runtime techniques that track and enforce dependencies among concurrent transactions. The use of static analysis simplifies IC3's runtime design, allowing it to scale to many cores. Evaluations on a 64-core machine using the TPC- C benchmark show that IC3 outperforms traditional con- currency control schemes under contention. It achieves the throughput of 434K transactions/sec on the TPC-C bench- mark configured with only one warehouse. It also scales better than several recent concurrent control schemes that also target contended workloads.

References

[1]

D. Agrawal, A. J. Bernstein, P. Gupta, and S. Sengupta. Distributed optimistic concurrency control with reduced rollback. Distributed Computing, 2(1):45--59, 1987.

Digital Library

[2]

D. Agrawal, A. El Abbadi, R. Jeffers, and L. Lin. Ordered shared locks for real-time databases. The VLDB Journal, 4(1):87--126, 1995.

Digital Library

[3]

P. Bailis, A. Fekete, M. J. Franklin, A. Ghodsi, J. M. Hellerstein, and I. Stoica. Coordination avoidance in database systems. Proceedings of the VLDB Endowment, 8(3), 2014.

Digital Library

[4]

A. J. Bernstein, D. S. Gerstl, and P. M. Lewis. Concurrency control for step-decomposed transactions. Information Systems, 24(8):673--698, 1999.

Digital Library

[5]

P. A. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency control and recovery in database systems. Addison-Wesley New York, 1987.

Digital Library

[6]

P. A. Bernstein and D. W. Shipman. The correctness of concurrency control mechanisms in a system for distributed databases (sdd-1). ACM Transactions on Database Systems (TODS), 5(1):52--68, 1980.

Digital Library

[7]

P. A. Bernstein, D. W. Shipman, and J. B. Rothnie Jr. Concurrency control in a system for distributed databases (SDD-1). ACM Transactions on Database Systems (TODS), 5(1):18--51, 1980.

Digital Library

[8]

H. Boral, W. Alexander, L. Clay, G. Copeland, S. Danforth, M. Franklin, B. Hart, M. Smith, and P. Valduriez. Prototyping bubba, a highly parallel database system. Knowledge and Data Engineering, IEEE Transactions on, 2(1):4--24, 1990.

Digital Library

[9]

S. Boyd-Wickizer, M. F. Kaashoek, R. Morris, and N. Zeldovich. Non-scalable locks are dangerous. In Linux Symposium, 2012.

[10]

M. J. Carey. Modeling and evaluation of database concurrency control algorithms. 1983.

Digital Library

[11]

A. Cheung, S. Madden, A. Solar-Lezama, O. Arden, and A. C. Myers. Using program analysis to improve database applications. IEEE Data Eng. Bull., 37(1):48--59, 2014.

[12]

J. Cowling and B. Liskov. Granola: low-overhead distributed transaction coordination. In Proceedings of the 2012 USENIX conference on Annual Technical Conference, pages 21--21. USENIX Association, 2012.

Digital Library

[13]

C. T. Davies. Data processing spheres of control. IBM Systems Journal, 17(2):179--198, 1978.

Digital Library

[14]

D. J. DeWitt, S. Ghandeharizadeh, D. A. Schneider, A. Bricker, H.-I. Hsiao, and R. Rasmussen. The gamma database machine project. Knowledge and Data Engineering, IEEE Transactions on, 2(1):44--62, 1990.

Digital Library

[15]

C. Diaconu, C. Freedman, E. Ismert, P.-A. Larson, P. Mittal, R. Stonecipher, N. Verma, and M. Zwilling. Hekaton: SQL Server's memory-optimized OLTP engine. In Proc. SIGMOD, 2013.

Digital Library

[16]

J. M. Faleiro, A. Thomson, and D. J. Abadi. Lazy evaluation of transactions in database systems. In Proceedings of the 2014 ACM SIGMOD international conference on Management of data, pages 15--26. ACM, 2014.

Digital Library

[17]

H. Garcia-Molina. Using semantic knowledge for transaction processing in a distributed database. ACM Transactions on Database Systems (TODS), 8(2):186--213, 1983.

Digital Library

[18]

H. Garcia-Molina and K. Salem. Sagas, volume 16. ACM, 1987.

[19]

J. Gray and A. Reuter. Transaction processing: concepts and techniques, 1993.

Digital Library

[20]

S. Harizopoulos, D. J. Abadi, S. Madden, and M. Stonebraker. OLTP through the looking glass, and what we found there. In Proc. SIGMOD, pages 981--992. ACM, 2008.

Digital Library

[21]

T. Horikawa. Latch-free data structures for DBMS: design, implementation, and evaluation. In Proc. SIGMOD, pages 409--420. ACM, 2013.

Digital Library

[22]

R. Johnson, I. Pandis, N. Hardavellas, A. Ailamaki, and B. Falsafi. Shore-MT: a scalable storage manager for the multicore era. In Proc. EDBT, pages 24--35. ACM, 2009.

Digital Library

[23]

H. Jung, H. Han, A. D. Fekete, G. Heiser, and H. Y. Yeom. A scalable lock manager for multicores. In Proc. SIGMOD, pages 73--84. ACM, 2013.

Digital Library

[24]

R. Kallman, H. Kimura, J. Natkins, A. Pavlo, A. Rasin, S. Zdonik, E. P. Jones, S. Madden, M. Stonebraker, Y. Zhang, et al. H-store: a high-performance, distributed main memory transaction processing system. VLDB Endowment, 1(2):1496--1499, 2008.

Digital Library

[25]

A. Kemper and T. Neumann. Hyper: A hybrid oltp&olap main memory database system based on virtual memory snapshots. In Data Engineering (ICDE), 2011 IEEE 27th International Conference on, pages 195--206. IEEE, 2011.

Digital Library

[26]

H.-T. Kung and J. T. Robinson. On optimistic methods for concurrency control. ACM Transactions on Database Systems (TODS), 6(2):213--226, 1981.

Digital Library

[27]

P.-Å. Larson, S. Blanas, C. Diaconu, C. Freedman, J. M. Patel, and M. Zwilling. High-performance concurrency control mechanisms for main-memory databases. In Proc. VLDB, 2011.

Digital Library

[28]

J. Lee, M. Muehle, N. May, F. Faerber, V. Sikka, H. Plattner, J. Krueger, and M. Grund. High-performance transaction processing in sap hana. IEEE Data Eng. Bull., 36(2):28--33, 2013.

[29]

V. Leis, A. Kemper, and T. Neumann. Exploiting Hardware Transactional Memory in Main-Memory Databases. In Proc. ICDE, 2014.

[30]

J. Lindström, V. Raatikka, J. Ruuth, P. Soini, and K. Vakkila. Ibm soliddb: In-memory database optimized for extreme speed and availability. IEEE Data Eng. Bull., 36(2):14--20, 2013.

[31]

Y. Mao, E. Kohler, and R. T. Morris. Cache craftiness for fast multicore key-value storage. In Proc. EuroSys, pages 183--196, 2012.

Digital Library

[32]

J. M. Mellor-Crummey and M. L. Scott. Algorithms for scalable synchronization on shared-memory multiprocessors. ACM Transactions on Computer Systems (TOCS), 9(1):21--65, 1991.

Digital Library

[33]

S. Mu, Y. Cui, Y. Zhang, W. Lloyd, and J. Li. Extracting more concurrency from distribted transactions. In Proceedings of the 11th USENIX conference on Operating Systems Design and Implementation, pages 479--494. USENIX Association, 2014.

Digital Library

[34]

N. Narula, C. Cutler, E. Kohler, and R. Morris. Phase reconciliation for contended in-memory transactions. In Proceedings of the 11th USENIX conference on Operating Systems Design and Implementation, pages 511--524. USENIX Association, 2014.

Digital Library

[35]

I. Pandis, R. Johnson, N. Hardavellas, and A. Ailamaki. Data-oriented transaction execution. VLDB Endowment, 3(1--2):928--939, 2010.

Digital Library

[36]

H. E. Ramadan, I. Roy, M. Herlihy, and E. Witchel. Committing conflicting transactions in an stm. In ACM Sigplan Notices, volume 44, pages 163--172. ACM, 2009.

Digital Library

[37]

Y. Raz. The principle of commitment ordering, or guaranteeing serializability in a heterogeneous environment of multiple autonomous resource managers using atomic commitment. In VLDB, volume 92, pages 292--312, 1992.

Digital Library

[38]

Y. Raz. Serializability by commitment ordering. Information processing letters, 51(5):257--264, 1994.

Digital Library

[39]

K. Ren, A. Thomson, and D. J. Abadi. Lightweight locking for main memory database systems. Proceedings of the VLDB Endowment, 6(2):145--156, 2012.

Digital Library

[40]

K. Ren, A. Thomson, and D. J. Abadi. An evaluation of the advantages and disadvantages of deterministic database systems. Proceedings of the VLDB Endowment, 7(10):821--832, 2014.

Digital Library

[41]

D. Shasha, F. Llirbat, E. Simon, and P. Valduriez. Transaction chopping: Algorithms and performance studies. ACM Transactions on Database Systems (TODS), 20(3):325--363, 1995.

Digital Library

[42]

M. Stonebraker, S. Madden, D. J. Abadi, S. Harizopoulos, N. Hachem, and P. Helland. The end of an architectural era:(it's time for a complete rewrite). In Proc. VLDB, pages 1150--1160, 2007.

Digital Library

[43]

T. Subasu and J. Alonso. Database engines on multicores, why parallelize when you can distribute. In Proc. Eurosys, 2011.

Digital Library

[44]

The Transaction Processing Council. TPC-C Benchmark (Revision 5.9.0). http://www.tpc.org/tpcc/, 2007.

[45]

A. Thomson and D. J. Abadi. The case for determinism in database systems. Proceedings of the VLDB Endowment, 3(1--2):70--80, 2010.

Digital Library

[46]

A. Thomson and D. J. Abadi. Modularity and Scalability in Calvin. IEEE Data Engineering Bulletin, page 48, 2013.

[47]

A. Thomson, T. Diamond, S.-C. Weng, K. Ren, P. Shao, and D. J. Abadi. Calvin: fast distributed transactions for partitioned database systems. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pages 1--12. ACM, 2012.

Digital Library

[48]

S. Tu, W. Zheng, E. Kohler, B. Liskov, and S. Madden. Speedy Transactions in Multicore In-Memory Databases. In Proc. SOSP, 2013.

Digital Library

[49]

L. VoltDB. Voltdb technical overview, 2010.

[50]

Z. Wang, S. Mu, Y. Cui, H. Yi, H. Chen, and J. Li. Scaling multicore databases via constrained parallel execution. Technical Report TR2016--981, New York University, 2016. http://ic3.news.cs.nyu.edu/techreport16.pdf.

[51]

Z. Wang, H. Qian, J. Li, and H. Chen. Using restricted transactional memory to build a scalable in-memory database. In Proc. EuroSys, 2014.

Digital Library

[52]

C. Xie, C. Su, C. Littley, L. Alvisi, M. Kapritsos, and Y. Wang. High-performance acid via modular concurrency control. In Proceedings of the 25th Symposium on Operating Systems Principles, pages 279--294. ACM, 2015.

Digital Library

[53]

X. Yu, G. Bezerra, A. Pavlo, S. Devadas, and M. Stonebraker. Staring into the abyss: An evaluation of concurrency control with one thousand cores. Proceedings of the VLDB Endowment, 8(3):209--220, 2014.

Digital Library

[54]

Y. Zhang, R. Power, S. Zhou, Y. Sovran, M. K. Aguilera, and J. Li. Transaction chains: achieving serializability with low latency in geo-distributed storage systems. In Proc. SOSP, pages 276--291. ACM, 2013.

Digital Library

[55]

W. Zheng, S. Tu, E. Kohler, and B. Liskov. Fast databases with fast durability and recovery through multicore parallelism. In Proceedings of the 11th USENIX conference on Operating Systems Design and Implementation, pages 465--477. USENIX Association, 2014.

Digital Library

Cited By

Cheng AKabcenell AChan JShi XBailis PCrooks NStoica I(2024)Towards Optimal Transaction SchedulingProceedings of the VLDB Endowment10.14778/3681954.368195617:11(2694-2707)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.14778/3681954.3681956
Wei JGu YLi TQi JLi CZhang YJensen CYu G(2024)LTPG: Large-Batch Transaction Processing on GPUs with Deterministic Concurrency Control2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00296(3865-3877)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00296
Dong ZWang ZZhang XXu XZhao CChen HPanda ALi J(2023)Fine-Grained Re-Execution for Efficient Batched Commit of Distributed TransactionsProceedings of the VLDB Endowment10.14778/3594512.359452316:8(1930-1943)Online publication date: 22-Jun-2023
https://dl.acm.org/doi/10.14778/3594512.3594523
Show More Cited By

Index Terms

Scaling Multicore Databases via Constrained Parallel Execution
1. Information systems
  1. Data management systems
    1. Database management system engines
      1. Database transaction processing

Recommendations

Alternative Correctness Criteria for Concurrent Execution of Transactions in Multilevel Secure Databases

This paper investigates issues related to transaction concurrency control in multilevel secure databases. It demonstrates how the conflicts between the correctness requirements and the secrecy requirements can be reconciled by proposing two different ...
A Non-Two-Phase Locking Protocol for Global Concurrency Control in Distributed Heterogeneous Database Systems

A concurrency control method is proposed for global transactions in a distributed heterogeneous database system. This method is applicable when the database sites are interconnected in a rooted tree fashion. It guarantees deadlock freedom in addition to ...
Efficient Execution of Read-Only Transactions in Replicated Multiversion Databases

Multiple versions of data are used in database systems to increase concurrency. The higher concurrency results since read-only transactions can be executed without any concurrency control overhead and, therefore, read-only transactions do not interfere ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '16: Proceedings of the 2016 International Conference on Management of Data

June 2016

2300 pages

ISBN:9781450335317

DOI:10.1145/2882903

General Chairs:
Fatma Özcan
IBM Research, USA
,
Georgia Koutrika
HP Labs, USA
,
Program Chair:
Sam Madden
Massachusetts Institute of Technology, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

SIGMOD/PODS'16

Sponsor:

SIGMOD

SIGMOD/PODS'16: International Conference on Management of Data

June 26 - July 1, 2016

California, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

50
Total Citations
View Citations
825
Total Downloads

Downloads (Last 12 months)80
Downloads (Last 6 weeks)7

Reflects downloads up to 14 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cheng AKabcenell AChan JShi XBailis PCrooks NStoica I(2024)Towards Optimal Transaction SchedulingProceedings of the VLDB Endowment10.14778/3681954.368195617:11(2694-2707)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.14778/3681954.3681956
Wei JGu YLi TQi JLi CZhang YJensen CYu G(2024)LTPG: Large-Batch Transaction Processing on GPUs with Deterministic Concurrency Control2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00296(3865-3877)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00296
Dong ZWang ZZhang XXu XZhao CChen HPanda ALi J(2023)Fine-Grained Re-Execution for Efficient Batched Commit of Distributed TransactionsProceedings of the VLDB Endowment10.14778/3594512.359452316:8(1930-1943)Online publication date: 22-Jun-2023
https://dl.acm.org/doi/10.14778/3594512.3594523
Lai ZLiu CLo E(2023)When Private Blockchain Meets Deterministic DatabaseProceedings of the ACM on Management of Data10.1145/35889521:1(1-28)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3588952
Zhu BHua YLong ZLiu X(2023)CATS: A Computation-Aware Transaction Processing System with Proactive Unlocking2023 IEEE/ACM 31st International Symposium on Quality of Service (IWQoS)10.1109/IWQoS57198.2023.10188780(1-10)Online publication date: 19-Jun-2023
https://doi.org/10.1109/IWQoS57198.2023.10188780
Yu QZhou J(2023)Associativity-Aware Transaction Processing Optimization for Web Applications2022 IEEE 28th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS56603.2022.00072(507-513)Online publication date: Jan-2023
https://doi.org/10.1109/ICPADS56603.2022.00072
Dong ZWang ZYi CXu XZhang JLi JChen H(2023)Database Deadlock Diagnosis for Large-Scale ORM-Based Web Applications2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00219(2864-2877)Online publication date: Apr-2023
https://doi.org/10.1109/ICDE55515.2023.00219
Masumura KHoshino TKawashima H(2022)Fast Concurrency Control with Thread Activity Management beyond BackoffJournal of Information Processing10.2197/ipsjjip.30.55230(552-561)Online publication date: 2022
https://doi.org/10.2197/ipsjjip.30.552
Liu YSu LShah VZhou YVaz Salles MIves ZBonifati AEl Abbadi A(2022)Hybrid Deterministic and Nondeterministic Execution of Transactions in Actor SystemsProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3526172(65-78)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3526172
Chen YYu XKoutris PArpaci-Dusseau AArpaci-Dusseau RShu JIves ZBonifati AEl Abbadi A(2022)Plor: General Transactions with Predictable, Low Tail LatencyProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3517879(19-33)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3517879
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents