research-article

Public Access

SolarDB: Toward a Shared-Everything Database on Distributed Log-Structured Storage

Authors:

Huiqi HuAuthors Info & Claims

ACM Transactions on Storage (TOS), Volume 15, Issue 2

Article No.: 11, Pages 1 - 26

https://doi.org/10.1145/3318158

Published: 25 June 2019 Publication History

All formats PDF

Abstract

Efficient transaction processing over large databases is a key requirement for many mission-critical applications. Although modern databases have achieved good performance through horizontal partitioning, their performance deteriorates when cross-partition distributed transactions have to be executed. This article presents SolarDB, a distributed relational database system that has been successfully tested at a large commercial bank. The key features of SolarDB include (1) a shared-everything architecture based on a two-layer log-structured merge-tree; (2) a new concurrency control algorithm that works with the log-structured storage, which ensures efficient and non-blocking transaction processing even when the storage layer is compacting data among nodes in the background; and (3) find-grained data access to effectively minimize and balance network communication within the cluster. According to our empirical evaluations on TPC-C, Smallbank, and a real-world workload, SolarDB outperforms the existing shared-nothing systems by up to 50x when there are close to or more than 5% distributed transactions.

References

[1]

Alibaba Oceanbase. 2015. Oceanbase. Retrieved April 4, 2019 from https://github.com/alibaba/oceanbase.

[2]

Hal Berenson, Phil Bernstein, Jim Gray, Jim Melton, Elizabeth O’Neil, and Patrick O’Neil. 1995. A critique of ANSI SQL isolation levels. In Proceedings of SIGMOD, Vol. 24. ACM, New York, NY, 1--10.

Digital Library

[3]

Philip A. Bernstein, Sudipto Das, Bailu Ding, and Markus Pilman. 2015. Optimizing optimistic concurrency control for tree-structured, log-structured databases. In Proceedings of SIGMOD. 1295--1309.

Digital Library

[4]

Sashikanth Chandrasekaran and Roger Bamford. 2003. Shared cache-the future of parallel databases. In Proceedings of ICDE. IEEE, Los Alamitos, CA, 840--850.

[5]

Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, et al. 2008. Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems 26, 2 (2008), 4.

Digital Library

[6]

Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, et al. 2007. Dynamo: Amazon’s highly available key-value store. In Proceedings of SOSP, Vol. 41. ACM, New York, NY, 205--220.

Digital Library

[7]

Cristian Diaconu, Craig Freedman, Erik Ismert, Per-Ake Larson, Pravin Mittal, Ryan Stonecipher, Nitin Verma, et al. 2013. Hekaton: SQL server’s memory-optimized OLTP engine. In Proceedings of SIGMOD. ACM, New York, NY, 1243--1254.

Digital Library

[8]

Aleksandar Dragojevic, Dushyanth Narayanan, Miguel Castro, and Orion Hodson. 2014. FaRM: Fast remote memory. In Proceedings of NSDI. 401--414.

[9]

Aleksandar Dragojevic, Dushyanth Narayanan, Edmund B. Nightingale, Matthew Renzelmann, Alex Shamis, Anirudh Badam, and Miguel Castro. 2015. No compromises: Distributed transactions with consistency, availability, and performance. In Proceedings of SOSP. 54--70.

Digital Library

[10]

Anil K. Goel, Jeffrey Pound, Nathan Auch, Peter Bumbulis, Scott MacLean, Franz Färber, Francis Gropengiesser, et al. 2015. Towards scalable real-time analytics: An architecture for scale-out of OLxP workloads. Proceedings of the VLDB Endowment 8, 12 (2015), 1716--1727.

Digital Library

[11]

J. W. Josten, C. Mohan, I. Narang, and J. Z. Teng. 1997. DB2’s use of the coupling facility for data sharing. IBM Systems Journal 36, 2 (1997), 327--351.

Digital Library

[12]

Robert Kallman, Hideaki Kimura, Jonathan Natkins, Andrew Pavlo, Alexander Rasin, Stanley Zdonik, Evan P. C. Jones, et al. 2008. H-store: A high-performance, distributed main memory transaction processing system. Proceedings of the VLDB Endowment 1, 2 (2008), 1496--1499.

Digital Library

[13]

Alfons Kemper and Thomas Neumann. 2011. HyPer: A hybrid OLTP8OLAP main memory database system based on virtual memory snapshots. In Proceedings of ICDE. IEEE, Los Alamitos, CA, 195--206.

Digital Library

[14]

Ken Kennedy and Kathryn S. McKinley. 1993. Maximizing Loop Parallelism and Improving Data Locality via Loop Fusion and Distribution. Springer.

Digital Library

[15]

Hsiang-Tsung Kung and John T. Robinson. 1981. On optimistic methods for concurrency control. ACM Transactions on Database Systems 2 (1981), 213--226.

Digital Library

[16]

Avinash Lakshman and Prashant Malik. 2010. Cassandra: A decentralized structured storage system. ACM SIGOPS Operating Systems Review 44, 2 (2010), 35--40.

Digital Library

[17]

Justin Levandoski, David Lomet, Sudipta Sengupta, Ryan Stutsman, and Rui Wang. 2015. High performance transactions in Deuteronomy. In Proceedings of CIDR. https://www.microsoft.com/en-us/research/publication/high-performance-transactions-in-deuteronomy/.

[18]

LevelDB. 2017. Home Page. Retrieved April 4, 2019 from http://leveldb.org/.

[19]

Simon Loesing, Markus Pilman, Thomas Etter, and Donald Kossmann. 2015. On the design and scalability of distributed shared-data databases. In Proceedings of SIGMOD. ACM, New York, NY, 663--676.

Digital Library

[20]

Shuai Mu, Yang Cui, Yang Zhang, Wyatt Lloyd, and Jinyang Li. 2014. Extracting more concurrency from distributed transactions. In Proceedings of OSDI. 479--494.

Digital Library

[21]

Steven S. Muchnick. 1997. Advanced Compiler Design Implementation. Morgan Kaufmann.

Digital Library

[22]

Patrick O’Neil, Edward Cheng, Dieter Gawlick, and Elizabeth O’Neil. 1996. The log-structured merge-tree (LSM-tree). Acta Informatica 33, 4 (1996), 351--385.

Digital Library

[23]

John Ousterhout, Parag Agrawal, David Erickson, Christos Kozyrakis, Jacob Leverich, David Mazières, Subhasish Mitra, et al. 2010. The case for RAMClouds: Scalable high-performance storage entirely in DRAM. ACM SIGOPS Operating Systems Review 43, 4 (2010), 92--105.

Digital Library

[24]

Wolf Rödiger, Tobias Mühlbauer, Alfons Kemper, and Thomas Neumann. 2015. High-speed query processing over high-speed networks. Proceedings of the VLDB Endowment 9, 4 (2015), 228--239.

Digital Library

[25]

Marco Serafini, Essam Mansour, Ashraf Aboulnaga, Kenneth Salem, Taha Rafiq, and Umar Farooq Minhas. 2014. Accordion: Elastic scalability for database systems supporting distributed transactions. Proceedings of the VLDB Endowment 7, 12 (2014), 1035--1046.

Digital Library

[26]

Michael Stonebraker, Samuel Madden, Daniel J. Abadi, Stavros Harizopoulos, Nabil Hachem, and Pat Helland. 2007. The end of an architectural era: (It’s time for a complete rewrite). In Proceedings of VLDB. 1150--1160.

[27]

Michael Stonebraker and Ariel Weisberg. 2013. The VoltDB main memory DBMS. IEEE Data Engineering Bulletin 36, 2 (2013), 21--27.

[28]

Rebecca Taft, Essam Mansour, Marco Serafini, Jennie Duggan, Aaron J. Elmore, Ashraf Aboulnaga, Andrew Pavlo, et al. 2014. E-store: Fine-grained elastic partitioning for distributed transaction processing systems. In Proceedings of VLDB. 245--256.

Digital Library

[29]

Alexander Thomson, Thaddeus Diamond, Shu-Chun Weng, Kun Ren, Philip Shao, and Daniel J. Abadi. 2012. Calvin: Fast distributed transactions for partitioned database systems. In Proceedings of SIGMOD. 1--12.

Digital Library

[30]

Stephen Tu, Wenting Zheng, Eddie Kohler, Barbara Liskov, and Samuel Madden. 2013. Speedy transactions in multicore in-memory databases. In Proceedings of SOSP. 18--32.

Digital Library

[31]

Hoang Tam Vo, Sheng Wang, Divyakant Agrawal, Gang Chen, and Beng Chin Ooi. 2012. LogBase: A scalable log-structured database system in the cloud. Proceedings of the VLDB Endowment 5, 10 (2012), 1004--1015.

Digital Library

[32]

VoltDB Inc. 2017. VoltDB. Retrieved April 4, 2019 from https://www.voltdb.com/.

[33]

Gottfried Vossen. 1995. Database transaction models. In Computer Science Today. Springer, 560--574.

[34]

Zhaoguo Wang, Shuai Mu, Yang Cui, Han Yi, Haibo Chen, and Jinyang Li. 2016. Scaling multicore databases via constrained parallel execution. In Proceedings of SIGMOD. ACM, New York, NY, 1643--1658.

Digital Library

[35]

Zhaoguo Wang, Hao Qian, Jinyang Li, and Haibo Chen. 2014. Using restricted transactional memory to build a scalable in-memory database. In Proceedings of EuroSys. 26:1--26:15.

Digital Library

[36]

Michael Wei, Amy Tai, Christopher J. Rossbach, Ittai Abraham, Maithem Munshed, Medhavi Dhawan, Jim Stabile, et al. 2017. vCorfu: A cloud-scale object store on a shared log. In Proceedings of USENIX NSDI. 35--49.

Digital Library

[37]

Xingda Wei, Sijie Shen, Rong Chen, and Haibo Chen. 2017. Replication-driven live reconfiguration for fast distributed transaction processing. In Proceedings of USENIX ATC. 335--347.

Digital Library

[38]

Xingda Wei, Jiaxin Shi, Yanzhe Chen, Rong Chen, and Haibo Chen. 2015. Fast in-memory transaction processing using RDMA and HTM. In Proceedings of SOSP. ACM, New York, NY, 87--104.

Digital Library

[39]

Brian White, Jay Lepreau, Leigh Stoller, Robert Ricci, Shashi Guruprasad, Mac Newbold, Mike Hibler, et al. 2002. An integrated experimental environment for distributed systems and networks. In Proceedings of OSDI. 255--270.

Digital Library

[40]

Yingjun Wu, Chee-Yong Chan, and Kian-Lee Tan. 2016. Transaction healing: Scaling optimistic concurrency control on multicores. In Proceedings of SIGMOD. ACM, New York, NY, 1689--1704.

Digital Library

[41]

Cong Yan and Alvin Cheung. 2016. Leveraging lock contention to improve OLTP application performance. In Proceedings of VLDB. 444--455.

Digital Library

[42]

Matei Zaharia, Mosharaf Chowdhury, Michael J. Franklin, Scott Shenker, and Ion Stoica. 2010. Spark: Cluster computing with working sets. In Proceedings of USENIX HotCloud.

Digital Library

Cited By

Margara ACugola GFelicioni NCilloni S(2023)A Model and Survey of Distributed Data-Intensive SystemsACM Computing Surveys10.1145/360480156:1(1-69)Online publication date: 26-Aug-2023
https://dl.acm.org/doi/10.1145/3604801
Hu DChen ZChe WSun JChen HIves ZBonifati AEl Abbadi A(2022)Halo: A Hybrid PMem-DRAM Persistent Hash Index with Fast RecoveryProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3517884(1049-1063)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3517884
Lu ZCao QJiang HWang SDong YBromberg YKermarrec AKozyrakis C(2022)p2KVSProceedings of the Seventeenth European Conference on Computer Systems10.1145/3492321.3519567(575-591)Online publication date: 28-Mar-2022
https://dl.acm.org/doi/10.1145/3492321.3519567
Show More Cited By

Index Terms

SolarDB: Toward a Shared-Everything Database on Distributed Log-Structured Storage
1. Information systems
  1. Data management systems
    1. Database management system engines
      1. Database transaction processing
      2. Parallel and distributed DBMSs
        Relational parallel and distributed DBMSs

Recommendations

Implementing Distributed Read-Only Transactions

This paper presents an efficient scheme for eliminating conflicts between distributed read-only transactions and distributed update transactions, thereby reducing synchronization delays. The scheme makes use of a multiversion mechanism in order to ...
The Exclusive-Writer Approach to Updating Replicated Files in Distributed Processing Systems

Consistency control protocols can be classified as either pessimistic or optimistic. Pessimistic protocols check for conflicting file accesses before a transaction references shared files; this prevents transaction restarts but adds intercomputer ...
An Optimistic Locking Technique for Concurrency Control in Distributed Databases

A method called optimistic method with dummy locks (ODL) is suggested for concurrency control in distributed databases. It is shown that by using long-term dummy locks, the need for the information about the write sets of validated transactions is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Storage

ACM Transactions on Storage Volume 15, Issue 2

Systor 2018 Special Section on ATC 2018, Special Section on OSDI 2018 and Regular Papers

May 2019

187 pages

ISSN:1553-3077

EISSN:1553-3093

DOI:10.1145/3326597

Editor:
Sam H. Noh
Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2019

Accepted: 01 March 2019

Received: 01 January 2019

Published in TOS Volume 15, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Key R&D Plan Project
863 Program
National Science Foundation
National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
1,136
Total Downloads

Downloads (Last 12 months)243
Downloads (Last 6 weeks)24

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Margara ACugola GFelicioni NCilloni S(2023)A Model and Survey of Distributed Data-Intensive SystemsACM Computing Surveys10.1145/360480156:1(1-69)Online publication date: 26-Aug-2023
https://dl.acm.org/doi/10.1145/3604801
Hu DChen ZChe WSun JChen HIves ZBonifati AEl Abbadi A(2022)Halo: A Hybrid PMem-DRAM Persistent Hash Index with Fast RecoveryProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3517884(1049-1063)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3517884
Lu ZCao QJiang HWang SDong YBromberg YKermarrec AKozyrakis C(2022)p2KVSProceedings of the Seventeenth European Conference on Computer Systems10.1145/3492321.3519567(575-591)Online publication date: 28-Mar-2022
https://dl.acm.org/doi/10.1145/3492321.3519567
Cao WLi FHuang GLou JZhao JHe DSun MZhang YWang SWu XLiao HChen ZFang XChen MLiang CLuo YWang HWang SMa ZYang XPeng XRuan YWang YZhou JWang JHu QKang J(2022)PolarDB-X: An Elastic Distributed Relational Database for Cloud-Native Applications2022 IEEE 38th International Conference on Data Engineering (ICDE)10.1109/ICDE53745.2022.00259(2859-2872)Online publication date: May-2022
https://doi.org/10.1109/ICDE53745.2022.00259
Pei OLi ZDu HLiu WGao J(2021)Dependence-Cognizant Locking Improvement for the Main Memory Database SystemsMathematical Problems in Engineering10.1155/2021/66544612021(1-12)Online publication date: 20-Feb-2021
https://doi.org/10.1155/2021/6654461
Huang HGhandeharizadeh SLi GLi ZIdreos SSrivastava D(2021)Nova-LSM: A Distributed, Component-based LSM-tree Key-value StoreProceedings of the 2021 International Conference on Management of Data10.1145/3448016.3457297(749-763)Online publication date: 9-Jun-2021
https://dl.acm.org/doi/10.1145/3448016.3457297
He XCai PZhou XZhou A(2021)Continuously Bulk Loading over Range Partitioned Tables for Large Scale Historical Data2021 IEEE 37th International Conference on Data Engineering (ICDE)10.1109/ICDE51399.2021.00088(960-971)Online publication date: Apr-2021
https://doi.org/10.1109/ICDE51399.2021.00088

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents