research-article

Taurus Database: How to be Fast, Available, and Frugal in the Cloud

Authors:

Alex Depoutovitch,

Yongjun HeAuthors Info & Claims

SIGMOD '20: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

Pages 1463 - 1478

https://doi.org/10.1145/3318464.3386129

Published: 31 May 2020 Publication History

Abstract

Using cloud Database as a Service (DBaaS) offerings instead of on-premise deployments is increasingly common. Key advantages include improved availability and scalability at a lower cost than on-premise alternatives. In this paper, we describe the design of Taurus, a new multi-tenant cloud database system. Taurus separates the compute and storage layers in a similar manner to Amazon Aurora and Microsoft Socrates and provides similar benefits, such as read replica support, low network utilization, hardware sharing and scalability. However, the Taurus architecture has several unique advantages. Taurus offers novel replication and recovery algorithms providing better availability than existing approaches using the same or fewer replicas. Also, Taurus is highly optimized for performance, using no more than one network hop on critical paths and exclusively using append-only storage, delivering faster writes, reduced device wear, and constant-time snapshots. This paper describes Taurus and provides a detailed description and analysis of the storage node architecture, which has not been previously available from the published literature.

Supplementary Material

MP4 File (3318464.3386129.mp4)

Presentation Video

Download
107.74 MB

References

[1]

Amazon. 2019. MySQL on Amazon RDS. Amazon. https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_MySQL.html Retrieved October 1, 2019 from

[2]

Panagiotis Antonopoulos, Alex Budovski, Cristian Diaconu, Alejandro Hernandez Saenz, Jack Hu, Hanuma Kodavalla, Donald Kossmann, Sandeep Lingam, Umar Farooq Minhas, Naveen Prakash, Vijendra Purohit, Hugh Qu, Chaitanya Sreenivas Ravella, Krystyna Reisteter, Sheetal Shrotri, Dixin Tang, and Vikram Wakade. 2019. Socrates: The New SQL Server in the Cloud. In Proceedings of the 2019 International Conference on Management of Data (SIGMOD '19). ACM, New York, NY, USA, 1743--1756. https://doi.org/10.1145/3299869.3314047

Digital Library

[3]

Ken Birman. 2007. The Promise, and Limitations, of Gossip Protocols. SIGOPS Oper. Syst. Rev., Vol. 41, 5 (Oct. 2007), 8--13.

Digital Library

[4]

Wei Cao, Zhenjun Liu, Peng Wang, Sen Chen, Caifeng Zhu, Song Zheng, Yuhui Wang, and Guoqing Ma. 2018. PolarFS: An Ultra-low Latency and Failure Resilient Distributed File System for Shared Storage Cloud Database. Proc. VLDB Endow., Vol. 11, 12 (Aug. 2018), 1849--1862. https://doi.org/10.14778/3229863.3229872

Digital Library

[5]

Robert J. Chansler (Ed.). 2012. Data Availability and Durability with the Hadoop Distributed File System. Login, Vol. 37, 1 (Feb. 2012).

[6]

James C Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, Jeffrey John Furman, Sanjay Ghemawat, Andrey Gubarev, Christopher Heiser, Peter Hochschild, et al. 2013. Spanner: Google's globally distributed database. ACM Transactions on Computer Systems (TOCS), Vol. 31, 3 (2013), 8.

Digital Library

[7]

Benoit Dageville, Thierry Cruanes, Marcin Zukowski, Vadim Antonov, Artin Avanes, Jon Bock, Jonathan Claybaugh, Daniel Engovatov, Martin Hentschel, Jiansheng Huang, Allison W. Lee, Ashish Motivala, Abdul Q. Munir, Steven Pelley, Peter Povinec, Greg Rahn, Spyridon Triantafyllis, and Philipp Unterbrunner. 2016. The Snowflake Elastic Data Warehouse. In Proceedings of the 2016 International Conference on Management of Data (SIGMOD '16). ACM, New York, NY, USA, 215--226. https://doi.org/10.1145/2882903.2903741

Digital Library

[8]

Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall, and Werner Vogels. 2007. Dynamo: Amazon's Highly Available Key-value Store. In Proceedings of Twenty-first ACM SIGOPS Symposium on Operating Systems Principles (SOSP '07). ACM, New York, NY, USA, 205--220.

Digital Library

[9]

Alan Demers, D Greene, C Hauser, W Irish, J Larson, S Shenker, H Sturgis, D Swinehart, and D Terry. 1987. Epidemic algorithms for replicated database management. In Proceedings of the 6th Annual ACM Symposium on Principles of Distributed Computing (PODC'87). ACM, New York, NY, USA, 1--12.

Digital Library

[10]

Armando Fox, Steven D. Gribble, Yatin Chawathe, Eric A. Brewer, and Paul Gauthier. 1997. Cluster-based Scalable Network Services. In Proceedings of the Sixteenth ACM Symposium on Operating Systems Principles (SOSP '97). ACM, New York, NY, USA, 78--91.

Digital Library

[11]

David K. Gifford. 1979. Weighted Voting for Replicated Data. In Proceedings of the Seventh ACM Symposium on Operating Systems Principles (SOSP '79). ACM, New York, NY, USA, 150--162.

Digital Library

[12]

Donald Kossmann, Tim Kraska, and Simon Loesing. 2010. An Evaluation of Alternative Architectures for Transaction Processing in the Cloud. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data (SIGMOD '10). ACM, New York, NY, USA, 579--590.

Digital Library

[13]

Leslie Lamport. 1978. Time, Clocks, and the Ordering of Events in a Distributed System. Commun. ACM, Vol. 21, 7 (July 1978), 558--565.

Digital Library

[14]

Changman Lee, Dongho Sim, Jooyoung Hwang, and Sangyeun Cho. 2015. F2FS: A New File System for Flash Storage. In 13th USENIX Conference on File and Storage Technologies (FAST 15). USENIX Association, Santa Clara, CA, 273--286. https://www.usenix.org/conference/fast15/technical-sessions/presentation/lee

Digital Library

[15]

J. J. Levandoski, D. B. Lomet, and S. Sengupta. 2013. The Bw-Tree: A B-tree for new hardware platforms. In 2013 IEEE 29th International Conference on Data Engineering (ICDE). IEEE, New York, NY, USA, 302--313.

[16]

Xuhui Li, Ashraf Aboulnaga, Kenneth Salem, Aamer Sachedina, and Shaobo Gao. 2005. Second-tier cache management using write hints. In Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies-Volume 4. USENIX Association, New York, NY, USA.

Digital Library

[17]

Maged M Michael. 2002. High performance dynamic lock-free hash tables and list-based sets. In Proceedings of the fourteenth annual ACM symposium on Parallel algorithms and architectures. ACM, ACM, New York, NY, USA, 73--82.

Digital Library

[18]

Microsoft. 2018. TPCC-Like Workload for Sysbench 1.0. Percona. https://www.percona.com/blog/2018/03/05/tpcc-like-workload-sysbench-1-0/ Retrieved October 1, 2019 from

[19]

Microsoft. 2019. Azure SQL Database. Microsoft. https://azure.microsoft.com/en-ca/services/sql-database/ Retrieved October 1, 2019 from

[20]

John Ousterhout, Parag Agrawal, David Erickson, Christos Kozyrakis, Jacob Leverich, David Mazières, Subhasish Mitra, Aravind Narayanan, Guru Parulkar, Mendel Rosenblum, et al. 2010. The case for RAMClouds: scalable high-performance storage entirely in DRAM. ACM SIGOPS Operating Systems Review, Vol. 43, 4 (2010), 92--105.

Digital Library

[21]

Mendel Rosenblum and John K Ousterhout. 1992. The design and implementation of a log-structured file system. ACM Transactions on Computer Systems (TOCS), Vol. 10, 1 (1992), 26--52.

Digital Library

[22]

Caetano Sauer, Goetz Graefe, and Theo H"arder. 2018. FineLine: Log-structured Transactional Storage and Recovery. Proc. VLDB Endow., Vol. 11, 13 (Sept. 2018), 2249--2262. https://doi.org/10.14778/3275366.3284969

Digital Library

[23]

Ben Vandiver, Shreya Prasad, Pratibha Rana, Eden Zik, Amin Saeidi, Pratyush Parimal, Styliani Pantela, and Jaimin Dave. 2018. Eon Mode: Bringing the Vertica Columnar Database to the Cloud. In Proceedings of the 2018 International Conference on Management of Data (SIGMOD '18). ACM, New York, NY, USA, 797--809.

Digital Library

[24]

Alexandre Verbitski, Anurag Gupta, Debanjan Saha, Murali Brahmadesam, Kamal Gupta, Raman Mittal, Sailesh Krishnamurthy, Sandor Maurice, Tengiz Kharatishvili, and Xiaofeng Bao. 2017. Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases. In Proceedings of the 2017 ACM International Conference on Management of Data (SIGMOD '17). ACM, New York, NY, USA, 1041--1052. https://doi.org/10.1145/3035918.3056101

Digital Library

[25]

Hoang Tam Vo, Sheng Wang, Divyakant Agrawal, Gang Chen, and Beng Chin Ooi. 2012. LogBase: A Scalable Log-structured Database System in the Cloud. Proc. VLDB Endow., Vol. 5, 10 (June 2012), 1004--1015. https://doi.org/10.14778/2336664.2336673

Digital Library

Cited By

Copik MCalotoiu AZhou PTaranov KHoefler TMencagli GDazzi PLowenthal DBadia R(2024)FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an ExampleProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658661(94-108)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658661
Wang RGao CWang JKadam PTamerÖzsu MAref W(2024)Optimizing LSM-based indexes for disaggregated memoryThe VLDB Journal10.1007/s00778-024-00863-yOnline publication date: 19-Jun-2024
https://doi.org/10.1007/s00778-024-00863-y
Depoutovitch AChen CLarson PNg JLin SXiong GLee PBoctor ERen SWu LZhang YSun C(2023)Taurus MM: Bringing Multi-Master to the CloudProceedings of the VLDB Endowment10.14778/3611540.361154216:12(3488-3500)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.14778/3611540.3611542
Show More Cited By

Index Terms

Taurus Database: How to be Fast, Available, and Frugal in the Cloud
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
    1. Availability
    2. Reliability
2. Information systems
  1. Data management systems
    1. Database management system engines
      1. DBMS engine architectures
      2. Parallel and distributed DBMSs
        Relational parallel and distributed DBMSs

Recommendations

Migrating a (large) science database to the cloud
HPDC '10: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing

We report on attempts to put an existing scientific (astronomical) database -- the Sloan Digital Sky Survey (SDSS) science archive [1] - in the cloud. Based on our experience, it is either very frustrating or impossible at this time to migrate an ...
Adaptive checkpointing with reliable storage in cloud environment

Cloud computing has recently gained popularity as a resource platform for on-demand, high availability and high scalability access to resources, while offering dynamic flexible infrastructures and QoS (Quality of Services) guaranteed services. In ...
Differentiated Availability in Cloud Computing SLAs
GRID '11: Proceedings of the 2011 IEEE/ACM 12th International Conference on Grid Computing

Cloud computing is the new trend in service delivery, and promises large cost savings and agility for the customers. However, some challenges still remain to be solved before widespread use can be seen. This is especially relevant for enterprises, which ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '20: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

June 2020

2925 pages

ISBN:9781450367356

DOI:10.1145/3318464

General Chairs:
David Maier
Portland State University, USA
,
Rachel Pottinger
University of British Columbia, Canada
,
Program Chairs:
AnHai Doan
University of Wisconsin, USA
,
Wang-Chiew Tan
Megagon Labs, USA
,
Publications Chairs:
Abdussalam Alawini
University of Illinois at Urbana-Champaign, USA
,
Hung Q. Ngo
RelationalAI, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGMOD/PODS '20

Sponsor:

SIGMOD

SIGMOD/PODS '20: International Conference on Management of Data

June 14 - 19, 2020

OR, Portland, USA

Acceptance Rates

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
2,773
Total Downloads

Downloads (Last 12 months)310
Downloads (Last 6 weeks)33

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Copik MCalotoiu AZhou PTaranov KHoefler TMencagli GDazzi PLowenthal DBadia R(2024)FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an ExampleProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658661(94-108)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658661
Wang RGao CWang JKadam PTamerÖzsu MAref W(2024)Optimizing LSM-based indexes for disaggregated memoryThe VLDB Journal10.1007/s00778-024-00863-yOnline publication date: 19-Jun-2024
https://doi.org/10.1007/s00778-024-00863-y
Depoutovitch AChen CLarson PNg JLin SXiong GLee PBoctor ERen SWu LZhang YSun C(2023)Taurus MM: Bringing Multi-Master to the CloudProceedings of the VLDB Endowment10.14778/3611540.361154216:12(3488-3500)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.14778/3611540.3611542
Wang JLi TSong HYang XZhou WLi FYan BWu QLiang YYing CWang YChen BCai CRuan YWeng XChen SYin LYang CCai XXing HYu NChen XHuang DSun J(2023)PolarDB-IMCI: A Cloud-Native HTAP Database System at AlibabaProceedings of the ACM on Management of Data10.1145/35897851:2(1-25)Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1145/3589785
Ruan CZhang YBi CMa XChen HLi FYang XLi CAboulnaga AXu YAamodt TJerger NSwift M(2023)Persistent Memory Disaggregation for Cloud-Native Relational DatabasesProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3582016.3582055(498-512)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3582016.3582055
Wang JZhang QDas SPandis ISelçuk Candan KAmer-Yahia S(2023)Disaggregated Database SystemsCompanion of the 2023 International Conference on Management of Data10.1145/3555041.3589403(37-44)Online publication date: 4-Jun-2023
https://dl.acm.org/doi/10.1145/3555041.3589403
Sun JMa HZhang LLiu HShi HLuo SWu KBruhwiler KZhu CNie YChen JZhang LLiang Y(2023)Accelerating Cloud-Native Databases with Distributed PMem Stores2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00233(3043-3057)Online publication date: Apr-2023
https://doi.org/10.1109/ICDE55515.2023.00233
Wang RWang JKadam PTamer Özsu MAref W(2023)dLSM: An LSM-Based Index for Memory Disaggregation2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00217(2835-2849)Online publication date: Apr-2023
https://doi.org/10.1109/ICDE55515.2023.00217
Pakrijauskas KMažeika D(2022)A Method of Transparent Graceful Failover in Low Latency Stateful MicroservicesElectronics10.3390/electronics1123393611:23(3936)Online publication date: 28-Nov-2022
https://doi.org/10.3390/electronics11233936
Wang RWang JIdreos SÖzsu MAref W(2022)The case for distributed shared-memory databases with RDMA-enabled memory disaggregationProceedings of the VLDB Endowment10.14778/3561261.356126316:1(15-22)Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.14778/3561261.3561263
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents