research-article

Bolt-on causal consistency

Authors:

Joseph M. Hellerstein,

Ion StoicaAuthors Info & Claims

SIGMOD '13: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data

Pages 761 - 772

https://doi.org/10.1145/2463676.2465279

Published: 22 June 2013 Publication History

Abstract

We consider the problem of separating consistency-related safety properties from availability and durability in distributed data stores via the application of a "bolt-on" shim layer that upgrades the safety of an underlying general-purpose data store. This shim provides the same consistency guarantees atop a wide range of widely deployed but often inflexible stores. As causal consistency is one of the strongest consistency models that remain available during system partitions, we develop a shim layer that upgrades eventually consistent stores to provide convergent causal consistency. Accordingly, we leverage widely deployed eventually consistent infrastructure as a common substrate for providing causal guarantees. We describe algorithms and shim implementations that are suitable for a large class of application-level causality relationships and evaluate our techniques using an existing, production-ready data store and with real-world explicit causality relationships.

References

[1]

Metafilter Infodump. http://stuff.metafilter.com/infodump/. Combination of all available comment datasets: mefi, askme, meta, music. User count from usernames.

[2]

D. J. Abadi. Consistency tradeoffs in modern distributed database system design: CAP is only part of the story. IEEE Computer, 45(2):37--42, 2012.

Digital Library

[3]

A. Adya. Weak consistency: a generalized theory and optimistic implementations for distributed transactions. PhD thesis, Massachusetts Institute of Technology, 1999.

Digital Library

[4]

M. Ahamad, G. Neiger, J. E. Burns, P. Kohli, and P. Hutto. Causal memory: Definitions, implementation and programming. Distributed Computing, 9(1), 1995.

[5]

B. Alpern and F. B. Schneider. Defining liveness. Inf. Process. Lett., 21(4):181--185, 1985.

[6]

P. Bailis, A. Fekete, A. Ghodsi, J. M. Hellerstein, and I. Stoica. The potential dangers of causal consistency and an explicit solution. In ACM SOCC 2012.

Digital Library

[7]

P. Bailis and A. Ghodsi. Eventual Consistency today: Limitations, extensions, and beyond. ACM Queue, 11(3), 2013.

Digital Library

[8]

P. Bailis, S. Venkataraman, M. J. Franklin, J. M. Hellerstein, and I. Stoica. Probabilistically Bounded Staleness for practical partial quorums. In VLDB 2012.

Digital Library

[9]

N. Belaramani, M. Dahlin, L. Gao, A. Nayate, A. Venkataramani, P. Yalagandula, and J. Zheng. PRACTI replication. In NSDI 2006.

Digital Library

[10]

D. Bermbach, J. Kuhlenkamp, B. Derre, M. Klems, and S. Tai. A middleware guaranteeing client-centric consistency on top of eventually consistent datastores. In IEEE IC2E 2013.

[11]

K. Birman. A response to Cheriton and Skeen's criticism of causal and totally ordered communication. SIGOPS Oper. Syst. Rev., 28(1):11--21, Jan. 1994.

Digital Library

[12]

M. Brantner, D. Florescu, D. Graf, D. Kossmann, and T. Kraska. Building a database on S3. In SIGMOD 2008.

Digital Library

[13]

J. Brzezinski, C. Sobaniec, and D. Wawrzyniak. From session causality to causal consistency. In PDP 2004.

[14]

D. G. Campbell, G. Kakivaya, and N. Ellis. Extreme scale with full SQL language support in Microsoft SQL Azure. In SIGMOD 2010.

Digital Library

[15]

M. Cha, A. Mislove, and K. P. Gummadi. A Measurement-driven Analysis of Information Propagation in the Flickr Social Network. In WWW 2009.

Digital Library

[16]

D. R. Cheriton and D. Skeen. Understanding the limitations of causally and totally ordered communication. In SOSP 1993.

Digital Library

[17]

B. F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H.-A. Jacobsen, N. Puz, D. Weaver, and R. Yerneni. PNUTS: Yahoo!'s hosted data serving platform. In VLDB 2008.

Digital Library

[18]

B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking cloud serving systems with YCSB. In ACM SOCC 2010.

Digital Library

[19]

S. Das, D. Agrawal, and A. El Abbadi. G-store: a scalable data store for transactional multi key access in the cloud. In ACM SOCC 2010.

Digital Library

[20]

S. Davidson, H. Garcia-Molina, and D. Skeen. Consistency in partitioned networks. ACM Computing Surveys, 17(3):341--370, 1985.

Digital Library

[21]

J. Dean. Designs, lessons, and advice from building large distributed systems. Keynote from LADIS 2009.

[22]

G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon's highly available key-value store. In SOSP 2007.

Digital Library

[23]

Facebook, Inc. SEC Form S-1, February 2012.

[24]

D. Farber. Twitter hits 400 million tweets per day, mostly mobile. CNET, 2012. http://cnet.co/KHlg8q.

[25]

A. Feinberg. Project Voldemort: Reliable distributed storage. In ICDE 2011.

[26]

A. Geppert and K. Dittrich. Component database systems: Introduction, foundations, and overview, in Component Database Systems, 2001.

Digital Library

[27]

S. Gilbert and N. Lynch. Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News, 33(2):51--59, June 2002.

Digital Library

[28]

M. Klems. cassandra-user mailing list: Benchmarking Cassandra with YCSB, February 2011. http://bit.ly/15iQfBD.

[29]

R. Ladin, B. Liskov, L. Shrira, and S. Ghemawat. Providing high availability using lazy replication. ACM Trans. Comput. Syst., 10(4):360--391, Nov. 1992.

Digital Library

[30]

A. Lakshman and P. Malik. Cassandra - a decentralized structured storage system. In LADIS 2008. Configuration: http://cassandra.apache.org (2013).

[31]

L. Lamport. Time, clocks, and the ordering of events in a distributed system. Commun. ACM, 21(7):558--565, July 1978.

Digital Library

[32]

J. Levandoski, D. Lomet, M. Mokbel, and K. Zhao. Deuteronomy: Transaction support for cloud data. In CIDR 2011.

[33]

G. Linden. Make data useful. https://sites.google.com/site/glinden/Home/StanfordDataMining.2006-11-29.ppt, 2006.

[34]

W. Lloyd, M. J. Freedman, M. Kaminsky, and D. G. Andersen. Don't settle for eventual: scalable causal consistency for wide-area storage with COPS. In SOSP 2011.

Digital Library

[35]

W. Lloyd, M. J. Freedman, M. Kaminsky, and D. G. Andersen. Stronger semantics for low-latency geo-replicated storage. In NSDI 2013.

Digital Library

[36]

P. Mahajan, L. Alvisi, and M. Dahlin. Consistency, availability, convergence. Technical Report TR-11-22, Computer Science Department, UT Austin, May 2011.

[37]

F. Mattern. Virtual time and global states of distributed systems. Parallel and Distributed Algorithms, 1(23):215--226, 1989.

[38]

J. S. Mill. A System of Logic, Ratiocinative and Inductive, Being a Connected View of the Principles of Evidence, and the Methods of Scientific Investigation, volume 1. John W. Parker, West Strand, London, 1843.

[39]

D. S. Parker, G. J. Popek, G. Rudisin, A. Stoughton, B. J. Walker, E. Walton, J. M. Chow, D. Edwards, S. Kiser, and C. Kline. Detection of mutual inconsistency in distributed systems. IEEE Trans. Softw. Eng., 9(3):240--247, May 1983.

Digital Library

[40]

M. Patiño-Martinez, R. Jiménez-Peris, B. Kemme, and G. Alonso. Consistent database replication at the middleware level. ACM TOCS, 23(4):375--423, 2005.

Digital Library

[41]

K. Petersen, M. J. Spreitzer, D. B. Terry, M. M. Theimer, and A. J. Demers. Flexible update propagation for weakly consistent replication. In SOSP 1997.

Digital Library

[42]

M. Rawashdeh. Bolstering our infrastructure. http://engineering.twitter.com/2012/11/bolstering-our-infrastructure.html (2013).

[43]

A. Ritter, C. Cherry, and B. Dolan. Unsupervised modeling of Twitter conversations. In HLT 2010.

Digital Library

[44]

J. H. Saltzer, D. P. Reed, and D. D. Clark. End-to-end arguments in system design. ACM Trans. Comput. Syst., 2(4):277--288, Nov. 1984.

Digital Library

[45]

A. Schiper, K. Birman, and P. Stephenson. Lightweight causal and atomic group multicast. ACM Trans. Comput. Syst., 9(3):272--314, Aug. 1991.

Digital Library

[46]

R. Schwarz and F. Mattern. Detecting causal relationships in distributed computations: In search of the holy grail. Distributed Computing, 7:149--174, 1994.

Digital Library

[47]

Y. Sovran, R. Power, M. K. Aguilera, and J. Li. Transactional storage for geo-replicated systems. In SOSP 2011.

Digital Library

[48]

D. B. Terry, A. J. Demers, K. Petersen, M. J. Spreitzer, M. M. Theimer, and B. B. Welch. Session guarantees for weakly consistent replicated data. In PDIS 1994.

Digital Library

[49]

W. Vogels. Eventually consistent. Commun. ACM, 52(1):40--44, Jan. 2009.

Digital Library

[50]

W. Vogels. Choosing consistency. http://www.allthingsdistributed.com/2010/02/strong_consistency_simpledb.html, 2010.

[51]

S. Ye and S. F. Wu. Measuring message propagation and social influence on Twitter.com. In SocInfo 2010.

Digital Library

[52]

R. Zafarani and H. Liu. Social computing data repository at ASU, 2009. TUAW dataset.

[53]

M. Zawirski, A. Bieniusa, V. Balegas, N. Preguica, S. Duarte, M. Shapiro, and C. Baquero. Geo-replication all the way to the edge. 2013. Personal communication and draft under submission. http://asc.di.fct.unl.pt/~nmp/swiftcomp/swiftcloud.html.

Cited By

Camilleri CVella JNezval V(2024)Horizontally Scalable Implementation of a Distributed DBMS Delivering Causal Consistency via the Actor ModelElectronics10.3390/electronics1317336713:17(3367)Online publication date: 24-Aug-2024
https://doi.org/10.3390/electronics13173367
Nikolic LDimitrieski VCelikovic M(2024)An approach for supporting transparent acid transactions over heterogeneous data stores in microservice architecturesComputer Science and Information Systems10.2298/CSIS221210006N21:1(167-202)Online publication date: 2024
https://doi.org/10.2298/CSIS221210006N
Psistakis AChaix FTorrellas J(2024)MINOS: Distributed Consistency and Persistency Protocol Implementation & Offloading to SmartNICs2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA57654.2024.00076(1-17)Online publication date: 2-Mar-2024
https://doi.org/10.1109/HPCA57654.2024.00076
Show More Cited By

Index Terms

Bolt-on causal consistency
1. Information systems
  1. Data management systems
    1. Database management system engines
      1. Parallel and distributed DBMSs

Recommendations

Causal consistency: beyond memory
PPoPP '16

In distributed systems where strong consistency is costly when not impossible, causal consistency provides a valuable abstraction to represent program executions as partial orders. In addition to the sequential program order of each computing entity, ...
Causal consistency: beyond memory
PPoPP '16: Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

In distributed systems where strong consistency is costly when not impossible, causal consistency provides a valuable abstraction to represent program executions as partial orders. In addition to the sequential program order of each computing entity, ...
Limitations of Highly-Available Eventually-Consistent Data Stores
PODC '15: Proceedings of the 2015 ACM Symposium on Principles of Distributed Computing

Modern replicated data stores aim to provide high availability, by immediately responding to client requests, often by implementing objects that expose concurrency. Such objects, for example, multi-valued registers (MVRs), do not have sequential ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '13: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data

June 2013

1322 pages

ISBN:9781450320375

DOI:10.1145/2463676

General Chairs:
Kenneth Ross
Columbia University
,
Divesh Srivastava
AT&T Research
,
Program Chair:
Dimitris Papadias
HKUST

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 June 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGMOD/PODS'13

Sponsor:

SIGMOD

SIGMOD/PODS'13: International Conference on Management of Data

June 22 - 27, 2013

New York, New York, USA

Acceptance Rates

SIGMOD '13 Paper Acceptance Rate 76 of 372 submissions, 20%;

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

166
Total Citations
View Citations
1,190
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)5

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Camilleri CVella JNezval V(2024)Horizontally Scalable Implementation of a Distributed DBMS Delivering Causal Consistency via the Actor ModelElectronics10.3390/electronics1317336713:17(3367)Online publication date: 24-Aug-2024
https://doi.org/10.3390/electronics13173367
Nikolic LDimitrieski VCelikovic M(2024)An approach for supporting transparent acid transactions over heterogeneous data stores in microservice architecturesComputer Science and Information Systems10.2298/CSIS221210006N21:1(167-202)Online publication date: 2024
https://doi.org/10.2298/CSIS221210006N
Psistakis AChaix FTorrellas J(2024)MINOS: Distributed Consistency and Persistency Protocol Implementation & Offloading to SmartNICs2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA57654.2024.00076(1-17)Online publication date: 2-Mar-2024
https://doi.org/10.1109/HPCA57654.2024.00076
Gessert FWingerath WRitter NGessert FWingerath WRitter N(2024)Caching in Forschung und IndustrieSchnelles und skalierbares Cloud-Datenmanagement10.1007/978-3-031-54388-3_5(91-140)Online publication date: 3-May-2024
https://doi.org/10.1007/978-3-031-54388-3_5
Gessert FWingerath WRitter NGessert FWingerath WRitter N(2024)Systeme für skalierbares DatenmanagementSchnelles und skalierbares Cloud-Datenmanagement10.1007/978-3-031-54388-3_4(61-90)Online publication date: 3-May-2024
https://doi.org/10.1007/978-3-031-54388-3_4
Kraft PLi QZhou XBailis PStonebraker MZaharia MYu X(2023)Epoxy: ACID Transactions across Diverse Data StoresProceedings of the VLDB Endowment10.14778/3611479.361148416:11(2742-2754)Online publication date: 24-Aug-2023
https://dl.acm.org/doi/10.14778/3611479.3611484
Ferreira MLoff JGarcia J(2023)Rendezvous: Where Serverless Functions Find ConsistencyProceedings of the 4th Workshop on Resource Disaggregation and Serverless10.1145/3605181.3626290(51-57)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3605181.3626290
Ferreira Loff JPorto DGarcia JMace JRodrigues RDruschel PKaufmann AMace JFlinn JSeltzer M(2023)Antipode: Enforcing Cross-Service Causal Consistency in Distributed ApplicationsProceedings of the 29th Symposium on Operating Systems Principles10.1145/3600006.3613176(298-313)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3600006.3613176
Liu YMa PTian J(2023)Research on Technology and Industry Situation of Lakehouse2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom)10.1109/TrustCom60117.2023.00308(2198-2203)Online publication date: 1-Nov-2023
https://doi.org/10.1109/TrustCom60117.2023.00308
Frey DMostefaoui APerrin MRoman PTaïani F(2023)Differentiated Consistency for Worldwide GossipsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2022.320915034:1(1-15)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TPDS.2022.3209150
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten