research-article

Serializability for eventual consistency: criterion, analysis, and applications

Authors:

Lucas Brutschy,

Dimitar Dimitrov,

Martin VechevAuthors Info & Claims

POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages

Pages 458 - 472

https://doi.org/10.1145/3009837.3009895

Published: 01 January 2017 Publication History

Abstract

Developing and reasoning about systems using eventually consistent data stores is a difficult challenge due to the presence of unexpected behaviors that do not occur under sequential consistency. A fundamental problem in this setting is to identify a correctness criterion that precisely captures intended application behaviors yet is generic enough to be applicable to a wide range of applications.

In this paper, we present such a criterion. More precisely, we generalize conflict serializability to the setting of eventual consistency. Our generalization is based on a novel dependency model that incorporates two powerful algebraic properties: commutativity and absorption. These properties enable precise reasoning about programs that employ high-level replicated data types, common in modern systems. To apply our criterion in practice, we also developed a dynamic analysis algorithm and a tool that checks whether a given program execution is serializable.

We performed a thorough experimental evaluation on two real-world use cases: debugging cloud-backed mobile applications and implementing clients of a popular eventually consistent key-value store. Our experimental results indicate that our criterion reveals harmful synchronization problems in applications, is more effective at finding them than prior approaches, and can be used for the development of practical, eventually consistent applications.

References

[1]

A. Adya, B. Liskov, and P. E. O’Neil. Generalized isolation level definitions. In Proceedings of the 16th International Conference on Data Engineering, ICDE ’00, pages 67–78, Washington, DC, USA, 2000. IEEE Computer Society. ISBN 0-7695-0506-6.

Digital Library

[2]

P. Bailis, A. Fekete, M. J. Franklin, A. Ghodsi, J. M. Hellerstein, and I. Stoica. Coordination avoidance in database systems. Proc. VLDB Endow., 8(3):185–196, Nov. 2014. ISSN 2150-8097.

Digital Library

[4]

G. Bernardi and A. Gotsman. Robustness against consistency models with atomic visibility. In CONCUR’16: International Conference on Concurrency Theory, 2016.

Digital Library

[5]

P. A. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency Control and Recovery in Database Systems. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1987. ISBN 0-201-10715-5.

Digital Library

[6]

A. Bouajjani, E. Derevenetc, and R. Meyer. Checking and enforcing robustness against TSO. In Proceedings of the 22nd European Conference on Programming Languages and Systems, ESOP’13, pages 533–553, Berlin, Heidelberg, 2013. Springer-Verlag. ISBN 978- 3-642-37035-9.

Digital Library

[7]

R. Brown, S. Cribbs, C. Meiklejohn, and S. Elliott. Riak DT map: A composable, convergent replicated dictionary. In Proceedings of the First Workshop on Principles and Practice of Eventual Consistency, PaPEC ’14, pages 1:1–1:1, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-2716-9.

Digital Library

[8]

S. Burckhardt. Principles of Eventual Consistency, volume 1 of Foundations and Trends in Programming Languages. now publishers, October 2014.

Digital Library

[9]

S. Burckhardt, A. Gotsman, H. Yang, and M. Zawirski. Replicated data types: Specification, verification, optimality. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’14, pages 271–284, New York, NY, USA, 2014.

[10]

ACM. ISBN 978-1-4503-2544-8.

[11]

S. Burckhardt, D. Leijen, J. Protzenko, and M. Fähndrich. Global sequence protocol: A robust abstraction for replicated shared state. In J. T. Boyland, editor, 29th European Conference on Object-Oriented Programming, volume 37 of ECOOP 2015, pages 568–590, Dagstuhl, Germany, 2015. Schloss Dagstuhl– Leibniz-Zentrum fuer Informatik. ISBN 978-3-939897-86-6.

[12]

A. Cerone, G. Bernardi, and A. Gotsman. A Framework for Transactional Consistency Models with Atomic Visibility. In L. Aceto and D. de Frutos Escrig, editors, 26th International Conference on Concurrency Theory, volume 42 of CONCUR 2015, pages 58–71, Dagstuhl, Germany, 2015. Schloss Dagstuhl– Leibniz-Zentrum fuer Informatik. ISBN 978-3-939897-91-0.

Digital Library

[13]

B. F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H.-A. Jacobsen, N. Puz, D. Weaver, and R. Yerneni. PNUTS: yahoo!’s hosted data serving platform. Proc. VLDB Endow., 1(2):1277– 1288, Aug. 2008. ISSN 2150-8097.

Digital Library

[14]

G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon’s highly available key-value store. In Proceedings of Twentyfirst ACM SIGOPS Symposium on Operating Systems Principles, SOSP ’07, pages 205–220, New York, NY, USA, 2007. ACM. ISBN 978-1- 59593-591-5.

Digital Library

[15]

D. Dimitrov, V. Raychev, M. Vechev, and E. Koskinen. Commutativity race detection. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, pages 305–315, New York, NY, USA, 2014. ACM. ISBN 978-1-4503- 2784-8.

Digital Library

[16]

A. Fekete, D. Liarokapis, E. O’Neil, P. O’Neil, and D. Shasha. Making snapshot isolation serializable. ACM Trans. Database Syst., 30(2): 492–528, June 2005. ISSN 0362-5915.

[17]

G. Gierz, K. Hofmann, K. Keimel, J. Lawson, M. Mislove, and D. Scott. Continuous Lattices and Domains. Number 93 in Encyclopedia of Mathematics and its Applications. Cambridge University Press, 2003. ISBN 9780521803380.

Digital Library

[18]

S. Gilbert and N. Lynch. Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News, 33 (2):51–59, June 2002. ISSN 0163-5700.

Digital Library

[19]

P. Hunt, M. Konar, F. P. Junqueira, and B. Reed. Zookeeper: Waitfree coordination for internet-scale systems. In Proceedings of the 2010 USENIX Conference on USENIX Annual Technical Conference, USENIXATC’10, pages 11–11, Berkeley, CA, USA, 2010. USENIX Association.

Digital Library

[20]

P. R. Johnson and R. H. Thomas. The maintenance of duplicate databases. RFC 677, RFC Editor, january 1975.

Digital Library

[21]

R. Klophaus. Riak core: Building distributed applications without shared state. In ACM SIGPLAN Commercial Users of Functional Programming, CUFP ’10, pages 14:1–14:1, New York, NY, USA, 2010.

[22]

ACM. ISBN 978-1-4503-0516-7.

Digital Library

[23]

A. Lakshman and P. Malik. Cassandra: A decentralized structured storage system. SIGOPS Oper. Syst. Rev., 44(2):35–40, Apr. 2010. ISSN 0163-5980.

Digital Library

[24]

L. Lamport. Time, clocks, and the ordering of events in a distributed system. Commun. ACM, 21(7):558–565, July 1978. ISSN 0001-0782.

Digital Library

[26]

L. Lamport. How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Trans. Comput., 28(9):690–691, Sept. 1979. ISSN 0018-9340.

Digital Library

[27]

W. Lloyd, M. J. Freedman, M. Kaminsky, and D. G. Andersen. Don’t settle for eventual: Scalable causal consistency for wide-area storage with cops. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP ’11, pages 401– 416, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0977-6.

[28]

W. Lloyd, M. J. Freedman, M. Kaminsky, and D. G. Andersen. Stronger semantics for low-latency geo-replicated storage. In Proceedings of the 10th USENIX Conference on Networked Systems Design and Implementation, nsdi’13, pages 313–328, Berkeley, CA, USA, 2013.

[29]

USENIX Association.

[30]

F. Mattern. Virtual time and global states of distributed systems. In M. Cosnard, editor, Proc. Workshop on Parallel and Distributed Algorithms, pages 215–226, North-Holland / Elsevier, 1989. (Reprinted in: Z. Yang, T.A. Marsland (Eds.), "Global States and Time in Distributed Systems", IEEE, 1994, pp. 123-133.).

Digital Library

[31]

A. Mazurkiewicz. Trace theory. In W. Brauer, W. Reisig, and G. Rozenberg, editors, Petri Nets: Applications and Relationships to Other Models of Concurrency, volume 255 of Lecture Notes in Computer Science, pages 278–324. Springer Berlin Heidelberg, 1987. ISBN 978-3-540-17906-1.

Digital Library

[32]

S. Owens. Reasoning about the implementation of concurrency abstractions on x86-TSO. In Proceedings of the 24th European Conference on Object-oriented Programming, ECOOP’10, pages 478– 503, Berlin, Heidelberg, 2010. Springer-Verlag. ISBN 3-642-14106-4, 978-3-642-14106-5.

Digital Library

[33]

C. H. Papadimitriou. The serializability of concurrent database updates. J. ACM, 26(4):631–653, Oct. 1979. ISSN 0004-5411.

[35]

N. Preguiça, M. Zawirski, A. Bieniusa, S. Duarte, V. Balegas, C. Baquero, and M. Shapiro. SwiftCloud: Fault-tolerant geo-replication integrated all the way to the client machine. In Proceedings of the 2014 IEEE 33rd International Symposium on Reliable Distributed Systems Workshops, SRDSW ’14, pages 30–33, Washington, DC, USA, 2014. IEEE Computer Society. ISBN 978-1-4799-7361-3.

Digital Library

[36]

M. Shapiro, N. Preguiça, C. Baquero, and M. Zawirski. A comprehensive study of Convergent and Commutative Replicated Data Types. Research Report RR-7506, Inria – Centre Paris-Rocquencourt ; INRIA, Jan. 2011.

Digital Library

[37]

M. Shapiro, N. Preguiça, C. Baquero, and M. Zawirski. Conflictfree replicated data types. In Proceedings of the 13th International Conference on Stabilization, Safety, and Security of Distributed Systems, SSS’11, pages 386–400, Berlin, Heidelberg, 2011. Springer-Verlag. ISBN 978-3-642-24549-7.

Digital Library

[38]

D. Shasha and M. Snir. Efficient and correct execution of parallel programs that share memory. ACM Trans. Program. Lang. Syst., 10(2): 282–312, Apr. 1988. ISSN 0164-0925.

Digital Library

[39]

S. Sivasubramanian. Amazon dynamoDB: A seamlessly scalable nonrelational database service. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, SIGMOD ’12, pages 729–730, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1247-9.

[40]

Y. Sovran, R. Power, M. K. Aguilera, and J. Li. Transactional storage for geo-replicated systems. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP ’11, pages 385– 400, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0977-6.

Digital Library

[41]

N. Tillmann, M. Moskal, J. de Halleux, and M. Fahndrich. TouchDevelop: Programming cloud-connected mobile devices via touchscreen. In Proceedings of the 10th SIGPLAN Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2011, pages 49–60, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0941-7.

Digital Library

[42]

Transaction Processing Performance Council. TPC-C benchmark, revision 5.11, Feb. 2010.

Digital Library

[43]

M. Vaziri, F. Tip, and J. Dolby. Associating synchronization constraints with data in an object-oriented language. In Conference Record of the 33rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’06, pages 334–345, New York, NY, USA, 2006. ACM. ISBN 1-59593-027-2.

[44]

W. E. Weihl. Commutativity-based concurrency control for abstract data types. IEEE Trans. Comput., 37(12):1488–1505, Dec. 1988. ISSN 0018-9340.

[45]

K. Zellag and B. Kemme. How consistent is your cloud application? In Proceedings of the Third ACM Symposium on Cloud Computing, SoCC ’12, pages 6:1–6:14, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1761-0.

Cited By

Geng CBlanas SBond MWang Y(2024)IsoPredict: Dynamic Predictive Analysis for Detecting Unserializable Behaviors in Weakly Isolated Data Store ApplicationsProceedings of the ACM on Programming Languages10.1145/36563918:PLDI(343-367)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656391
Cui ZDou WGao YWang DSong JZheng YWang TYang RXu KHu YWei JHuang TRoychoudhury APaiva AAbreu RStorey M(2024)Understanding Transaction Bugs in Database SystemsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639207(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639207
Cai YCheng RZhou YZhang SXiao JJin H(2024)Enabling Complete Atomicity for Cross-Chain Applications Through Layered State Commitments2024 43rd International Symposium on Reliable Distributed Systems (SRDS)10.1109/SRDS64841.2024.00032(248-259)Online publication date: 30-Sep-2024
https://doi.org/10.1109/SRDS64841.2024.00032
Show More Cited By

Index Terms

Serializability for eventual consistency: criterion, analysis, and applications
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles

Recommendations

Static serializability analysis for causal consistency
PLDI 2018: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation

Many distributed databases provide only weak consistency guarantees to reduce synchronization overhead and remain available under network partitions. However, this leads to behaviors not possible under stronger guarantees. Such behaviors can easily defy ...
Serializability for eventual consistency: criterion, analysis, and applications
POPL '17

Developing and reasoning about systems using eventually consistent data stores is a difficult challenge due to the presence of unexpected behaviors that do not occur under sequential consistency. A fundamental problem in this setting is to identify a ...
Constraining the eventual in eventual consistency
PaPoC '18: Proceedings of the 5th Workshop on the Principles and Practice of Consistency for Distributed Data

CRDTs are highly available replicated data structures which offer strong eventual consistency in the face of concurrent operations [3]. By their definition, CRDTs eventually converge to a consistent state given enough time. However, this is not strict ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages

January 2017

901 pages

ISBN:9781450346603

DOI:10.1145/3009837

General Chair:
Giuseppe Castagna
Paris Diderot University, France / CNRS, France
,
Program Chair:
Andrew D. Gordon
Microsoft Research, UK / University of Edinburgh, UK

ACM SIGPLAN Notices Volume 52, Issue 1
POPL '17
January 2017
901 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/3093333
Editor:
Matthew Fluet
Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 January 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

POPL '17

Sponsor:

SIGPLAN

POPL '17: The 44th Annual ACM SIGPLAN Symposium on Principles of Programming Languages

January 15 - 21, 2017

Paris, France

Acceptance Rates

Overall Acceptance Rate 824 of 4,130 submissions, 20%

Upcoming Conference

POPL '25

Sponsor:
sigplan

The 52nd Annual ACM SIGPLAN Symposium on Principles of Programming Languages

January 19 - 25, 2025

Denver , CO , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

52
Total Citations
View Citations
498
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Geng CBlanas SBond MWang Y(2024)IsoPredict: Dynamic Predictive Analysis for Detecting Unserializable Behaviors in Weakly Isolated Data Store ApplicationsProceedings of the ACM on Programming Languages10.1145/36563918:PLDI(343-367)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656391
Cui ZDou WGao YWang DSong JZheng YWang TYang RXu KHu YWei JHuang TRoychoudhury APaiva AAbreu RStorey M(2024)Understanding Transaction Bugs in Database SystemsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639207(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639207
Cai YCheng RZhou YZhang SXiao JJin H(2024)Enabling Complete Atomicity for Cross-Chain Applications Through Layered State Commitments2024 43rd International Symposium on Reliable Distributed Systems (SRDS)10.1109/SRDS64841.2024.00032(248-259)Online publication date: 30-Sep-2024
https://doi.org/10.1109/SRDS64841.2024.00032
Eilers MDardinier TMüller P(2023)CommCSL: Proving Information Flow Security for Concurrent Programs using Abstract CommutativityProceedings of the ACM on Programming Languages10.1145/35912897:PLDI(1682-1707)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1145/3591289
Bouajjani AEnea CRomán-Calvo E(2023)Dynamic Partial Order Reduction for Checking Correctness against Transaction Isolation LevelsProceedings of the ACM on Programming Languages10.1145/35912437:PLDI(565-590)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1145/3591243
Zhang JJi YMu STan CFedorova ANarayanan DDi Luna GQuerzoni L(2023)Viper: A Fast Snapshot Isolation CheckerProceedings of the Eighteenth European Conference on Computer Systems10.1145/3552326.3567492(654-671)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3552326.3567492
Dou WCui ZDai QSong JWang DGao YWang WWei JChen LWang HZhong HHuang T(2023)Detecting Isolation Bugs via Transaction Oracle Construction2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00101(1123-1135)Online publication date: May-2023
https://doi.org/10.1109/ICSE48619.2023.00101
Hackett FRowe JKuppe MLiu AMuccini H(2023)Understanding Inconsistency in Azure Cosmos DB with TLA+Proceedings of the 45th International Conference on Software Engineering: Software Engineering in Practice10.1109/ICSE-SEIP58684.2023.00006(1-12)Online publication date: 17-May-2023
https://dl.acm.org/doi/10.1109/ICSE-SEIP58684.2023.00006
Beillahi SBouajjani AEnea C(2023)Comparing Causal Convergence Consistency ModelsNetworked Systems10.1007/978-3-031-37765-5_5(62-77)Online publication date: 7-Jul-2023
https://doi.org/10.1007/978-3-031-37765-5_5
Cui ZDou WDai QSong JWang WWei JYe D(2022)Differentially Testing Database Transactions for Fun and ProfitProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering10.1145/3551349.3556924(1-12)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3551349.3556924
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents