research-article

Open access

CLOTHO: directed test generation for weakly consistent database systems

Authors:

Benjamin Delaware,

Suresh JagannathanAuthors Info & Claims

Proceedings of the ACM on Programming Languages, Volume 3, Issue OOPSLA

Article No.: 117, Pages 1 - 28

https://doi.org/10.1145/3360543

Published: 10 October 2019 Publication History

Abstract

Relational database applications are notoriously difficult to test and debug. Concurrent execution of database transactions may violate complex structural invariants that constraint how changes to the contents of one (shared) table affect the contents of another. Simplifying the underlying concurrency model is one way to ameliorate the difficulty of understanding how concurrent accesses and updates can affect database state with respect to these sophisticated properties. Enforcing serializable execution of all transactions achieves this simplification, but it comes at a significant price in performance, especially at scale, where database state is often replicated to improve latency and availability.

To address these challenges, this paper presents a novel testing framework for detecting serializability violations in (SQL) database-backed Java applications executing on weakly-consistent storage systems. We manifest our approach in a tool, CLOTHO, that combines a static analyzer and model checker to generate abstract executions, discover serializability violations in these executions, and translate them back into concrete test inputs suitable for deployment in a test environment. To the best of our knowledge, CLOTHO, is the first automated test generation facility for identifying serializability anomalies of Java applications intended to operate in geo-replicated distributed environments. An experimental evaluation on a set of industry-standard benchmarks demonstrates the utility of our approach.

Supplementary Material

a117-rahmani (a117-rahmani.webm)

Presentation at OOPSLA '19

Download
126.67 MB

References

[1]

2010. TPC-C Benchmark. http://www.tpc.org/tpc_documents_current_versions/pdf/tpc-c_v5.11.0.pdf . Online; Accessed April 2019.

[2]

Atul Adya. 1999. Weak Consistency: A Generalized Theory and Optimistic Implementations for Distributed Transactions. Ph.D. Dissertation. Cambridge, MA, USA. AAI0800775.

[3]

Atul Adya, Barbara Liskov, and Patrick E. O’Neil. 2000. Generalized Isolation Level Definitions. In Proceedings of the 16th International Conference on Data Engineering, San Diego, California, USA, February 28 - March 3, 2000 . 67–78.

[4]

Peter Alvaro, Neil Conway, Joe Hellerstein, and William R. Marczak. 2011. Consistency Analysis in Bloom: a CALM and Collected Approach. In CIDR 2011, Fifth Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 9-12, 2011, Online Proceedings . 249–260.

[5]

Peter Bailis, Alan Fekete, Michael J. Franklin, Ali Ghodsi, Joseph M. Hellerstein, and Ion Stoica. 2014a. Coordination Avoidance in Database Systems. Proc. VLDB Endow. 8, 3 (Nov. 2014), 185–196.

Digital Library

[6]

Peter Bailis, Alan Fekete, Joseph M. Hellerstein, Ali Ghodsi, and Ion Stoica. 2014b. Scalable Atomic Visibility with RAMP Transactions. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data (SIGMOD ’14). ACM, New York, NY, USA, 27–38.

Digital Library

[7]

Valter Balegas, Nuno Preguiça, Rodrigo Rodrigues, Sérgio Duarte, Carla Ferreira, Mahsa Najafzadeh, and Marc Shapiro. 2015. Putting the Consistency back into Eventual Consistency. In Proceedings of the Tenth European Conference on Computer System (EuroSys ’15) . Bordeaux, France. http://lip6.fr/Marc.Shapiro/papers/putting-consistency-back-EuroSys-2015.pdf

Digital Library

[8]

Hal Berenson, Phil Bernstein, Jim Gray, Jim Melton, Elizabeth O’Neil, and Patrick O’Neil. 1995. A Critique of ANSI SQL Isolation Levels. In Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data (SIGMOD ’95). ACM, New York, NY, USA, 1–10.

Digital Library

[9]

Giovanni Bernardi and Alexey Gotsman. 2016. Robustness against Consistency Models with Atomic Visibility. In 27th International Conference on Concurrency Theory, CONCUR 2016, August 23-26, 2016, Québec City, Canada . 7:1–7:15.

[10]

Blockade. 2019. Blockade. https://github.com/worstcase/blockade accessed on 2019-03-29.

[11]

Lucas Brutschy, Dimitar Dimitrov, Peter Müller, and Martin T. Vechev. 2017. Serializability for eventual consistency: criterion, analysis, and applications. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages, POPL 2017, Paris, France, January 18-20, 2017 . 458–472. http://dl.acm.org/citation.cfm?id=3009895

[12]

Lucas Brutschy, Dimitar Dimitrov, Peter Müller, and Martin T. Vechev. 2018. Static serializability analysis for causal consistency. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2018, Philadelphia, PA, USA, June 18-22, 2018 . 90–104.

Digital Library

[13]

Sebastian Burckhardt, Alexey Gotsman, Hongseok Yang, and Marek Zawirski. 2014. Replicated Data Types: Specification, Verification, Optimality. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’14) . ACM, New York, NY, USA, 271–284.

Digital Library

[14]

Andrea Cerone, Giovanni Bernardi, and Alexey Gotsman. 2015. A Framework for Transactional Consistency Models with Atomic Visibility. In 26th International Conference on Concurrency Theory, CONCUR 2015, Madrid, Spain, September 1.4, 2015 . 58–71.

[15]

Andrea Cerone and Alexey Gotsman. 2016. Analysing Snapshot Isolation. In Proceedings of the 2016 ACM Symposium on Principles of Distributed Computing, PODC 2016, Chicago, IL, USA, July 25-28, 2016 . 55–64.

Digital Library

[16]

Andrea Cerone, Alexey Gotsman, and Hongseok Yang. 2017. Algebraic Laws for Weak Consistency. In 28th International Conference on Concurrency Theory, CONCUR 2017, September 5-8, 2017, Berlin, Germany . 26:1–26:18.

[17]

Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Tools and Algorithms for the Construction and Analysis of Systems, C. R. Ramakrishnan and Jakob Rehof (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 337–340.

Digital Library

[18]

Djellel Eddine Difallah, Andrew Pavlo, Carlo Curino, and Philippe Cudre-Mauroux. 2013. OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases. Proc. VLDB Endow. 7, 4 (Dec. 2013), 277–288.

Digital Library

[19]

Alan Fekete, Dimitrios Liarokapis, Elizabeth J. O’Neil, and Patrick E. O’Neil a fnd Dennis E. Shasha. 2005. Making snapshot isolation serializable. ACM Trans. Database Syst. 30, 2 (2005), 492–528.

Digital Library

[20]

Cormac Flanagan, Cormac Flanagan, and Stephen N Freund. 2004. Atomizer: A Dynamic Atomicity Checker for Multithreaded Programs. In Proceedings of the 31st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’04) . ACM, New York, NY, USA, 256–267.

Digital Library

[21]

Cormac Flanagan and Shaz Qadeer. 2003. A Type and Effect System for Atomicity. In Proceedings of the ACM SIGPLAN 2003 Conference on Programming Language Design and Implementation (PLDI ’03) . ACM, New York, NY, USA, 338–349.

Digital Library

[22]

Alexey Gotsman, Hongseok Yang, Carla Ferreira, Mahsa Najafzadeh, and Marc Shapiro. 2016. ’Cause I’m Strong Enough: Reasoning About Consistency Choices in Distributed Systems. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL 2016) . ACM, New York, NY, USA, 371–384.

Digital Library

[23]

Christian Hammer, Julian Dolby, Mandana Vaziri, and Frank Tip. 2008. Dynamic Detection of Atomic-set-serializability Violations. In Proceedings of the 30th International Conference on Software Engineering (ICSE ’08). ACM, New York, NY, USA, 231–240.

Digital Library

[24]

Jeff Huang, Charles Zhang, and Julian Dolby. 2013. CLAP: Recording Local Executions to Reproduce Concurrency Failures. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’13). ACM, New York, NY, USA, 141–152.

Digital Library

[25]

Jepsen 2018. https://jepsen.io/

[26]

Sudhir Jorwekar, Alan Fekete, Krithi Ramamritham, and S. Sudarshan. 2007. Automating the Detection of Snapshot Isolation Anomalies. In Proceedings of the 33rd International Conference on Very Large Data Bases, University of Vienna, Austria, September 23-27, 2007 . 1263–1274. http://www.vldb.org/conf/2007/papers/industrial/p1263-jorwekar.pdf

[27]

Gowtham Kaki, Kapil Earanky, KC Sivaramakrishnan, and Suresh Jagannathan. 2018. Safe Replication Through Bounded Concurrency Verification. Proc. ACM Program. Lang. 2, OOPSLA, Article 164 (Oct. 2018), 27 pages.

Digital Library

[28]

Charles Killian, James W. Anderson, Ranjit Jhala, and Amin Vahdat. 2007. Life, Death, and the Critical Transition: Finding Liveness Bugs in Systems Code. In Proceedings of the 4th USENIX Conference on Networked Systems Design &#38; Implementation (NSDI’07) . USENIX Association, Berkeley, CA, USA, 18–18. http://dl.acm.org/citation.cfm?id=1973430. 1973448

Digital Library

[29]

Mohsen Lesani, Christian J. Bell, and Adam Chlipala. 2016. Chapar: Certified Causally Consistent Distributed Key-value Stores. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’16) . ACM, New York, NY, USA, 357–370.

Digital Library

[30]

Cheng Li, João Leitão, Allen Clement, Nuno Preguiça, Rodrigo Rodrigues, and Viktor Vafeiadis. 2014. Automating the Choice of Consistency Levels in Replicated Systems. In Proceedings of USENIX Annual Technical Conference (USENIX ATC’14) . USENIX Association, Berkeley, CA, USA, 281–292. http://dl.acm.org/citation.cfm?id=2643634.2643664

[31]

Cheng Li, Daniel Porto, Allen Clement, Johannes Gehrke, Nuno Preguiça, and Rodrigo Rodrigues. 2012. Making Georeplicated Systems Fast As Possible, Consistent when Necessary. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (OSDI’12) . USENIX Association, Berkeley, CA, USA, 265–278. http: //dl.acm.org/citation.cfm?id=2387880.2387906

Digital Library

[32]

Shan Lu, Soyeon Park, Chongfeng Hu, Xiao Ma, Weihang Jiang, Zhenmin Li, Raluca A. Popa, and Yuanyuan Zhou. 2007. MUVI: Automatically Inferring Multi-variable Access Correlations and Detecting Related Semantic and Concurrency Bugs. In Proceedings of Twenty-first ACM SIGOPS Symposium on Operating Systems Principles (SOSP ’07). ACM, New York, NY, USA, 103–116.

Digital Library

[33]

Shan Lu, Joseph Tucek, Feng Qin, and Yuanyuan Zhou. 2006. AVIO: Detecting Atomicity Violations via Access Interleaving Invariants. In Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XII) . ACM, New York, NY, USA, 37–48.

Digital Library

[34]

Brandon Lucia, Luis Ceze, and Karin Strauss. 2010. ColorSafe: Architectural Support for Debugging and Dynamically Avoiding Multi-variable Atomicity Violations. In Proceedings of the 37th Annual International Symposium on Computer Architecture (ISCA ’10) . ACM, New York, NY, USA, 222–233.

Digital Library

[35]

Nuno Machado, Brandon Lucia, and Luís Rodrigues. 2015. Concurrency Debugging with Differential Schedule Projections. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’15). ACM, New York, NY, USA, 586–595.

Digital Library

[36]

Bill McCloskey, Feng Zhou, David Gay, and Eric Brewer. 2006. Autolocker: Synchronization Inference for Atomic Sections. In Conference Record of the 33rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’06). ACM, New York, NY, USA, 346–358.

Digital Library

[37]

Kartik Nagar and Suresh Jagannathan. 2018. Automated Detection of Serializability Violations Under Weak Consistency. In 29th International Conference on Concurrency Theory, CONCUR 2018, September 4-7, 2018, Beijing, China. 41:1–41:18.

[38]

Christos H. Papadimitriou. 1979. The Serializability of Concurrent Database Updates. J. ACM 26, 4 (Oct. 1979), 631–653.

Digital Library

[39]

Kia Rahmani, Gowtham Kaki, and Suresh Jagannathan. 2018. Fine-grained Distributed Consistency Guarantees with Effect Orchestration. In Proceedings of the 5th Workshop on the Principles and Practice of Consistency for Distributed Data (PaPoC ’18) . ACM, New York, NY, USA, Article 6, 5 pages.

Digital Library

[40]

Kia Rahmani, Kartik Nagar, Benjamin Delaware, and Suresh Jagannatha. 2019. CLOTHO: Directed Test Generation for Weakly Consistent Database Systtems (Extended Version). https://arxiv.org/pdf/1908.05655.pdf

[41]

Sriram Rajamani, G. Ramalingam, Venkatesh Prasad Ranganath, and Kapil Vaswani. 2009. ISOLATOR: Dynamically Ensuring Isolation in Comcurrent Programs. In Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XIV) . ACM, New York, NY, USA, 181–192.

Digital Library

[42]

Dennis Shasha and Philippe Bonnet. 2003. Database Tuning: Principles, Experiments, and Troubleshooting Techniques. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

[43]

KC Sivaramakrishnan, Gowtham Kaki, and Suresh Jagannathan. 2015. Declarative Programming over Eventually Consistent Data Stores. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2015) . ACM, New York, NY, USA, 413–424.

Digital Library

[44]

Ruby Y. Tahboub, Grégory M. Essertel, and Tiark Rompf. 2018. How to Architect a Query Compiler, Revisited. In Proceedings of the 2018 International Conference on Management of Data (SIGMOD ’18) . ACM, New York, NY, USA, 307–322.

Digital Library

[45]

Raja Vallée-Rai, Phong Co, Etienne Gagnon, Laurie Hendren, Patrick Lam, and Vijay Sundaresan. 1999. Soot - a Java Bytecode Optimization Framework. In Proceedings of the 1999 Conference of the Centre for Advanced Studies on Collaborative Research (CASCON ’99) . IBM Press, 13–. http://dl.acm.org/citation.cfm?id=781995.782008

Digital Library

[46]

Todd Warszawski and Peter Bailis. 2017. ACIDRain: Concurrency-Related Attacks on Database-Backed Web Applications. In Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD Conference 2017, Chicago, IL, USA, May 14-19, 2017 . 5–20.

Digital Library

[47]

Min Xu, Rastislav Bodík, and Mark D. Hill. 2005. A Serializability Violation Detector for Shared-memory Server Programs. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’05). ACM, New York, NY, USA, 1–14.

Digital Library

[48]

Yang Zhang, Russell Power, Siyuan Zhou, Yair Sovran, Marcos K. Aguilera, and Jinyang Li. 2013. Transaction chains: achieving serializability with low latency in geo-distributed storage systems. In ACM SIGOPS 24th Symposium on Operating Systems Principles, SOSP ’13, Farmington, PA, USA, November 3-6, 2013 . 276–291.

Digital Library

Cited By

Geng CBlanas SBond MWang Y(2024)IsoPredict: Dynamic Predictive Analysis for Detecting Unserializable Behaviors in Weakly Isolated Data Store ApplicationsProceedings of the ACM on Programming Languages10.1145/36563918:PLDI(343-367)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656391
Bouajjani AEnea CRomán-Calvo E(2023)Dynamic Partial Order Reduction for Checking Correctness against Transaction Isolation LevelsProceedings of the ACM on Programming Languages10.1145/35912437:PLDI(565-590)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1145/3591243
Dou WCui ZDai QSong JWang DGao YWang WWei JChen LWang HZhong HHuang T(2023)Detecting Isolation Bugs via Transaction Oracle Construction2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00101(1123-1135)Online publication date: May-2023
https://doi.org/10.1109/ICSE48619.2023.00101
Show More Cited By

Index Terms

CLOTHO: directed test generation for weakly consistent database systems
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Repairing serializability bugs in distributed database programs via automated schema refactoring
PLDI 2021: Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation

Serializability is a well-understood concurrency control mechanism that eases reasoning about highly-concurrent database programs. Unfortunately, enforcing serializability has a high performance cost, especially on geographically distributed database ...
Fast Serializable Multi-Version Concurrency Control for Main-Memory Database Systems
SIGMOD '15: Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data

Multi-Version Concurrency Control (MVCC) is a widely employed concurrency control mechanism, as it allows for execution modes where readers never block writers. However, most systems implement only snapshot isolation (SI) instead of full ...
Supporting consistent updates in replicated multidatabase systems

Replication is useful in multidatabase systems (MDBSs) because, as in traditional distributed database systems, it increases data availability in the presence of failures and decreases data retrieval costs by reading local or close copies of data. ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages

Proceedings of the ACM on Programming Languages Volume 3, Issue OOPSLA

October 2019

2077 pages

EISSN:2475-1421

DOI:10.1145/3366395

Issue’s Table of Contents

Copyright © 2019 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2019

Published in PACMPL Volume 3, Issue OOPSLA

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Air Force Research Lab
National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
883
Total Downloads

Downloads (Last 12 months)136
Downloads (Last 6 weeks)31

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Geng CBlanas SBond MWang Y(2024)IsoPredict: Dynamic Predictive Analysis for Detecting Unserializable Behaviors in Weakly Isolated Data Store ApplicationsProceedings of the ACM on Programming Languages10.1145/36563918:PLDI(343-367)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656391
Bouajjani AEnea CRomán-Calvo E(2023)Dynamic Partial Order Reduction for Checking Correctness against Transaction Isolation LevelsProceedings of the ACM on Programming Languages10.1145/35912437:PLDI(565-590)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1145/3591243
Dou WCui ZDai QSong JWang DGao YWang WWei JChen LWang HZhong HHuang T(2023)Detecting Isolation Bugs via Transaction Oracle Construction2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00101(1123-1135)Online publication date: May-2023
https://doi.org/10.1109/ICSE48619.2023.00101
Cui ZDou WDai QSong JWang WWei JYe D(2022)Differentially Testing Database Transactions for Fun and ProfitProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering10.1145/3551349.3556924(1-12)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3551349.3556924
Houshmand FSaberlatibari JLesani MJhala RDillig I(2022)Hamband: RDMA replicated data typesProceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation10.1145/3519939.3523426(348-363)Online publication date: 9-Jun-2022
https://dl.acm.org/doi/10.1145/3519939.3523426
Biswas RKakwani DVedurada JEnea CLal A(2021)MonkeyDB: effectively testing correctness under weak isolation levelsProceedings of the ACM on Programming Languages10.1145/34855465:OOPSLA(1-27)Online publication date: 20-Oct-2021
https://dl.acm.org/doi/10.1145/3485546
Lykhenko TSoares RRodrigues LZhang KGherbi AVenkatasubramanian NVeiga L(2021)FaaSTCCProceedings of the 22nd International Middleware Conference10.1145/3464298.3493392(159-171)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.1145/3464298.3493392
Rahmani KNagar KDelaware BJagannathan SFreund SYahav E(2021)Repairing serializability bugs in distributed database programs via automated schema refactoringProceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation10.1145/3453483.3454028(32-47)Online publication date: 19-Jun-2021
https://dl.acm.org/doi/10.1145/3453483.3454028
Tan CZhao CMu SWalfish MLu SHowell J(2020)COBRAProceedings of the 14th USENIX Conference on Operating Systems Design and Implementation10.5555/3488766.3488770(63-80)Online publication date: 4-Nov-2020
https://dl.acm.org/doi/10.5555/3488766.3488770
Seybold DWesner SDomaschka JHung CCerny TShin DBechini A(2020)King louieProceedings of the 35th Annual ACM Symposium on Applied Computing10.1145/3341105.3373968(144-153)Online publication date: 30-Mar-2020
https://dl.acm.org/doi/10.1145/3341105.3373968
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents