research-article

Open access

How Hard Is Weak-Memory Testing?

Authors:

Soham Chakraborty,

Shankara Narayanan Krishna,

Andreas PavlogiannisAuthors Info & Claims

Proceedings of the ACM on Programming Languages, Volume 8, Issue POPL

Article No.: 66, Pages 1978 - 2009

https://doi.org/10.1145/3632908

Published: 05 January 2024 Publication History

Abstract

Weak-memory models are standard formal specifications of concurrency across hardware, programming languages, and distributed systems. A fundamental computational problem is consistency testing: is the observed execution of a concurrent program in alignment with the specification of the underlying system? The problem has been studied extensively across Sequential Consistency (SC) and weak memory, and proven to be NP-complete when some aspect of the input (e.g., number of threads/memory locations) is unbounded. This unboundedness has left a natural question open: are there efficient parameterized algorithms for testing?

The main contribution of this paper is a deep hardness result for consistency testing under many popular weak-memory models: the problem remains NP-complete even in its bounded setting, where candidate executions contain a bounded number of threads, memory locations, and values. This hardness spreads across several Release-Acquire variants of C11, a popular variant of its Relaxed fragment, popular Causal Consistency models, and the POWER architecture. To our knowledge, this is the first result that fully exposes the hardness of weak-memory testing and proves that the problem admits no parameterization under standard input parameters. It also yields a computational separation of these models from SC, x86-TSO, PSO, and Relaxed, for which bounded consistency testing is either known (for SC), or shown here (for the rest), to be in polynomial time.

References

[1]

Parosh Abdulla, Mohamed Faouzi Atig, S. Krishna, Ashutosh Gupta, and Omkar Tuppe. 2023. Optimal Stateless Model Checking for Causal Consistency. In Tools and Algorithms for the Construction and Analysis of Systems, Sriram Sankaranarayanan and Natasha Sharygina (Eds.). Springer Nature Switzerland, Cham. 105–125. isbn:978-3-031-30823-9

[2]

Parosh Aziz Abdulla, Mohamed Faouzi Atig, Bengt Jonsson, Magnus Lång, Tuan Phong Ngo, and Konstantinos Sagonas. 2019. Optimal stateless model checking for reads-from equivalence under sequential consistency. Proc. ACM Program. Lang., 3, OOPSLA (2019), 150:1–150:29. https://doi.org/10.1145/3360576

[3]

Parosh Aziz Abdulla, Mohamed Faouzi Atig, Bengt Jonsson, and Tuan Phong Ngo. 2018. Optimal Stateless Model Checking under the Release-Acquire Semantics. Proc. ACM Program. Lang., 2, OOPSLA (2018), Article 135, oct, 29 pages. https://doi.org/10.1145/3276505

[4]

Sarita V. Adve and Mark D. Hill. 1990. Weak Ordering&Mdash;a New Definition. In ISCA ’90. 2–14. https://doi.org/10.1145/325164.325100

[5]

Pratyush Agarwal, Krishnendu Chatterjee, Shreya Pathak, Andreas Pavlogiannis, and Viktor Toman. 2021. Stateless Model Checking Under a Reads-Value-From Equivalence. In CAV’21. Springer International Publishing, 341–366.

[6]

Mustaque Ahamad, Gil Neiger, James E. Burns, Prince Kohli, and Phillip W. Hutto. 1995. Causal Memory: Definitions, Implementation, and Programming. Distrib. Comput., 9, 1 (1995), mar, 37–49. issn:0178-2770 https://doi.org/10.1007/BF01784241

[7]

Jade Alglave, Will Deacon, Richard Grisenthwaite, Antoine Hacquard, and Luc Maranget. 2021. Armed Cats: Formal Concurrency Modelling at Arm. ACM Trans. Program. Lang. Syst., 43, 2 (2021), Article 8, 54 pages. https://doi.org/10.1145/3458926

[8]

Jade Alglave, Luc Maranget, Susmit Sarkar, and Peter Sewell. 2011. Litmus: Running Tests against Hardware. In Tools and Algorithms for the Construction and Analysis of Systems, Parosh Aziz Abdulla and K. Rustan M. Leino (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 41–44. isbn:978-3-642-19835-9

[9]

Jade Alglave, Luc Maranget, and Michael Tautschnig. 2014. Herding cats: modelling, simulation, testing, and data-mining for weak memory. ACM Trans. Program. Lang. Syst., 36, 2 (2014), 7:1–7:74. https://doi.org/10.1145/2627752

[10]

Mark Batty, Mike Dodds, and Alexey Gotsman. 2013. Library abstraction for C/C++ concurrency. In POPL’13. ACM, 235–248. https://doi.org/10.1145/2429069.2429099

[11]

Ahmed Bouajjani, Constantin Enea, Rachid Guerraoui, and Jad Hamza. 2017. On Verifying Causal Consistency. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL ’17). Association for Computing Machinery, New York, NY, USA. 626–638. isbn:9781450346603 https://doi.org/10.1145/3009837.3009888

[12]

Truc Lam Bui, Krishnendu Chatterjee, Tushar Gautam, Andreas Pavlogiannis, and Viktor Toman. 2021. The Reads-from Equivalence for the TSO and PSO Memory Models. Proc. ACM Program. Lang., https://doi.org/10.1145/3485541

[13]

Sebastian Burckhardt. 2014. Principles of Eventual Consistency. Found. Trends Program. Lang., 1, 1–2 (2014), oct, 1–150. issn:2325-1107 https://doi.org/10.1561/2500000011

[14]

Jason F. Cantin, Mikko H. Lipasti, and James E. Smith. 2005. The Complexity of Verifying Memory Coherence and Consistency. IEEE Trans. Parallel Distrib. Syst., 16, 7 (2005), jul, 663–671. issn:1045-9219 https://doi.org/10.1109/TPDS.2005.86

[15]

Soham Chakraborty, Shankaranarayanan Krishna, Umang Mathur, and Andreas Pavlogiannis. 2023. How Hard is Weak-Memory Testing? arxiv:2311.04302.

[16]

Marek Chalupa, Krishnendu Chatterjee, Andreas Pavlogiannis, Nishant Sinha, and Kapil Vaidya. 2017. Data-Centric Dynamic Partial Order Reduction. Proc. ACM Program. Lang., 2, POPL (2017), Article 31, dec, 30 pages. https://doi.org/10.1145/3158119

[17]

Krishnendu Chatterjee, Andreas Pavlogiannis, and Viktor Toman. 2019. Value-centric dynamic partial order reduction. Proc. ACM Program. Lang., 3, OOPSLA (2019), 124:1–124:29. https://doi.org/10.1145/3360550

[18]

Yunji Chen, Yi Lv, Weiwu Hu, Tianshi Chen, Haihua Shen, Pengyu Wang, and Hong Pan. 2009. Fast complete memory consistency verification. In 2009 IEEE 15th International Symposium on High Performance Computer Architecture. 381–392. https://doi.org/10.1109/HPCA.2009.4798276

[19]

C. J. Fidge. 1988. Timestamps in message-passing systems that preserve the partial ordering. Proceedings of the 11th Australian Computer Science Conference, 10, 1 (1988), 56–66.

[20]

Florian Furbach, Roland Meyer, Klaus Schneider, and Maximilian Senftleben. 2015. Memory-Model-Aware Testing: A Unified Complexity Analysis. ACM Trans. Embed. Comput. Syst., 14, 4 (2015), Article 63, sep, 25 pages. issn:1539-9087 https://doi.org/10.1145/2753761

[21]

Michael R. Garey and David S. Johnson. 1990. Computers and Intractability; A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., USA. isbn:0716710455

[22]

Phillip B. Gibbons and Ephraim Korach. 1994. On Testing Cache-Coherent Shared Memories. In Proceedings of the Sixth Annual ACM Symposium on Parallel Algorithms and Architectures (SPAA ’94). Association for Computing Machinery, New York, NY, USA. 177–188. isbn:0897916719 https://doi.org/10.1145/181014.181328

[23]

Phillip B. Gibbons and Ephraim Korach. 1997. Testing Shared Memories. SIAM J. Comput., 26, 4 (1997), 1208–1244. https://doi.org/10.1137/S0097539794279614

[24]

Alex Gonthmakher, Sergey Polyakov, and Assaf Schuster. 2003. Complexity of Verifying Java Shared Memory Execution. Parallel Processing Letters, 13, 04 (2003), 721–733. https://doi.org/10.1142/S0129626403001628

[25]

Jeff Huang, Patrick O’Neil Meredith, and Grigore Rosu. 2014. Maximal Sound Predictive Race Detection with Control Flow Abstraction. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’14). ACM, New York, NY, USA. 337–348. isbn:978-1-4503-2784-8 https://doi.org/10.1145/2594291.2594315

[26]

P.W. Hutto and M. Ahamad. 1990. Slow memory: weakening consistency to enhance concurrency in distributed shared memories. In Proceedings.,10th International Conference on Distributed Computing Systems. 302–309. https://doi.org/10.1109/ICDCS.1990.89297

[27]

Christian Gram Kalhauge and Jens Palsberg. 2018. Sound Deadlock Prediction. Proc. ACM Program. Lang., 2, OOPSLA (2018), Article 146, oct, 29 pages. https://doi.org/10.1145/3276516

[28]

Dileep Kini, Umang Mathur, and Mahesh Viswanathan. 2017. Dynamic Race Prediction in Linear Time. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2017). ACM, New York, NY, USA. 157–170. isbn:978-1-4503-4988-8 https://doi.org/10.1145/3062341.3062374

[29]

Michalis Kokologiannakis, Ori Lahav, Konstantinos Sagonas, and Viktor Vafeiadis. 2017. Effective Stateless Model Checking for C/C++ Concurrency. Proc. ACM Program. Lang., 2, POPL (2017), https://doi.org/10.1145/3158105

[30]

Michalis Kokologiannakis, Ori Lahav, and Viktor Vafeiadis. 2023. Kater: Automating Weak Memory Model Metatheory and Consistency Checking. Proc. ACM Program. Lang., 7, POPL (2023), Article 19, jan, 29 pages. https://doi.org/10.1145/3571212

[31]

Michalis Kokologiannakis, Iason Marmanis, Vladimir Gladstein, and Viktor Vafeiadis. 2022. Truly Stateless, Optimal Dynamic Partial Order Reduction. Proc. ACM Program. Lang., 6, POPL (2022), https://doi.org/10.1145/3498711

[32]

Michalis Kokologiannakis, Azalea Raad, and Viktor Vafeiadis. 2019. Model Checking for Weakly Consistent Libraries. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2019). Association for Computing Machinery, New York, NY, USA. 96–110. isbn:9781450367127 https://doi.org/10.1145/3314221.3314609

[33]

Ori Lahav and Udi Boker. 2022. What’s Decidable About Causally Consistent Shared Memory? ACM Trans. Program. Lang. Syst., 44, 2 (2022), Article 8, apr, 55 pages. https://doi.org/10.1145/3505273

[34]

Ori Lahav, Nick Giannarakis, and Viktor Vafeiadis. 2016. Taming release-acquire consistency. In POPL’16. ACM, 649–662. https://doi.org/10.1145/2837614.2837643

[35]

Ori Lahav and Roy Margalit. 2019. Robustness against Release/Acquire Semantics. In PLDI 2019. 126–141. https://doi.org/10.1145/3314221.3314604

[36]

Leslie Lamport. 1978. Time, Clocks, and the Ordering of Events in a Distributed System. Commun. ACM, 21, 7 (1978), jul, 558–565. issn:0001-0782 https://doi.org/10.1145/359545.359563

[37]

Sung-Hwan Lee, Minki Cho, Roy Margalit, Chung-Kil Hur, and Ori Lahav. 2023. Putting Weak Memory in Order via a Promising Intermediate Representation. Proc. ACM Program. Lang., 7, PLDI (2023), Article 183, jun, 24 pages. https://doi.org/10.1145/3591297

[38]

Weiyu Luo and Brian Demsky. 2021. C11Tester: A Race Detector for C/C++ Atomics. Association for Computing Machinery, New York, NY, USA. 630–646. https://doi.org/10.1145/3445814.3446711

[39]

C. Manovit and S. Hangal. 2006. Completely verifying memory consistency of test program executions. In The Twelfth International Symposium on High-Performance Computer Architecture, 2006. 166–175. https://doi.org/10.1109/HPCA.2006.1598123

[40]

Roy Margalit and Ori Lahav. 2021. Verifying Observational Robustness against a C11-Style Memory Model. Proc. ACM Program. Lang., 5, POPL (2021), Article 4, 33 pages. https://doi.org/10.1145/3434285

[41]

Umang Mathur, Andreas Pavlogiannis, and Mahesh Viswanathan. 2020. The Complexity of Dynamic Data Race Prediction. LICS ’20. Association for Computing Machinery, New York, NY, USA. 713–727. isbn:9781450371049 https://doi.org/10.1145/3373718.3394783

[42]

Umang Mathur, Andreas Pavlogiannis, and Mahesh Viswanathan. 2021. Optimal Prediction of Synchronization-Preserving Races. Proc. ACM Program. Lang., 5, POPL (2021), Article 36, 29 pages. https://doi.org/10.1145/3434317

[43]

Brian Norris and Brian Demsky. 2013. CDSChecker: Checking Concurrent Data Structures Written with C/C++ Atomics. In OOPSLA’13.

[44]

Scott Owens. 2010. Reasoning about the Implementation of Concurrency Abstractions on x86-TSO. In ECOOP. 478–503.

[45]

Andreas Pavlogiannis. 2019. Fast, Sound, and Effectively Complete Dynamic Race Prediction. Proc. ACM Program. Lang., 4, POPL (2019), Article 17, dec, 29 pages. https://doi.org/10.1145/3371085

[46]

Matthieu Perrin, Achour Mostefaoui, and Claude Jard. 2016. Causal Consistency: Beyond Memory. In Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’16). Association for Computing Machinery, New York, NY, USA. Article 26, 12 pages. isbn:9781450340922 https://doi.org/10.1145/2851141.2851170

[47]

S. Qadeer. 2003. Verifying sequential consistency on shared-memory multiprocessors by model checking. IEEE Transactions on Parallel and Distributed Systems, 14, 8 (2003), 730–741. https://doi.org/10.1109/TPDS.2003.1225053

[48]

Peter Sewell, Susmit Sarkar, Scott Owens, Francesco Zappa Nardelli, and Magnus O. Myreen. 2010. X86-TSO: A Rigorous and Usable Programmer’s Model for X86 Multiprocessors. Commun. ACM, 53, 7 (2010), jul, 89–97. issn:0001-0782 https://doi.org/10.1145/1785414.1785443

[49]

CORPORATE SPARC International, Inc. 1994. The SPARC Architecture Manual (Version 9). Prentice-Hall, Inc., USA. isbn:0130992275

[50]

Hünkar Can Tunç, Parosh Aziz Abdulla, Soham Chakraborty, Shankaranarayanan Krishna, Umang Mathur, and Andreas Pavlogiannis. 2023. Optimal Reads-From Consistency Checking for C11-Style Memory Models. Proc. ACM Program. Lang., 7, PLDI (2023), Article 137, jun, 25 pages. https://doi.org/10.1145/3591251

[51]

John Wickerson, Mark Batty, Tyler Sorensen, and George A. Constantinides. 2017. Automatically Comparing Memory Consistency Models. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL ’17). Association for Computing Machinery, New York, NY, USA. 190–204. isbn:9781450346603 https://doi.org/10.1145/3009837.3009838

[52]

Matt Windsor, Alastair F. Donaldson, and John Wickerson. 2022. High-coverage metamorphic testing of concurrency support in C compilers. Software Testing, Verification and Reliability, e1812.

Cited By

Tunç HDeshmukh ACirisci BEnea CPavlogiannis ATsafrir DMusuvathi MGupta RAbu-Ghazaleh N(2024)CSSTs: A Dynamic Data Structure for Partial Orders in Concurrent Execution AnalysisProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3620666.3651358(223-238)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3620666.3651358

Index Terms

How Hard Is Weak-Memory Testing?
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
2. Theory of computation
  1. Semantics and reasoning
    1. Program reasoning
      1. Program analysis
  2. Theory and algorithms for application domains

Recommendations

Optimal Reads-From Consistency Checking for C11-Style Memory Models

Over the years, several memory models have been proposed to capture the subtle concurrency semantics of C/C++. One of the most fundamental problems associated with a memory model M is consistency checking: given an execution X, is X consistent with M? ...
Verifying read-copy-update in a logic for weak memory
PLDI '15

Read-Copy-Update (RCU) is a technique for letting multiple readers safely access a data structure while a writer concurrently modifies it. It is used heavily in the Linux kernel in situations where fast reads are important and writes are infrequent. ...
Verifying read-copy-update in a logic for weak memory
PLDI '15: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation

Read-Copy-Update (RCU) is a technique for letting multiple readers safely access a data structure while a writer concurrently modifies it. It is used heavily in the Linux kernel in situations where fast reads are important and writes are infrequent. ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages

Proceedings of the ACM on Programming Languages Volume 8, Issue POPL

January 2024

2820 pages

EISSN:2475-1421

DOI:10.1145/3554315

Editor:
Michael Hicks
Amazon, USA

Issue’s Table of Contents

Copyright © 2024 Owner/Author.

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 January 2024

Published in PACMPL Volume 8, Issue POPL

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
221
Total Downloads

Downloads (Last 12 months)221
Downloads (Last 6 weeks)29

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tunç HDeshmukh ACirisci BEnea CPavlogiannis ATsafrir DMusuvathi MGupta RAbu-Ghazaleh N(2024)CSSTs: A Dynamic Data Structure for Partial Orders in Concurrent Execution AnalysisProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3620666.3651358(223-238)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3620666.3651358

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents