research-article

Minimizing Test Suites with Unfoldings of Multithreaded Programs

Authors:

Olli Saarikivi,

Hernán Ponce-De-León,

Kari Kähkönen,

Keijo Heljanko,

Javier EsparzaAuthors Info & Claims

ACM Transactions on Embedded Computing Systems (TECS), Volume 16, Issue 2

Article No.: 45, Pages 1 - 24

https://doi.org/10.1145/3012281

Published: 03 February 2017 Publication History

Abstract

This article focuses on computing minimal test suites for multithreaded programs. Based on previous work on test case generation for multithreaded programs using unfoldings, this article shows how this unfolding can be used to generate minimal test suites covering all local states of the program. Generating such minimal test suites is shown to be NP-complete in the size of the unfolding. We propose an SMT encoding for this problem and two methods based on heuristics which only approximate the solution, but scale better in practice. Finally, we apply our methods to compute the minimal test suites for several benchmarks.

References

[1]

Parosh Aziz Abdulla, Stavros Aronis, Mohamed Faouzi Atig, Bengt Jonsson, Carl Leonardsson, and Konstantinos F. Sagonas. 2015. Stateless model checking for TSO and PSO. In Proceedings of the 21st International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS 2015). Lecture Notes in Computer Science, Vol. 9035. Springer, 353--367.

Digital Library

[2]

Parosh Aziz Abdulla, Stavros Aronis, Bengt Jonsson, and Konstantinos F. Sagonas. 2014. Optimal dynamic partial order reduction. In Proceedings of the 41st Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’14). ACM, 373--384.

Digital Library

[3]

Parosh Aziz Abdulla, Mohamed Faouzi Atig, Bengt Jonsson, and Carl Leonardsson. 2016. Stateless model checking for POWER. In Proceedings of the 28th International Conference on Computer Aided Verification (CAV’16). Lecture Notes in Computer Science, Vol. 9780. Springer, 134--156.

[4]

Paolo Baldan, Andrea Corradini, and Ugo Montanari. 2001. Contextual petri nets, asymmetric event structures, and processes. Inf. Comput. 171, 1 (2001), 1--49.

Digital Library

[5]

Sébastien Bornot, Rémi Morin, Peter Niebert, and Sarah Zennou. 2002. Black box unfolding with local first search. In Proceedings of the 8th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’02). Lecture Notes in Computer Science, Vol. 2280. Springer, 386--400.

Digital Library

[6]

Leonardo Mendonça de Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In Proceedings of the 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS ’08). Lecture Notes in Computer Science, Vol. 4963. Springer, 337--340.

Digital Library

[7]

Volker Diekert and Grzegorz Rozenberg (Eds.). 1995. The Book of Traces. World Scientific Publishing Co., Inc.

Digital Library

[8]

Javier Esparza and Keijo Heljanko. 2008. Unfoldings—Partial-Order Approach to Model Checking. Springer.

Digital Library

[9]

Azadeh Farzan, Andreas Holzer, Niloofar Razavi, and Helmut Veith. 2013. Con2colic testing. In Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE’13). ACM, 37--47.

Digital Library

[10]

Azadeh Farzan and P. Madhusudan. 2006. Causal atomicity. In Proceedings of the 18th International Conference on Computer Aided Verification (CAV’06). Lecture Notes in Computer Science, Vol. 4144. Springer, 315--328.

Digital Library

[11]

Cormac Flanagan and Patrice Godefroid. 2005. Dynamic partial-order reduction for model checking software. In Proceedings of the 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’05). ACM, 110--121.

Digital Library

[12]

Patrice Godefroid. 1996. Partial-Order Methods for the Verification of Concurrent Systems—An Approach to the State-Explosion Problem. Lecture Notes in Computer Science, Vol. 1032. Springer.

Digital Library

[13]

Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed automated random testing. In Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation. ACM, 213--223.

Digital Library

[14]

Kari Kähkönen. 2015. Automated Systematic Testing Methods for Multithreaded Programs. Doctoral Dissertation. School of Science, Aalto University.

[15]

Kari Kähkönen and Keijo Heljanko. 2014a. Lightweight state capturing for automated testing of multithreaded programs. In Proceedings of the 8th International Conference on Tests and Proofs (TAP’14). Lecture Notes in Computer Science, Vol. 8570. Springer, 187--203.

[16]

Kari Kähkönen and Keijo Heljanko. 2014b. Testing multithreaded programs with contextual unfoldings and dynamic symbolic execution. In Proceedings of the 14th International Conference on Application of Concurrency to System Design (ACSD’14). IEEE Computer Society, 142--151.

Digital Library

[17]

Kari Kähkönen, Olli Saarikivi, and Keijo Heljanko. 2012. Using unfoldings in automated testing of multithreaded programs. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE’12). ACM, 150--159.

Digital Library

[18]

Victor Khomenko, Alex Kondratyev, Maciej Koutny, and Walter Vogler. 2006. Merged processes: A new condensed representation of petri net behaviour. Acta Inf. 43, 5 (2006), 307--330.

Digital Library

[19]

Kenneth L. McMillan. 1995. A technique of state space search based on unfolding. Formal Methods Syst. Des. 6, 1 (1995), 45--65.

Digital Library

[20]

Ugo Montanari and Francesca Rossi. 1995. Contextual nets. Acta Inf. 32, 6 (1995), 545--596.

Digital Library

[21]

Peter Niebert, Michaela Huhn, Sarah Zennou, and Denis Lugiez. 2001. Local first search - a new paradigm for partial order reductions. In Proceedings of the 12th International Conference on Concurrency Theory (CONCUR’01). Lecture Notes in Computer Science, Vol. 2154. Springer, 396--410.

Digital Library

[22]

Mogens Nielsen, Gordon D. Plotkin, and Glynn Winskel. 1981. Petri nets, event structures and domains, part i. Theor. Comput. Sci. 13 (1981), 85--108.

[23]

Sampo Niskanen and Patric Östergård. 2002. Cliquer - routines for clique searching. Retrieved from http://users.aalto.fi/&sim;pat/cliquer.html.

[24]

Patric Östergård. 2002. A fast algorithm for the maximum clique problem. Discr. Appl. Math. 120, 1--3 (2002), 197--207.

Digital Library

[25]

Hernán Ponce de León and Andrey Mokhov. 2015. Building bridges between sets of partial orders. In Proceedings of the 9th International Conference on Language and Automata Theory and Applications (LATA’15). Lecture Notes in Computer Science, Vol. 8977. Springer, 145--160.

[26]

Hernán Ponce de León, Olli Saarikivi, Kari Kähkönen, Keijo Heljanko, and Javier Esparza. 2015. Unfolding based minimal test suites for testing multithreaded programs. In Proceedings of the 15th International Conference on Application of Concurrency to System Design (ACSD’15). IEEE Computer Society, 40--49.

Digital Library

[27]

César Rodríguez. 2013. Verification Based on Unfoldings of Petri Nets with Read Arcs. Thèse de doctorat. Laboratoire Spécification et Vérification, ENS Cachan, France.

[28]

César Rodríguez, Marcelo Sousa, Subodh Sharma, and Daniel Kroening. 2015. Unfolding-based partial order reduction. In Proceedings of the 26th International Conference on Concurrency Theory (CONCUR’15). 456--469.

[29]

Koushik Sen. 2006. Scalable Automated Methods for Dynamic Program Analysis. Ph.D. Thesis. University of Illinois.

Digital Library

[30]

Antti Valmari. 1996. The state explosion problem. In Lectures on Petri Nets I: Basic Models, Advances in Petri Nets, the Volumes Are Based on the Advanced Course on Petri Nets. 429--528.

Digital Library

Cited By

Yang XYe CChen Y(2023)Depth-First Net Unfoldings and Equivalent ReductionSymmetry10.3390/sym1509177515:9(1775)Online publication date: 16-Sep-2023
https://doi.org/10.3390/sym15091775
Xiang DLin SWang XLiu G(2023)Checking Missing-Data Errors in Cyber-Physical Systems Based on the Merged Process of Petri NetsIEEE Transactions on Industrial Informatics10.1109/TII.2022.318166919:3(3047-3056)Online publication date: Mar-2023
https://doi.org/10.1109/TII.2022.3181669
Xiang DTao XLiu Y(2021)An Incremental and Backward-Conflict Guided Method for Unfolding Petri NetsSymmetry10.3390/sym1303039213:3(392)Online publication date: 28-Feb-2021
https://doi.org/10.3390/sym13030392
Show More Cited By

Index Terms

Minimizing Test Suites with Unfoldings of Multithreaded Programs
1. Software and its engineering
  1. Software creation and management
    1. Search-based software engineering
    2. Software verification and validation
2. Theory of computation
  1. Computational complexity and cryptography
    1. Proof complexity
  2. Models of computation
    1. Concurrency

Recommendations

Unfolding Based Minimal Test Suites for Testing Multithreaded Programs
ACSD '15: Proceedings of the 2015 15th International Conference on Application of Concurrency to System Design

This paper focuses on the problem of computing the minimal test suite for a terminating multithreaded program that covers all its executable statements. We have in previous work shown how to use unfoldings to capture the true concurrency semantics of ...
Prioritizing random combinatorial test suites
SAC '17: Proceedings of the Symposium on Applied Computing

The behaviour of a system under test can be influenced by several factors, such as system configurations, user inputs, and so on. It has also been observed that many failures are caused by only a small number of factors. Combinatorial testing aims at ...
Adaptive random prioritization for interaction test suites
SAC '14: Proceedings of the 29th Annual ACM Symposium on Applied Computing

Combinatorial interaction testing (CIT), a black-box testing method, has been well studied in recent years. It aims at constructing an effective interaction test suites, so as to identify the faults that are caused by interactions among parameters. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Embedded Computing Systems

ACM Transactions on Embedded Computing Systems Volume 16, Issue 2

Special Issue on LCETES 2015, Special Issue on ACSD 2015 and Special Issue on Embedded Devise Forensics and Security

May 2017

705 pages

ISSN:1539-9087

EISSN:1558-3465

DOI:10.1145/3025020

Editor:
Sandeep K. Shukla
Indian Institute of Technology, India

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 03 February 2017

Accepted: 01 October 2016

Revised: 01 September 2016

Received: 01 February 2016

Published in TECS Volume 16, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
145
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang XYe CChen Y(2023)Depth-First Net Unfoldings and Equivalent ReductionSymmetry10.3390/sym1509177515:9(1775)Online publication date: 16-Sep-2023
https://doi.org/10.3390/sym15091775
Xiang DLin SWang XLiu G(2023)Checking Missing-Data Errors in Cyber-Physical Systems Based on the Merged Process of Petri NetsIEEE Transactions on Industrial Informatics10.1109/TII.2022.318166919:3(3047-3056)Online publication date: Mar-2023
https://doi.org/10.1109/TII.2022.3181669
Xiang DTao XLiu Y(2021)An Incremental and Backward-Conflict Guided Method for Unfolding Petri NetsSymmetry10.3390/sym1303039213:3(392)Online publication date: 28-Feb-2021
https://doi.org/10.3390/sym13030392
Xiang DZhao FLiu Y(2021)DICER 2.0: A New Model Checker for Data-Flow Errors of Concurrent Software SystemsMathematics10.3390/math90909669:9(966)Online publication date: 25-Apr-2021
https://doi.org/10.3390/math9090966
Lu FTao RDu YZeng QBao Y(2019)Deadlock Detection-oriented Unfolding of Unbounded Petri NetsInformation Sciences10.1016/j.ins.2019.05.021Online publication date: May-2019
https://doi.org/10.1016/j.ins.2019.05.021
Tao RLu FZhang XZhu GBen-Othman JYu HUnger HArai M(2018)An introduction and review of petri net unfolding technologyProceedings of the 4th International Conference on Communication and Information Processing10.1145/3290420.3290430(102-106)Online publication date: 2-Nov-2018
https://dl.acm.org/doi/10.1145/3290420.3290430
Ponce De León HMokhov A(2018)Compact and efficiently verifiable models for concurrent systemsFormal Methods in System Design10.1007/s10703-018-0316-053:3(407-431)Online publication date: 1-Dec-2018
https://dl.acm.org/doi/10.1007/s10703-018-0316-0

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents