Article

Applications of synchronization coverage

Authors:

Shmuel UrAuthors Info & Claims

PPoPP '05: Proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming

Pages 206 - 212

https://doi.org/10.1145/1065944.1065972

Published: 15 June 2005 Publication History

Abstract

Coverage analysis is a useful testing technology. However, some coverage models are more acceptable to the industry than others. In the field of testing multi-threaded applications, there is a need for a coverage model that can be used to evaluate tests for concurrent completeness and to find new testing requirements. We present a new coverage model: synchronization coverage. This model is simple to understand and the action items generated by each uncovered task are clear to testers and developers. We propose that synchronization coverage could, and should, become one of the more commonly used coverage models.

References

[1]

Software test and evaluation guidelines. Department of the Army, Pamphlet 73--7.]]

[2]

Beizer. Software Testing Techniques. Van Nostrand Reinhold, New York, 1990.]]

Digital Library

[3]

F. P. Brooks. The Mythical Man-Month: Essays on Software Engineering. Addison Wesley, 1995.]]

Digital Library

[4]

E. Buchnik and S. Ur. Compacting regression-suites on-the-fly. In Proceedings of the 4th Asia Pacific Software Engineering Conference, December 1997.]]

Digital Library

[5]

H. Chockler, O. Kupferman, and M.Y. Vardi. Coverage metrics for formal verification. In 12th Advanced Research Working Conference on Correct Hardware Design and Verification Methods, volume 2860 of Lecture Notes in Computer Science, pages 111--125. Springer-Verlag, 2003.]]

[6]

S. Cornett. Software test coverage analysis,. http://www.bullseye.com/webCoverage.html.]]

[7]

O. Edelstein, E. Farchi, Y. Nir G Ratzaby, and S Ur. Multithreaded java program test generation. IBM Systems Journal, 41(3):111--125, 2002.]]

Digital Library

[8]

M. Factor, E. Farchi, Y. Lichtenstein, and Y. Malka. Testing concurrent programs: a formal evaluation of coverage criteria. In Proceedings of the Seventh Israeli Conference on Computer Systems and Software Engineering, pages 119 -- 126, June 1996.]]

Digital Library

[9]

Eitan Farchi, Yarden Nir, and Shmuel Ur. Concurrent bug patterns and how to test them. In PADTAD II in the 17th International Parallel and Distributed Processing Symposium (IPDPS'03), April 2003.]]

Digital Library

[10]

Amiram Hayardeny, Shachar Fienblit, and Eitan Farchi. Concurrent and distributed desk checking. In 18th International Parallel and Distributed Processing Symposium (IPDPS'04) - Workshop 16, April 2004.]]

[11]

C. Kaner. Software negligence and testing coverage. In proceedings of STAR 96: the Fifth International Conference, Software Testing, Analysis and Review, pages 299--327, June 1996.]]

[12]

Oded Lachish, Eitan Marcus, Shmuel Ur, and Avi Ziv. Hole analysis for functional coverage data. In Proceedings of the 39th conference on Design automation, pages 807--812. ACM Press, 2002.]]

Digital Library

[13]

B. Marick. The Craft of Software Testing, Subsystem testing Including Object-Based and Object-Oriented Testing. Prentice-Hall, 1985.]]

Digital Library

[14]

A. Mathur and W. Wong. Comparing the fault detection effectiveness of mutation and data flow testing: An empirical study, 1993.]]

[15]

G. Ratzaby, S. Ur, and Y. Wolfsthal. Coverability analysis using symbolic model checking. Submitted to Charme 2001.]]

Digital Library

[16]

R. Stewart. Unit test coverage as leading indicator of rework. In proceedings of EuroSTAR 97, NOvember 1997.]]

[17]

E. J. Weyuker. The evaluation of program-based software test data adequacy criteria. Commun. ACM, 31(6):668--675, 1988.]]

Digital Library

[18]

Cheer-Sun D. Yang, Amie L. Souter, and Lori L. Pollock. All-du-path coverage for parallel programs. In Proceedings of the 1998 ACM SIGSOFT international symposium on Software testing and analysis, pages 153--162. ACM Press, 1998.]]

Digital Library

Cited By

Stoica BLu SMusuvathi MNath SFedorova ANarayanan DDi Luna GQuerzoni L(2023)WAFFLE: Exposing Memory Ordering Bugs Efficiently with Active Delay InjectionProceedings of the Eighteenth European Conference on Computer Systems10.1145/3552326.3567507(111-126)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3552326.3567507
Elyasaf AFarchi EMargalit OWeiss GWeiss Y(2023)Generalized Coverage Criteria for Combinatorial Sequence TestingIEEE Transactions on Software Engineering10.1109/TSE.2023.3279570(1-12)Online publication date: 2023
https://doi.org/10.1109/TSE.2023.3279570
Yuan MLee YZhang CLi YCai YZhao BCadar CZhang X(2021)RAProducer: efficiently diagnose and reproduce data race bugs for binaries via trace analysisProceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3460319.3464831(593-606)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3460319.3464831
Show More Cited By

Index Terms

Applications of synchronization coverage
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

State coverage: a structural test adequacy criterion for behavior checking
ESEC-FSE '07: Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering

We propose a new language-independent, structural test adequacy criterion called state coverage. State coverage measures whether unit-level tests check the outputs and sideeffects of a program.

State coverage differs in several respects from existing ...
State coverage: a structural test adequacy criterion for behavior checking
ESEC-FSE companion '07: The 6th Joint Meeting on European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering: companion papers

We propose a new language-independent, structural test adequacy criterion called state coverage. State coverage measures whether unit-level tests check the outputs and side effects of a program.

State coverage differs in several respects from existing ...
Exploring output-based coverage for testing PHP web applications

In software testing, different testers focus on different aspects of the software such as functionality, performance, design, and other attributes. While many tools and coverage metrics exist to support testers at the code level, not much support is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PPoPP '05: Proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming

June 2005

310 pages

ISBN:1595930809

DOI:10.1145/1065944

General Chair:
Keshav Pingali
Cornell University
,
Program Chairs:
Katherine Yelick
University of California, Berkeley and LBNL
,
Andrew Grimshaw
University of Virginia

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 June 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

PPoPP05

Sponsor:

PPoPP05: ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming 2005

June 15 - 17, 2005

IL, Chicago, USA

Acceptance Rates

Overall Acceptance Rate 230 of 1,014 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

73
Total Citations
View Citations
911
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Stoica BLu SMusuvathi MNath SFedorova ANarayanan DDi Luna GQuerzoni L(2023)WAFFLE: Exposing Memory Ordering Bugs Efficiently with Active Delay InjectionProceedings of the Eighteenth European Conference on Computer Systems10.1145/3552326.3567507(111-126)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3552326.3567507
Elyasaf AFarchi EMargalit OWeiss GWeiss Y(2023)Generalized Coverage Criteria for Combinatorial Sequence TestingIEEE Transactions on Software Engineering10.1109/TSE.2023.3279570(1-12)Online publication date: 2023
https://doi.org/10.1109/TSE.2023.3279570
Yuan MLee YZhang CLi YCai YZhao BCadar CZhang X(2021)RAProducer: efficiently diagnose and reproduce data race bugs for binaries via trace analysisProceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3460319.3464831(593-606)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3460319.3464831
Taheri SGopalakrishnan G(2021)GoAT: Automated Concurrency Analysis and Debugging Tool for Go2021 IEEE International Symposium on Workload Characterization (IISWC)10.1109/IISWC53511.2021.00023(138-150)Online publication date: Nov-2021
https://doi.org/10.1109/IISWC53511.2021.00023
Shirole MKumar R(2021)Concurrency coverage criteria for activity diagramsIET Software10.1049/sfw2.1200915:1(43-54)Online publication date: 19-Jan-2021
https://doi.org/10.1049/sfw2.12009
Chen DJiang YXu CMa X(2021)On interleaving space exploration of multi-threaded programsFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-020-9501-615:4Online publication date: 1-Aug-2021
https://dl.acm.org/doi/10.1007/s11704-020-9501-6
Shirole MKumar R(2021)Concurrent behavioral coverage criteria for sequence diagramsInnovations in Systems and Software Engineering10.1007/s11334-021-00413-719:2(157-176)Online publication date: 25-Nov-2021
https://doi.org/10.1007/s11334-021-00413-7
Guo JLi ZShi CZhao R(2020)Thread Scheduling Sequence Generation Based on All Synchronization Pair Coverage CriteriaInternational Journal of Software Engineering and Knowledge Engineering10.1142/S021819402050005930:01(97-118)Online publication date: 27-Feb-2020
https://doi.org/10.1142/S0218194020500059
Chin KTsay RWu H(2020)A Heuristic Region-based Concurrency Bug Testing Approach2020 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom)10.1109/ISPA-BDCloud-SocialCom-SustainCom51426.2020.00168(1126-1135)Online publication date: Dec-2020
https://doi.org/10.1109/ISPA-BDCloud-SocialCom-SustainCom51426.2020.00168
Abrecht SAkila MGannamaneni SGroh KHeinzemann CHouben SWoehrle M(2020)Revisiting Neuron Coverage and Its Application to Test GenerationComputer Safety, Reliability, and Security. SAFECOMP 2020 Workshops10.1007/978-3-030-55583-2_21(289-301)Online publication date: 8-Sep-2020
https://doi.org/10.1007/978-3-030-55583-2_21
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents