tutorial

Improving developers' confidence in test results of multi-threaded systems: avoiding early and late assertions

Author:

OOPSLA Companion '08: Companion to the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applications

Pages 899 - 900

https://doi.org/10.1145/1449814.1449898

Published: 19 October 2008 Publication History

Get Access

Abstract

Testing multi-threaded systems is challenging, especially due to their inherent non-determinism. As a result, considering the same test, we can have success and failure results for different test executions. This may happen due to bugs that are only revealed on certain thread interleavings, and which are difficult to reproduce. However, this may also be caused by a problem with the test. When the latter case happens, developers may waste a lot of time searching a bug that does not exist, or they may lose trust in the test, maybe not investigating real bugs. With this work, we want to support tests development through an approach to avoid test failures caused by a common problem with multi-threaded systems testing: assertions performed too early or too late. The approach's basic idea is to use thread monitoring and other techniques to avoid such problems during the development of tests and hence increase the developers' confidence in test results.

References

[1]

Shady Copty and Shmuel Ur. Multi-threaded Testing with AOP is Easy, and It Finds Bugs. Proc. 11th International Euro-Par Conference, LNCS, 3648:740--749, 2005.

Digital Library

Google Scholar

[2]

Ayla Dantas, Francisco Brasileiro, and Walfredo Cirne. Improving Automated Testing of Multithreaded Software. ICST, 0:521--524, 2008.

Digital Library

Google Scholar

[3]

B. Goetz and T. Peierls. Java concurrency in practice. Addison-Wesley, 2006.

Digital Library

Google Scholar

[4]

M.J. Harrold. Testing: a roadmap. Proceedings of the Conference on The Future of Software Engineering, pages 61--72, 2000.

Digital Library

Google Scholar

[5]

Gregor Kiczales, John Lamping, Anurag Mendhekar, ChrisMaeda, Cristina Videira Lopes, Jean-Marc Loingtier, and John Irwin. Aspect-Oriented Programming. In European Conference on Object-Oriented Programming, ECOOP'97, LNCS 1241, pages 220--242, Finland, June 1997. Springer-Verlag.

Google Scholar

[6]

Doug Lea. Concurrent Programming in Java (tm): Design Principles and Patterns. Addison-Wesley, 2000.

Digital Library

Google Scholar

[7]

S.MacDonald, J. Chen, and D. Novillo. Choosing Among Alternative Futures. Proc. Haifa Verification Conference, LNCS, 3875: 247--264, 2005.

Digital Library

Google Scholar

[8]

G.Meszaros. XUnit Test Patterns: Refactoring Test Code. Addison-Wesley, 2007.

Digital Library

Google Scholar

[9]

Marcelo Iury S. Oliveira. OurBackup: Uma Solução P2P de Backup Baseada em Redes Sociais. Master's thesis, Universidade Federal de Campina Grande, 2007.

Google Scholar

[10]

Scott D. Stoller. Testing concurrent Java programs using randomized scheduling. In Proc. Second Workshop on Runtime Verification (RV), volume 70(4) of Electronic Notes in Theoretical Computer Science. Elsevier, July 2002.

Google Scholar

[11]

H. Sutter and J. Larus. Software and the concurrency revolution. Queue, 3(7):54--62, 2005.

Digital Library

Google Scholar

Cited By

View all

Souza SSouza PBrito MSimao AZaluska E(2015)Empirical evaluation of a new composite approach to the coverage criteria and reachability testing of concurrent programsSoftware Testing, Verification & Reliability10.1002/stvr.156825:3(310-332)Online publication date: 1-May-2015
https://dl.acm.org/doi/10.1002/stvr.1568
Souza PSouza SZaluska E(2012)Structural testing for message‐passing concurrent programs: an extended test modelConcurrency and Computation: Practice and Experience10.1002/cpe.293726:1(21-50)Online publication date: 10-Oct-2012
https://doi.org/10.1002/cpe.2937

Index Terms

Improving developers' confidence in test results of multi-threaded systems: avoiding early and late assertions
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Improving Automated Testing of Multi-threaded Software
ICST '08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation

This paper discusses an approach to avoid incorrect results in the execution of automatic tests of multi-threaded systems. We argue that such incorrect results have two main sources. First, it is typically difficult to determine when all threads have ...
Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction

Software testing is a critical part of software development. As new test cases are generated over time due to software modifications, test suite sizes may grow significantly. Because of time and resource constraints for testing, test suite minimization ...
How (much) do developers test?
ICSE '15: Proceedings of the 37th International Conference on Software Engineering - Volume 2

What do we know about software testing in the real world? It seems we know from Fred Brooks' seminal work "The Mythical Man-Month" that 50% of project effort is spent on testing. However, due to the enormous advances in software engineering in the past ...

Comments

Information & Contributors

Information

Published In

OOPSLA Companion '08: Companion to the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applications

October 2008

306 pages

ISBN:9781605582207

DOI:10.1145/1449814

General Chair:
Gail E. Harris
Instantiated Software Inc.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial

Conference

OOPSLA08

Sponsor:

OOPSLA08: ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications

October 19 - 23, 2008

TN, Nashville, USA

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
109
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Souza SSouza PBrito MSimao AZaluska E(2015)Empirical evaluation of a new composite approach to the coverage criteria and reachability testing of concurrent programsSoftware Testing, Verification & Reliability10.1002/stvr.156825:3(310-332)Online publication date: 1-May-2015
https://dl.acm.org/doi/10.1002/stvr.1568
Souza PSouza SZaluska E(2012)Structural testing for message‐passing concurrent programs: an extended test modelConcurrency and Computation: Practice and Experience10.1002/cpe.293726:1(21-50)Online publication date: 10-Oct-2012
https://doi.org/10.1002/cpe.2937

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Improving Automated Testing of Multi-threaded Software

Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction

How (much) do developers test?

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Upcoming Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Improving Automated Testing of Multi-threaded Software

Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction

How (much) do developers test?

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations