research-article

Change-aware preemption prioritization

Authors:

Vilas Jagannath,

Darko MarinovAuthors Info & Claims

ISSTA '11: Proceedings of the 2011 International Symposium on Software Testing and Analysis

Pages 133 - 143

https://doi.org/10.1145/2001420.2001437

Published: 17 July 2011 Publication History

Abstract

Successful software evolves as developers add more features, respond to requirements changes, and fix faults. Regression testing is widely used for ensuring the validity of evolving software. As regression test suites grow over time, it becomes expensive to execute them. The problem is exacerbated when test suites contain multithreaded tests. These tests are generally long running as they explore many different thread schedules searching for concurrency faults such as dataraces, atomicity violations, and deadlocks. While many techniques have been proposed for regression test prioritization, selection, and minimization for sequential tests, there is not much work for multithreaded code.

We present a novel technique, called Change-Aware Preemption Prioritization (CAPP), that uses information about the changes in software evolution to prioritize the exploration of schedules in a multithreaded regression test. We have implemented CAPP in two frameworks for systematic exploration of multithreaded Java code. We evaluated CAPP on the detection of 15 faults in multithreaded Java programs, including large open-source programs. The results show that CAPP can substantially reduce the exploration required to detect multithreaded regression faults.

References

[1]

Apache Software Foundation. DIRMINA-803. https://issues.apache.org/jira/browse/DIRMINA-803.

[2]

Apache Software Foundation. LANG-481. https://issues.apache.org/jira/browse/LANG-481.

[3]

Apache Software Foundation. POOL-107. https://issues.apache.org/jira/browse/POOL-107.

[4]

Apache Software Foundation. POOL-120. https://issues.apache.org/jira/browse/POOL-120.

[5]

Apache Software Foundation. POOL-146. https://issues.apache.org/jira/browse/POOL-146.

[6]

A. Arcuri and L. Briand. A practical guide for using statistical tests to assess randomized algorithms in software engineering. In To appear ICSE, 2011.

Digital Library

[7]

T. Ball, S. Burckhardt, K. Coons, M. Musuvathi, and S. Qadeer. Preemption sealing for efficient concurrency testing. In TACAS, 2010.

Digital Library

[8]

S. Burckhardt, C. Dern, M. Musuvathi, and R. Tan. Line-Up: A complete and automatic linearizability checker. In PLDI, 2010.

Digital Library

[9]

C. L. Conway, K. S. Namjoshi, D. Dams, and S. A. Edwards. Incremental algorithms for inter-procedural analysis of safety properties. In CAV, 2005.

Digital Library

[10]

K. Coons, S. Burckhardt, and M. Musuvathi. Gambit: Effective unit testing for concurrency libraries. In PPoPP, 2010.

Digital Library

[11]

H. Do, S. G. Elbaum, and G. Rothermel. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. ESE, 2005.

Digital Library

[12]

M. B. Dwyer, S. G. Elbaum, S. Person, and R. Purandare. Parallel randomized state-space search. In ICSE, 2007.

Digital Library

[13]

M. B. Dwyer, S. Person, and S. G. Elbaum. Controlling factors in evaluating path-sensitive error detection techniques. In FSE, 2006.

Digital Library

[14]

S. Elbaum, A. G. Malishevsky, and G. Rothermel. Test case prioritization: A family of empirical studies. IEEE TSE, 2002.

Digital Library

[15]

C. Flanagan and S. N. Freund. FastTrack: efficient and precise dynamic race detection. In PLDI, 2009.

Digital Library

[16]

M. Gligoric, V. Jagannath, and D. Marinov. MuTMuT: Efficient exploration for mutation testing of multithreaded code. In ICST, 2010.

Digital Library

[17]

M. J. Harrold, J. A. Jones, T. Li, D. Liang, and A. Gujarathi. Regression test selection for Java software. In OOPSLA, 2001.

Digital Library

[18]

T. A. Henzinger, R. Jhala, R. Majumdar, and M. A. A. Sanvido. Extreme model checking. In Verification: Theory and Practice, 2003.

[19]

H.-Y. Hsu and A. Orso. MINTS: A general framework and tool for supporting test-suite minimization. In ICSE, 2009.

Digital Library

[20]

J. A. Jones and M. J. Harrold. Test-suite reduction and prioritization for modified condition/decision coverage. In ICSM, 2001.

Digital Library

[21]

P. Joshi, M. Naik, and K. Sen. An effective dynamic analysis for detecting generalized deadlocks. In FSE, 2010.

Digital Library

[22]

JPF home page. http://babelfish.arc.nasa.gov/trac/jpf/.

[23]

S. Lauterburg, A. Sobeih, D. Marinov, and M. Viswanathan. Incremental state-space exploration for programs with dynamically allocated data. In ICSE, 2008.

Digital Library

[24]

J. Makowsky and E. Rawe. Incremental model checking for fixed point properties on decomposable structures. In MFCS, 1995.

Digital Library

[25]

M. Musuvathi and S. Qadeer. Iterative context bounding for systematic testing of multithreaded programs. In PLDI, 2007.

Digital Library

[26]

A. Orso, T. Apiwattanapong, and M. J. Harrold. Leveraging field data for impact analysis and regression testing. In ESEC/FSE, 2003.

Digital Library

[27]

X. Ren, F. Shah, F. Tip, B. Ryder, and O. Chesley. Chianti: A tool for change impact analysis of Java programs. In OOPSLA, 2004.

Digital Library

[28]

G. Rothermel and M. J. Harrold. A safe, efficient regression test selection technique. TOSEM, 1997.

Digital Library

[29]

A. Rountev. Precise identification of side-effect-free methods in Java. In ICSM, 2004.

Digital Library

[30]

D. Saff and M. D. Ernst. An experimental evaluation of continuous testing during development. In ISSTA, 2004.

Digital Library

[31]

O. Sokolsky and S. A. Smolka. Incremental model checking in the modal mu-calculus. In CAV, 1994.

Digital Library

[32]

SpringSource and Groovy Community. GROOVY-1890. http://jira.codehaus.org/browse/GROOVY-1890.

[33]

A. Srivastava and J. Thiagarajan. Effectively prioritizing tests in development environment. In ISSTA, 2002.

Digital Library

[34]

The Eclipse Foundation. Eclipse JDT UI. http://www.eclipse.org/jdt/ui/.

[35]

University of Nebraska Lincoln. Software-artifact Infrastructure Repository. http://sir.unl.edu.

[36]

W. Visser, K. Havelund, G. P. Brat, S. Park, and F. Lerda. Model checking programs. J-ASE, 2003.

Digital Library

[37]

G. H. Xu and A. Rountev. Regression test selection for AspectJ software. In ICSE, 2007.

Digital Library

[38]

G. Yang, M. B. Dwyer, and G. Rothermel. Regression model checking. In ICSM, 2009.

[39]

S. Yoo and M. Harman. Regression testing minimization, selection and prioritization: A survey. STVR, 2010.

Digital Library

[40]

L. Zhang, S.-S. Hou, C. Guo, T. Xie, and H. Mei. Time-aware test-case prioritization using integer linear programming. In ISSTA, 2009.

Digital Library

Cited By

Yu THuang ZWang C(2020) ConTesa : Directed Test Suite Augmentation for Concurrent Software IEEE Transactions on Software Engineering10.1109/TSE.2018.286139246:4(405-419)Online publication date: 1-Apr-2020
https://doi.org/10.1109/TSE.2018.2861392
Sung CLahiri SEnea CWang CHuchard MKästner CFraser G(2018)Datalog-based scalable semantic diffing of concurrent programsProceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering10.1145/3238147.3238211(656-666)Online publication date: 3-Sep-2018
https://dl.acm.org/doi/10.1145/3238147.3238211
Bianchi FMargara APezze M(2018)A Survey of Recent Trends in Testing Concurrent Software SystemsIEEE Transactions on Software Engineering10.1109/TSE.2017.270708944:8(747-783)Online publication date: 1-Aug-2018
https://dl.acm.org/doi/10.1109/TSE.2017.2707089
Show More Cited By

Recommendations

Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage

Software testing is particularly expensive for developers of high-assurance software, such as software that is produced for commercial airborne systems. One reason for this expense is the Federal Aviation Administration's requirement that test suites be ...
An empirical study of incorporating cost into test suite reduction and prioritization
SAC '09: Proceedings of the 2009 ACM symposium on Applied Computing

Software developers use testing to gain and maintain confidence in the correctness of a software system. Automated reduction and prioritization techniques attempt to decrease the time required to detect faults during test suite execution. This paper ...
SmarTest: A Test Case Prioritization Tool for Drupal
SPLC '17: Proceedings of the 21st International Systems and Software Product Line Conference - Volume B

Test case prioritization techniques aim to identify the optimal ordering of tests to accelerate the detection of faults. The importance of these techniques has been recognized in the context of Software Product Lines (SPLs), where the potentially huge ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA '11: Proceedings of the 2011 International Symposium on Software Testing and Analysis

July 2011

394 pages

ISBN:9781450305624

DOI:10.1145/2001420

General Chair:
Matthew Dwyer
University of Nebraska
,
Program Chair:
Frank Tip
IBM T.J. Watson Research Center

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 July 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

ISSTA '11

Sponsor:

SIGSOFT

ISSTA '11: International Symposium on Software Testing and Analysis

July 17 - 21, 2011

Ontario, Toronto, Canada

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
257
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yu THuang ZWang C(2020) ConTesa : Directed Test Suite Augmentation for Concurrent Software IEEE Transactions on Software Engineering10.1109/TSE.2018.286139246:4(405-419)Online publication date: 1-Apr-2020
https://doi.org/10.1109/TSE.2018.2861392
Sung CLahiri SEnea CWang CHuchard MKästner CFraser G(2018)Datalog-based scalable semantic diffing of concurrent programsProceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering10.1145/3238147.3238211(656-666)Online publication date: 3-Sep-2018
https://dl.acm.org/doi/10.1145/3238147.3238211
Bianchi FMargara APezze M(2018)A Survey of Recent Trends in Testing Concurrent Software SystemsIEEE Transactions on Software Engineering10.1109/TSE.2017.270708944:8(747-783)Online publication date: 1-Aug-2018
https://dl.acm.org/doi/10.1109/TSE.2017.2707089
Krishna Murthy DPradel M(2018)Change-Aware Dynamic Program Analysis for JavaScript2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME.2018.00023(127-137)Online publication date: Sep-2018
https://doi.org/10.1109/ICSME.2018.00023
Wang SNam JTan LBodden ESchäfer WDeursen AZisman A(2017)QTEP: quality-aware test case prioritizationProceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering10.1145/3106237.3106258(523-534)Online publication date: 21-Aug-2017
https://dl.acm.org/doi/10.1145/3106237.3106258
Yu T(2017)SimEvo: Testing Evolving Multi-process Software Systems2017 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME.2017.29(204-215)Online publication date: Sep-2017
https://doi.org/10.1109/ICSME.2017.29
Chen JBai YHao DXiong YZhang HXie BUchitel SOrso ARobillard M(2017)Learning to prioritize test programs for compiler testingProceedings of the 39th International Conference on Software Engineering10.1109/ICSE.2017.70(700-711)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.1109/ICSE.2017.70
Choudhary ALu SPradel MUchitel SOrso ARobillard M(2017)Efficient detection of thread safety violations via coverage-guided generation of concurrent testsProceedings of the 39th International Conference on Software Engineering10.1109/ICSE.2017.32(266-277)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.1109/ICSE.2017.32
Guo SKusano MWang CLo DApel SKhurshid S(2016)Conc-iSE: incremental symbolic execution of concurrent softwareProceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering10.1145/2970276.2970332(531-542)Online publication date: 25-Aug-2016
https://dl.acm.org/doi/10.1145/2970276.2970332
Shi QHuang JChen ZXu B(2016)Verifying Synchronization for Atomicity Violation FixingIEEE Transactions on Software Engineering10.1109/TSE.2015.247782042:3(280-296)Online publication date: 1-Mar-2016
https://doi.org/10.1109/TSE.2015.2477820
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten