Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3533767.3543288acmconferencesArticle/Chapter ViewAbstractPublication PagesisstaConference Proceedingsconference-collections
short-paper
Open access

Faster mutation analysis with MeMu

Published: 18 July 2022 Publication History

Abstract

Mutation analysis is a program analysis method with applications in assessing the quality of test cases, fault localization, test input generation, security analysis, etc. The method involves repeated running of test suites against a large number of program mutants, often leading to poor scalability. A large body of research is aimed at accelerating mutation analysis via a variety of approaches such as, reducing the number of mutants, reducing the number of test cases to run, or reducing the execution time of individual mutants. This paper presents the implementation of a novel technique, named MeMu, for reducing mutant execution time, through memoizing the most expensive methods in the system. Memoization is a program optimization technique that allows bypassing the execution of expensive methods and reusing pre-calculated results, when repeated inputs are detected. MeMu can be used on its own or alongside existing mutation analysis acceleration techniques. The current implementation of MeMu achieves, on average, an 18.15% speed-up for PITest JVM-based mutation testing tool.

References

[1]
Iftekhar Ahmed, Carlos Jensen, Alex Groce, and Paul E McKenney. 2017. Applying mutation analysis on kernel test suites: an experience report. In ICSTW. 110–115.
[2]
Paul Ammann and Jeff Offutt. 2016. Introduction to software testing. Cambridge University Press.
[3]
Oracle Corporation. 2004. Java Instrumentation API. https://bit.ly/3czmzFV Accessed: 105/22.
[4]
Vidroha Debroy and W Eric Wong. 2010. Using mutation to automatically suggest fixes for faulty programs. In ICST. 65–74.
[5]
Richard A DeMillo, Richard J Lipton, and Frederick G Sayward. 1978. Hints on test data selection: Help for the practicing programmer. IEEE Computer, 34–41.
[6]
Richard A DeMillo and A Jefferson Offutt. 1991. Constraint-based automatic test data generation. TSE, 900–910.
[7]
Juan P Galeotti, Carlo A Furia, Eva May, Gordon Fraser, and Andreas Zeller. 2015. Inferring loop invariants by mutation, dynamic analysis, and static checking. TSE, 1019–1037.
[8]
Ali Ghanbari, Samuel Benton, and Lingming Zhang. 2019. Practical program repair via bytecode mutation. In ISSTA. 19–30.
[9]
Ali Ghanbari and Andrian Marcus. 2020. Faster Mutation Analysis with MeMu. https://github.com/ali-ghanbari/memu-demo Accessed: 05/22.
[10]
Ali Ghanbari and Andrian Marcus. 2021. Toward Speeding up Mutation Analysis by Memoizing Expensive Methods. In ICSE-NIER. 71–75.
[11]
Milos Gligoric, Vilas Jagannath, and Darko Marinov. 2010. MuTMuT: Efficient exploration for mutation testing of multithreaded code. In ICST. 55–64.
[12]
René Just, Michael D. Ernst, and Gordon Fraser. 2014. Efficient mutation analysis by propagating and partitioning infected execution states. In ISSTA. 315–326.
[13]
Rene Just, Franz Schweiggert, and Gregory M Kapfhammer. 2011. MAJOR: An efficient and extensible tool for mutation analysis in a Java compiler. In ASE. 612–615.
[14]
Kim N King and A Jefferson Offutt. 1991. A fortran language system for mutation-based software testing. SPE, 685–718.
[15]
Donald Michie. 1968. “Memo” functions and machine learning. Nature, 19–22.
[16]
OW2 Consortium. 2020. ASM Bytecode Manipulation Framework. http://bit.ly/3fsPL2r Accessed: 05/22.
[17]
Mike Papadakis and Yves Le Traon. 2015. Metallaxis-FL: mutation-based fault localization. Software Testing, Verification and Reliability, 25, 5-7 (2015), 605–628.
[18]
PIT Contributors. 2022. PIT Mutation Testing. https://bit.ly/36Xvhho Accessed: 05/22.
[19]
Alessandro Viola Pizzoleto, Fabiano Cutigi Ferrari, Jeff Offutt, Leo Fernandes, and Márcio Ribeiro. 2019. A systematic literature review of techniques and metrics to reduce the cost of mutation testing. JSS, 157 (2019), 110388.
[20]
Donghwan Shin, Shin Yoo, Mike Papadakis, and Doo-Hwan Bae. 2019. Empirical evaluation of mutation-based test case prioritization techniques. STVR, e1695.
[21]
Roland H Untch, A Jefferson Offutt, and Mary Jean Harrold. 1993. Mutation analysis using mutant schemata. In ISSTA. 139–148.
[22]
Bo Wang, Yingfei Xiong, Yangqingwei Shi, Lu Zhang, and Dan Hao. 2017. Faster Mutation Analysis via Equivalence modulo States. In ISSTA. 295–306.
[23]
Jie Zhang, Ziyi Wang, Lingming Zhang, Dan Hao, Lei Zang, Shiyang Cheng, and Lu Zhang. 2016. Predictive Mutation Testing. In ISSTA. 342–353.
[24]
Lingming Zhang, Darko Marinov, and Sarfraz Khurshid. 2013. Faster mutation testing inspired by test prioritization and reduction. In ISSTA. 235–245.

Cited By

View all
  • (2023)Guiding Greybox Fuzzing with Mutation TestingProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598107(929-941)Online publication date: 12-Jul-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ISSTA 2022: Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis
July 2022
808 pages
ISBN:9781450393799
DOI:10.1145/3533767
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Memoization
  2. Method
  3. Mutant
  4. Mutation Analysis
  5. Test Case

Qualifiers

  • Short-paper

Conference

ISSTA '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)99
  • Downloads (Last 6 weeks)20
Reflects downloads up to 11 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Guiding Greybox Fuzzing with Mutation TestingProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598107(929-941)Online publication date: 12-Jul-2023

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media