Mutation Operators for Java Streams
Authors: 
Manoel Aranda III, 
Elvys Soares, Márcio Ribeiro, 
Fabiano Ferrari, Rohit Gheyi, Arthur LimaAuthors Info & Claims
Manoel Aranda III, Elvys Soares, Márcio Ribeiro, Fabiano Ferrari, Rohit Gheyi, Arthur LimaAuthors Info & ClaimsPages 59 - 68
Abstract
Mutation testing analyzes test suites to verify their capability to detect artificially injected faults. Mutation testing tools rely on mutation operators to simulate those faults by modifying language constructs. The popularization of Streaming APIs, which enable parallel processing of native data structures with relatively succinct constructs, presents challenges related to functional programming, and faults from the APIs’ misuse are already objects of study. However, no comprehensive mutation operators have been defined for this purpose. We propose seven mutation operators to simulate stream-related faults. To evaluate our operators, we mined 22 open-source projects from different domains (i.e., applications for smart cities and messaging frameworks) to identify faults our operators could simulate. We analyzed 357 commits, raising 91 fixes for stream-related faults in GitHub Java projects. Our operators can simulate 96.7% of the analyzed faults, and we verified five of our proposals in practice. Our mutation operators can enhance the capabilities of current mutation testing tools and help developers to improve their test suites by avoiding stream-related faults.
References
[1]
Allen T Acree, Timothy A Budd, Richard A DeMillo, Richard J Lipton, and Frederick G Sayward. 1979. Mutation Analysis. Technical Report. School of Information and Computer Science, Georgia Institute of Technology.
[2]
Hiralal Agrawal, Richard A DeMillo, R_ Hathaway, William Hsu, Wynne Hsu, Edward W Krauser, Rhonda J Martin, Aditya P Mathur, and Eugene Spafford. 1989. Design of Mutant Operators for the C Programming Language. Technical Report. Software Engineering Research Center, Purdue University.
[3]
Syed Ahmed and Mehdi Bagherzadeh. 2018. What Do Concurrency Developers Ask about? A Large-Scale Study Using Stack Overflow. In 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement(ESEM). ACM, New York, NY, USA, 1–10.
[4]
Paul Ammann, Marcio Eduardo Delamaro, and Jeff Offutt. 2014. Establishing Theoretical Minimal Sets of Mutants. In International Conference on Software Testing, Verification, and Validation(ICST). IEEE, New York, NY, USA, 21–30.
[5]
Android. 2022. Stream Android Developers. Google LLC. Retrieved Mar 18, 2022 from https://developer.android.com/reference/java/util/stream/Stream
[6]
Manoel Aranda III, Elvys Soares, Márcio Ribeiro, Fabiano Ferrari, Rohit Gheyi, and Arthur Ramos. 2022. Mutation Operators for Java Streams: Dataset. GitHub Inc. https://github.com/easy-software-ufal/MutationOperatorsForJavaStreams
[7]
James E. Bartlett II, Joe W. Kotrlik, and Chadwick C. Higgins. 2001. Organizational research: Determining appropriate sample size in survey research appropriate sample size in survey research. Information technology, learning, and performance journal 19, 1(2001), 43–50.
[8]
Aggelos Biboudis, Nick Palladinos, George Fourtounis, and Yannis Smaragdakis. 2015. Streams à la carte: Extensible pipelines with object algebras. In 29th European Conference on Object-Oriented Programming(ECOOP). Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 591–613.
[9]
Jeremy S. Bradbury, James R. Cordy, and Juergen Dingel. 2006. Mutation Operators for Concurrent Java (J2SE 5.0). In 2nd Workshop on Mutation Analysis(MUTATION). IEEE, New York, NY, USA, 11–11.
[10]
Luiz Carvalho, Marcio Augusto Guimarães, Márcio Ribeiro, Leonardo Fernandes, Mustafa Al-Hajjaji, Rohit Gheyi, and Thomas Thüm. 2018. Equivalent Mutants in Configurable Systems: An Empirical Study. In 12th International Workshop on Variability Modelling of Software-Intensive Systems(VaMoS). ACM, New York, NY, USA, 11–18.
[11]
Thierry Titcheu Chekam, Mike Papadakis, Yves Le Traon, and Mark Harman. 2017. An Empirical Study on Mutation, Statement and Branch Coverage Fault Revelation That Avoids the Unreliable Clean Program Assumption. In 39th International Conference on Software Engineering(ICSE). IEEE, New York, NY, USA, 597–608.
[12]
Henry Coles, Thomas Laurent, Christopher Henard, Mike Papadakis, and Anthony Ventresque. 2016. PIT: A Practical Mutation Testing Tool for Java (Demo). In 25th International Symposium on Software Testing and Analysis(ISSTA). ACM, New York, NY, USA, 449–452.
[13]
Henry Coles, Thomas Laurent, Christopher Henard, Mike Papadakis, and Anthony Ventresque. 2022. PITest Mutation Testing Tool for Java. arcmutate. Retrieved Mar 18, 2022 from http://pitest.org/
[14]
Henry Coles, Jon Tizzard, Mark Trudinger, and Mark Meszaros. 2022. Mutation Testing Tools Improved. arcmutate. Retrieved Mar 18, 2022 from https://www.arcmutate.com/
[15]
Henrique Neves da Silva, Silvia Regina Vergilio, and André Takeshi Endo. 2022. Accessibility Mutation Testing of Android Applications. Journal of Software Engineering Research and Development 10, 8(2022), 1–11.
[16]
Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: Simplified Data Processing on Large Clusters. Commun. ACM 51, 1 (2008), 107––113.
[17]
Márcio Delamaro, Mauro Pezze, Auri MR Vincenzi, and José Carlos Maldonado. 2001. Mutant Operator for Testing Concurrent Java Programs. In 15th Brazilian Symposium on Software Engineering(SBES). UFRJ, Rio de Janeiro, Brazil, 386–391.
[18]
Marcio Eduardo Delamaro, Jeff Offutt, and Paul Ammann. 2014. Designing Deletion Mutation Operators. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation(ICST). IEEE, New York, NY, USA, 11–20.
[19]
Pedro Delgado-Pérez, Inmaculada Medina-Bulo, Francisco Palomo-Lozano, Antonio García-Domínguez, and Juan José Domínguez-Jiménez. 2017. Assessment of class mutation operators for C++ with the MuCPP mutation system. Information and Software Technology 81 (2017), 169–184.
[20]
Richard A DeMillo, Richard J Lipton, and Frederick G Sayward. 1978. Hints on Test Data Selection: Help for the Practicing Programmer. IEEE Computer 11, 4 (1978), 34–41.
[21]
Anna Derezinska and Karol Kowalski. 2011. Object-Oriented Mutation Applied in Common Intermediate Language Programs Originated from C#. In 4th International Conference on Software Testing, Verification and Validation Workshops(ICSTW). IEEE, New York, NY, USA, 342–350.
[22]
Leonardo Fernandes, Márcio Ribeiro, Luiz Carvalho, Rohit Gheyi, Melina Mongiovi, André Santos, Ana Cavalcanti, Fabiano Ferrari, and José Carlos Maldonado. 2017. Avoiding Useless Mutants. In 16th International Conference on Generative Programming: Concepts & Experience(GPCE). ACM, New York, NY, USA, 187–198.
[23]
Rohit Gheyi, Márcio Ribeiro, Beatriz Souza, Marcio Guimarães, Leo Fernandes, Marcelo d’Amorim, Vander Alves, Leopoldo Teixeira, and Baldoino Fonseca. 2021. Identifying method-level mutation subsumption relations using Z3. Information and Software Technology 132 (2021), 106496.
[24]
GitHub. 2022. GitHub Search REST API v3. GitHub Inc. Retrieved Mar 18, 2022 from https://developer.github.com/v3/search/
[25]
Marcio Augusto Guimarães, Leo Fernandes, Márcio Ribeiro, Marcelo d’Amorim, and Rohit Gheyi. 2020. Optimizing Mutation Testing by Discovering Dynamic Mutant Subsumption Relations. In IEEE 13th International Conference on Software Testing, Validation and Verification(ICST). IEEE, New York, NY, USA, 198–208.
[26]
Richard G. Hamlet. 1977. Testing Programs with the Aid of a Compiler. IEEE Transactions on Software Engineering 3, 4 (1977), 279–290.
[27]
Yue Jia and Mark Harman. 2009. Higher Order Mutation Testing. Information and Software Technology 51, 10 (2009), 1379–1393.
[28]
Yue Jia and Mark Harman. 2011. An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering 37, 5 (2011), 649–678.
[29]
René Just. 2014. The Major Mutation Framework: Efficient and Scalable Mutation Analysis for Java. In 23rd International Symposium on Software Testing and Analysis(ISSTA). ACM, New York, NY, USA, 433–436.
[30]
René Just, Darioush Jalali, Laura Inozemtseva, Michael D Ernst, Reid Holmes, and Gordon Fraser. 2014. Are Mutants a Valid Substitute for Real Faults in Software Testing?. In 22nd SIGSOFT International Symposium on Foundations of Software Engineering(FSE). ACM, New York, NY, USA, 654–665.
[31]
Raffi Khatchadourian, Yiming Tang, Mehdi Bagherzadeh, and Baishakhi Ray. 2020. An Empirical Study on the Use and Misuse of Java 8 Streams. In FASE. SpringerOpen, New York, NY, USA, 97–118.
[32]
Sunwoo Kim, John A Clark, and John A McDermid. 2000. Class mutation: Mutation testing for object-oriented programs. In Proc. Net. ObjectDays(FMES). Net. Objects Erfurt, Germany, 9–12.
[33]
Walter Lucas, Rodrigo Bonifácio, Edna Dias Canedo, Diego Marcílio, and Fernanda Lima. 2019. Does the Introduction of Lambda Expressions Improve the Comprehension of Java Programs?. In 33rd Brazilian Symposium on Software Engineering(SBES). ACM, New York, NY, USA, 187–196.
[34]
Yu-Seung Ma, Yong-Rae Kwon, and Jeff Offutt. 2002. Inter-class Mutation Operators for Java. In 13th International Symposium on Software Reliability Engineering(ISSRE). IEEE, New York, NY, USA, 352–366.
[35]
Yu-Seung Ma, Jeff Offutt, and Yong Rae Kwon. 2005. MuJava: An Automated Class Mutation System. Software Testing, Verification and Reliability 15, 2(2005), 97–133.
[36]
Lech Madeyski, Wojciech Orzeszyna, Richard Torkar, and Mariusz Jozala. 2014. Overcoming the Equivalent Mutant Problem: A Systematic Literature Review and a Comparative Experiment of Second Order Mutation. IEEE Transactions on Software Engineering 40, 1 (2014), 23–42.
[37]
Sebastian Nielebock, Robert Heumüller, and Frank Ortmeier. 2019. Programmers do not favor lambda expressions for concurrent object-oriented code. Empirical Software Engineering 24, 1 (2019), 103–138.
[38]
A. Jefferson Offutt and Roland H. Untch. 2001. Mutation 2000: Uniting the Orthogonal. Vol. 1. Springer US, New York, NY, USA, 34–44.
[39]
Oracle. 2022. java.util.stream (Java SE 18 & JDK 18). Oracle Corp. Retrieved Mar 18, 2022 from https://docs.oracle.com/en/java/javase/18/docs/api/java.base/java/util/stream/package-summary.html
[40]
Mike Papadakis, Marinos Kintis, Jie Zhang, Yue Jia, Yves Le Traon, and Mark Harman. 2019. Mutation Testing Advances: An Analysis and Survey. In Advances in Computers. Vol. 112. Elsevier, Amsterdam, The Netherlands, 275–378.
[41]
Liz Parody. 2022. Understanding Streams in Node.js. NodeSource. Retrieved Mar 18, 2022 from https://nodesource.com/blog/understanding-streams-in-nodejs/
[42]
Goran Petrovic, Marko Ivankovic, Bob Kurtz, Paul Ammann, and René Just. 2018. An Industrial Application of Mutation Testing: Lessons, Challenges, and Research Directions. In 13th International Workshop on Mutation Analysis(MUTATION). IEEE, New York, NY, USA, 47–53.
[43]
Pedro Pinheiro, José Carlos Viana, Leonardo Fernandes, Márcio Ribeiro, Fabiano Ferrari, Baldoino Fonseca, and Rohit Gheyi. 2018. Mutation Operators for Code Annotations. In III Brazilian Symposium on Systematic and Automated Software Testing(SAST). ACM, New York, NY, USA, 77–86.
[44]
Pedro Pinheiro, José Carlos Viana, Márcio Ribeiro, Leo Fernandes, Fabiano Ferrari, Rohit Gheyi, and Baldoino Fonseca. 2020. Mutating code annotations: An empirical evaluation on Java and C# programs. Science of Computer Programming 191 (2020), 102418.
[45]
Hussachai Puripunpinyo. 2022. Java 8 vs. Scala, Part II: the Streams API. DZone. Retrieved Mar 18, 2022 from https://dzone.com/articles/java-8-vs-scalapart-ii-streams-api
[46]
Elvys Soares, Márcio Ribeiro, Guilherme Amaral, Rohit Gheyi, Leo Fernandes, Alessandro Garcia, Baldoino Fonseca, and André Santos. 2020. Refactoring Test Smells: A Perspective from Open-Source Developers. In 5th Brazilian Symposium on Systematic and Automated Software Testing(SAST). ACM, New York, NY, USA, 50–59.
[47]
Elvys Soares, Marcio Ribeiro, Rohit Gheyi, Guilherme Amaral, and Andre Medeiros Santos. 2022. Refactoring Test Smells With JUnit 5: Why Should Developers Keep Up-to-Date?IEEE Transactions on Software Engineering(2022), 1–1.
[48]
Allison Sullivan, Kaiyuan Wang, Razieh Nokhbeh Zaeem, and Sarfraz Khurshid. 2017. Automated Test Generation and Mutation Testing for Alloy. In International Conference on Software Testing, Verification and Validation(ICST). IEEE, New York, NY, USA, 264–275.
[49]
Arzu Behiye Tarımcı and Hasan Sözer. 2022. Mutation testing of PL/SQL programs. Journal of Systems and Software 192 (2022), 111399.
[50]
Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, and Anders Wesslén. 2000. Experimentation in software engineering: an introduction. Springer Science & Business Media, Berlin/Heidelberg, Germany.
Index Terms
- Mutation Operators for Java Streams
Recommendations
JDAMA: Java database application mutation analyser
Database application programs are ubiquitous; hence, good techniques for testing them are needed. Several approaches to testing database applications have been proposed recently, including test generation techniques and adequacy criteria. Tuya et al. ...
Mutation Operators for Actor Systems
ICSTW '10: Proceedings of the 2010 Third International Conference on Software Testing, Verification, and Validation WorkshopsMutation testing is a well known technique for estimating and improving the quality of test suites. Given a test suite T for a system S, mutation testing systematically creates mutants of S and executes T to measure how many mutants T detects. If T does ...
On guiding the augmentation of an automated test suite via mutation analysis
Mutation testing has traditionally been used as a defect injection technique to assess the effectiveness of a test suite as represented by a "mutation score." Recently, mutation testing tools have become more efficient, and industrial usage of mutation ...
Comments
Information & Contributors
Information
Published In
October 2022
78 pages
ISBN:9781450397537
DOI:10.1145/3559744
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 03 October 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Fundação de Amparo à Pesquisa do Estado de Alagoas
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
Conference
SAST 2022
SAST 2022: 7th Brazilian Symposium on Systematic and Automated Software Testing
October 3 - 7, 2022
Uberlandia, Brazil
Acceptance Rates
Overall Acceptance Rate 45 of 92 submissions, 49%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 95Total Downloads
- Downloads (Last 12 months)18
- Downloads (Last 6 weeks)1
Reflects downloads up to 17 Oct 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format