research-article

Refactoring Test Smells: A Perspective from Open-Source Developers

Authors:

Márcio Ribeiro,

Guilherme Amaral,

Alessandro Garcia,

Baldoino Fonseca,

André SantosAuthors Info & Claims

SAST '20: Proceedings of the 5th Brazilian Symposium on Systematic and Automated Software Testing

Pages 50 - 59

https://doi.org/10.1145/3425174.3425212

Published: 22 October 2020 Publication History

Abstract

Test smells are symptoms in the test code that indicate possible design or implementation problems. Their presence, along with their harmfulness, has already been demonstrated by previous researches. However, we do not know to what extent developers acknowledge the presence of test smells and how to refactor existing code to eliminate them in practice. This study aims to assess open-source developers' awareness about the existence of test smells and their refactoring strategies. We conducted a mixed-method study with two parts: (i) a survey with 73 experienced open-source developers to assess their preference and motivation to choose between 10 different smelly test code samples, found in 272 open-source projects, and their refactored versions; and (ii) the submission of 50 pull requests to assess developers' acceptance of the proposed refactorings. As a result, most surveyed developers preferred the refactored proposal for 78% of the investigated test smells, and the pull requests had an average acceptance of 75% among respondents. Additionally, we were able to provide empiric validation for literature-proposed refactoring strategies. This study demonstrates that although not always using the academic terminology, developers acknowledge both the negative impact of test smells presence and most of the literature's proposals for their removal.

References

[1]

Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? An empirical study. Empirical Software Engineering 20, 4 (2015), 1052--1094.

Digital Library

[2]

Martin Fowler. 2018. Refactoring: improving the design of existing code. Addison-Wesley Professional.

Digital Library

[3]

Vahid Garousi and Barış Küçük. 2018. Smells in software test code: A survey of knowledge in industry and academia. Journal of Systems and Software 138 (2018), 52--81.

[4]

Nildo Silva Junior, Larissa Rocha, Luana Almeida Martins, and Ivan Machado. 2020. A survey on test practitioners' awareness of test smells. arXiv:2003.05613 [cs.SE]

[5]

Stefano Lambiase, Andrea Cupito, Fabiano Pecorelli, Andrea De Lucia, and Fabio Palomba. 2018. Just-In-Time Test Smell Detection and Refactoring: The DARTS Project. (2018).

[6]

Gerard Meszaros. 2007. xUnit test patterns: Refactoring test code. Pearson Education.

[7]

Emerson Murphy-Hill, Chris Parnin, and Andrew P Black. 2011. How we refactor, and how we know it. IEEE Transactions on Software Engineering 38, 1 (2011), 5--18.

Digital Library

[8]

F. Palomba, G. Bavota, M. D. Penta, R. Oliveto, and A. D. Lucia. 2014. Do They Really Smell Bad? A Study on Developers' Perception of Bad Code Smells. In 2014 IEEE International Conference on Software Maintenance and Evolution. 101--110.

[9]

Fabio Palomba, Dario Di Nucci, Annibale Panichella, Rocco Oliveto, and Andrea De Lucia. 2016. On the diffusion of test smells in automatically generated test code: An empirical study. In 2016 IEEE/ACM 9th International Workshop on Search-Based Software Testing (SBST). IEEE, 5--14.

Digital Library

[10]

Fabio Palomba and Andy Zaidman. 2019. The smell of fear: On the relation between test smells and flaky tests. Empirical Software Engineering 24, 5 (2019), 2907--2946.

Digital Library

[11]

Anthony Peruma, Khalid Almalki, Christian D Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2019. On the distribution of test smells in open source Android applications: an exploratory study. In Proceedings of the 29th Annual International Conference on Computer Science and Software Engineering. IBM Corp., 193--202.

Digital Library

[12]

Anthony Peruma, Christian D Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2020. An Exploratory Study on the Refactoring of Unit Test Files in Android Applications. In Conference on Software Engineering Workshops (ICSEW'20).

[13]

Anthony Shehan Ayam Peruma. 2018. What the Smell? An Empirical Investigation on the Distribution and Severity of Test Smells in Open Source Android Applications. Ph.D. Dissertation.

[14]

Martin Schvarcbacher, Davide Spadini, Magiel Bruntink, and Ana Oprescu. 2019. Investigating developer perception on test smells using better code hub-Work in progress. In 2019 Seminar Series on Advanced Techniques and Tools for Software Evolution, SATTOSE.

[15]

Davide Spadini, Martin Schvarcbacher, Ana-Maria Oprescu, Magiel Bruntink, and Alberto Bacchelli. [n.d.]. Investigating Severity Thresholds for Test Smells. ([n.d.]).

[16]

David Thomas and Andrew Hunt. 2019. The Pragmatic Programmer: your journey to mastery. Addison-Wesley Professional.

[17]

Michele Tufano, Fabio Palomba, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, Andrea De Lucia, and Denys Poshyvanyk. 2016. An empirical investigation into the nature of test smells. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. 4--15.

Digital Library

[18]

Arie Van Deursen, Leon Moonen, Alex Van Den Bergh, and Gerard Kok. 2001. Refactoring test code. In Proceedings of the 2nd international conference on extreme programming and flexible processes in software engineering (XP). 92--95.

[19]

Bart Van Rompaey, Bart Du Bois, Serge Demeyer, and Matthias Rieger. 2007. On the detection of test smells: A metrics-based approach for general fixture and eager test. IEEE Transactions on Software Engineering 33, 12 (2007), 800--817.

Digital Library

Cited By

Aranda MOliveira NSoares ERibeiro MRomão DPatriota UGheyi RSouza EMachado I(2024)A Catalog of Transformations to Remove Smells From Natural Language TestsProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661225(7-16)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661225
Martins LGhaleb TCosta HMachado I(2024)A comprehensive catalog of refactoring strategies to handle test smells in Java-based systemsSoftware Quality Journal10.1007/s11219-024-09663-732:2(641-679)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s11219-024-09663-7
Teixeira TSilveira FGuerra E(2023)Moving towards a Mutant-Based Testing Tool for Verifying Behavior Maintenance in Test Code RefactoringsComputers10.3390/computers1211023012:11(230)Online publication date: 13-Nov-2023
https://doi.org/10.3390/computers12110230
Show More Cited By

Index Terms

Refactoring Test Smells: A Perspective from Open-Source Developers
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Empirical software validation
      2. Software defect analysis
        Software testing and debugging

Recommendations

Prioritising Refactoring Using Code Bad Smells
ICSTW '11: Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops

We investigated the relationship between six of Fowler et al.'s Code Bad Smells (Duplicated Code, Data Clumps, Switch Statements, Speculative Generality, Message Chains, and Middle Man) and software faults. In this paper we discuss how our results can ...
Analyzing Test Smells Refactoring from a Developers Perspective
SBQS '22: Proceedings of the XXI Brazilian Symposium on Software Quality

Test smells represent a set of poorly designed tests, which can harm a test code’s maintenance and quality criteria. Although fundamental steps to understand test smells have been investigated, there is still an evident lack of studies evaluating the ...
On experimenting refactoring tools to remove code smells
XP '15 workshops: Scientific Workshop Proceedings of the XP2015

When we develop a software project of a certain complexity, source code maintainability could become a problem, in particular if developers do not use a consolidate development process that simplifies the management of the entire project. When source ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SAST '20: Proceedings of the 5th Brazilian Symposium on Systematic and Automated Software Testing

October 2020

126 pages

ISBN:9781450387552

DOI:10.1145/3425174

General Chairs:
Everton Cavalcante
UFRN, Brazil
,
Francisco Dantas
UERN, Brazil
,
Thais Batista
UFRN, Brazil

Copyright © 2020 ACM.

© 2020 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

In-Cooperation

SBC: Brazilian Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
Fundação de Amparo à Pesquisa do Estado de Alagoas
Conselho Nacional de Desenvolvimento Científico e Tecnológico

Conference

SAST 20

SAST 20: 5th Brazilian Symposium on Systematic and Automated Software Testing

October 20 - 21, 2020

Natal, Brazil

Acceptance Rates

Overall Acceptance Rate 45 of 92 submissions, 49%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
283
Total Downloads

Downloads (Last 12 months)51
Downloads (Last 6 weeks)7

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Aranda MOliveira NSoares ERibeiro MRomão DPatriota UGheyi RSouza EMachado I(2024)A Catalog of Transformations to Remove Smells From Natural Language TestsProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661225(7-16)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661225
Martins LGhaleb TCosta HMachado I(2024)A comprehensive catalog of refactoring strategies to handle test smells in Java-based systemsSoftware Quality Journal10.1007/s11219-024-09663-732:2(641-679)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s11219-024-09663-7
Teixeira TSilveira FGuerra E(2023)Moving towards a Mutant-Based Testing Tool for Verifying Behavior Maintenance in Test Code RefactoringsComputers10.3390/computers1211023012:11(230)Online publication date: 13-Nov-2023
https://doi.org/10.3390/computers12110230
Pizzini AReinehr SMalucelli A(2023)Sentinel: A process for automatic removing of Test SmellsProceedings of the XXII Brazilian Symposium on Software Quality10.1145/3629479.3630019(80-89)Online publication date: 7-Nov-2023
https://dl.acm.org/doi/10.1145/3629479.3630019
Lima RCosta KSouza JTeixeira LFonseca BD'Amorim MRibeiro MMiranda B(2023)Do you see any problem? On the Developers Perceptions in Test Smells DetectionProceedings of the XXII Brazilian Symposium on Software Quality10.1145/3629479.3629485(21-30)Online publication date: 7-Nov-2023
https://dl.acm.org/doi/10.1145/3629479.3629485
Soares ERibeiro MGheyi RAmaral GSantos A(2023)Refactoring Test Smells With JUnit 5: Why Should Developers Keep Up-to-Date?IEEE Transactions on Software Engineering10.1109/TSE.2022.317265449:3(1152-1170)Online publication date: 1-Mar-2023
https://doi.org/10.1109/TSE.2022.3172654
Martins LCosta HRibeiro MPalomba FMachado I(2023)Automating Test-Specific Refactoring Mining: A Mixed-Method Investigation2023 IEEE 23rd International Working Conference on Source Code Analysis and Manipulation (SCAM)10.1109/SCAM59687.2023.00013(13-24)Online publication date: 2-Oct-2023
https://doi.org/10.1109/SCAM59687.2023.00013
Soares EAranda MOliveira NRibeiro MGheyi RSouza EMachado ISantos AFonseca BBonifácio R(2023)Manual Tests Do Smell! Cataloging and Identifying Natural Language Test Smells2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)10.1109/ESEM56168.2023.10304800(1-11)Online publication date: 26-Oct-2023
https://doi.org/10.1109/ESEM56168.2023.10304800
Martins LCampos DSantana RJunior JCosta HMachado I(2023)Hearing the voice of experts: Unveiling Stack Exchange communities’ knowledge of test smells2023 IEEE/ACM 16th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE)10.1109/CHASE58964.2023.00017(80-91)Online publication date: May-2023
https://doi.org/10.1109/CHASE58964.2023.00017
Martins LCosta HMachado I(2023)On the diffusion of test smells and their relationship with test code quality of Java projectsJournal of Software: Evolution and Process10.1002/smr.2532Online publication date: 18-Jan-2023
https://doi.org/10.1002/smr.2532
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents