Developers’ perception matters: machine learning to detect developer-sensitive smells
Abstract
Code smells are symptoms of poor design that hamper software evolution and maintenance. Hence, code smells should be detected as early as possible to avoid software quality degradation. However, the notion of whether a design and/or implementation choice is smelly is subjective, varying for different projects and developers. In practice, developers may have different perceptions about the presence (or not) of a smell, which we call developer-sensitive smell detection. Although Machine Learning (ML) techniques are promising to detect smells, there is little knowledge regarding the accuracy of these techniques to detect developer-sensitive smells. Besides, companies may change developers frequently, and the models should adapt quickly to the preferences of new developers, i.e., using few training instances. Based on that, we present an investigation of the behavior of ML techniques in detecting developer-sensitive smells. We evaluated seven popular ML techniques based on their accuracy and efficiency for identifying 10 smell types according to individual perceptions of 63 developers, with some divergent agreement on the presence of smells. The results showed that five out of seven techniques had statistically similar behavior, being able to properly detect smells. However, the accuracy of all ML techniques was affected by developers’ opinion agreement and smell types. We also observed that the detection rules generated for developers individually have more metrics than in related studies. We can conclude that code smells detection tools should consider the individual perception of each developer to reach higher accuracy. However, untrained developers or developers with high disagreement can introduce bias in the smell detection, which can be risky for overall software quality. Moreover, our findings shed light on improving the state of the art and practice for the detection of code smells, contributing to multiple stakeholders.
References
[1]
Abbes M, Khomh F, Gueheneuc Y G, Antoniol G (2011) An empirical study of the impact of two antipatterns, blob and spaghetti code, on program comprehension. In: 15th European conference on software maintenance and reengineering (CSMR). IEEE, pp 181–190
[2]
Amorim L, Costa E, Antunes N, Fonseca B, Ribeiro M (2015) Experience report: evaluating the effectiveness of decision trees for detecting code smells. In: Proceedings of the 2015 IEEE 26th international symposium on software reliability engineering (ISSRE), ISSRE ’15. IEEE Computer Society, Washington, DC, pp 261–269
[3]
Arcelli Fontana F, Mäntylä MV, Zanoni M, and Marino A Comparing and experimenting machine learning techniques for code smell detection Empir Softw Eng 2016 21 3 1143-1191
[4]
Arcoverde R, Guimarães ET, Bertran IM, Garcia A, Cai Y (2013) Prioritization of code anomalies based on architecture sensitiveness. In: 27th Brazilian symposium on software engineering, SBES 2013, Brasilia, Brazil, October 1-4, 2013. IEEE Computer Society, pp 69–78
[5]
Azeem MI, Palomba F, Shi L, and Wang QMachine learning techniques for code smell detection: a systematic literature review and meta-analysisInf Softw Technol2019108115-138https://doi.org/10.1016/j.infsof.2018.12.009
[6]
Bertran IM (2011) Detecting architecturally-relevant code smells in evolving software systems. In: Taylor RN, Gall HC, Medvidovic N (eds) Proceedings of the 33rd international conference on software engineering, ICSE 2011, Waikiki, Honolulu, HI, USA, May 21-28, 2011. ACM, pp 1090–1093
[7]
Bertran IM, Arcoverde R, Garcia A, Chavez C, von Staa A (2012a) On the relevance of code anomalies for identifying architecture degradation symptoms. In: Mens T, Cleve A, Ferenc R (eds) 16th European conference on software maintenance and reengineering, CSMR 2012, Szeged, Hungary, March 27-30, 2012. IEEE Computer Society, pp 277–286
[8]
Bertran IM, Garcia J, Popescu D, Garcia A, Medvidovic N, von Staa A (2012b) Are automatically-detected code anomalies relevant to architectural modularity?: an exploratory analysis of evolving systems. In: Hirschfeld R, Tanter É, Sullivan KJ, Gabriel RP (eds) Proceedings of the 11th International Conference on Aspect-oriented Software Development, AOSD 2012, Potsdam, Germany, March 25-30, 2012. ACM, pp 167–178
[9]
Bertran IM, Garcia A, Chavez C, von Staa A (2013) Enhancing the detection of code anomalies with architecture-sensitive strategies. In: Cleve A, Ricca F, Cerioli M (eds) 17th European conference on software maintenance and reengineering, CSMR 2013, Genova, Italy, March 5-8, 2013. IEEE Computer Society, pp 177–186
[10]
Bigonha MA, Ferreira K, Souza P, Sousa B, Januário M, and Lima D The usefulness of software metric thresholds for detection of bad smells and fault prediction Inf Softw Technol 2019 115 79-92
[11]
Breiman L, Friedman J H, Olshen R A, Stone C J (1984) Classification and regression trees, Wadsworth and Brooks, Monterey
[12]
Cohen W W (1995) Fast effective rule induction. In: Twelfth international conference on machine learning. Morgan Kaufmann, pp 115–123
[13]
de Mello RM, Oliveira RF, Garcia A (2017) On the influence of human factors for identifying code smells: a multi-trial empirical study. In: 2017 ACM/IEEE International symposium on empirical software engineering and measurement (ESEM)., pp 68–77
[14]
Di Nucci D, Palomba F, Tamburri D A, Serebrenik A, De Lucia A (2018) Detecting code smells using machine learning techniques: Are we there yet?. In: IEEE 25th international conference on software analysis, evolution and reengineering (SANER), pp 612–621.
[15]
Fernandes E, Vale G, da Silva Sousa L, Figueiredo E, Garcia A, Lee J (2017) No code anomaly is an island - anomaly agglomeration as sign of product line instabilities. In: Botterweck G, Werner CML (eds) Mastering scale and complexity in software reuse - 16th international conference on software reuse, ICSR 2017, Salvador, Brazil, May 29-31, 2017, proceedings. Lecture Notes in Computer Science, vol 10221, pp 48–64.
[16]
Fleiss JL Measuring nominal scale agreement among many raters Psychol Bull 1971 76 5 378
[17]
Fontana F A, Mariani E, Mornioli A, Sormani R, Tonello A (2011) An experience report on using code smells detection tools. In: 2011 IEEE Fourth international conference on software testing, verification and validation workshops, pp 450–457. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5954446
[18]
Fontana F A, Zanoni M, Marino A, Mäntylä MV (2013) code smell detection: towards a machine learning-based approach. In: 2013 IEEE International conference on software maintenance, pp 396–399. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6676916
[19]
Fowler M Refactoring: improving the design of existing code 1999 Boston Addison-Wesley
[20]
Friedman M The use of ranks to avoid the assumption of normality implicit in the analysis of variance J Am Stat Assoc 1937 32 200 675-701
[21]
Gopalan R (2012) Automatic detection of code smells in java source code. Ph.D. thesis, Dissertation for Honour Degree The University of Western Australia
[22]
Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, and Witten IH The weka data mining software: an update ACM SIGKDD Expl Newsl 2009 11 1 10-18
[23]
Ho T K (1995) Random decision forests. In: Proceedings of the third international conference on document analysis and recognition, vol 1. IEEE, pp 278–282
[24]
Holte R Very simple classification rules perform well on most commonly used datasets Mach Learn 1993 11 63-91
[25]
Hozano M, Antunes N, Fonseca B, Costa E (2017a) Evaluating the accuracy of machine learning algorithms on detecting code smells for different developers. In: Proceedings of the 19th international conference on enterprise information systems, pp 474–482
[26]
Hozano M, Garcia A, Antunes N, Fonseca B, Costa E (2017b) Smells are sensitive to developers!: on the efficiency of (un)guided customized detection. In: Proceedings of the 25th international conference on program comprehension, ICPC ’17. IEEE Press, Piscataway, pp 110–120.
[27]
Hozano M, Garcia A, Fonseca B, and Costa EAre you smelling it? Investigating how similar developers detect code smellsInf Softw Technol201893C130-146https://doi.org/10.1016/j.infsof.2017.09.002
[28]
Khomh F, Vaucher S, Guéhéneuc Y G, Sahraoui H (2009) A bayesian approach for the detection of code and design smells. In: 9th international conference on quality software. QSIC’09. IEEE, pp 305–314
[29]
Khomh F, Penta MD, Guéhéneuc YG, and Antoniol GAn exploratory study of the impact of antipatterns on class change- and fault-pronenessEmpir Softw Eng2011173243-275https://doi.org/10.1007/s10664-011-9171-y
[30]
Khomh F, Vaucher S, Guéhéneuc YG, and Sahraoui HBdtex: a gqm-based bayesian approach for the detection of antipatternsJ Syst Softw2011844559-572https://doi.org/10.1016/j.jss.2010.11.921
[31]
Lantz B (2019) Machine learning with R: expert techniques for predictive modeling. Packt Publishing Ltd
[32]
Lanza M, Marinescu R, Ducasse S (2005) Object-oriented metrics in practice, Springer, New York
[33]
Maiga A, Ali N, Bhattacharya N, Sabane A, Gueheneuc Y G, Aimeur E (2012) SMURF: a SVM-based incremental anti-pattern detection approach. In: 2012 19th Working conference on reverse engineering, pp 466–475.
[34]
Maneerat N, Muenchaisri P (2011) Bad-smell prediction from software design model using machine learning techniques. In: 2011 Eighth international joint conference on computer science and software engineering (JCSSE), pp 331–336. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5930143
[35]
Mäntylä M V (2005) An experiment on subjective evolvability evaluation of object-oriented software: explaining factors and interrater agreement. In: 2005 International symposium on empirical software engineering, p 10.
[36]
Mäntylä M V, Lassenius C (2006) Subjective evaluation of software evolvability using code smells: an empirical study, vol 11, Springer.
[37]
Marinescu R (2004) Detection strategies: metrics-based rules for detecting design flaws. In: Proceedings of the 20th IEEE international conference on software maintenance, ICSM ’04. IEEE Computer Society, Washington, DC, pp 350–359. http://dl.acm.org/citation.cfm?id=1018431.1021443
[38]
Mitchell T M (1997) Machine learning. McGraw-Hill series in computer science, McGraw-Hill, Boston. http://opac.inria.fr/record=b1093076
[39]
Moha N, Guéhéneuc YG, Meur AFL, Duchien L, and Tiberghien AFrom a domain analysis to the specification and detection of code and design smellsForm Asp Comput2009223345-361https://doi.org/10.1007/s00165-009-0115-x. http://link.springer.com/10.1007/s00165-009-0115-x
[40]
Moha N, Gueheneuc YG, Duchien L, and Le Meur AFDECOR: a method for the specification and detection of code and design smellsIEEE Trans Softw Eng201036120-36https://doi.org/10.1109/TSE.2009.50 https://doi.org/10.1109/TSE.2009.50
[41]
Munro M (2005) Product metrics for automatic identification of “Bad smell” design problems in java Source-Code. In: 11th IEEE International software metrics symposium (METRICS’05), pp 15–15.
[42]
Oizumi WN, Garcia AF, Sousa LS, Cafeo BBP, Zhao Y (2016) Code anomalies flock together: exploring code anomaly agglomerations for locating design problems. In: Dillon LK, Visser W, Williams LA (eds) Proceedings of the 38th international conference on software engineering, ICSE 2016, Austin, TX, USA, May 14-22, 2016. ACM, pp 440–451.
[43]
Oizumi WN, Sousa LS, Oliveira A, Carvalho L, Garcia A, Colanzi TE, Oliveira RF (2019) On the density and diversity of degradation symptoms in refactored classes: a multi-case study. In: Katinka Wolter, Schieferdecker I, Gallina B, Cukier M, Natella R, Ivaki NR, Laranjeiro N (eds) 30th IEEE International symposium on software reliability engineering, ISSRE 2019, Berlin, Germany, October 28-31, 2019. IEEE, pp 346–357.
[44]
Oliveira D, Assunção W K G, Souza L, Oizumi W, Garcia A, Fonseca B (2020) Applying machine learning to customized smell detection: a multi-project study. In: 34th Brazilian symposium on software engineering, SBES ’20. Association for computing machinery, New York, pp 233–242.
[45]
Oliveira D, Assunção W K G, Garcia A, Fonseca B, Ribeiro M (2022) Supplementary material—developers’ perception matters: Machine learning to detect developer-sensitive smells. https://github.com/smellsensitive/smellsensitive.github.io/raw/main/dataset.rar
[46]
Palomba F, Bavota G, Di Penta M, Oliveto R, De Lucia A, Poshyvanyk D (2013) Detecting bad smells in source code using change history information. In: 2013 28th IEEE/ACM international conference on automated software engineering (ASE). IEEE, pp 268–278. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6693086
[47]
Palomba F, Bavota G, Di Penta M, Oliveto R, Poshyvanyk D, and De Lucia AMining version histories for detecting code smellsIEEE Trans Softw Eng20145589c1-1https://doi.org/10.1109/TSE.2014.2372760 https://doi.org/10.1109/TSE.2014.2372760
[48]
Palomba F, Bavota G, Penta M D, Oliveto R, Lucia A D (2014b) Do they really smell bad? A study on developers’ perception of bad code smells. In: 2014 IEEE International conference on software maintenance and evolution, pp 101–110.
[49]
Pecorelli F, Di Nucci D, De Roover C, De Lucia A (2019) On the role of data balancing for machine learning-based code smell detection. In: Proceedings of the 3rd ACM SIGSOFT international workshop on machine learning techniques for software quality evaluation, MaLTeSQuE 2019. Association for Computing Machinery, New York, pp 19–24
[50]
Pecorelli F, Di Nucci D, De Roover C, and De Lucia AA large empirical assessment of the role of data balancing in machine-learning-based code smell detectionJ Syst Softw2020169110693https://doi.org/10.1016/j.jss.2020.110693. http://www.sciencedirect.com/science/article/pii/S0164121220301448
[51]
Platt J (1998) Fast training of support vector machines using sequential minimal optimization. In: Schoelkopf B, Burges C, Smola A (eds) Advances in kernel methods—support vector learning. MIT Press. http://research.microsoft.com/~jplatt/smo.html
[52]
Quinlan R C4.5: programs for machine learning 1993 San Mateo Morgan Kaufmann Publishers
[53]
Rasool G and Arshad Z A review of code smell mining techniques J Softw: Evol Process 2015 27 11 867-895
[54]
Santos J A M, de Mendonça M G, Silva C V A (2013) An exploratory study to investigate the impact of conceptualization in god class detection. In: Proceedings of the 17th international conference on evaluation and assessment in software engineering, EASE ’13. ACM, New York, pp 48–59. http://doi.acm.org/10.1145/2460999.2461007
[55]
Schumacher J, Zazworka N, Shull F, Seaman C, Shaw M (2010) Building empirical support for automated code smell detection. In: Proceedings of the 2010 ACM-IEEE international symposium on empirical software engineering and measurement—ESEM ’10, p 1.
[56]
Silva AL, Garcia A, Cirilo EJR, de Lucena CJP (2013) Are domain-specific detection strategies for code anomalies reusable? An industry multi-project study. In: 27th Brazilian symposium on software engineering, SBES 2013, Brasilia, Brazil, October 1-4, 2013. IEEE Computer Society, pp 79–88.
[57]
Sousa LS, Oliveira A, Oizumi WN, Barbosa SDJ, Garcia A, Lee J, Kalinowski M, de Mello RM, Fonseca B, Oliveira RF, Lucena C, de Paes RB (2018) Identifying design problems in the source code: a grounded theory. In: Chaudron M, Crnkovic I, Chechik M, Harman M (eds) Proceedings of the 40th international conference on software engineering, ICSE 2018, Gothenburg, Sweden, May 27 - June 03, 2018. ACM, pp 921–931.
[58]
Sousa LS, Oizumi WN, Garcia A, Oliveira A, Cedrim D, Lucena C (2020) When are smells indicators of architectural refactoring opportunities: a study of 50 software projects. In: ICPC ’20: 28th international conference on program comprehension, Seoul, Republic of Korea, July 13-15, 2020. ACM, pp 354–365.
[59]
Spearman C The proof and measurement of association between two things Am J Psychol 1904 15 1 72-101
[60]
Steinwart I, Christmann A (2008) Support vector machines. Springer Science & Business Media
[61]
Surhone LM, Timpledon MT, Marseken SF (2010) Shapiro-Wilk test. VDM Publishing
[62]
van Solingen R, Basili V, Caldiera G, Rombach H D (2002) Goal question metric (GQM) approach, Wiley, New York
[63]
Vargha A and Delaney HD A critique and improvement of the cl common language effect size statistics of McGraw and Wong J Educ Behav Stat 2000 25 2 101-132
[64]
Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, and Wesslén A Experimentation in software engineering: an introduction 2000 Norwell Kluwer Academic Publishers
[65]
Yamashita A, Moonen L (2013) Exploring the impact of inter-smell relations on software maintainability: an empirical study. In: Proceedings of the 2013 international conference on software engineering, ICSE ’13. IEEE Press, Piscataway, pp 682–691. http://dl.acm.org/citation.cfm?id=2486788.2486878
Index Terms
- Developers’ perception matters: machine learning to detect developer-sensitive smells
Index terms have been assigned to the content through auto-classification.
Recommendations
Applying Machine Learning to Customized Smell Detection: A Multi-Project Study
SBES '20: Proceedings of the XXXIV Brazilian Symposium on Software EngineeringCode smells are considered symptoms of poor implementation choices, which may hamper the software maintainability. Hence, code smells should be detected as early as possible to avoid software quality degradation. Unfortunately, detecting code smells is ...
Automatic detection of Feature Envy and Data Class code smells using machine learning
AbstractCode smells in software indicate poor design and implementation choices. Detecting and removing them is critical for sustainable software development. Machine learning (ML) can automate code smell detection. Most ML solutions train models from ...
Highlights- We detect Feature Envy and Data Class code smells using pre-trained code embeddings.
- We compare handcrafted code metrics with automatically inferred code embeddings.
- We test the performance of smell detectors on the large manually ...
Are you smelling it? Investigating how similar developers detect code smells
ContextA code smell indicates a poor implementation choice that often worsens software quality. Thus, code smell detection is an elementary technique to identify refactoring opportunities in software systems. Unfortunately, there is limited knowledge on ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022. Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 01 December 2022
Accepted: 29 August 2022
Author Tags
Qualifiers
- Research-article
Funding Sources
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro
- Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro
- Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro
- Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in