research-article

Metrics to quantify software developer experience: a systematic mapping

Authors:

Renata Brasil-Silva,

Fábio Levy SiqueiraAuthors Info & Claims

SAC '22: Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing

Pages 1562 - 1569

https://doi.org/10.1145/3477314.3507304

Published: 06 May 2022 Publication History

Abstract

The quality of the developers has a direct impact on the software product. Besides the code they write, an experienced developer may define architectural solutions essential to the system's maintainability. This condition is necessary to develop new features and fix defects in a timely and cost-effective manner. Some works have studied the impact of developer experience and use metrics to study the relationship with the maintainability and evolution of software. In this systematic mapping, we present a catalog of metrics that have been used to quantify developers' experience, found in 34 works. These metrics are classified by the type of experience they represent and the purpose of using them. The results showed that most of the selected studies use metrics based on counting developer activities in code.

References

[1]

2019. Influence of Developer Factors on Code Quality: A Data Study. 2019 IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS-C), 120--125.

[2]

Shadi Banitaan and Mamdouh Alenezi. 2013. DECOBA: Utilizing Developers Communities in Bug Assignment. In 2013 12th International Conference on Machine Learning and Applications, Vol. 2. IEEE, 66--71.

Digital Library

[3]

Gunnar R. Bergersen, Jo E. Hannay, Dag I.K. Sjoberg, Tore Dyba, and Amela Karahasanovic. 2011. Inferring Skill from Tests of Programming Performance: Combining Time and Quality. In 2011 International Symposium on Empirical Software Engineering and Measurement. IEEE, 305--314.

Digital Library

[4]

Pamela Bhattacharya, Iulian Neamtiu, and Michalis Faloutsos. 2014. Determining Developers' Expertise and Role: A Graph Hierarchy-Based Approach. In 2014 IEEE International Conference on Software Maintenance and Evolution. IEEE, 11--20. arXiv:arXiv:1108.2059v1

Digital Library

[5]

Christian Bird, Nachiappan Nagappan, Brendan Murphy, Harald Gall, and Premkumar Devanbu. 2011. Don't Touch My Code!: Examining the Effects of Ownership on Software Quality. Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (2011), 4.

Digital Library

[6]

Gul Calikli, Ayse Bener, and Berna Arslan. 2010. An analysis of the effects of company culture, education and experience on confirmation bias levels of software developers and testers. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - ICSE '10, Vol. 2. ACM Press, 187.

Digital Library

[7]

H. Alperen Çetin and Eray Tüzün. 2020. Identifying key developers using artifact traceability graphs. In Proceedings of the 16th ACM International Conference on Predictive Models and Data Analytics in Software Engineering. ACM, New York, NY, USA, 51--60.

Digital Library

[8]

Eleni Constantinou and Georgia M. Kapitsaki. 2016. Identifying Developers' Expertise in Social Coding Platforms. In 2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE, 63--67.

[9]

Théo Coulin, Maxence Detante, William Mouchère, and Fabio Petrillo. 2019. Software Architecture Metrics: a literature review. CoRR abs/1901.0 (jan 2019), 1--15.

[10]

Jose Ricardo da Silva, Esteban Clua, Leonardo Murta, and Anita Sarma. 2015. Niche vs. breadth: Calculating expertise over time through a fine-grained analysis. In 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER). IEEE, 409--418.

[11]

Mivian Ferreira, Marco Tulio Valente, and Kecia Ferreira. 2017. A Comparison of Three Algorithms for Computing Truck Factors. IEEE International Conference on Program Comprehension (2017), 207--217.

Digital Library

[12]

Matthieu Foucault, Jean-Rémy Falleri, and Xavier Blanc. 2014. Code ownership in open-source software. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering - EASE '14. ACM Press, 1--9.

Digital Library

[13]

Matthieu Foucault, Cédric Teyton, David Lo, Xavier Blanc, and Jean-rémy Falleri. 2015. On the usefulness of ownership metrics in open-source software projects. Information and Software Technology 64 (2015), 102--112.

Digital Library

[14]

Michaela Greiler, Kim Herzig, and Jacek Czerwonka. 2015. Code Ownership and Software Quality: A Replication Study. In 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories, Vol. 2015-Augus. IEEE, 2--12.

[15]

Lile Hattori and Michele Lanza. 2009. Mining the history of synchronous changes to refine code ownership. Proceedings of the 2009 6th IEEE International Working Conference on Mining Software Repositories, MSR 2009 (2009), 141--150.

Digital Library

[16]

Lile Palma Hattori, Michele Lanza, and Romain Robbes. 2012. Refining code ownership with synchronous changes. Empirical Software Engineering 17, 4--5 (2012), 467--499.

Digital Library

[17]

Mitchell Joblin, Sven Apel, Claus Hunsen, and Wolfgang Mauerer. 2017. Classifying Developers into Core and Peripheral: An Empirical Study on Count and Network Metrics. Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering, ICSE 2017 (2017), 164--174.

Digital Library

[18]

Georgia M. Kapitsaki and Panagiotis Foutros. 2017. Dear Developers, your Expertise in One Place. In 2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE, 371--374.

[19]

B. Kitchenham and S. Charters. 2007. Guidelines for performing Systematic Literature Reviews in Software Engineering. Technical report 2, 3 (2007).

[20]

Eduard Kuric and Mária Bieliková. 2014. Estimation of student's programming expertise. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement - ESEM '14. ACM Press, New York, New York, USA, 1--4.

Digital Library

[21]

Roberto Latorre. 2014. Effects of Developer Experience on Learning and Applying Unit Test-Driven Development. IEEE Transactions on Software Engineering 40, 4 (apr 2014), 381--395.

Digital Library

[22]

Audris Mockus, Roy T Fielding, and James D Herbsleb. 2002. Two case studies of open source software development: Apache and Mozilla. ACM Transactions on Software Engineering and Methodology 11, 3 (jul 2002), 309--346.

Digital Library

[23]

Audris Mockus and James D Herbsleb. 2002. Expertise Browser: A Quantitative Approach to Identify Expertise. Proceedings of the 24th International Conference on Software Engineering. ICSE 2002 93 (2002), 379--380.

[24]

Johnatan Oliveira, Denis Pinheiro, and Eduardo Figueiredo. 2020. JExpert: A Tool for Library Expert Identification. Proceedings of the 34th Brazilian Symposium on Software Engineering, 386--392. Issue Dcc.

Digital Library

[25]

Tobias Olsson, Morgan Ericsson, and Anna Wingkvist. 2017. The relationship of code churn and architectural violations in the open source software JabRef. (2017), 152--158.

Digital Library

[26]

Matteo Orrú and Michele Marchesi. 2016. A case study on the relationship between code ownership and refactoring activities in a Java software system. In Proceedings of the 7th International Workshop on Emerging Trends in Software Metrics - WETSoM '16. ACM Press, 43--49.

Digital Library

[27]

Seldag Ozcan Kini and Ayse Tosun. 2018. Periodic Developer Metrics in Software Defect Prediction. In 2018 IEEE 18th International Working Conference on Source Code Analysis and Manipulation (SCAM). IEEE, 72--81.

[28]

Kai Petersen, Sairam Vakkalanka, and Ludwik Kuzniarz. 2015. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology 64 (aug 2015), 1--18.

Digital Library

[29]

Yilin Qiu, Weiqiang Zhang, Weiqin Zou, Jia Liu, and Qin Liu. 2015. An Empirical Study of Developer Quality. 2015 IEEE International Conference on Software Quality, Reliability and Security - Companion, 202--209.

Digital Library

[30]

Foyzur Rahman and Premkumar Devanbu. 2011. Ownership, Experience and Defects: A Fine-Grained Study of Authorship. In Proceeding of the 33rd international conference on Software engineering - ICSE '11. ACM Press, 491.

Digital Library

[31]

Mehvish Rashid, Paul M. Clarke, and Rory V. O'Connor. 2019. A systematic examination of knowledge loss in open source software projects. International Journal of Information Management 46, December 2018 (2019), 104--123.

Digital Library

[32]

Romain Robbes and David Rothlisberger. 2013. Using developer interaction data to compare expertise metrics. In 2013 10th Working Conference on Mining Software Repositories (MSR). IEEE, 297--300.

[33]

Pierre N. Robillard. 1999. The role of knowledge in software development. Commun. ACM 42, 1 (jan 1999), 87--92.

Digital Library

[34]

Vinicius Schettino, Vitor Horta, Marco Antonio P. Araujo, and Victor Stroele. 2019. Towards Community and Expert Detection in Open Source Global Development. In 2019 IEEE 23rd International Conference on Computer Supported Cooperative Work in Design (CSCWD). IEEE, 350--355.

[35]

Andreas Schilling, Sven Laumer, and Tim Weitzel. 2012. Who Will Remain? An Evaluation of Actual Person-Job and Person-Team Fit to Predict Developer Retention in FLOSS Projects. In 2012 45th Hawaii International Conference on System Sciences. IEEE, 3446--3455.

Digital Library

[36]

Tamanna Siddiqui and Ausaf Ahmad. 2018. Data Mining Tools and Techniques for Mining Software Repositories : A Data Mining Tools and Techniques for Mining Software Repositories : A Systematic Review. May (2018).

[37]

Srdjan Stevanetic and Uwe Zdun. 2015. Software metrics for measuring the understandability of architectural structures. In Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering - EASE '15. ACM Press, 1--14.

Digital Library

[38]

Ralf Teusner, Christoph Matthies, and Philipp Giese. 2017. Should I Bug You? Identifying Domain Experts in Software Projects Using Code Complexity Metrics. In 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS). IEEE, 418--425.

[39]

Cédric Teyton, Marc Palyart, Jean-Rémy Falleri, Floréal Morandat, and Xavier Blanc. 2014. Automatic extraction of developer expertise. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering - EASE '14. ACM Press, New York, New York, USA, 1--10.

Digital Library

[40]

Patanamon Thongtanunam, Shane McIntosh, Ahmed E. Hassan, and Hajimu Iida. 2016. Revisiting code ownership and its relationship with software quality in the scope of modern code review. Proceedings of the 38th International Conference on Software Engineering - ICSE '16 1 (2016), 1039--1050.

Digital Library

[41]

Christoph Treude, Fernando Figueira Filho, and Uirá Kulesza. 2015. Summarizing and measuring development activity. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering - ESEC/FSE 2015. ACM Press, 625--636.

Digital Library

[42]

Wenjin Wu, Wen Zhang, Ye Yang, and Qing Wang. 2011. DREX: Developer Recommendation with K-Nearest-Neighbor Search and Expertise Ranking. In 2011 18th Asia-Pacific Software Engineering Conference. IEEE, 389--396.

Digital Library

[43]

Yangsong Wu, Yibiao Yang, Yangyang Zhao, Hongmin Lu, Yuming Zhou, and Baowen Xu. 2014. The Influence of Developer Quality on Software Fault-Proneness Prediction. 2014 Eighth International Conference on Software Security and Reliability, 11--19.

Digital Library

[44]

Marvin Wyrich, Daniel Graziotin, and Stefan Wagner. 2019. A theory on individual characteristics of successful coding challenge solvers. PeerJ Computer Science 5, 2 (feb 2019), e173.

[45]

Asmita Yadav, Sandeep Kumar Singh, and Jasjit S. Suri. 2019. Ranking of software developers based on expertise score for bug triaging. Information and Software Technology 112, March (aug 2019), 1--17.

Digital Library

[46]

Feng Zhang, Foutse Khomh, Ying Zou, and Ahmed E. Hassan. 2014. An empirical study of the effect of file editing patterns on software quality. Journal of Software: Evolution and Process 26, 11 (nov 2014), 996--1029.

Digital Library

[47]

Gang Zhou, Yafeng Wu, Ting Yan, Tian He, Chengdu Huang, John A. Stankovic, and Tarek F. Abdelzaher. 2010. A multifrequency MAC specially designed for wireless sensor network applications. ACM Trans. Embed. Comput. Syst. 9, 4, Article 39 (April 2010), 41 pages.

Digital Library

[48]

Minghui Zhou and Audris Mockus. 2010. Developer fluency. In Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering - FSE '10. ACM Press, 137.

Digital Library

Cited By

Lopes JOliveira JFigueiredo E(2024)Evaluating the Impact of Developer Experience on Code Quality: A Systematic Literature ReviewAnais do XXVII Congresso Ibero-Americano em Engenharia de Software (CIbSE 2024)10.5753/cibse.2024.28446(166-180)Online publication date: 6-May-2024
https://doi.org/10.5753/cibse.2024.28446
de Campos VDavid JStröele VBraga R(2024)Aligning technical knowledge to an industry domain in global software development: A systematic mappingJournal of Software: Evolution and Process10.1002/smr.2713Online publication date: 15-Jul-2024
https://doi.org/10.1002/smr.2713

Index Terms

Metrics to quantify software developer experience: a systematic mapping
1. Software and its engineering
  1. Software creation and management

Recommendations

Software metrics in agile software: an empirical study
Agile Processes in Software Engineering and Extreme Programming

This paper presents a software metrics analysis of eight object-oriented systems. Five systems had been developed using Agile methodologies and three using plan-driven methodologies; three systems were written in Python and five in Java. For each system,...
Developer experience: concept and definition
ICSSP '12: Proceedings of the International Conference on Software and System Process

New ways of working such as globally distributed development or the integration of self-motivated external developers into software ecosystems will require a better and more comprehensive understanding of developers' feelings, perceptions, motivations ...
Coupling and Cohesion Metrics for Object-Oriented Software: A Systematic Mapping Study
ISEC '18: Proceedings of the 11th Innovations in Software Engineering Conference

Coupling and Cohesion are two fundamental concepts that can be applied to design better modular object-oriented software. This study aims at reviewing existing research on coupling and cohesion metrics in order to identify the potential ones and needs ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '22: Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing

April 2022

2099 pages

ISBN:9781450387132

DOI:10.1145/3477314

Conference Chairs:
Jiman Hong
Soongsil University
,
Miroslav Bures
Czech Technical University, Czechia
,
Program Chairs:
Juw Won Park
University of Louisville
,
Tomas Cerny
Baylor University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 May 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior

Conference

SAC '22

Sponsor:

SIGAPP

SAC '22: The 37th ACM/SIGAPP Symposium on Applied Computing

April 25 - 29, 2022

Virtual Event

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
248
Total Downloads

Downloads (Last 12 months)118
Downloads (Last 6 weeks)14

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lopes JOliveira JFigueiredo E(2024)Evaluating the Impact of Developer Experience on Code Quality: A Systematic Literature ReviewAnais do XXVII Congresso Ibero-Americano em Engenharia de Software (CIbSE 2024)10.5753/cibse.2024.28446(166-180)Online publication date: 6-May-2024
https://doi.org/10.5753/cibse.2024.28446
de Campos VDavid JStröele VBraga R(2024)Aligning technical knowledge to an industry domain in global software development: A systematic mappingJournal of Software: Evolution and Process10.1002/smr.2713Online publication date: 15-Jul-2024
https://doi.org/10.1002/smr.2713

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents