short-paper

An Empirical Study of Reviewer Recommendation in Pull-based Development Model

Authors:

Huaimin WangAuthors Info & Claims

Internetware '17: Proceedings of the 9th Asia-Pacific Symposium on Internetware

Article No.: 14, Pages 1 - 6

https://doi.org/10.1145/3131704.3131718

Published: 23 September 2017 Publication History

Abstract

Code review is an important process to reduce code defects and improve software quality. However, in social coding communities using the pull-based model, everyone can submit code changes, which increases the required code review efforts. Therefore, there is a great need of knowing the process of code review and analyzing the pre-existing reviewer recommendation algorithms. In this paper, we do an empirical study about the PRs and their reviewers in Rails project. Moreover, we reproduce a popular and effective IR-based code reviewer recommendation algorithm and validate it on our dataset which contains 16,049 PRs. We find that the inactive reviewers are very important to code reviewing process, however, the pre-existing method's recommendation result strongly depends on the activeness of reviewers.

References

[1]

Vipin Balachandran. 2013. Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation. In Software Engineering (ICSE), 2013 35th International Conference on. IEEE, 931--940.

[2]

Barry Boehm and Victor R Basili. 2005. Software defect reduction top 10 list. Foundations of empirical software engineering: the legacy of Victor R. Basili 426 (2005), 37.

[3]

Amiangshu Bosu and Jeffrey C Carver. 2013. Peer code review to prevent security vulnerabilities: An empirical evaluation. In Software Security and Reliability-Companion (SERE-C), 2013 IEEE 7th International Conference on. IEEE, 229--230.

Digital Library

[4]

Gerardo Canfora and Luigi Cerulo. 2006. Supporting change request assignment in open source development. In Proceedings of the 2006 ACM symposium on Applied computing. ACM, 1767--1772.

Digital Library

[5]

Michael E Fagan. 2001. Design and code inspections to reduce errors in program development. In Pioneers and Their Contributions to Software Engineering. Springer, 301--334.

[6]

Gaeul Jeong, Sunghun Kim, and Thomas Zimmermann. 2009. Improving bug triage with bug tossing graphs. In Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering. ACM, 111--120.

Digital Library

[7]

Gaeul Jeong, Sunghun Kim, Thomas Zimmermann, and Kwangkeun Yi. 2009. Improving code review by predicting reviewers and acceptance of patches. Research on Software Analysis for Error-free Computing Center Tech-Memo (ROSAEC MEMO 2009-006) (2009), 1--18.

[8]

Huzefa Kagdi and Denys Poshyvanyk. 2009. Who can help me with this change request?. In Program Comprehension, 2009. ICPC'09. IEEE 17th International Conference on. IEEE, 273--277.

[9]

Sami Kollanus and Jussi Koskinen. 2009. Survey of software inspection research. The Open Software Engineering Journal 3, 1 (2009), 15--34.

[10]

Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E Hassan. 2014. The impact of code review coverage and code review participation on software quality: A case study of the qt, vtk, and itk projects. In Proceedings of the 11th Working Conference on Mining Software Repositories. ACM, 192--201.

Digital Library

[11]

Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E Hassan. 2016. An empirical study of the impact of modern code review practices on software quality. Empirical Software Engineering 21, 5 (2016), 2146--2189.

Digital Library

[12]

Rodrigo Morales, Shane McIntosh, and Foutse Khomh. 2015. Do code review practices impact design quality? a case study of the qt, vtk, and itk projects. In Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22nd International Conference on. IEEE, 171--180.

[13]

Murtuza Mukadam, Christian Bird, and Peter C Rigby. 2013. Gerrit software code review data from android. In Mining Software Repositories (MSR), 2013 10th IEEE Working Conference on. IEEE, 45--48.

[14]

Mohammad Masudur Rahman, Chanchal K Roy, and Jason A Collins. 2016. CoRReCT: code reviewer recommendation in GitHub based on cross-project and technology experience. In Proceedings of the 38th International Conference on Software Engineering Companion. ACM, 222--231.

Digital Library

[15]

Peter C Rigby and Margaret-Anne Storey. 2011. Understanding broadcast based peer review on open source software projects. In Proceedings of the 33rd International Conference on Software Engineering. ACM, 541--550.

Digital Library

[16]

Patanamon Thongtanunam, Chakkrit Tantithamthavorn, Raula Gaikovina Kula, Norihiro Yoshida, Hajimu Iida, and Ken-ichi Matsumoto. 2015. Who should review my code? A file location-based code-reviewer recommendation approach for modern code review. In Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22nd International Conference on. IEEE, 141--150.

[17]

Yue Yu, Huaimin Wang, Vladimir Filkov, Premkumar Devanbu, and Bogdan Vasilescu. 2015. Wait For It: Determinants of Pull Request Evaluation Latency on GitHub. In MSR.

[18]

Yue Yu, Huaimin Wang, Gang Yin, and Charles X Ling. 2014. Reviewer recommender of pull-requests in GitHub. In Software Maintenance and Evolution (ICSME), 2014 IEEE International Conference on. IEEE, 609--612.

Digital Library

[19]

Yue Yu, Huaimin Wang, Gang Yin, and Charles X Ling. 2014. Who should review this pull-request: Reviewer recommendation to expedite crowd collaboration. In Software Engineering Conference (APSEC), 2014 21st Asia-Pacific, Vol. 1. IEEE, 335--342.

Digital Library

[20]

Yue Yu, Huaimin Wang, Gang Yin, and Tao Wang. 2016. Reviewer recommendation for pull-requests in GitHub: What can we learn from code review and bug assignment? Information and Software Technology 74 (2016), 204--218.

Digital Library

[21]

Yue Yu, Gang Yin, Tao Wang, Cheng Yang, and Huaimin Wang. 2016. Determinants of pull-based development in the context of continuous integration. Science China Information Sciences 59, 8 (2016), 080104.

[22]

Xunhui Zhang, Tao Wang, Gang Yin, Cheng Yang, and Huaimin Wang. 2017. Who Will be Interested in? A Contributor Recommendation Approach for Open Source Projects. In Proceedings of the 29th International Conference on Software Engineering & Knowledge Engineering Accepted.

[23]

Xunhui Zhang, Tao Wang, Gang Yin, Cheng Yang, Yue Yu, and Huaimin Wang. 2017. DevRec: A Developer Recommendation System for Open Source Repositories. In International Conference on Software Reuse. Springer, 3--11.

Cited By

Davila NNunes IWiese I(2025)A fine-grained taxonomy of code review feedback in TypeScript projectsEmpirical Software Engineering10.1007/s10664-024-10604-y30:2Online publication date: 14-Jan-2025
https://doi.org/10.1007/s10664-024-10604-y
Rong GYu YZhang YZhang HShen HShao DKuang HWang MWei ZXu YWang J(2024)Distilling Quality Enhancing Comments From Code Reviews to Underpin Reviewer RecommendationIEEE Transactions on Software Engineering10.1109/TSE.2024.335681950:7(1658-1674)Online publication date: Jul-2024
https://doi.org/10.1109/TSE.2024.3356819
Badampudi DUnterkalmsteiner MBritto R(2023)Modern Code Reviews—Survey of Literature and PracticeACM Transactions on Software Engineering and Methodology10.1145/358500432:4(1-61)Online publication date: 26-May-2023
https://dl.acm.org/doi/10.1145/3585004
Show More Cited By

Recommendations

CoRReCT: code reviewer recommendation in GitHub based on cross-project and technology experience
ICSE '16: Proceedings of the 38th International Conference on Software Engineering Companion

Peer code review locates common coding rule violations and simple logical errors in the early phases of software development, and thus reduces overall cost. However, in GitHub, identifying an appropriate code reviewer for a pull request is a non-trivial ...
CORRECT: code reviewer recommendation at GitHub for Vendasta technologies
ASE '16: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

Peer code review locates common coding standard violations and simple logical errors in the early phases of software development, and thus, reduces overall cost. Unfortunately, at GitHub, identifying an appropriate code reviewer for a pull request is ...
Code reviewer recommendation in tencent: practice, challenge, and direction
ICSE-SEIP '22: Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice

Code review is essential for assuring system quality in software engineering. Over decades in practice, code review has evolved to be a lightweight tool-based process focusing on code change: the smallest unit of the development cycle, and we refer to it ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

Internetware '17: Proceedings of the 9th Asia-Pacific Symposium on Internetware

September 2017

172 pages

ISBN:9781450353137

DOI:10.1145/3131704

Conference Chairs:
Hong Mei,
Jian Lyu,
Zhi Jin,
Wenyun Zhao

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Fudan University
SIGSOFT: ACM Special Interest Group on Software Engineering
CCF: China Computer Federation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 September 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Funding Sources

National Natural Science Foundation of China
national key R&D program of China

Conference

Internetware'17

Internetware'17: The Eighth Asia-Pacific Symposium on Internetware

September 23, 2017

Shanghai, China

Acceptance Rates

Overall Acceptance Rate 55 of 111 submissions, 50%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
193
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Davila NNunes IWiese I(2025)A fine-grained taxonomy of code review feedback in TypeScript projectsEmpirical Software Engineering10.1007/s10664-024-10604-y30:2Online publication date: 14-Jan-2025
https://doi.org/10.1007/s10664-024-10604-y
Rong GYu YZhang YZhang HShen HShao DKuang HWang MWei ZXu YWang J(2024)Distilling Quality Enhancing Comments From Code Reviews to Underpin Reviewer RecommendationIEEE Transactions on Software Engineering10.1109/TSE.2024.335681950:7(1658-1674)Online publication date: Jul-2024
https://doi.org/10.1109/TSE.2024.3356819
Badampudi DUnterkalmsteiner MBritto R(2023)Modern Code Reviews—Survey of Literature and PracticeACM Transactions on Software Engineering and Methodology10.1145/358500432:4(1-61)Online publication date: 26-May-2023
https://dl.acm.org/doi/10.1145/3585004
Panichella SCanfora GDi Sorbo A(2021)“Won’t We Fix this Issue?” Qualitative characterization and automated identification of wontfix issues on GitHubInformation and Software Technology10.1016/j.infsof.2021.106665139:COnline publication date: 23-Aug-2021
https://dl.acm.org/doi/10.1016/j.infsof.2021.106665
Hu YWang JHou JLi SWang Q(2020)Is There A "Golden" Rule for Code Reviewer Recommendation? : —An Experimental Evaluation2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)10.1109/QRS51102.2020.00069(497-508)Online publication date: Dec-2020
https://doi.org/10.1109/QRS51102.2020.00069
Zhang WPan ZWang Z(2020)Prediction Method of Code Review Time Based on Hidden Markov ModelWeb Information Systems and Applications10.1007/978-3-030-60029-7_15(168-175)Online publication date: 22-Sep-2020
https://doi.org/10.1007/978-3-030-60029-7_15
Badampudi DBritto RUnterkalmsteiner MAli SGarousi V(2019)Modern code reviews - Preliminary results of a systematic mapping studyProceedings of the 23rd International Conference on Evaluation and Assessment in Software Engineering10.1145/3319008.3319354(340-345)Online publication date: 15-Apr-2019
https://dl.acm.org/doi/10.1145/3319008.3319354
Yang CZhang XZeng LFan QWang TYu YYin GWang H(2018)RevRec: A two-layer reviewer recommendation algorithm in pull-based development modelRevREC:一个基于Pull-Request 开发模型的双层审阅人推荐算法Journal of Central South University10.1007/s11771-018-3812-x25:5(1129-1143)Online publication date: 18-May-2018
https://doi.org/10.1007/s11771-018-3812-x

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents