Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3510003.3510213acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Modeling review history for reviewer recommendation: a hypergraph approach

Published: 05 July 2022 Publication History

Abstract

Modern code review is a critical and indispensable practice in a pull-request development paradigm that prevails in Open Source Software (OSS) development. Finding a suitable reviewer in projects with massive participants thus becomes an increasingly challenging task. Many reviewer recommendation approaches (recommenders) have been developed to support this task which apply a similar strategy, i.e. modeling the review history first then followed by predicting/recommending a reviewer based on the model. Apparently, the better the model reflects the reality in review history, the higher recommender's performance we may expect. However, one typical scenario in a pull-request development paradigm, i.e. one Pull-Request (PR) (such as a revision or addition submitted by a contributor) may have multiple reviewers and they may impact each other through publicly posted comments, has not been modeled well in existing recommenders. We adopted the hypergraph technique to model this high-order relationship (i.e. one PR with multiple reviewers herein) and developed a new recommender, namely HGRec, which is evaluated by 12 OSS projects with more than 87K PRs, 680K comments in terms of accuracy and recommendation distribution. The results indicate that HGRec outperforms the state-of-the-art recommenders on recommendation accuracy. Besides, among the top three accurate recommenders, HGRec is more likely to recommend a diversity of reviewers, which can help to relieve the core reviewers' workload congestion issue. Moreover, since HGRec is based on hypergraph, which is a natural and interpretable representation to model review history, it is easy to accommodate more types of entities and realistic relationships in modern code review scenarios. As the first attempt, this study reveals the potentials of hypergraph on advancing the pragmatic solutions for code reviewer recommendation.

References

[1]
Wisam Haitham Abbood Al-Zubaidi, Patanamon Thongtanunam, Hoa Khanh Dam, Chakkrit Tantithamthavorn, and Aditya Ghose. 2020. Workload-aware reviewer recommendation using a multi-objective search-based approach. In Proceedings of the 16th International Conference on Predictive Models and Data Analytics in Software Engineering (PROMISE' 20). ACM, 21--30.
[2]
Omar Alonso, Premkumar T Devanbu, and Michael Gertz. 2008. Expertise identification and visualization from cvs. In Proceedings of the 5th international working conference on Mining software repositories (MSR '08). IEEE, 125--128.
[3]
John Anvik, Lyndon Hiew, and Gail C Murphy. 2006. Who should fix this bug?. In Proceedings of the 28th International Conference on Software Engineering (ICSE' 06). ACM, 361--370.
[4]
Sumit Asthana, Rahul Kumar, Ranjita Bhagwan, Christian Bird, Chetan Bansal, Chandra Maddila, Sonu Mehta, and B Ashok. 2019. WhoDo: automating reviewer suggestions at scale. In Proceedings of the 27th Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE' 19). ACM, 937--945.
[5]
Alberto Bacchelli and Christian Bird. 2013. Expectations, outcomes, and challenges of modern code review. In Proceedings of the 35th International Conference on Software Engineering (ICSE' 13). IEEE, 712--721.
[6]
Olga Baysal, Oleksii Kononenko, Reid Holmes, and Michael W Godfrey. 2013. The influence of non-technical factors on code review. In Proceedings of the 20th Working Conference on Reverse Engineering (WCRE' 13). IEEE, 122--131.
[7]
Moritz Beller, Alberto Bacchelli, Andy Zaidman, and Elmar Juergens. 2014. Modern code reviews in open-source projects: Which problems do they fix?. In Proceedings of the 11th Working Conference on Mining Software Repositories (MSR' 14). ACM, 202--211.
[8]
Amiangshu Bosu and Jeffrey C Carver. 2013. Impact of peer code review on peer impression formation: a survey. In Proceedings of the 7th International Symposium on Empirical Software Engineering and Measurement (ESEM' 13). IEEE, 133--142.
[9]
Amiangshu Bosu, Jeffrey C Carver, Christian Bird, Jonathan Orbeck, and Christopher Chockley. 2016. Process aspects and social dynamics of contemporary code review: insights from open source development and industrial practice at microsoft. IEEE Transactions on Software Engineering 43, 1 (2016), 56--75.
[10]
Jiajun Bu, Shulong Tan, Chun Chen, Can Wang, Hao Wu, Lijun Zhang, and Xiaofei He. 2010. Music recommendation by unified hypergraph: combining social media information and music content. In Proceedings of the 18th International Conference on Multimedia (MM' 10). ACM, 391--400.
[11]
Manoel Limeira de Lima Júnior, Daricélio Moreira Soares, Alexandre Plastino, and Leonardo Murta. 2015. Developers assignment for analyzing pull requests. In Proceedings of the 30th annual ACM symposium on applied computing (SAC' 15). ACM, 1567--1572.
[12]
Manoel Limeira de Lima Júnior, Daricélio Moreira Soares, Alexandre Plastino, and Leonardo Murta. 2018. Automatic assignment of integrators to pull requests: the importance of selecting appropriate attributes. Journal of Systems and Software 144 (2018), 181--196.
[13]
Olivier Duchenne, Francis Bach, In-So Kweon, and Jean Ponce. 2011. A tensor-based algorithm for high-order graph matching. IEEE transactions on pattern analysis and machine intelligence 33, 12 (2011), 2383--2395.
[14]
Tudor Girba, Adrian Kuhn, Mauricio Seeberger, and Stéphane Ducasse. 2005. How developers drive software evolution. In Proceedings of the 8th international workshop on principles of software evolution (IWPSE'05). IEEE, 113--122.
[15]
Nils Göde and Rainer Koschke. 2011. Frequency and risks of changes to clones. In Proceedings of the 33rd International Conference on Software Engineering (ICSE '11). 311--320.
[16]
Christoph Hannebauer, Michael Patalas, Sebastian Stünkel, and Volker Gruhn. 2016. Automatically recommending code reviewers based on their expertise: an empirical comparison. In Proceedings of the 31st International Conference on Automated Software Engineering (ASE' 16). IEEE/ACM, 99--110.
[17]
Gaeul Jeong, Sunghun Kim, Thomas Zimmermann, and Kwangkeun Yi. 2009. Improving code review by predicting reviewers and acceptance of patches. Research on software analysis for error-free computing center Tech-Memo (2009), 1--18.
[18]
Jing Jiang, JiaHuan He, and XueYuan Chen. 2015. Coredevrec: automatic core member recommendation for contribution evaluation. Journal of Computer Science and Technology 30, 5 (2015), 998--1016.
[19]
Jing Jiang, David Lo, Jiateng Zheng, Xin Xia, Yun Yang, and Li Zhang. 2019. Who should make decision on this pull request? Analyzing time-decaying relationships and file similarities for integrator prediction. Journal of Systems and Software 154 (2019), 196--210.
[20]
Jiajun Jiang, Luyao Ren, Yingfei Xiong, and Lingming Zhang. 2019. Inferring program transformations from singular examples via big code. In Proceedings of the 34th International Conference on Automated Software Engineering (ASE '19). IEEE, 255--266.
[21]
Jing Jiang, Yun Yang, Jiahuan He, Xavier Blanc, and Li Zhang. 2017. Who should comment on this pull request? analyzing attributes for more accurate commenter recommendation in pull-based development. Information and Software Technology 84 (2017), 48--62.
[22]
Jirayus Jiarpakdee, Chakkrit Tantithamthavorn, Hoa Khanh Dam, and John Grundy. 2020. An empirical study of model-agnostic techniques for defect prediction models. IEEE Transactions on Software Engineering (2020), 1--1.
[23]
Jirayus Jiarpakdee, Chakkrit Kla Tantithamthavorn, and John Grundy. 2021. Practitioners' perceptions of the goals and visual explanations of defect prediction models. In Proceedings of the 18th International Conference on Mining Software Repositories (MSR'21). IEEE, 432--443.
[24]
Oleksii Kononenko, Olga Baysal, and Michael W Godfrey. 2016. Code review quality: how developers see it. In Proceedings of the 38th International Conference on Software Engineering (ICSE' 16). ACM, 1028--1038.
[25]
Oleksii Kononenko, Tresa Rose, Olga Baysal, Michael Godfrey, Dennis Theisen, and Bart De Water. 2018. Studying pull request merges: a case study of shopify's active merchant. In Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP' 18). IEEE/ACM, 124--133.
[26]
Zhifang Liao, ZeXuan Wu, Yanbing Li, Yan Zhang, Xiaoping Fan, and Jinsong Wu. 2019. Core-reviewer recommendation based on pull request topic model and collaborator social network. Soft Computing 24, 8 (2019), 1--11.
[27]
Tsau Young Lin and I-Jen Chiang. 2005. A simplicial complex, a hypergraph, structure in the latent semantic space of document clustering. International Journal of approximate reasoning 40, 1--2 (2005), 55--80.
[28]
Jakub Lipcak and Bruno Rossi. 2018. A large-scale study on source code reviewer recommendation. In Proceedings of the 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA' 18). IEEE, 378--387.
[29]
Anam Luqman, Muhammad Akram, Ahmad N Al-Kenani, and José Carlos R Alcantud. 2019. A study on hypergraph representations of complex fuzzy information. Symmetry 11, 11 (2019), 1381--1408.
[30]
Laura MacLeod, Michaela Greiler, Margaret-Anne Storey, Christian Bird, and Jacek Czerwonka. 2017. Code reviewing in the trenches: challenges and best practices. IEEE Software 35, 4 (2017), 34--42.
[31]
David W McDonald and Mark S Ackerman. 2000. Expertise recommender: a flexible recommendation system and architecture. In Proceedings of the 12th ACM conference on Computer supported cooperative work (CSCW'00). ACM, 231--240.
[32]
Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E Hassan. 2014. The impact of code review coverage and code review participation on software quality: a case study of the qt, vtk, and itk projects. In Proceedings of the 11th Working Conference on Mining Software Repositories (MSR '14). ACM, 192--201.
[33]
Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E Hassan. 2016. An empirical study of the impact of modern code review practices on software quality. Empirical Software Engineering 21, 5 (2016), 2146--2189.
[34]
Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence 267 (2019), 1--38.
[35]
Ehsan Mirsaeedi and Peter C Rigby. 2020. Mitigating turnover with code review recommendation: balancing expertise, workload, and knowledge distribution. In Proceedings of the 42nd International Conference on Software Engineering (ICSE' 20). IEEE/ACM, 1183--1195.
[36]
Ali Ouni, Raula Gaikovina Kula, and Katsuro Inoue. 2016. Search-based peer reviewers recommendation in modern code review. In Proceedings of the 32nd International Conference on Software Maintenance and Evolution (ICSME '16). IEEE, 367--377.
[37]
Xavier Ouvrard, Jean-Marie Le Goff, and Stéphane Marchand-Maillet. 2018. Hypergraph modeling and visualisation of complex co-occurence networks. Electronic Notes in Discrete Mathematics 70 (2018), 65--70.
[38]
Mohammad Masudur Rahman, Chanchal K Roy, Jesse Redl, and Jason A Collins. 2016. CORRECT: code reviewer recommendation at github for vendasta technologies. In Proceedings of the 31st International Conference on Automated Software Engineering (ASE' 16). IEEE/ACM, 792--797.
[39]
Soumaya Rebai, Abderrahmen Amich, Somayeh Molaei, Marouane Kessentini, and Rick Kazman. 2020. Multi-objective code reviewer recommendations: balancing expertise, availability and collaborations. Automated Software Engineering 27, 3 (2020), 301--328.
[40]
Shade Ruangwan, Patanamon Thongtanunam, Akinori Ihara, and Kenichi Matsumoto. 2019. The impact of human factors on the participation decision of reviewers in modern code review. Empirical Software Engineering 24, 2 (2019), 973--1016.
[41]
Caitlin Sadowski, Emma Söderberg, Luke Church, Michal Sipko, and Alberto Bacchelli. 2018. Modern code review: a case study at google. In Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP' 18). IEEE/ACM, 181--190.
[42]
David Schuler and Thomas Zimmermann. 2008. Mining usage expertise from version archives. In Proceedings of the 5th Working Conference on Mining Software Repositories (MSR' 08). IEEE, 121--124.
[43]
Junji Shimagaki, Yasutaka Kamei, Shane McIntosh, Ahmed E Hassan, and Naoyasu Ubayashi. 2016. A study of the quality-impacting practices of modern code review at sony mobile. In Proceedings of the 38th International Conference on Software Engineering Companion (ICSE-C '16). ACM, 212--221.
[44]
Emre Sülün, Eray Tüzün, and Uğur Doğrusöz. 2019. Reviewer recommendation using software artifact traceability graphs. In Proceedings of the 15th International Conference on Predictive Models and Data Analytics in Software Engineering (PROMISE' 19). ACM, 66--75.
[45]
Emre Sülün, Eray Tüzün, and Uğur Doğrusöz. 2021. RSTrace+: reviewer suggestion using software artifact traceability graphs. Information and Software Technology 130 (2021), 106455.
[46]
Shulong Tan, Jiajun Bu, Chun Chen, Bin Xu, Can Wang, and Xiaofei He. 2011. Using rich social media information for music recommendation via hypergraph model. ACM Transactions on Multimedia Computing, Communications, and Applications 7, 1 (2011), 1--22.
[47]
Chakkrit Kla Tantithamthavorn and Jirayus Jiarpakdee. 2021. Explainable ai for software engineering. In Proceedings of the 36th International Conference on Automated Software Engineering (ASE'21). IEEE, 1--2.
[48]
Julian Thomé, Lwin Khin Shar, Domenico Bianculli, and Lionel Briand. 2017. Search-driven string constraint solving for vulnerability detection. In Proceedings of the 39th International Conference on Software Engineering (ICSE '17). IEEE, 198--208.
[49]
Christopher Thompson and David Wagner. 2017. A large-scale study of modern code review and security in open source projects. In Proceedings of the 13th International Conference on Predictive Models and Data Analytics in Software Engineering (PROMISE' 17). IEEE, 83--92.
[50]
Patanamon Thongtanunam, Raula Gaikovina Kula, Ana Erika Camargo Cruz, Norihiro Yoshida, and Hajimu Iida. 2014. Improving code review effectiveness through reviewer recommendations. In Proceedings of the 7th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE' 14). ACM, 119--122.
[51]
Patanamon Thongtanunam, Chakkrit Tantithamthavorn, Raula Gaikovina Kula, Norihiro Yoshida, Hajimu Iida, and Ken-ichi Matsumoto. 2015. Who should review my code? a file location-based code-reviewer recommendation approach for modern code review. In Proceedings of the 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER' 15). IEEE, 141--150.
[52]
Jason Tsay, Laura Dabbish, and James Herbsleb. 2014. Let's talk about it: evaluating contributions through discussion in github. In Proceedings of the 22nd Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE' 14). ACM, 144--154.
[53]
Zhenglin Xia, Hailong Sun, Jing Jiang, Xu Wang, and Xudong Liu. 2017. A hybrid approach to code reviewer recommendation with collaborative filtering. In Proceedings of the 6th International Workshop on Software Mining (SoftwareMining' 17). IEEE, 24--31.
[54]
Cheng Yang, Xunhui Zhang, Lingbin Zeng, Qiang Fan, Tao Wang, Yue Yu, Gang Yin, and Huaimin Wang. 2018. RevRec: a two-layer reviewer recommendation algorithm in pull-based development model. Journal of Central South University 25, 5 (2018), 1129--1143.
[55]
Haochao Ying, Liang Chen, Tingting Liang, and Jian Wu. 2016. Earec: leveraging expertise and authority for pull-request reviewer recommendation in github. In Proceedings of the 3rd International Workshop on CrowdSourcing in Software Engineering (CSI-SE' 16). IEEE/ACM, 29--35.
[56]
Yue Yu, Huaimin Wang, Vladimir Filkov, Premkumar Devanbu, and Bogdan Vasilescu. 2015. Wait for it: determinants of pull request evaluation latency on github. In Proceedings of the 12th Working Conference on Mining Software Repositories (MSR '15). IEEE, 367--371.
[57]
Yue Yu, Huaimin Wang, Gang Yin, and Charles X Ling. 2014. Who should review this pull-request: reviewer recommendation to expedite crowd collaboration. In Proceedings of the 21st Asia-Pacific Software Engineering Conference (APSEC '14, Vol. 1). IEEE, 335--342.
[58]
Yue Yu, Huaimin Wang, Gang Yin, and Tao Wang. 2016. Reviewer recommendation for pull-requests in github: what can we learn from code review and bug assignment? Information and Software Technology 74 (2016), 204--218.
[59]
Fiorella Zampetti, Gabriele Bavota, Gerardo Canfora, and Massimiliano Di Penta. 2019. A study on the interplay between pull request review and continuous integration builds. In Proceedings of the 26th International Conference on Software Analysis, Evolution and Reengineering (SANER' 19). IEEE, 38--48.
[60]
Motahareh Bahrami Zanjani, Huzefa Kagdi, and Christian Bird. 2015. Automatically recommending peer reviewers in modern code review. IEEE Transactions on Software Engineering 42, 6 (2015), 530--543.
[61]
Luming Zhang, Yue Gao, Chaoqun Hong, Yinfu Feng, Jianke Zhu, and Deng Cai. 2013. Feature correlation hypergraph: exploiting high-order potentials for multimodal recognition. IEEE transactions on cybernetics 44, 8 (2013), 1408--1419.
[62]
Wei Zhao, Shulong Tan, Ziyu Guan, Boxuan Zhang, Maoguo Gong, Zhengwen Cao, and Quan Wang. 2018. Learning to map social network users by unified manifold alignment on hypergraph. IEEE transactions on neural networks and learning systems 29, 12 (2018), 5834--5846.

Cited By

View all
  • (2025)A time-aware developer recommendation approach based on multi-feature fusionApplied Soft Computing10.1016/j.asoc.2024.112609169(112609)Online publication date: Jan-2025
  • (2024)Unity Is Strength: Collaborative LLM-Based Agents for Code Reviewer RecommendationProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695291(2235-2239)Online publication date: 27-Oct-2024
  • (2024)Code Impact Beyond Disciplinary Boundaries: Constructing a Multidisciplinary Dependency Graph and Analyzing Cross-Boundary ImpactProceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice10.1145/3639477.3639726(122-133)Online publication date: 14-Apr-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICSE '22: Proceedings of the 44th International Conference on Software Engineering
May 2022
2508 pages
ISBN:9781450392211
DOI:10.1145/3510003
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

  • IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. hypergraph
  2. modern code review
  3. reviewer recommendation

Qualifiers

  • Research-article

Funding Sources

  • the National Key Research and Development Program of China
  • the National Natural Science Foundation of China
  • Intergovernmental Bilateral Innovation Project of Jiangsu Province

Conference

ICSE '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)96
  • Downloads (Last 6 weeks)5
Reflects downloads up to 04 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2025)A time-aware developer recommendation approach based on multi-feature fusionApplied Soft Computing10.1016/j.asoc.2024.112609169(112609)Online publication date: Jan-2025
  • (2024)Unity Is Strength: Collaborative LLM-Based Agents for Code Reviewer RecommendationProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695291(2235-2239)Online publication date: 27-Oct-2024
  • (2024)Code Impact Beyond Disciplinary Boundaries: Constructing a Multidisciplinary Dependency Graph and Analyzing Cross-Boundary ImpactProceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice10.1145/3639477.3639726(122-133)Online publication date: 14-Apr-2024
  • (2024)Factoring Expertise, Workload, and Turnover Into Code Review RecommendationIEEE Transactions on Software Engineering10.1109/TSE.2024.336675350:4(884-899)Online publication date: Apr-2024
  • (2024)Distilling Quality Enhancing Comments From Code Reviews to Underpin Reviewer RecommendationIEEE Transactions on Software Engineering10.1109/TSE.2024.335681950:7(1658-1674)Online publication date: Jul-2024
  • (2024)Code Reviewer Recommendation Based on a Hypergraph with Multiplex Relationships2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00049(417-428)Online publication date: 12-Mar-2024
  • (2024)Prioritizing code review requests to improve review efficiency: a simulation studyEmpirical Software Engineering10.1007/s10664-024-10575-030:1Online publication date: 12-Nov-2024
  • (2024)A preliminary investigation on using multi-task learning to predict change performance in code reviewsEmpirical Software Engineering10.1007/s10664-024-10526-929:6Online publication date: 28-Sep-2024
  • (2024)Hypergraphs with Attention on Reviews for Explainable RecommendationAdvances in Information Retrieval10.1007/978-3-031-56027-9_14(230-246)Online publication date: 24-Mar-2024
  • (2023)Evaluating Learning-to-Rank Models for Prioritizing Code Review Requests using Process Simulation2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER56733.2023.00050(461-472)Online publication date: Mar-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media