research-article

Factors influencing code review processes in industry

Authors:

Kurt SchneiderAuthors Info & Claims

FSE 2016: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering

Pages 85 - 96

https://doi.org/10.1145/2950290.2950323

Published: 01 November 2016 Publication History

Abstract

Code review is known to be an efficient quality assurance technique. Many software companies today use it, usually with a process similar to the patch review process in open source software development. However, there is still a large fraction of companies performing almost no code reviews at all. And the companies that do code reviews have a lot of variation in the details of their processes. For researchers trying to improve the use of code reviews in industry, it is important to know the reasons for these process variations. We have performed a grounded theory study to clarify process variations and their rationales. The study is based on interviews with software development professionals from 19 companies. These interviews provided insights into the reasons and influencing factors behind the adoption or non-adoption of code reviews as a whole as well as for different process variations. We have condensed these findings into seven hypotheses and a classification of the influencing factors. Our results show the importance of cultural and social issues for review adoption. They trace many process variations to differences in development context and in desired review effects.

References

[1]

S. Adolph, W. Hall, and P. Kruchten. Using grounded theory to study the experience of software development. Empirical Software Engineering, 16(4):487–513, 2011.

Digital Library

[2]

A. Bacchelli and C. Bird. Expectations, outcomes, and challenges of modern code review. In Proceedings of the 2013 International Conference on Software Engineering, pages 712–721. IEEE Press, 2013.

Digital Library

[3]

R. A. Baker Jr. Code reviews enhance software quality. In Proceedings of the 19th International Conference on Software Engineering, pages 570–571. ACM, 1997.

Digital Library

[4]

T. Baum, O. Liskin, K. Niklas, and K. Schneider. A faceted classification scheme for change-based industrial code review processes. In Software Quality, Reliability and Security (QRS), 2016 IEEE International Conference on. IEEE, 2016.

[5]

M. Bernhart and T. Grechenig. On the understanding of programs with continuous code reviews. In Program Comprehension (ICPC), 2013 IEEE 21st International Conference on, pages 192–198. IEEE, 2013.

[6]

K. Charmaz. Constructing Grounded Theory. Introducing Qualitative Methods series. SAGE Publications, 2014.

[7]

M. Ciolkowski, O. Laitenberger, and S. Biffl. Software reviews: The state of the practice. IEEE software, 20(6):46–51, 2003.

Digital Library

[8]

P. Clarke and R. V. O’Connor. The situational factors that affect the software development process: Towards a comprehensive reference framework. Information and Software Technology, 54(5):433–447, 2012.

Digital Library

[9]

G. Coleman and R. O’Connor. Using grounded theory to understand software process improvement: A study of irish software product companies. Information and Software Technology, 49(6):654–667, 2007.

Digital Library

[10]

G. Coleman and R. V. O’Connor. An investigation into software development process formation in software start-ups. Journal of Enterprise Information Management, 21(6):633–648, 2008.

[11]

J. Corbin and A. Strauss. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. SAGE Publications, 3e edition, 2007.

[12]

C. Denger and F. Shull. A practical approach for quality-driven inspections. Software, IEEE, 24(2):79–86, 2007.

Digital Library

[13]

M. Fagan. Design and code inspections to reduce errors in program development. IBM Systems Journal, 15(3):182–211, 1976.

Digital Library

[14]

M. Fagan. A history of software inspections. In Software pioneers, pages 562–573. Springer, 2002.

Digital Library

[15]

S. Friese. Qualitative Data Analysis with ATLAS.ti. SAGE Publications, 2012.

[16]

B. Glaser and A. Strauss. The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine, 1967.

[17]

P. Green II, T. Menzies, S. Williams, and O. El-Rawas. Understanding the value of software engineering technologies. In Proceedings of the 2009 IEEE/ACM International Conference on Automated Software Engineering, pages 52–61. IEEE Computer Society, 2009.

Digital Library

[18]

L. Harjumaa, I. Tervonen, and A. Huttunen. Peer reviews in real life-motivators and demotivators. In Quality Software, 2005.(QSIC 2005). Fifth International Conference on, pages 29–36. IEEE, 2005.

Digital Library

[19]

J. Iisakka and I. Tervonen. The darker side of inspection. In Proceedings of Workshop on Inspection in Software Engineering (WISE 2001), Paris, 2001.

[20]

P. Jalote and M. Haragopal. Overcoming the nah syndrome for inspection deployment. In Proceedings of the 20th International Conference on Software Engineering, pages 371–378. IEEE Computer Society, 1998.

Digital Library

[21]

M. Komssi, M. Kauppinen, M. Pyhajarvi, J. Talvio, and T. Mannisto. Persuading software development teams to document inspections: success factors and challenges in practice. In Requirements Engineering Conference (RE), 2010 18th IEEE International, pages 283–288. IEEE, 2010.

Digital Library

[22]

L. P. W. Land and J. Higgs. Factors contributing to software quality practices-an australian case study. In Wireless Communications, Networking and Mobile Computing, 2007. WiCom 2007. International Conference on, pages 5149–5152. IEEE, 2007.

[23]

M. V. Mantyla and C. Lassenius. What types of defects are really discovered in code reviews? Software Engineering, IEEE Transactions on, 35(3):430–448, 2009.

Digital Library

[24]

J. G. March and H. A. Simon. Organizations. John Wiley & Sons, Inc., 1958.

[25]

E. Mustonen-Ollila and K. Lyytinen. Why organizations adopt information system process innovations: a longitudinal study using diffusion of innovation theory. Information Systems Journal, 13(3):275–297, 2003.

[26]

A. J. Onwuegbuzie and N. L. Leech. Validity and qualitative research: An oxymoron? Quality & Quantity, 41(2):233–249, 2007.

[27]

W. J. Orlikowski. Case tools as organizational change: Investigating incremental and radical changes in systems development. MIS quarterly, pages 309–340, 1993.

Digital Library

[28]

A. Porter, H. Siy, A. Mockus, and L. Votta. Understanding the sources of variation in software inspections. ACM Transactions on Software Engineering and Methodology (TOSEM), 7(1):41–79, 1998.

Digital Library

[29]

P. C. Rigby, A. Bacchelli, G. Gousios, and M. Mukadam. A mixed methods approach to mining code review data: Examples and a study of multi-commit reviews and pull requests. In C. Bird, T. Menzies, and T. Zimmermann, editors, The Art and Science of Analyzing Software Data. Morgan Kaufmann, 2015.

[30]

P. C. Rigby and C. Bird. Convergent contemporary software peer review practices. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, pages 202–212. ACM, 2013.

Digital Library

[31]

P. C. Rigby, B. Cleary, F. Painchaud, M. Storey, and D. M. German. Contemporary peer review in action: Lessons from open source development. Software, IEEE, 29(6):56–61, 2012.

Digital Library

[32]

P. C. Rigby and M.-A. Storey. Understanding broadcast based peer review on open source software projects. In Proceedings of the 33rd International Conference on Software Engineering, pages 541–550. ACM, 2011.

Digital Library

[33]

E. M. Rogers. Diffusion of Innovations. Free Press, 5th edition, 2003.

[34]

M.-L. Sánchez-Gordón and R. V. O’Connor. Understanding the gap between software process practices and actual practice in very small companies. Software Quality Journal, pages 1–22, 2015.

[35]

C. Sauer, D. R. Jeffery, L. Land, and P. Yetton. The effectiveness of software development technical reviews: A behaviorally motivated program of research. Software Engineering, IEEE Transactions on, 26(1):1–14, 2000.

Digital Library

[36]

K. Spohrer, T. Kude, C. T. Schmidt, and A. Heinzl. Knowledge creation in information systems development teams: The role of pair programming and peer code review. In ECIS, page 213, 2013.

Cited By

Sarma AChen N(2024)Effective Teaching through Code Reviews: Patterns and Anti-patternsProceedings of the ACM on Software Engineering10.1145/36607641:FSE(1262-1283)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660764
Davila NMelegati JWiese I(2024)Tales From the Trenches: Expectations and Challenges From Practice for Code Review in the Generative AI EraIEEE Software10.1109/MS.2024.342843941:6(38-45)Online publication date: Nov-2024
https://doi.org/10.1109/MS.2024.3428439
Papotti AParamitha RMassacci F(2024)On the acceptance by code reviewers of candidate security patches suggested by Automated Program Repair toolsEmpirical Software Engineering10.1007/s10664-024-10506-z29:5Online publication date: 3-Aug-2024
https://doi.org/10.1007/s10664-024-10506-z
Show More Cited By

Index Terms

Factors influencing code review processes in industry
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Project and people management
2. Software and its engineering
  1. Software creation and management
    1. Software development process management
    2. Software verification and validation
      1. Process validation
        Walkthroughs

Recommendations

The impact of code review coverage and code review participation on software quality: a case study of the qt, VTK, and ITK projects
MSR 2014: Proceedings of the 11th Working Conference on Mining Software Repositories

Software code review, i.e., the practice of having third-party team members critique changes to a software system, is a well-established best practice in both open source and proprietary software domains. Prior work has shown that the formal code ...
An Empirical Study on Interaction Factors Influencing Bug Reopenings
APSEC '14: Proceedings of the 2014 21st Asia-Pacific Software Engineering Conference - Volume 02

Bugs can be reopened after they have been closed due to identification of the actual cause, previous incorrect fixing, or better reproducing, etc. Reopened bugs may increase the cost in maintenance, degrade the overall quality of the software product, ...
Adopting Code Reviews for Agile Software Development
AGILE '10: Proceedings of the 2010 Agile Conference

Code reviews have many benefits, most importantly to find bugs early in the development phase and to enforce coding standards. Still, it is widely accepted that formal code reviews are time-consuming and the practical applicability in agile development ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

FSE 2016: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering

November 2016

1156 pages

ISBN:9781450342186

DOI:10.1145/2950290

General Chair:
Thomas Zimmermann
Microsoft Research, USA
,
Program Chairs:
Jane Cleland-Huang
University of Notre Dame, USA
,
Zhendong Su
University of California at Davis, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

FSE'16

Sponsor:

SIGSOFT

FSE'16: 24nd ACM SIGSOFT International Symposium on the Foundations of Software Engineering

November 13 - 18, 2016

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 17 of 128 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
1,087
Total Downloads

Downloads (Last 12 months)95
Downloads (Last 6 weeks)10

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sarma AChen N(2024)Effective Teaching through Code Reviews: Patterns and Anti-patternsProceedings of the ACM on Software Engineering10.1145/36607641:FSE(1262-1283)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660764
Davila NMelegati JWiese I(2024)Tales From the Trenches: Expectations and Challenges From Practice for Code Review in the Generative AI EraIEEE Software10.1109/MS.2024.342843941:6(38-45)Online publication date: Nov-2024
https://doi.org/10.1109/MS.2024.3428439
Papotti AParamitha RMassacci F(2024)On the acceptance by code reviewers of candidate security patches suggested by Automated Program Repair toolsEmpirical Software Engineering10.1007/s10664-024-10506-z29:5Online publication date: 3-Aug-2024
https://doi.org/10.1007/s10664-024-10506-z
Nanthaamornphong ABoonchieng E(2023)An Exploratory Study on Code Smells during Code Review in OSS Projects: A Case Study on OpenStack and WikiMediaRecent Advances in Computer Science and Communications10.2174/266625581666623022211231316:7Online publication date: Sep-2023
https://doi.org/10.2174/2666255816666230222112313
Yang LXu JZhang YZhang HBacchelli AChandra SBlincoe KTonella P(2023)EvaCRC: Evaluating Code Review CommentsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616245(275-287)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616245
Badampudi DUnterkalmsteiner MBritto R(2023)Modern Code Reviews—Survey of Literature and PracticeACM Transactions on Software Engineering and Methodology10.1145/358500432:4(1-61)Online publication date: 26-May-2023
https://dl.acm.org/doi/10.1145/3585004
Lin HThongtanunam P(2023)Towards Automated Code Reviews: Does Learning Code Structure Help?2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER56733.2023.00075(703-707)Online publication date: Mar-2023
https://doi.org/10.1109/SANER56733.2023.00075
Huang F(2023)Promoting Students' Cognitive Ability to Identify Human Error-Prone Scenarios in Programs2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10343466(1-9)Online publication date: 18-Oct-2023
https://doi.org/10.1109/FIE58773.2023.10343466
Fregnan EFröhlich JSpadini DBacchelli A(2023)Graph-based visualization of merge requests for code reviewJournal of Systems and Software10.1016/j.jss.2022.111506195(111506)Online publication date: Jan-2023
https://doi.org/10.1016/j.jss.2022.111506
Gunawardena STempero EBlincoe K(2023)Concerns identified in code reviewInformation and Software Technology10.1016/j.infsof.2022.107054153:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.infsof.2022.107054
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents