research-article

Open access

Software security during modern code review: the developer’s perspective

Authors:

Alberto BacchelliAuthors Info & Claims

ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Pages 810 - 821

https://doi.org/10.1145/3540250.3549135

Published: 09 November 2022 Publication History

Abstract

To avoid software vulnerabilities, organizations are shifting security to earlier stages of the software development, such as at code review time. In this paper, we aim to understand the developers’ perspective on assessing software security during code review, the challenges they encounter, and the support that companies and projects provide. To this end, we conduct a two-step investigation: we interview 10 professional developers and survey 182 practitioners about software security assessment during code review. The outcome is an overview of how developers perceive software security during code review and a set of identified challenges. Our study revealed that most developers do not immediately report to focus on security issues during code review. Only after being asked about software security, developers state to always consider it during review and acknowledge its importance. Most companies do not provide security training, yet expect developers to still ensure security during reviews. Accordingly, developers report the lack of training and security knowledge as the main challenges they face when checking for security issues. In addition, they have challenges with third-party libraries and to identify interactions between parts of code that could have security implications. Moreover, security may be disregarded during reviews due to developers’ assumptions about the security dynamic of the application they develop. Preprint: https://arxiv.org/abs/2208.04261 Data and materials: https://doi.org/10.5281/zenodo.6969369

References

[1]

2020. GitLab: Mapping the DevSecOps Landscape - 2020 Survey. https://about.gitlab.com/developer-survey

[2]

A. Ackerman, L. Buchwald, and F. Lewski. 1989. Software inspections: an effective verification process. IEEE Software, 6, 3 (1989), 31–36.

Digital Library

[3]

A. Ackerman, P. Fowler, and R. Ebenau. 1984. Software Inspections and the Industrial Production of Software. In Proceedings of the Symposium on Software Validation: Inspection-Testing-Verification-Alternatives. 13–40.

[4]

Google Alphabet. Last accessed in March 2022. Vulnerability Rewards Program. https://bughunters.google.com/.

[5]

H. Assal and S. Chiasson. 2018. Security in the software development lifecycle. In Proceedings of the symposium on usable privacy and security. 281–296.

[6]

N. Ayewah and W. Pugh. 2008. A report on a survey and study of static analysis users. In Proceedings of the workshop on Defects in large software systems. 1–5.

[7]

N. Ayewah, W. Pugh, D. Hovemeyer, J. Morgenthaler, and J. Penix. 2008. Using static analysis to find bugs. IEEE software, 25, 5 (2008), 22–29.

[8]

A. Bacchelli and C. Bird. 2013. Expectations, outcomes, and challenges of modern code review. In Proceedings of the International Conference on Software Engineering. 712–721.

[9]

V. Balachandran. 2013. Reducing Human Effort and Improving Quality in Peer Code Reviews Using Automatic Static Analysis and Reviewer Recommendation. In Proceedings of the International Conference on Software Engineering. 931–940.

[10]

D. Balfanz, G. Durfee, D. Smetters, and R. Grinter. 2004. In search of usable security: Five lessons from the field. IEEE Security & Privacy, 2, 5 (2004), 19–24.

Digital Library

[11]

T. Baum, O. Liskin, K. Niklas, and K. Schneider. 2016. Factors influencing code review processes in industry. In Proceedings of the international symposium on foundations of software engineering. 85–96.

[12]

B. Boehm and V. Basili. 2001. Software Defect Reduction Top 10 List. 34, 1 (2001), 135–137.

[13]

S. Bratus. 2007. What Hackers Learn that the Rest of Us Don’t: Notes on Hacker Curriculum. IEEE Security Privacy, 5, 4 (2007), 72–75.

Digital Library

[14]

L. Braz, C. Aeberhard, G. Çalikli, and A. Bacchelli. 2022. Less is more: supporting developers in vulnerability detection during code review. In Proceedings of the 44th International Conference on Software Engineering. 1317–1329.

[15]

L. Braz and A. Bacchelli. 2022. Replication Package - “Software Security during Modern Code Review: The Developer’s Perspective”. https://doi.org/10.5281/zenodo.6969369

Digital Library

[16]

L. Braz, E. Fregnan, G. Çalikli, and A. Bacchelli. 2021. Why Don’t Developers Detect Improper Input Validation? ’; DROP TABLE Papers;–. In Proceedings of the International Conference on Software Engineering. 499–511.

[17]

M. Christakis and C. Bird. 2016. What developers want and need from program analysis: an empirical study. In Proceedings of the international conference on automated software engineering. 332–343.

[18]

Cisco. 2014. The Cisco 2014 Annual Security Reporte. http://www.cisco.com/web/offers/lp/2014-annual-securityreport/index.html.

[19]

J. Cohen. 2010. Modern Code Review. In Making Software. O’Reilly, 329–338.

[20]

T. Cook and D. Campbell. 1979. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Houghton Mifflin Company.

[21]

A. Danilova, A. Naiakshina, S. Horstmann, and M. Smith. 2021. Do You Really Code? Designing and Evaluating Screening Questions for Online Surveys with Programmers. 537–548.

[22]

Defcon. Last accessed in March 2022. Capture the Flag Competition. https://defcon.org/html/links/dc-ctf-history.html.

[23]

K. Dempsey, P. Eavy, and G. Moore. 2017. Automation Support for Security Control Assessments. Technical Report NISTIR 8011, National Institute of Standards and Technology.

[24]

A. Edmundson, B. Holtkamp, E. Rivera, M. Finifter, A. Mettler, and D. Wagner. 2013. An empirical study on the effectiveness of security code review. In Proceedings of the International Symposium on Engineering Secure Software and Systems. 197–212.

[25]

S. Eick, J. Steffen, and E. Sumner. 1992. Seesoft-a tool for visualizing line oriented software statistics. Transactions on Software Engineering, 18, 11 (1992), 957–968.

Digital Library

[26]

D. Falessi, N. Juristo, C. Wohlin, B. Turhan, J. Münch, A. Jedlitschka, and M. Oivo. 2018. Empirical Software Engineering Experts on the Use of Students and Professionals in Experiments. Empirical Software Engineering, 23, 1 (2018), 452–489.

Digital Library

[27]

World Economic Forum. 2022. Global Risks Report 2022. https://www.weforum.org/reports/global-risks-report-2022/in-full/chapter-3-digital-dependencies-and-cyber-vulnerabilities/.

[28]

The OWASP Foundation. Last accessed March 2022. OWASP Foundation. https://owasp.org/

[29]

A. Furnham. 1986. Response bias, social desirability and dissimulation. Personality and individual differences, 7, 3 (1986), 385–400.

[30]

M. Galesic and M. Bosnjak. 2009. Effects of questionnaire length on participation and indicators of response quality in a web survey. Public opinion quarterly, 73, 2 (2009), 349–360.

[31]

L. Gasparini, E. Fregnan, L. Braz, T. Baum, and A. Bacchelli. 2021. ChangeViz: Enhancing the GitHub Pull Request Interface with Method Call Information. In Proceedings of the Working Conference on Software Visualization. 115–119.

[32]

N. Golafshani. 2003. Understanding reliability and validity in qualitative research. The qualitative report, 8, 4 (2003), 597–607.

[33]

M. Green and M. Smith. 2016. Developers are not the enemy!: The need for usable security apis. IEEE Security & Privacy, 14, 5 (2016), 40–46.

Digital Library

[34]

D. Hildum and R. Brown. 1956. Verbal reinforcement and interviewer bias. The Journal of Abnormal and Social Psychology, 53, 1 (1956), 108.

[35]

P. Khaloo, M. Maghoumi, E. Taranta, D. Bettner, and J. Laviola. 2017. Code park: A new 3d code visualization tool. In Proceedings of the Working Conference on Software Visualization. 43–53.

[36]

V. Kovalenko, N. Tintarev, E. Pasynkov, C. Bird, and A. Bacchelli. 2018. Does reviewer recommendation help developers? Transactions on Software Engineering, 46, 7 (2018), 710–731.

[37]

T. Lindlof and B. Taylor. 2002. Qualitative communication research methods. Sage.

[38]

S. Lipner. 2004. The trustworthy computing security development lifecycle. In Proceedings of the Annual Computer Security Applications Conference. 2–13.

Digital Library

[39]

A. Mattila, P. Ihantola, T. Kilamo, A. Luoto, M. Nurminen, and H. Väätäjä. 2016. Software visualization today: Systematic literature review. In Proceedings of the 20th International Academic Mindtrek Conference. 262–271.

[40]

McAfee. 2018. The Economic Impact of Cybercrime—No Slowing Down.

[41]

G. McGraw. 2004. Software security. IEEE Security Privacy, 2, 2 (2004), 80–83.

Digital Library

[42]

A. Meneely and L. Williams. 2010. Strengthening the Empirical Analysis of the Relationship between Linus’ Law and Software Security. In Proceedings of the International Symposium on Empirical Software Engineering and Measurement. 1–10.

[43]

A. Meneely and O. Williams. 2012. Interactive Churn Metrics: Socio-Technical Variants of Code Churn. Software Engineering Notes, 37, 6 (2012), 1–6.

Digital Library

[44]

Microsoft. Last accessed in March 2022. Insider risk management in Microsoft 365. https://docs.microsoft.com/en-us/microsoft-365/compliance/insider-risk-management?view=o365-worldwide.

[45]

A. Nichols and J. Maner. 2008. The Good-Subject Effect: Investigating Participant Demand Characteristics. Journal of General Psychology, 135, 2 (2008), 151–165.

[46]

A. Onwuegbuzie and N. Leech. 2007. Validity and qualitative research: An oxymoron? Quality & quantity, 41, 2 (2007), 233–249.

[47]

O. Pieczul, S. Foley, and M. Zurko. 2017. Developer-Centered Security and the Symmetry of Ignorance. In Proceedings of the New Security Paradigms Workshop. Association for Computing Machinery, 46–56.

[48]

XM Platform. Last accessed March 2022. Qualtrics. https://www.qualtrics.com.

[49]

A. Poller, L. Kocksch, S. Trpe, F. Epp, and K. Kinder-Kurlanda. 2017. Can security become a routine? A study of organizational change in an agile software development group. In Proceedings of the conference on computer supported cooperative work and social computing. 2489–2503.

[50]

CWE Project. 2021. CWE-327: Use of a Broken or Risky Cryptographic Algorithm. https://cwe.mitre.org/data/definitions/327.html

[51]

OWASP Project. 2017. OWASP Code Review Guide 2.0. https://owasp.org/www-pdf-archive/OWASP_Code_Review_Guide_v2.pdf

[52]

OWASP Project. 2017. OWASP Top Ten. https://owasp.org/www-project-top-ten

[53]

M. Rahman, C. K. Roy, and J. Collins. 2016. CORRECT: Code Reviewer Recommendation in GitHub Based on Cross-Project and Technology Experience. In Proceedings of the International Conference on Software Engineering Companion (ICSE-C). 222–231.

[54]

P. Rigby and C. Bird. 2013. Convergent contemporary software peer review practices. In Proceedings of the Joint Meeting on Foundations of Software Engineering. 202–212.

[55]

P. Rigby, D. German, L. Cowen, and M. Storey. 2014. Peer review on open-source software projects: Parameters, statistical models, and theory. Transactions on Software Engineering and Methodology, 23, 4 (2014), 1–33.

Digital Library

[56]

C. Sadowski, E. Söderberg, L. Church, M. Sipko, and A. Bacchelli. 2018. Modern code review: a case study at Google. In Proceedings of the International Conference on Software Engineering: Software Engineering in Practice. 181–190.

[57]

Y. Shin, A. Meneely, L. Williams, and J. Osborne. 2011. Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vulnerabilities. Transactions on Software Engineering, 37 (2011), 772–787.

Digital Library

[58]

L. Sigelaman. 1981. Question-order effects on presidential popularity. Public Opinion Quarterly, 45, 2 (1981), 199–207.

[59]

J. Smith, B. Johnson, E. Murphy-Hill, B. Chu, and H. Lipford. 2015. Questions developers ask while diagnosing potential security vulnerabilities with static analysis. In Proceedings of the Joint Meeting on Foundations of Software Engineering. 248–259.

[60]

J. Smith, B. Johnson, E. Murphy-Hill, B. Chu, and H. Lipford. 2018. How developers diagnose potential security vulnerabilities with a static analysis tool. Transactions on Software Engineering, 45, 9 (2018), 877–897.

Digital Library

[61]

Snyk. Last accessed in March 2022. Snyk Code. https://snyk.io/product/snyk-code/.

[62]

Sonar Source. Last accessed in March 2022. SonarQube. https://www.sonarqube.org/.

[63]

D. Spencer. 2009. Card sorting: Designing usable categories. Rosenfeld Media.

[64]

M. Tahaei and K. Vaniea. 2019. A survey on developer-centred security. In Proceedings of the Symposium on Security and Privacy Workshops. 129–138.

[65]

C. Theisen, L. Williams, K. Oliver, and E. Murphy-Hill. 2016. Software security education at scale. In Proceedings of the International Conference on Software Engineering Companion. 346–355.

[66]

T. Thomas, M. Tabassum, B. Chu, and H. Lipford. 2018. Security during application development: An application security expert perspective. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.

[67]

C. Thompson and D. Wagner. 2017. A Large-Scale Study of Modern Code Review and Security in Open Source Projects. In Proceedings of the International Conference on Predictive Models and Data Analytics in Software Engineering. 83–92.

[68]

P. Thongtanunam, C. Tantithamthavorn, R. Kula, N. Yoshida, H. Iida, and K. Matsumoto. 2015. Who should review my code? A file location-based code-reviewer recommendation approach for Modern Code Review. In Proceedings of the International Conference on Software Analysis, Evolution, and Reengineering. 141–150.

[69]

C. Weir, A. Rashid, and J. Noble. 2016. How to improve the security skills of mobile app developers? Comparing and contrasting expert views. In Proceedings of the Symposium on Usable Privacy and Security.

[70]

R. Weiss. 1995. Learning from Strangers: The art and method of qualitative interview studies. The Free Press.

[71]

C. Wijayarathna and N. Arachchilage. 2018. Why Johnny Can’t Store Passwords Securely? A Usability Evaluation of Bouncycastle Password Hashing. In Proceedings of the International Conference on Evaluation and Assessment in Software Engineering. 205–210.

[72]

Wikipedia. Last accessed in 2022. Massive open online course. https://en.wikipedia.org/wiki/Massive_open_online_course

[73]

I. Woon and A. Kankanhalli. 2007. Investigation of IS professionals’ intention to practise secure development of applications. International Journal of Human-Computer Studies, 65, 1 (2007), 29–41.

Digital Library

[74]

Info World. 2004. Microsoft: More secure but mission not over. https://www.infoworld.com/article/2618608/microsoft–more-secure-but-mission-not-over.html

[75]

G. Wurster and P. Van Oorschot. 2008. The developer is the. In Proceedings of the New Security Paradigms Workshop. 89–97.

[76]

S. Xiao, J. Witschey, and E. Murphy-Hill. 2014. Social influences on secure development tool adoption: why security tools spread. In Proceedings of the Conference on Computer supported cooperative work and social computing. 1095–1106.

[77]

J. Xie, H. Lipford, and B. Chu. 2011. Why do programmers make security errors? In Proceedings of the Symposium on Visual Languages and Human-Centric Computing. 161–164.

Cited By

Karanassios CHadan HZhang-Kennedy LAssal H(2024)Towards Security-Focused Developer PersonasProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685406(1-18)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685406
Steinmacher IPenney JFelizardo KGarcia AGerosa M(2024)Can ChatGPT emulate humans in software engineering surveys?Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3690744(414-419)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3674805.3690744
Charoenwet WThongtanunam PPham VTreude CChristakis MPradel M(2024)An Empirical Study of Static Analysis Tools for Secure Code ReviewProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680313(691-703)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680313
Show More Cited By

Index Terms

Software security during modern code review: the developer’s perspective

Index terms have been assigned to the content through auto-classification.

Recommendations

Software security in DevOps: synthesizing practitioners' perceptions and practices
CSED '16: Proceedings of the International Workshop on Continuous Software Evolution and Delivery

In organizations that use DevOps practices, software changes can be deployed as fast as 500 times or more per day. Without adequate involvement of the security team, rapidly deployed software changes are more likely to contain vulnerabilities due to ...
Improving the effectiveness of peer code review in identifying security defects
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Prior studies found peer code review useful in identifying security defects. That is why most of the commercial and open-source software (OSS) projects embraced peer code review and mandated the use of it in the software development life cycle. However, ...
Synthesis of secure software development controls
SIN '15: Proceedings of the 8th International Conference on Security of Information and Networks

A study of the available approaches aimed at mitigating vulnerabilities in the software development, and their applicability during the software compliance evaluation was carried out. Having systematized the standards and guidelines on the development ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 2022

1822 pages

ISBN:9781450394130

DOI:10.1145/3540250

General Chair:
Abhik Roychoudhury
National University of Singapore, Singapore
,
Program Chairs:
Cristian Cadar
Imperial College London, UK
,
Miryung Kim
University of California at Los Angeles, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Artifacts Available / v1.1

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

ESEC/FSE '22

Sponsor:

ESEC/FSE '22: 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 14 - 18, 2022

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
926
Total Downloads

Downloads (Last 12 months)521
Downloads (Last 6 weeks)75

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Karanassios CHadan HZhang-Kennedy LAssal H(2024)Towards Security-Focused Developer PersonasProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685406(1-18)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685406
Steinmacher IPenney JFelizardo KGarcia AGerosa M(2024)Can ChatGPT emulate humans in software engineering surveys?Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3690744(414-419)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3674805.3690744
Charoenwet WThongtanunam PPham VTreude CChristakis MPradel M(2024)An Empirical Study of Static Analysis Tools for Secure Code ReviewProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680313(691-703)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680313
Thool ABrown C(2024)Harnessing the Power of LLMs: LLM Summarization for Human-Centric DAST Reports2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL/HCC60511.2024.00014(33-39)Online publication date: 2-Sep-2024
https://doi.org/10.1109/VL/HCC60511.2024.00014
Iosif ALechner UPinto-Albuquerque MGasiba T(2024)Serious Game for Industrial Cybersecurity: Experiential Learning Through Code Review2024 36th International Conference on Software Engineering Education and Training (CSEE&T)10.1109/CSEET62301.2024.10663058(1-6)Online publication date: 29-Jul-2024
https://doi.org/10.1109/CSEET62301.2024.10663058
Charoenwet WThongtanunam PPham VTreude C(2024)Toward effective secure code reviews: an empirical study of security-related coding weaknessesEmpirical Software Engineering10.1007/s10664-024-10496-y29:4Online publication date: 8-Jun-2024
https://dl.acm.org/doi/10.1007/s10664-024-10496-y
Krichen M(2023)Formal Methods and Validation Techniques for Ensuring Automotive Systems SecurityInformation10.3390/info1412066614:12(666)Online publication date: 18-Dec-2023
https://doi.org/10.3390/info14120666
Andrade RTorres JOrtiz-Garcés IMiño JAlmeida L(2023)An Exploratory Study Gathering Security Requirements for the Software Development ProcessElectronics10.3390/electronics1217359412:17(3594)Online publication date: 25-Aug-2023
https://doi.org/10.3390/electronics12173594
Nocera SRomano SFrancese RBurlon RScanniello G(2023)Managing Vulnerabilities in Software Projects: the Case of NTT Data2023 49th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA60479.2023.00046(247-253)Online publication date: 6-Sep-2023
https://doi.org/10.1109/SEAA60479.2023.00046
Charoenwet WGrundy J(2023)Complementing Secure Code Review with Automated Program AnalysisProceedings of the 45th International Conference on Software Engineering: Companion Proceedings10.1109/ICSE-Companion58688.2023.00052(189-191)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE-Companion58688.2023.00052
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents