research-article

Public Access

An empirical investigation into merge conflicts and their effect on software quality

Authors:

Caius Brindescu,

Iftekhar Ahmed,

Anita SarmaAuthors Info & Claims

Empirical Software Engineering, Volume 25, Issue 1

Pages 562 - 590

https://doi.org/10.1007/s10664-019-09735-4

Published: 01 January 2020 Publication History

Abstract

Merge conflicts are known to cause extra effort for developers, but little is known about their effect on software. While some research has been done, many questions remain. To better understand merge conflicts and their impact we performed an empirical study about the types, frequency, and impact of merge conflicts, where impact is measured in terms of bug fixing commits associated with conflicts. We analyzed 143 open source projects and found that almost 1 in 5 merges cause conflicts. In 75.23% of these cases, a developer needed to reflect on the program logic to resolve it. We also found that the code associated with a merge conflict is twice as likely to have a bug. When the code associated with merge conflicts require manual intervention, the code is 26× more likely to have a bug.

References

[1]

Accioly P, Borba P, and Cavalcanti G Understanding semi-structured merge conflict characteristics in open-source java projects Empir Softw Eng 2018 23 4 2051-2085

[2]

Ahmed I, Gopinath R, Brindescu C, Groce A, Jensen C (2016) Can testedness be effectively measured?. In: Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering, FSE 2016. ACM, Seattle, pp 547–558, 10.1145/2950290.2950324

[3]

Ahmed I, Brindescu C, Mannan UA, Jensen C, Sarma A (2017) An empirical examination of the relationship between code smells and merge conflicts. In: Proceedings of the 11th ACM/IEEE international symposium on empirical software engineering and measurement, ESEM ’17. IEEE Press, Piscataway, pp 58–67, 10.1109/ESEM.2017.12

[4]

Apache Maven (2018) Maven. The Apache Software Foundation. https://maven.apache.org/. Accessed 1 May 2018

[5]

Apel S, Liebig J, Brandl B, Lengauer C, Kästner C (2011) Semistructured merge: rethinking merge in revision control systems. In: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on foundations of software engineering, ESEC/FSE ’11. ACM, New York, pp 190–200, 10.1145/2025113.2025141

[6]

Apel S, Lessenich O, Lengauer C (2012) Structured Merge with auto-tuning: balancing precision and performance. In: Proceedings of the 27th IEEE/ACM international conference on automated software engineering, ASE 2012. ACM, Essen, pp 120–129, 10.1145/2351676.2351694

[7]

Biehl JT, Czerwinski M, Smith G, Robertson GG (2007) FASTDash: a visual dashboard for fostering awareness in software teams. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’07. ACM, New York, pp 1313–1322, 10.1145/1240624.1240823

[8]

Bird C, Bachmann A, Aune E, Duffy J, Bernstein A, Filkov V, Devanbu P (2009) Fair and balanced?: bias in bug-fix datasets. In: Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering, pp 121–130

[9]

Boehm BW, Brown JR, Lipow M (1976) Quantitative evaluation of software quality. In: Proceedings of the 2nd international conference on software engineering, ICSE ’76. IEEE Computer Society Press, Los Alamitos, pp 592–605

[10]

Brun Y, Holmes R, Ernst MD, Notkin D (2011) Proactive detection of collaboration conflicts. In: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on foundations of software engineering, ESEC/FSE ’11. ACM, Szeged, pp 168–178, 10.1145/2025113.2025139

[11]

Brun Y, Holmes R, Ernst MD, and Notkin D Early detection of collaboration conflicts and risks IEEE Trans Softw Eng 2013 39 10 1358-1375

[12]

Buckley J, Mens T, Zenger M, Rashid A, and Kniesel G Towards a taxonomy of software change J Softw Maint Evol Res Pract 2005 17 5 309-332

[13]

Canfora G, Cerulo L, Penta MD (2007) Identifying changed source code lines from version repositories. In: Fourth international workshop on mining software repositories (MSR’07:ICSE Workshops 2007), pp 14–14, 10.1109/MSR.2007.14

[14]

Cataldo M and Herbsleb JD Coordination breakdowns and their impact on development productivity and software failures IEEE Trans Softw Eng 2013 39 3 343-360

[15]

Cavalcanti G, Borba P, and Accioly P Evaluating and improving semistructured merge Proc ACM Program Lang 2017 1 OOPSLA 59:1-59:27

[16]

Cohen J, Cohen P, West S, Aiken L (2013) Applied multiple regression/correlation analysis for the behavioral sciences. Taylor & Francis

[17]

Companion website (2016) http://caius.brindescu.com/research/mc-classification

[18]

Corbet J, Kroah-Hartman G (2016) 2016 Linux Kernel Development Report. Tech. rep., The Linux Foundation

[19]

Costa C, Figueiredo JJC, Ghiotto G, and Murta LGP Characterizing the problem of developers’ assignment for merging branches Int J Softw Eng Knowl Eng 2014 24 10 1489-1508

[20]

Cruzes DS, Dyba T (2011) Recommended steps for thematic synthesis in software engineering. In: 2011 international symposium on empirical software engineering and measurement, pp 275–284, 10.1109/ESEM.2011.36

[21]

da Silva IA, Chen PH, Van der Westhuizen C, Ripley RM, van der Hoek A (2006) Lighthouse: coordination through emerging design. In: Proceedings of the 2006 OOPSLA workshop on Eclipse Technology eXchange, eclipse ’06. ACM, New York, pp 11–15, 10.1145/1188835.1188838

[22]

de Souza CRB, Redmiles D, Dourish P (2003) “Breaking the code”, Moving between private and public work in collaborative software development. In: Proceedings of the 2003 International ACM SIGGROUP Conference on Supporting Group Work, GROUP ’03. ACM, New York, pp 105–114, 10.1145/958160.958177

[23]

De Souza LBL, Maia MDA (2013) Do software categories impact coupling metrics?. In: Proceedings of the 10th working conference on mining software repositories, MSR ’13. IEEE Press, Piscataway, pp 217–220

[24]

Dewan P, Hegde R (2007) Semi-synchronous conflict detection and resolution in asynchronous software development. In: Bannon LJ, Wagner I, Gutwin C, Harper RHR, Schmidt K (eds) ECSCW 2007. Springer, London, Limerick, pp 159–178, 10.1007/978-1-84800-031-5_9

[25]

El Emam K, Benlarbi S, Goel N, and Rai SN The confounding effect of class size on the validity of object-oriented metrics IEEE Trans Softw Eng 2001 27 7 630-650

[26]

Falleri JR, Morandat F, Blanc X, Martinez M, Montperrus M (2014) Fine-grained and accurate source code differencing. In: Proceedings of the 29th ACM/IEEE international conference on automated software engineering, ASE ’14. ACM, New York, pp 313–324, 10.1145/2642937.2642982

[27]

GitHub (2017) https://www.github.com/

[28]

Godfrey MW and Zou L Using origin analysis to detect merging and splitting of source code entities IEEE Trans Softw Eng 2005 31 2 166-181

[29]

Gorton I, Liu A (2002) Software component quality assessment in practice: successes and practical impediments. In: Proceedings of the 24th international conference on software engineering, ICSE ’02. ACM, New York, pp 555–558, 10.1145/581339.581408

[30]

Grinter RE (1995) Using a configuration management tool to coordinate software development. In: Proceedings of conference on organizational computing systems, COCS ’95. ACM, New York, pp 168–177, 10.1145/224019.224036

[31]

Groce A, Zhang C, Eide E, Chen Y, Regehr J (2012) Swarm Testing. In: Proceedings of the 2012 international symposium on software testing and analysis, ISSTA 2012. ACM, Minneapolis, pp 78–88, 10.1145/2338965.2336763

[32]

Guimarães ML, Silva AR (2012) Improving early detection of software merge conflicts. In: Proceedings of the 34th international conference on software engineering, ICSE ’12. IEEE Press, Zurich, pp 342–352

[33]

Hensher D, Stopher P (1979) Behavioural travel modelling, Croom Helm, London

[34]

Holzmann GJ, Joshi R, and Groce A Swarm verification techniques IEEE Trans Softw Eng 2011 37 6 845-857

[35]

Johnson Eric J. and Goldstein Daniel Do Defaults Save Lives? Science 2003 302 5649 1338-1339

[36]

Johnson EJ, Bellman S, and Lohse GL Defaults, framing and privacy: why opting in-opting out1 Mark Lett 2002 13 1 5-15

[37]

Kalliamvakou E, Gousios G, Blincoe K, Singer L, German DM, Damian D (2014) The promises and perils of mining GitHub. In: Proceedings of the 11th working conference on mining software repositories, MSR 2014. ACM, New York, pp 92–101, 10.1145/2597073.2597074

[38]

Kasi BK, Sarma A (2013) Cassandra: proactive conflict minimization through optimized task scheduling. In: Proceedings of the 2013 international conference on software engineering, ICSE ’13. IEEE Press, Piscataway, pp 732–741

[39]

Kim S, Zimmermann T, Pan K, Whitehead EJJ (2006) Automatic identification of bug-introducing changes. In: Proceedings of the 21st IEEE/ACM international conference on automated software engineering, ASE ’06. IEEE Computer Society, Washington, DC, pp 81–90, 10.1109/ASE.2006.23

[40]

Kim S, Whitehead EJJr, and Zhang Y Classifying software changes: clean or buggy? IEEE Trans Softw Eng 2008 34 2 181-196

[41]

Lippe E, van Oosterom N (1992) Operation-based merging. In: Proceedings of the fifth ACM SIGSOFT symposium on software development environments, SDE 5. ACM, New York, pp 78–87, 10.1145/142868.143753

[42]

Marinescu R (2001) Detecting design flaws via metrics in object-oriented systems. In: Proceedings of the 39th international conference and exhibition on technology of object-oriented languages and systems (TOOLS39), TOOLS ’01. IEEE Computer Society, Washington, DC, p 173

[43]

Marinescu R (2004) Detection strategies: metrics-based rules for detecting design flaws. In: 20th IEEE international conference on software maintenance, 2004. Proceedings, pp 350–359

[44]

McKee S, Nelson N, Sarma A, Dig D (2017) Software practitioner perspectives on merge conflicts and resolutions. In: 2017 IEEE international conference on software maintenance and evolution, ICSME 2017, Shanghai, China, September 17–22, 2017, pp 467–478, 10.1109/ICSME.2017.53

[45]

Menezes GGLD (2016) On the nature of software merge conflicts. Ph.D. thesis, Universidade Federal Fluminense

[46]

Mens T A state-of-the-art survey on software merging IEEE Trans Softw Eng 2002 28 5 449-462

[47]

Mockus A and Weiss DM Predicting risk of software changes Bell Labs Tech J 2000 5 2 169-180

[48]

Mockus A, Fielding RT, and Herbsleb JD Two case studies of open source software development: Apache and Mozilla ACM Trans Softw Eng Methodol 2002 11 3 309-346

[49]

Nieminen A (2012) Real-time collaborative resolving of merge conflicts. In: Proceedings of the 2012 8th international conference on collaborative computing: networking, applications and worksharing (CollaborateCom 2012), COLLABORATECOM ’12. IEEE Computer Society, Washington, DC, pp 540–543, 10.4108/icst.collaboratecom.2012.250435

[50]

Parsons J and Saunders C Cognitive heuristics in software engineering applying and extending anchoring and adjustment to artifact reuse IEEE Trans Softw Eng 2004 30 12 873-888

[51]

Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, and Duchesnay EScikit-learn: machine learning in pythonJ Mach Learn Res2011122825-283028543481280.68189

[52]

Perry DE, Siy HP, and Votta LG Parallel changes in large-scale software development: an observational case study ACM Trans Softw Eng Methodol 2001 10 3 308-337

[53]

Raymond ES The Cathedral and the bazaar, 1st edn 1999 Sebastopol O’Reilly & Associates, Inc.

[54]

Ripley RM, Yasui RY, Sarma A, van der Hoek A (2004) Workspace awareness in application development. In: Proceedings of the 2004 OOPSLA workshop on Eclipse Technology eXchange, eclipse ’04. ACM, New York, pp 17–21, 10.1145/1066129.1066133

[55]

Samuelson W and Zeckhauser R Status quo bias in decision making J Risk Uncertain 1988 1 1 7-59

[56]

Sarma A, Noroozi Z, Hoek AVD (2003) Palantír: raising awareness among configuration management workspaces. In: Proceedings of the 25th international conference on software engineering, May 3-10, 2003, Portland, Oregon, USA, pp 444–454. 10.1109/ICSE.2003.1201222

[57]

Servant F, Jones JA, van der Hoek A (2010) CASI: preventing indirect conflicts through a live visualization. In: Proceedings of the 2010 ICSE workshop on cooperative and human aspects of software engineering, CHASE ’10. ACM, New York, pp 39–46, 10.1145/1833310.1833317

[58]

Shihab E, Bird C, Zimmermann T (2012) The effect of branching strategies on software quality. In: Proceedings of the ACM-IEEE international symposium on empirical software engineering and measurement, ESEM ’12. ACM, New York, pp 301–310, 10.1145/2372251.2372305

[59]

Sun Z, Song Q, and Zhu X Using coding-based ensemble learning to improve software defect prediction IEEE Trans Syst Man Cybern Part C Appl Rev 2012 42 6 1806-1817

[60]

Sunstein CR, Thaler RH (2003) Libertarian paternalism is not an oxymoron. The University of Chicago Law Review, pp 1159–1202

[61]

Tian Y, Lawall J, Lo D (2012) Identifying linux bug fixing patches. In: 2012 34th international conference on software engineering (ICSE), pp 386–396

[62]

Understand ™ (2017) Static Code Analysis Tool

[63]

Weyuker EJ, Ostrand TJ, and Bell RM Do too many cooks spoil the broth? Using the number of developers to enhance defect prediction models Empir Softw Eng 2008 13 5 539-559

[64]

Zimmermann T, Kim S, Zeller A, Whitehead EJ Jr (2006) Mining version archives for co-changed lines. In: Proceedings of the 2006 international workshop on mining software repositories, MSR ’06. ACM, New York, pp 72–75, 10.1145/1137983.1138001

Cited By

Aldndni WServant FMeng NBaysal OLinares-Vasquez MMoran KSteinmacher I(2024)Understanding the Impact of Branch Edit Features for the Automatic Prediction of Merge Conflict ResolutionsProceedings of the 32nd IEEE/ACM International Conference on Program Comprehension10.1145/3643916.3644433(149-160)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643916.3644433
Shen BMeng N(2024)ConflictBenchJournal of Systems and Software10.1016/j.jss.2024.112084214:COnline publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.112084
Teles ASantos AStorm THirschfeld R(2023)Code Merging using Transformations and Member IdentityProceedings of the 2023 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3622758.3622891(71-88)Online publication date: 18-Oct-2023
https://dl.acm.org/doi/10.1145/3622758.3622891
Show More Cited By

Index Terms

An empirical investigation into merge conflicts and their effect on software quality
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Project and people management
      2. Software management
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
  2. Software notations and tools
    1. Software configuration management and version control systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Causes of merge conflicts: a case study of ElasticSearch
VaMoS '20: Proceedings of the 14th International Working Conference on Variability Modelling of Software-Intensive Systems

Software branching and merging allows collaborative development and creating software variants, commonly referred to as clone & own. While simple and cheap, a trade-off is the need to merge code and to resolve merge conflicts, which frequently occur in ...
Indicators for merge conflicts in the wild: survey and empirical study

While the creation of new branches and forks is easy and fast with modern version-control systems, merging is often time-consuming. Especially when dealing with many branches or forks, a prediction of merge costs based on lightweight indicators would be ...
An empirical examination of the relationship between code smells and merge conflicts
ESEM '17: Proceedings of the 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

Background: Merge conflicts are a common occurrence in software development. Researchers have shown the negative impact of conflicts on the resulting code quality and the development workflow. Thus far, no one has investigated the effect of bad design (...

Comments

Information & Contributors

Information

Published In

cover image Empirical Software Engineering

Empirical Software Engineering Volume 25, Issue 1

Jan 2020

1031 pages

ISSN:1382-3256

Issue’s Table of Contents

© Springer Science+Business Media, LLC, part of Springer Nature 2019.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 January 2020

Author Tags

Qualifiers

Research-article

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Aldndni WServant FMeng NBaysal OLinares-Vasquez MMoran KSteinmacher I(2024)Understanding the Impact of Branch Edit Features for the Automatic Prediction of Merge Conflict ResolutionsProceedings of the 32nd IEEE/ACM International Conference on Program Comprehension10.1145/3643916.3644433(149-160)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643916.3644433
Shen BMeng N(2024)ConflictBenchJournal of Systems and Software10.1016/j.jss.2024.112084214:COnline publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.112084
Teles ASantos AStorm THirschfeld R(2023)Code Merging using Transformations and Member IdentityProceedings of the 2023 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3622758.3622891(71-88)Online publication date: 18-Oct-2023
https://dl.acm.org/doi/10.1145/3622758.3622891
Oliveira JMongiovi MSouto S(2023)An empirical study of the relationship between refactorings and merge conflicts in Javascript codeProceedings of the XXXVII Brazilian Symposium on Software Engineering10.1145/3613372.3613402(89-98)Online publication date: 25-Sep-2023
https://dl.acm.org/doi/10.1145/3613372.3613402
Graeff CFarias KCarbonera C(2023)On the Prediction of Software Merge Conflicts: A Systematic Review and Meta-analysisProceedings of the XIX Brazilian Symposium on Information Systems10.1145/3592813.3592931(404-411)Online publication date: 29-May-2023
https://dl.acm.org/doi/10.1145/3592813.3592931
Shen BGulzar MHe FMeng N(2023)A Characterization Study of Merge Conflicts in Java ProjectsACM Transactions on Software Engineering and Methodology10.1145/354694432:2(1-28)Online publication date: 31-Mar-2023
https://dl.acm.org/doi/10.1145/3546944
Wattanakriengkrai SWang DKula RTreude CThongtanunam PIshio TMatsumoto K(2023)Giving Back: Contributions Congruent to Library Dependency Changes in a Software EcosystemIEEE Transactions on Software Engineering10.1109/TSE.2022.322519749:4(2566-2579)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TSE.2022.3225197
Aldndni WMeng NServant F(2023)Automatic prediction of developers’ resolutions for software merge conflictsJournal of Systems and Software10.1016/j.jss.2023.111836206:COnline publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1016/j.jss.2023.111836
Campos Junior Hde Menezes GBarros Mvan der Hoek AMurta L(2022)Towards Merge Conflict Resolution by Combining Existing Lines of CodeProceedings of the XXXVI Brazilian Symposium on Software Engineering10.1145/3555228.3555229(425-434)Online publication date: 5-Oct-2022
https://dl.acm.org/doi/10.1145/3555228.3555229
Towqir SShen BGulzar MMeng N(2022)Detecting Build Conflicts in Software Merge for Java Programs via Static AnalysisProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering10.1145/3551349.3556950(1-13)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3551349.3556950
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents