research-article

Co-evolution of infrastructure and source code: an empirical study

Authors:

Bram AdamsAuthors Info & Claims

MSR '15: Proceedings of the 12th Working Conference on Mining Software Repositories

Pages 45 - 55

Published: 16 May 2015 Publication History

Abstract

Infrastructure-as-code automates the process of configuring and setting up the environment (e.g., servers, VMs and databases) in which a software system will be tested and/or deployed, through textual specification files in a language like Puppet or Chef. Since the environment is instantiated automatically by the infrastructure languages' tools, no manual intervention is necessary apart from maintaining the infrastructure specification files. The amount of work involved with such maintenance, as well as the size and complexity of infrastructure specification files, have not yet been studied empirically. Through an empirical study of the version control system of 265 OpenStack projects, we find that infrastructure files are large and churn frequently, which could indicate a potential of introducing bugs. Furthermore, we found that the infrastructure code files are coupled tightly with the other files in a project, especially test files, which implies that testers often need to change infrastructure specifications when making changes to the test framework and tests.

References

[1]

J. Humble and D. Farley, Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation, 1st ed. Addison-Wesley Professional, 2010.

Digital Library

[2]

J. Turnbull and J. McCune, Pro Puppet, 1st ed. Paul Manning, 2011.

Digital Library

[3]

M. Taylor and S. Vargo, Learning Chef-A Guide To Configuration Management And Automation, 1st ed., 2014.

Digital Library

[4]

S. McIntosh, B. Adams, T. H. Nguyen, Y. Kamei, and A. E. Hassan, "An empirical study of build maintenance effort," in Proceedings of the 33rd International Conference on Software Engineering, ser. ICSE '11, 2011, pp. 141--150.

Digital Library

[5]

A. Hindle, M. W. Godfrey, and R. C. Holt, "Release pattern discovery: A case study of database systems," in 23rd IEEE International Conference on Software Maintenance (ICSM 2007), October 2-5, 2007, Paris, France, 2007, pp. 285--294. {Online}. Available: http://dx.doi.org/10.1109/ICSM.2007.4362641

[6]

Herzig, Kim and Nagappan, Nachiappan, "Empirically Detecting False Test Alarms Using Association Rules," in Companion Proceedings of the 37th International Conference on Software Engineering, May 2015.

Digital Library

[7]

Z. Lubsen, A. Zaidman, and M. Pinzger, "Studying co-evolution of production and test code using association rule mining," in Proceedings of the 6th Working Conference on Mining Software Repositories (MSR 2009), M. W. Godfrey and J. Whitehead, Eds. Washington, DC, USA: IEEE Computer Society, 2009, pp. 151--154.

Digital Library

[8]

H. Gall, K. Hajek, and M. Jazayeri, "Detection of logical coupling based on product release history," in Proceedings of the International Conference on Software Maintenance, ser. ICSM '98. Washington, DC, USA: IEEE Computer Society, 1998, pp. 190--.

Digital Library

[9]

B. Adams, K. De Schutter, H. Tromp, and W. De Meuter, "The evolution of the linux build system," Electronic Communications of the ECEASST, vol. 8, February 2008.

[10]

S. Mcintosh, B. Adams, and A. E. Hassan, "The evolution of java build systems," Empirical Softw. Engg., vol. 17, no. 4-5, pp. 578--608, Aug. 2012.

Digital Library

[11]

M. Serrano Zanetti, "The co-evolution of socio-technical structures in sustainable software development: Lessons from the open source software communities," in Proceedings of the 34th International Conference on Software Engineering, ser. ICSE '12. Piscataway, NJ, USA: IEEE Press, 2012, pp. 1587--1590.

Digital Library

[12]

F. Rahman and P. Devanbu, "Ownership, experience and defects: A fine-grained study of authorship," in Proceedings of the 33rd International Conference on Software Engineering, ser. ICSE '11. New York, NY, USA: ACM, 2011, pp. 491--500.

Digital Library

[13]

E. J. Weyuker, T. J. Ostrand, and R. M. Bell, "Do too many cooks spoil the broth? using the number of developers to enhance defect prediction models," Empirical Softw. Engg., vol. 13, no. 5, pp. 539--559, Oct. 2008.

Digital Library

[14]

A. Meneely and L. Williams, "Secure open source collaboration: An empirical study of linus' law," in Proceedings of the 16th ACM Conference on Computer and Communications Security, ser. CCS '09. New York, NY, USA: ACM, 2009, pp. 453--462. {Online}. Available: http://doi.acm.org/10.1145/1653662.1653717

Digital Library

[15]

S. Karus and M. Dumas, "Code churn estimation using organisational and code metrics: An experimental comparison," Information & Software Technology, vol. 54, no. 2, pp. 203--211, 2012. {Online}. Available: http://dx.doi.org/10.1016/j.infsof.2011.09.004

Digital Library

[16]

C. Bird and T. Zimmermann, "Assessing the value of branches with what-if analysis," in Proceedings of the 20th International Symposium on Foundations of Software Engineering (FSE 2012). Association for Computing Machinery, Inc., November 2012. {Online}. Available: http://research.microsoft.com/apps/pubs/default.aspx?id=172572

Digital Library

[17]

N. Nagappan, B. Murphy, and V. Basili, "The influence of organizational structure on software quality: An empirical case study," Microsoft Research, Tech. Rep. MSR-TR-2008-11, January 2008. {Online}. Available: http://research.microsoft.com/apps/pubs/default.aspx?id=70535

[18]

N. Nagappan and T. Ball, "Use of relative code churn measures to predict system defect density." Association for Computing Machinery, Inc., May 2005.

Digital Library

[19]

M. Shridhar, B. Adams, and F. Khomh, "A qualitative analysis of software build system changes and build ownership styles," in Proceedings of the 8th International Symposium on Empirical Software Engineering and Measurement (ESEM), Torino, Italy, September 2014.

Digital Library

[20]

B. Curtis, H. Krasner, and N. Iscoe, "A field study of the software design process for large systems," Commun. ACM, vol. 31, no. 11, pp. 1268--1287, Nov. 1988. {Online}. Available: http://doi.acm.org/10.1145/50087.50089

Digital Library

[21]

P. N. Robillard, "The role of knowledge in software development." Commun. ACM, vol. 42, no. 1, pp. 87--92, 1999. {Online}. Available: http://dblp.uni-trier.de/db/journals/cacm/cacm42.html#Robillard99

Digital Library

[22]

A. Mockus and D. M. Weiss, "Predicting risk of software changes." Bell Labs Technical Journal, vol. 5, no. 2, pp. 169--180, 2000.

[23]

A. Mockus, "Organizational volatility and its effects on software defects," in Proceedings of the Eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering, ser. FSE '10. New York, NY, USA: ACM, 2010, pp. 117--126. {Online}. Available: http://doi.acm.org/10.1145/1882291.1882311

Digital Library

[24]

M. B. Miles and A. M. Huberman, Qualitative data analysis: an expanded sourcebook, 2nd ed. Thousand Oaks, Calif.: Sage Publications, 1994, includes indexes.

[25]

L. Barker, "Android and the linux kernel community," http://www.steptwo.com.au/papers/kmc_whatisinfoarch/, May 2005.

[26]

A. Bacchelli and C. Bird, "Expectations, outcomes, and challenges of modern code review," in Proceedings of the 2013 International Conference on Software Engineering, ser. ICSE '13. Piscataway, NJ, USA: IEEE Press, 2013, pp. 712--721.

Digital Library

[27]

H. Hemmati, S. Nadi, O. Baysal, O. Kononenko, W. Wang, R. Holmes, and M. W. Godfrey, "The msr cookbook: Mining a decade of research," in Proceedings of the 10th Working Conference on Mining Software Repositories, ser. MSR '13. Piscataway, NJ, USA: IEEE Press, 2013, pp. 343--352. {Online}. Available: http://dl.acm.org/citation.cfm?id=2487085.2487150

Digital Library

[28]

"Kruskal-wallis test," http://www.r-tutor.com/elementary-statistics/non-parametric-methods/kruskal-wallis-test.

[29]

P. Kampstra, "Beanplot: A boxplot alternative for visual comparison of distributions," Journal of Statistical Software, Code Snippets, vol. 28, no. 1, pp. 1--9, October 2008. {Online}. Available: http://www.jstatsoft.org/v28/c01

[30]

"puppetlabspuppetlabs-openstack," https://github.com/puppetlabs/puppetlabs-openstack.

[31]

"stackforgeopenstack-chef-repo," https://github.com/stackforge/openstack-chef-repo.

[32]

L. Aversano, L. Cerulo, and C. Del Grosso, "Learning from bug-introducing changes to prevent fault prone code," in Ninth International Workshop on Principles of Software Evolution: In Conjunction with the 6th ESEC/FSE Joint Meeting, ser. IWPSE '07. New York, NY, USA: ACM, 2007, pp. 19--26. {Online}. Available: http://doi.acm.org/10.1145/1294948.1294954

Digital Library

[33]

S. Kim, T. Zimmermann, K. Pan, and E. J. W. Jr., "Automatic identification of bug-introducing changes." in ASE. IEEE Computer Society, 2006, pp. 81--90. {Online}. Available: http://dblp.uni-trier.de/db/conf/kbse/ase2006.html#KimZPW06

Digital Library

[34]

E. Shihab, B. Adams, A. E. Hassan, and Z. M. Jiang, "An industrial study on the risk of software changes," in Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (FSE), Research Triangle Park, NC, US, November 2012, pp. 62:1--62:11.

Digital Library

[35]

S. Kirbas, A. Sen, B. Caglayan, A. Bener, and R. Mahmutogullari, "The effect of evolutionary coupling on software defects: An industrial case study on a legacy system," in Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ser. ESEM '14. New York, NY, USA: ACM, 2014, pp. 6:1--6:7. {Online}. Available: http://doi.acm.org/10.1145/2652524.2652577

Digital Library

[36]

K. Herzig and N. Nagappan, "The impact of test ownership and team structure on the reliability and effectiveness of quality test runs," in Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ser. ESEM '14. New York, NY, USA: ACM, 2014, pp. 2:1--2:10.

Digital Library

Cited By

Henkel JSilva DTeixeira Ld'Amorim MReps T(2021)ShipwrightProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00106(1148-1160)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00106
Rahman AFarhana EParnin CWilliams LRothermel GBae D(2020)Gang of eightProceedings of the ACM/IEEE 42nd International Conference on Software Engineering10.1145/3377811.3380409(752-764)Online publication date: 27-Jun-2020
https://dl.acm.org/doi/10.1145/3377811.3380409
Rahman AParnin CWilliams LAtlee JBultan TWhittle J(2019)The seven sinsProceedings of the 41st International Conference on Software Engineering10.1109/ICSE.2019.00033(164-175)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/ICSE.2019.00033
Show More Cited By

Index Terms

Co-evolution of infrastructure and source code: an empirical study

Recommendations

Detect Related Bugs from Source Code Using Bug Information
COMPSAC '10: Proceedings of the 2010 IEEE 34th Annual Computer Software and Applications Conference

Open source projects often maintain open bug repositories during development and maintenance, and the reporters often point out straightly or implicitly the reasons why bugs occur when they submit them. The comments about a bug are very valuable for ...
Towards Semantic Detection of Smells in Cloud Infrastructure Code
WIMS 2020: Proceedings of the 10th International Conference on Web Intelligence, Mining and Semantics

Automated deployment and management of Cloud applications relies on descriptions of their deployment topologies, often referred to as Infrastructure Code. As the complexity of applications and their deployment models increases, developers inadvertently ...
Infrastructure in code: towards developer-friendly cloud applications
ASE '21: Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering

The popularity of cloud technologies has led to the development of a new type of applications that specifically target cloud environments. Such applications require a lot of cloud infrastructure to run, which brought about the Infrastructure as Code ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MSR '15: Proceedings of the 12th Working Conference on Mining Software Repositories

May 2015

542 pages

ISBN:9780769555942

General Chair:
Massimiliano Di Penta
University of Sannio, Italy
,
Program Chairs:
Martin Pinzger
University of Klagenfurt, Austria
,
Romain Robbes
University of Chile, Chile

Sponsors

ACM: Association for Computing Machinery
SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS\DATC: IEEE Computer Society
TCSE: IEEE Computer Society's Tech. Council on Software Engin.

Publisher

IEEE Press

Publication History

Published: 16 May 2015

Check for updates

Qualifiers

Research-article

Conference

ICSE '15

Sponsor:

ACM
SIGSOFT
IEEE-CS\DATC
TCSE

ICSE '15: 37th International Conference on Software Engineering

May 16 - 24, 2015

Florence, Italy

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
343
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Henkel JSilva DTeixeira Ld'Amorim MReps T(2021)ShipwrightProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00106(1148-1160)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00106
Rahman AFarhana EParnin CWilliams LRothermel GBae D(2020)Gang of eightProceedings of the ACM/IEEE 42nd International Conference on Software Engineering10.1145/3377811.3380409(752-764)Online publication date: 27-Jun-2020
https://dl.acm.org/doi/10.1145/3377811.3380409
Rahman AParnin CWilliams LAtlee JBultan TWhittle J(2019)The seven sinsProceedings of the 41st International Conference on Software Engineering10.1109/ICSE.2019.00033(164-175)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/ICSE.2019.00033
Masek PStusek MKrejci JZeman KPokorny JKudlacek M(2018)Unleashing Full Potential of Ansible Framework: University Labs AdministrationProceedings of the 22st Conference of Open Innovations Association FRUCT10.5555/3266365.3266385(144-150)Online publication date: 21-May-2018
https://dl.acm.org/doi/10.5555/3266365.3266385
Schermann GZumberi SCito JZaidman AKamei YHill E(2018)Structured information on state and evolution of dockerfiles on githubProceedings of the 15th International Conference on Mining Software Repositories10.1145/3196398.3196456(26-29)Online publication date: 28-May-2018
https://dl.acm.org/doi/10.1145/3196398.3196456
Rahman APartho AMorrison PWilliams LBosch JFitzgerald BGoedicke MOlsson HKonersmann MKrusche S(2018)What questions do programmers ask about configuration as code?Proceedings of the 4th International Workshop on Rapid Continuous Software Engineering10.1145/3194760.3194769(16-22)Online publication date: 29-May-2018
https://dl.acm.org/doi/10.1145/3194760.3194769
Rahman AStallings JWilliams LChaudron MCrnkovic IChechik MHarman M(2018)Defect prediction metrics for infrastructure as code scripts in DevOpsProceedings of the 40th International Conference on Software Engineering: Companion Proceeedings10.1145/3183440.3195034(414-415)Online publication date: 27-May-2018
https://dl.acm.org/doi/10.1145/3183440.3195034
Rahman AChaudron MCrnkovic IChechik MHarman M(2018)Characteristics of defective infrastructure as code scripts in DevOpsProceedings of the 40th International Conference on Software Engineering: Companion Proceeedings10.1145/3183440.3183452(476-479)Online publication date: 27-May-2018
https://dl.acm.org/doi/10.1145/3183440.3183452
Cito JSchermann GWittern JLeitner PZumberi SGall HGonzalez-Barahona JHindle ATan L(2017)An empirical analysis of the docker container ecosystem on GitHubProceedings of the 14th International Conference on Mining Software Repositories10.1109/MSR.2017.67(323-333)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.1109/MSR.2017.67
Chahal KSaini M(2016)Open Source Software EvolutionInternational Journal of Open Source Software and Processes10.4018/IJOSSP.20160101027:1(28-48)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.4018/IJOSSP.2016010102
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten