poster

Defect prediction metrics for infrastructure as code scripts in DevOps

Authors:

Akond Rahman,

Jonathan Stallings,

Laurie WilliamsAuthors Info & Claims

ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

Pages 414 - 415

https://doi.org/10.1145/3183440.3195034

Published: 27 May 2018 Publication History

Get Access

Abstract

Use of infrastructure as code (IaC) scripts helps software teams manage their configuration and infrastructure automatically. Information technology (IT) organizations use IaC scripts to create and manage automated deployment pipelines to deliver services rapidly. IaC scripts can be defective, resulting in dire consequences, such as creating wide-scale service outages for end-users. Prediction of defective IaC scripts can help teams to mitigate defects in these scripts by prioritizing their inspection efforts. The goal of this paper is to help software practitioners in prioritizing their inspection efforts for infrastructure as code (IaC) scripts by proposing defect prediction model-related metrics. IaC scripts use domain specific languages (DSL) that are fundamentally different from object-oriented programming (OOP) languages. Hence, the OOP-based metrics that researchers used in defect prediction might not be applicable for IaC scripts. We apply Constructivist Grounded Theory (CGT) on defect-related commits mined from version control systems to identify metrics suitable for IaC scripts. By applying CGT, we identify 18 metrics. Of these metrics, 13 are related to IaC, for example, count of string occurrences in a script. Four of the identified metrics are related to churn, and one metric is lines of code.

References

[1]

N. F. Alanna Brown, Jez Humble, Nigel Kersten, and Gene Kim. 2017. 2016 State of DevOps Report. https://puppet.com/resources/whitepaper/2016-state-of-devops-report. (2017). {Online; accessed 15-August-2017}.

Google Scholar

[2]

Jürgen Cito, Philipp Leitner, Thomas Fritz, and Harald C. Gall. 2015. The Making of Cloud Applications: An Empirical Study on Software Development for the Cloud. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015). ACM, New York, NY, USA, 393--403.

Digital Library

Google Scholar

[3]

James Fryman. 2014. DNS Outage Post Mortem-Github. https://github.com/blog/1759-dns-outage-post-mortem. (2014). {Online; accessed 24-August-2017}.

Google Scholar

[4]

Baljinder Ghotra, Shane McIntosh, and Ahmed E. Hassan. 2015. Revisiting the Impact of Classification Techniques on the Performance of Defect Prediction Models. In Proceedings of the 37th International Conference on Software Engineering - Volume 1 (ICSE '15). IEEE Press, Piscataway, NJ, USA, 789--800. http://dl.acm.org/citation.cfm?id=2818754.2818850

Digital Library

Google Scholar

[5]

Viral Gupta, P.K. Kapur, and Deepak Kumar. 2017. Modeling and measuring attributes influencing DevOps implementation in an enterprise using structural equation modeling. Information and Software Technology (2017).

Google Scholar

[6]

Oliver Hanappi, Waldemar Hummer, and Schahram Dustdar. 2016. Asserting Reliable Convergence for Configuration Management Scripts. SIGPLAN Not. 51, 10 (Oct. 2016), 328--343.

Digital Library

Google Scholar

[7]

Jez Humble and David Farley. 2010. Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation (1st ed.). Addison-Wesley Professional.

Digital Library

Google Scholar

[8]

IEEE. 2010. IEEE Standard Classification for Software Anomalies. IEEE Std 1044-2009 (Revision of IEEE Std 1044-1993) (Jan 2010), 1--23.

Google Scholar

[9]

Yujuan Jiang and Bram Adams. 2015. Co-evolution of Infrastructure and Source Code: An Empirical Study. In Proceedings of the 12th Working Conference on Mining Software Repositories (MSR '15). IEEE Press, Piscataway, NJ, USA, 45--55. http://dl.acm.org/citation.cfm?id=2820518.2820527

Digital Library

Google Scholar

[10]

C. Parnin, E. Helms, C. Atlee, H. Boughton, M. Ghattas, A. Glover, J. Holman, J. Micco, B. Murphy, T. Savor, M. Stumm, S. Whitaker, and L. Williams. 2017. The Top 10 Adages in Continuous Deployment. IEEE Software 34, 3 (May 2017), 86--95.

Digital Library

Google Scholar

[11]

D. Spinellis. 2012. Don't Install Software by Hand. IEEE Software 29, 4 (July 2012), 86--87.

Digital Library

Google Scholar

[12]

C. Tantithamthavorn, S. McIntosh, A. E. Hassan, A. Ihara, and K. Matsumoto. 2015. The Impact of Mislabelling on the Performance and Interpretation of Defect Prediction Models. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 1. 812--823.

Digital Library

Google Scholar

Cited By

View all

Simkin AKopp AOlkhovyi O(2023)Research on the optimization model for building an efficient IT infrastructure using the AWS platformInterConf10.51582/interconf.19-20.10.2023.027(300-315)Online publication date: 19-Oct-2023
https://doi.org/10.51582/interconf.19-20.10.2023.027
Lee DSeo Y(2022)Identification of propagated defects to reduce software testing cost via mutation testingMathematical Biosciences and Engineering10.3934/mbe.202228619:6(6124-6140)Online publication date: 2022
https://doi.org/10.3934/mbe.2022286
Chapetta WTravassos G(2020)Towards an evidence-based theoretical framework on factors influencing the software development productivityEmpirical Software Engineering10.1007/s10664-020-09844-525:5(3501-3543)Online publication date: 1-Sep-2020
https://dl.acm.org/doi/10.1007/s10664-020-09844-5

Index Terms

Defect prediction metrics for infrastructure as code scripts in DevOps
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis

Recommendations

Characteristics of defective infrastructure as code scripts in DevOps
ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

Defects in infrastructure as code (IaC) scripts can have serious consequences for organizations who adopt DevOps. By identifying which characteristics of IaC scripts correlate with defects, we can identify anti-patterns, and help software practitioners ...
Infrastructure as code for dynamic deployments
ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Modern DevOps organizations require a high degree of automation to achieve software stability at frequent changes. Further, there is a need for flexible, timely reconfiguration of the infrastructure, e.g., to use pay-per-use infrastructure efficiently ...
Testing practices for infrastructure as code
LANGETI 2020: Proceedings of the 1st ACM SIGSOFT International Workshop on Languages and Tools for Next-Generation Testing

Infrastructure as code (IaC) helps practitioners to rapidly deploy software services to end-users. Despite reported benefits, IaC scripts are susceptible to defects. Defects in IaC scripts can cause serious consequences, for example, creating large-...

Comments

Information & Contributors

Information

Published In

ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

May 2018

231 pages

ISBN:9781450356633

DOI:10.1145/3183440

Conference Chair:
Michel Chaudron
Chalmers University of Technology, University of Gothenburg, Sweden
,
General Chair:
Ivica Crnkovic
Chalmers University of Technology, University of Gothenburg, Sweden
,
Program Chairs:
Marsha Chechik
University of Toronto, Canada
,
Mark Harman
Facebook and University College London, United Kingdom

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2018

Check for updates

Author Tags

Qualifiers

Poster

Conference

ICSE '18

Sponsor:

SIGSOFT
IEEE-CS

ICSE '18: 40th International Conference on Software Engineering

May 27 - June 3, 2018

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
436
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)1

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Simkin AKopp AOlkhovyi O(2023)Research on the optimization model for building an efficient IT infrastructure using the AWS platformInterConf10.51582/interconf.19-20.10.2023.027(300-315)Online publication date: 19-Oct-2023
https://doi.org/10.51582/interconf.19-20.10.2023.027
Lee DSeo Y(2022)Identification of propagated defects to reduce software testing cost via mutation testingMathematical Biosciences and Engineering10.3934/mbe.202228619:6(6124-6140)Online publication date: 2022
https://doi.org/10.3934/mbe.2022286
Chapetta WTravassos G(2020)Towards an evidence-based theoretical framework on factors influencing the software development productivityEmpirical Software Engineering10.1007/s10664-020-09844-525:5(3501-3543)Online publication date: 1-Sep-2020
https://dl.acm.org/doi/10.1007/s10664-020-09844-5

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Characteristics of defective infrastructure as code scripts in DevOps

Infrastructure as code for dynamic deployments

Testing practices for infrastructure as code