survey

A Survey of DevOps Concepts and Challenges

Authors:

Leonardo Leite,

Dejan Milojicic,

Paulo MeirellesAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 52, Issue 6

Article No.: 127, Pages 1 - 35

https://doi.org/10.1145/3359981

Published: 14 November 2019 Publication History

Abstract

DevOpsis a collaborative and multidisciplinary organizational effort to automate continuous delivery of new software updates while guaranteeing their correctness and reliability. The present survey investigates and discusses DevOps challenges from the perspective of engineers, managers, and researchers. We review the literature and develop a DevOps conceptual map, correlating the DevOps automation tools with these concepts. We then discuss their practical implications for engineers, managers, and researchers. Finally, we critically explore some of the most relevant DevOps challenges reported by the literature.

References

[1]

X. Bai, M. Li, D. Pei, S. Li, and D. Ye. 2018. Continuous delivery of personalized assessment and feedback in agile software engineering projects. In Proceedings of the 40th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET’18). 58--67. Code: I818.

[2]

Armin Balalaie, Abbas Heydarnoori, and Pooyan Jamshidi. 2016. Microservices architecture enables DevOps: Migration to a cloud-native architecture. IEEE Softw. 33, 3 (2016), 42--52. Code: A2.

Digital Library

[3]

A. Basiri, N. Behnam, R. de Rooij, L. Hochstein, L. Kosewski, J. Reynolds, and C. Rosenthal. 2016. Chaos engineering. IEEE Softw. 33, 3 (2016), 35--41. Code: A76.

Digital Library

[4]

Len Bass. 2018. The software architect and DevOps. IEEE Softw. 35, 1 (2018), 8--10. Code: I33.

[5]

Kyle Brown and Bobby Woolf. 2016. Implementation patterns for microservices architectures. In Proceedings of the 23rd Conference on Pattern Languages of Programs (PLoP’16). The Hillside Group, Article 7, 7:1--7:35 pages. Code: A104.

Digital Library

[6]

Matt Callanan and Alexandra Spillane. 2016. DevOps: Making it easy to do the right thing. IEEE Softw. 33, 3 (2016), 53--59. Code: A67.

Digital Library

[7]

Lianping Chen. 2015. Continuous delivery: Huge benefits, but challenges too. IEEE Softw. 32, 2 (2015), 50--54. Code: B15.

Digital Library

[8]

Henrik Bærbak Christensen. 2016. Teaching DevOps and cloud computing using a cognitive apprenticeship and story-telling approach. In Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE’16). ACM, 174--179. Code: A47.

Digital Library

[9]

Sam Chung and Soon Bang. 2016. Identifying knowledge, skills, and abilities (KSA) for DevOps-aware server side web application with the grounded theory. J. Comput. Sci. Coll. 32, 1 (2016), 110--116. Code: A16.

Digital Library

[10]

Gerry Gerard Claps, Richard Berntsson Svensson, and Aybüke Aurum. 2015. On the journey to continuous deployment: Technical and social challenges along the way. Inf. Softw. Technol. 57 (2015), 21--31. Code: B13.

[11]

Daniel Cukier. 2013. DevOps patterns to scale web applications using cloud services. In Proceedings of the Companion Publication for Conference on Systems, Programming, 8 Applications: Software for Humanity (SPLASH’13). ACM, 143--152. Code: A35.

Digital Library

[12]

Maximilien de Bayser, Leonardo G. Azevedo, and Renato Cerqueira. 2015. ResearchOps: The case for DevOps in scientific applications. In Proceedings of the IFIP/IEEE International Symposium on Integrated Network Management (IM’15). 1398--1404. Code: I40.

[13]

Rico de Feijter, Sietse Overbeek, Rob van Vliet, Erik Jagroep, and Sjaak Brinkkemper. 2018. DevOps competences and maturity for software producing organizations. In Enterprise, Business-Process and Information Systems Modeling. Springer, 244--259. Code: S805.

[14]

Patrick Debois. 2011. DevOps: A software revolution in the making. Cutter IT J. 24, 8 (2011), 3--5. Code: B4.

[15]

Elisa Diel, Sabrina Marczak, and Daniela S. Cruzes. 2016. Communication challenges and strategies in distributed DevOps. In Proceedings of the 11th IEEE International Conference on Global Software Engineering (ICGSE’16). 24--28. Code: I19.

[16]

Andrej Dyck, Ralf Penners, and Horst Lichter. 2015. Towards definitions for release engineering and DevOps. In Proceedings of the IEEE/ACM 3rd International Workshop on Release Engineering. 3--3. Code: B26.

Digital Library

[17]

C. Ebert, G. Gallardo, J. Hernantes, and N. Serrano. 2016. DevOps. IEEE Softw. 33, 3 (2016), 94--100. Code: A54.

Digital Library

[18]

Dror G. Feitelson, Eitan Frachtenberg, and Kent L. Beck. 2013. Development and deployment at Facebook. IEEE Int. Comput. 17, 4 (2013), 8--17. Code: B7.

Digital Library

[19]

Nicole Forsgren and Mik Kersten. 2018. DevOps metrics. Commun. ACM 61, 4 (2018), 44--48. Code: B21.

Digital Library

[20]

Jim Gray. 2006. A conversation with Werner Vogels. ACM Queue 4, 4 (2006), 14--22. Code: B3.

Digital Library

[21]

Jez Humble. 2017. Continuous delivery sounds great, but will it work here?Queue 15, 6 (2017), 57--76. Code: B22.

[22]

Jez Humble and Joanne Molesky. 2011. Why enterprises must adopt DevOps to enable continuous delivery. Cutter IT J. 24, 8 (2011), 6. Code: B5.

[23]

Waqar Hussain, Tony Clear, and Stephen MacDonell. 2017. Emerging trends for global DevOps: A New Zealand perspective. In Proceedings of the 12th International Conference on Global Software Engineering (ICGSE’17). IEEE Press, 21--30. Code: A25.

Digital Library

[24]

Martin Gilje Jaatun. 2018. Software security activities that support incident management in secure DevOps. In Proceedings of the 13th International Conference on Availability, Reliability and Security (ARES’18). ACM, 8:1--8:6. Code: A803.

Digital Library

[25]

Martin Gilje Jaatun, Daniela S. Cruzes, and Jesus Luna. 2017. DevOps for better software security in the cloud. In Proceedings of the 12th International Conference on Availability, Reliability and Security (ARES’17). ACM, Article 69, 69:1--69:6 pages. Code: A85.

Digital Library

[26]

Hui Kang, Michael Le, and Shu Tao. 2016. Container and microservice driven design for cloud infrastructure DevOps. In Proceedings of the IEEE International Conference on Cloud Engineering (IC2E’16). 202--211. Code: I58.

[27]

Mik Kersten. 2018. A Cambrian explosion of DevOps tools. IEEE Softw. 35, 2 (2018), 14--17. Code: I808.

[28]

Teemu Laukkarinen, Kati Kuusinen, and Tommi Mikkonen. 2017. DevOps in regulated software development: Case medical devices. In Proceedings of the 39th International Conference on Software Engineering: New Ideas and Emerging Results Track (ICSE-NIER’17). IEEE Press, 15--18. Code: A26.

Digital Library

[29]

M. Leppanen, S. Makinen, M. Pagels, V. Eloranta, J. Itkonen, M. V. Mantyla, and T. Mannisto. 2015. The highways and country roads to continuous deployment. IEEE Softw. 32, 2 (2015), 64--72. Code: B14.

Digital Library

[30]

Z. Li, Q. Lu, L. Zhu, X. Xu, Y. Liu, and W. Zhang. 2018. An empirical study of cloud API issues. IEEE Cloud Comput. 5, 2 (2018), 58--72. Code: I802.

[31]

Lucy Ellen Lwakatare, Teemu Karvonen, Tanja Sauvola, Pasi Kuvaja, Helena Holmström Olsson, Jan Bosch, and Markku Oivo. 2016. Towards DevOps in the embedded systems domain: Why is it so hard? In Proceedings of the 49th Hawaii International Conference on System Sciences (HICSS’16). IEEE, 5437--5446. Code: A42.

Digital Library

[32]

Kostas Magoutis, Christos Papoulas, Antonis Papaioannou, Flora Karniavoura, Dimitrios-Georgios Akestoridis, Nikos Parotsidis, Maria Korozi, Asterios Leonidis, Stavroula Ntoa, and Constantine Stephanidis. 2015. Design and implementation of a social networking platform for cloud deployment specialists. J. Int. Serv. Appl. 6, 1 (2015). Code: S3.

[33]

Steve Neely and Steve Stolt. 2013. Continuous delivery? Easy! Just change everything (well, maybe it is not that easy). In Proceedings of the Agile Conference. 121--128. Code: B23.

Digital Library

[34]

Kristian Nybom, Jens Smeds, and Ivan Porres. 2016. On the impact of mixing responsibilities between devs and ops. In Proceedings of the International Conference on Agile Software Development (XP’16). Springer International Publishing, 131--143. Code: S18.

[35]

Helena H. Olsson, Hiva Alahyari, and Jan Bosch. 2012. Climbing the “stairway to heaven”—A mulitiple-case study exploring barriers in the transition from agile development towards continuous deployment of software. In Proceedings of the 38th Euromicro Conference on Software Engineering and Advanced Applications. 392--399. Code: B17.

Digital Library

[36]

Candy Pang and Abram Hindle. 2016. Continuous maintenance. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME’16). 458--462. Code: I55.

[37]

Rahul Punjabi and Ruhi Bajaj. 2016. User stories to user reality: A DevOps approach for the cloud. In Proceedings of the IEEE International Conference on Recent Trends in Electronics, Information Communication Technology (RTEICT’16). 658--662. Code: I17.

[38]

Akond Rahman. 2018. Characteristics of defective infrastructure as code scripts in DevOps. In Proceedings of the 40th International Conference on Software Engineering (ICSE’18). ACM, 476--479. Code: A806.

Digital Library

[39]

M. Rajkumar, A. K. Pole, V. S. Adige, and P. Mahanta. 2016. DevOps culture and its impact on cloud delivery and software development. In Proceedings of the International Conference on Advances in Computing, Communication, Automation (ICACCA’16). 1--6. Code: I48.

[40]

James Roche. 2013. Adopting DevOps practices in quality assurance. Commun. ACM 56, 11 (2013), 38--43. Code: A74.

Digital Library

[41]

S. Van Rossem, W. Tavernier, D. Colle, M. Pickavet, and P. Demeester. 2018. Introducing development features for virtualized network services. IEEE Commun. Mag. 56, 8 (2018), 184--192. Code: I77.

Digital Library

[42]

Mojtaba Shahin, Muhammad Ali Babar, and Liming Zhu. 2016. The intersection of continuous deployment and architecting process: Practitioners’ perspectives. In Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’16). ACM, 44:1--44:10. Code: A64.

Digital Library

[43]

Alan Sill. 2014. Cloud standards and the spectrum of development. IEEE Cloud Comput. 1, 3 (2014), 15--19. Code: I67.

[44]

Rodrigo Siqueira, Diego Camarinha, Melissa Wen, Paulo Meirelles, and Fabio Kon. 2018. Continuous delivery: Building trust in a large-scale, complex government organization. IEEE Softw. 35, 2 (2018), 38--43. Code: B29.

[45]

Barry Snyder and Bill Curtis. 2018. Using analytics to guide improvement during an agile/DevOps transformation. IEEE Softw. 35, 1 (2018), 78--83. Code: I7.

[46]

Johannes Wettinger, Vasilios Andrikopoulos, and Frank Leymann. 2015. Automated capturing and systematic usage of DevOps knowledge for cloud applications. In Proceedings of the IEEE International Conference on Cloud Engineering. IEEE, 60--65. Code: A61.

Digital Library

[47]

Johannes Wettinger, Vasilios Andrikopoulos, and Frank Leymann. 2015. Enabling DevOps collaboration and continuous delivery using diverse application environments. In Proceedings of the On the Move to Meaningful Internet Systems Conferences (OTM’15). Springer International Publishing, 348--358. Code: A17.

Digital Library

[48]

Eoin Woods. 2016. Operational: The forgotten architectural view. IEEE Softw. 33, 3 (2016), 20--23. Code: A82.

Digital Library

[49]

Hasan Yasar and Kiriakos Kontostathis. 2016. Where to integrate security practices on DevOps platform. Int. J. Sec. Softw. Eng. 7, 4 (2016), 39--50. Code: A58.

Digital Library

[50]

L. Zhu, D. Xu, A. B. Tran, X. Xu, L. Bass, I. Weber, and S. Dwarakanathan. 2015. Achieving reliable high-frequency releases in cloud environments. IEEE Softw. 32, 2 (2015), 73--80. Code: B12.

Digital Library

[51]

xMatters. 2017. xMatters Atlassian DevOps Maturity Survey Report 2017. Retrieved from: https://www.xmatters.com/press-release/xmatters-atlassian-2017-devops-maturity-survey-report/.

[52]

YouTube. 2018. How Netflix Thinks of DevOps. Retrieved from: https://www.youtube.com/watch?v=UTKIT6STSVM.

[53]

Nicole Forsgren Velasquez Alanna Brown, Gene Kim, Nigel Kersten, and Jez Humble. 2016. 2016 State of DevOps Report. Retrieved from: https://puppet.com/resources/whitepaper/2016-state-of-devops-report.

[54]

Hrishikesh Barua. 2015. The Role of Configuration Management in a Containerized World. Retrieved from: https://www.infoq.com/news/2015/12/containers-vs-config-mgmt.

[55]

Len Bass, Ingo Weber, and Liming Zhu. 2015. DevOps: A Software Architect’s Perspective. Addison-Wesley Professional.

Digital Library

[56]

Helen Beal. 2015. Where are you on the DevOps Maturity Scale Webcast. Retrieved from: https://www.youtube.com/watch?v=a50ArHzVRqk.

[57]

Kent Beck and Cynthia Andres. 2004. Extreme Programming Explained: Embrace Change. Addison-Wesley Professional.

Digital Library

[58]

Betsy Beyer, Chris Jones, Jennifer Petoff, and Niall Richard Murphy. 2016. Site Reliability Engineering: How Google Runs Production Systems. O’Reilly Media.

[59]

Jonas Bonér, Dave Farley, Roland Kuhn, and Martin Thompson. 2014. The Reactive Manifesto. Retrieved from: https://www.reactivemanifesto.org/.

[60]

Rob Brigham. 2015. DevOps at Amazon: A Look at Our Tools and Processes. Retrieved from: https://www.youtube.com/watch?v=esEFaY0FDKc.

[61]

Donovan Brown. 2018. Our DevOps Journey—Microsoft’s Internal Transformation Story. Retrieved from: https://www.youtube.com/watch?v=cbFzojQOjyA.

[62]

David Budgen and Pearl Brereton. 2006. Performing systematic literature reviews in software engineering. In Proceedings of the 28th International Conference on Software Engineering (ICSE’06). ACM, 1051--1052.

Digital Library

[63]

Necco Ceresani. 2016. The Periodic Table of DevOps Tools v.2 Is Here. Retrieved from: https://blog.xebialabs.com/2016/06/14/periodic-table-devops-tools-v-2/.

[64]

Kathy Charmaz. 2008. Chapter 7: Grounded theory as an emergent method. In Handbook of Emergent Methods. The Guilford Press.

[65]

Gerry Coleman and Rory O’Connor. 2008. Investigating software process in practice: A grounded theory perspective. J. Syst. Softw. 81, 5 (2008), 772--784.

Digital Library

[66]

Juliet Corbin and Anselm Strauss. 2014. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory (4th ed.). SAGE Publications, Inc.

[67]

Breno B. Nicolau de França, Helvio Jeronimo, Jr., and Guilherme Horta Travassos. 2016. Characterizing DevOps by hearing multiple voices. In Proceedings of the 30th Brazilian Symposium on Software Engineering (SBES’16). ACM, 53--62.

[68]

Patrick Debois. 2008. Agile Infrastructure Operations. At Agile 2008 Toronto. Retrieved on October 2019 from http://www.jedi.be/presentations/agile-infrastructure-agile-2008.pdf.

[69]

Phil Dougherty. 2015. Containers vs. Config Management. Retrieved from: https://blog.containership.io/containers-vs-config-management-e64cbb744a94.

[70]

Floris Erich, Chintan Amrit, and Maya Daneva. 2014. A mapping study on cooperation between information system development and operations. In Product-Focused Software Process Improvement, Andreas Jedlitschka, Pasi Kuvaja, Marco Kuhrmann, Tomi Männistö, Jürgen Münch, and Mikko Raatikainen (Eds.). Springer International Publishing, Cham, 277--280.

[71]

F. M. A. Erich, C. Amrit, and M. Daneva. 2017. A qualitative study of DevOps usage in practice. J. Softw.: Evol. Proc. 29, 6 (2017), e1885.

Digital Library

[72]

Nicole Forsgren, Jez Humble, and Gene Kim. 2018. Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations. IT Revolution Press.

Digital Library

[73]

Martin Fowler. 2004. Strangler Application. Retrieved from: https://www.martinfowler.com/bliki/Strangler-Application.html.

[74]

Martin Fowler. 2010. Blue Green Deployment. Retrieved from: https://martinfowler.com/bliki/BlueGreen-Deployment.html.

[75]

Georges Bou Ghantous and Asif Gill. 2017. DevOps: Concepts, practices, tools, benefits and challenges. In Proceedings of the 21st Pacific Asia Conference on Information Systems (PACIS’17). 96:1--96:12.

[76]

Peter J. Hager, Howard Jeffrey Scheiber, and Nancy C. Corbin. 1997. Designing 8 Delivering: Scientific, Technical, and Managerial Presentations. John Wiley 8 Sons.

[77]

James Hamilton. 2007. On designing and deploying internet-scale services. In Proceedings of the 21st Large Installation System Administration Conference (LISA’07). USENIX, 231--242.

Digital Library

[78]

Pete Hodgson. 2017. Feature Toggles (aka Feature Flags). Retrieved from: https://martinfowler.com/articles/feature-toggles.html.

[79]

Jonah Horowitz. 2017. Configuration Management Is an Antipattern. Retrieved from: https://hackernoon.com/configuration-management-is-an-antipattern-e677e34be64c.

[80]

Jez Humble. 2010. Continuous Delivery vs Continuous Deployment. Retrieved from: https://continuousdelivery.com/2010/08/continuous-delivery-vs-continuous-deployment/.

[81]

Jez Humble. 2012. There’s No Such Thing as a “DevOps Team.” Retrieved from: https://continuousdelivery.com/2012/10/theres-no-such-thing-as-a-devops-team/.

[82]

Jez Humble and David Farley. 2010. Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation. Addison-Wesley Professional.

Digital Library

[83]

Ramtin Jabbari, Nauman bin Ali, Kai Petersen, and Binish Tanveer. 2016. What is DevOps?: A systematic mapping study on definitions and practices. In Proceedings of the Scientific Workshop Proceedings of XP2016 (XP’16 Workshops). ACM, 12:1--12:11.

Digital Library

[84]

Adam Jacob. 2015. Chef Style DevOps Kungfu. Retrieved from: https://www.youtube.com/watch?v=_DEToXsgrPc.

[85]

Dan Kelly. 2016. Configuration Management and Containers: Which Is Better? Retrieved from: https://blog.containership.io/configuration-management-and-containers-which-is-better.

[86]

N. Kerzazi and B. Adams. 2016. Botched releases: Do we need to roll back? Empirical study on a commercial web app. In Proceedings of the IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER’16), Vol. 1. 574--583.

[87]

N. Kerzazi and B. Adams. 2016. Who needs release and DevOps engineers, and why? In Proceedings of the IEEE/ACM International Workshop on Continuous Software Evolution and Delivery (CSED’16). 77--83.

[88]

Gene Kim. 2012. The Three Ways: The Principles Underpinning DevOps. Retrieved from: http://itrevolution.com/the-three-ways-principles-underpinning-devops/.

[89]

Gene Kim, Kevin Behr, and Kim Spafford. 2014. The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win. IT Revolution.

[90]

Gene Kim, Jez Humble, Patrick Debois, and John Willis. 2016. The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations. IT Revolution Press.

Digital Library

[91]

Henrik Kniberg. 2014. Spotify engineering culture (part 1). Retrieved from: https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1.

[92]

Per Kroll and Philippe Kruchten. 2003. The Rational Unified Process Made Easy: A Practitioner’s Guide to the RUP. Addison-Wesley Professional.

[93]

Patrick Kua. 2013. An Appropriate Use of Metrics. Retrieved from: https://martinfowler.com/articles/useOfMetrics.html.

[94]

James Lewis and Martin Fowler. 2014. Microservices. Retrieved from: https://www.martinfowler.com/articles/microservices.html.

[95]

Lucy Ellen Lwakatare, Pasi Kuvaja, and Markku Oivo. 2015. Dimensions of DevOps. In Agile Processes in Software Engineering and Extreme Programming. Springer International Publishing, 212--217.

[96]

Robert C. Martin. 2008. Chapter 12: Emergence. In Clean Code: A Handbook of Agile Software Craftsmanship. Prentice Hall.

[97]

M. Douglas McIlroy, J. M. Buxton, Peter Naur, and Brian Randell. 1968. Mass-produced software components. In Proceedings of the NATO Conference on Software Engineering, Software Engineering Concepts and Techniques. 88--98.

[98]

Peter Mell and Timothy Grance. 2011. The NIST definition of cloud computing. Retrieved from: http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf.

[99]

Matthew B. Miles and A. Michael Huberman. 1994. Chapter 2: Focusing and bounding the collection of data—The substantive start. In Qualitative Data Analysis: An Expanded Sourcebook (6th ed.). SAGE Publications.

[100]

Dejan Milojicic. 2011. Autograding in the cloud: Interview with David O’Hallaron. IEEE Int. Comput. 15, 1 (2011), 9--12.

Digital Library

[101]

Kief Morris. 2016. Infrastructure as Code: Managing Servers in the Cloud. O’Reilly Media.

[102]

Eueung Mulyana, Rifqy Hakimi, and Hendrawan. 2018. Bringing automation to the classroom: A ChatOps-based approach. In Proceedings of the 4th International Conference on Wireless and Telematics (ICWT’18). 1--6.

[103]

Michael T. Nygard. 2009. Release It! Design and Deploy Production-Ready Software. Pragmatic Bookshelf.

[104]

Mary Poppendieck and Tom Poppendieck. 2006. Implementing Lean Software Development: From Concept to Cash. Addison-Wesley Professional.

[105]

Roger S. Pressman. 2005. Software Engineering: A Practitioner’s Approach (6th ed.). Palgrave Macmillan.

Digital Library

[106]

Mike Roberts. 2018. Serverless Architectures. Retrieved from: https://martinfowler.com/articles/serverless.html.

[107]

Kevin Roebuck. 2011. DevOps: High-impact Strategies—What You Need to Know: Definitions, Adoptions, Impact, Benefits, Maturity, Vendors. Tebbo.

[108]

Margaret Rouse. 2015. What Is NoOps?—Definition from WhatIs.com. Retrieved from: https://searchcloudapplications.techtarget.com/definition/noops.

[109]

Danilo Sato. 2014. CanaryRelease. Retrieved from: https://martinfowler.com/bliki/CanaryRelease.html.

[110]

Danilo Sato. 2014. Chapter 12: Infrastructure as code. In DevOps in Practice: Reliable and Automated Software Delivery. Casa do Código.

[111]

Alexandra Sbaraini, Stacy M. Carter, R. Wendell Evans, and Anthony Blinkhorn. 2011. How to do a grounded theory study: A worked example of a study of dental practices. BMC Med. Res. Methodol. 11, 128 (2011), 1--20.

[112]

Julia Silge. 2017. How Much Do Developers Earn? Find Out with the Stack Overflow Salary Calculator. Retrieved from: https://stackoverflow.blog/2017/09/19/much-developers-earn-find-stack-overflow-salary-calculator/.

[113]

Matthew Skelton and Manuel Pais. 2013. DevOps Topologies. Retrieved from: https://web.devopstopologies.com/.

[114]

Jens Smeds, Kristian Nybom, and Ivan Porres. 2015. DevOps: A definition and perceived adoption impediments. In Agile Processes in Software Engineering and Extreme Programming. Springer International Publishing, 166--177.

[115]

Ian Sommerville. 2011. Software Engineering (9th ed.). Addison-Wesley.

Digital Library

[116]

Daniel Stahl, Torvald Martensson, and Jan Bosch. 2017. Continuous practices and DevOps: Beyond the buzz, what does it all mean? In Proceedings of the 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA’17). 440--448.

[117]

Klaas-Jan Stol, Paul Ralph, and Brian Fitzgerald. 2016. Grounded theory in software engineering research: A critical review and guidelines. In Proceedings of the IEEE/ACM 38th International Conference on Software Engineering (ICSE’16). 120--131.

Digital Library

[118]

Antonio Terceiro, Joenio Costa, João Miranda, Paulo Meirelles, Luiz Romário Rios, Lucianna Almeida, Christina Chavez, and Fabio Kon. 2010. Analizo: An extensible multi-language source code analysis and visualization toolkit. In Proceedings of the Brazilian Conference on Software: Theory and Practice (Tools Session) (CBSoft’10), Vol. 29.

[119]

J. C. van Winkel. 2017. Life of an SRE at Google. Retrieved from: https://www.youtube.com/watch?v=7Oe8mYPBZmw.

[120]

Nicole Forsgren Velasquez, Gene Kim, Nigel Kersten, and Jez Humble. 2014. 2014 State of DevOps Report. Retrieved from: https://puppet.com/resources/whitepaper/2014-state-devops-report.

[121]

Adam Wiggins. 2011. The Twelve-Factor App. Retrieved from: https://12factor.net/.

[122]

Claes Wohlin. 2014. Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering (EASE’14). ACM, 38:1--38:10.

Digital Library

Cited By

Nasir Qureshi JShoaib Farooq MAli UKhelifi AAtal Z(2025)Exploring the Integration of Blockchain and Distributed DevOps for Secure, Transparent, and Traceable Software DevelopmentIEEE Access10.1109/ACCESS.2024.350903613(15489-15502)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2024.3509036
Zhang XZhao PJaskolka J(2025)Navigating the DevOps landscapeJournal of Systems and Software10.1016/j.jss.2024.112331(112331)Online publication date: Jan-2025
https://doi.org/10.1016/j.jss.2024.112331
Amrit CNarayanappa A(2025)An analysis of the challenges in the adoption of MLOpsJournal of Innovation & Knowledge10.1016/j.jik.2024.10063710:1(100637)Online publication date: Jan-2025
https://doi.org/10.1016/j.jik.2024.100637
Show More Cited By

Index Terms

A Survey of DevOps Concepts and Challenges
1. Software and its engineering
  1. Software creation and management

Recommendations

DevOps in practice: an exploratory case study
XP '18: Proceedings of the 19th International Conference on Agile Software Development: Companion

DevOps is a cultural movement and technical solution that plays a fundamental role for software-intensive organizations whose business greatly depends on how efficient development and operation are. DevOps is relatively recent, and thus little is known ...
Assessing the Maturity of DevOps Practices in Software Industry: An Empirical Study of HELENA2 Dataset
EASE '22: Proceedings of the 26th International Conference on Evaluation and Assessment in Software Engineering

Currently, the software development organizations are adopting DevOps practices in order to develop quality product. Due to the lack of definition of DevOps, the principles, practices, and methods adopted in DevOps to determine success have changed ...
CMMI guided process improvement for DevOps projects: an exploratory case study
ICSSP '16: Proceedings of the International Conference on Software and Systems Process

Very recently, an increasing number of software companies adopted DevOps to adapt themselves to the ever-changing business environment. While it is important to mature adoption of the DevOps for these companies, no dedicated maturity models for DevOps ...

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 52, Issue 6

November 2020

806 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3368196

Editor:
Sartaj Sahni
Department of Computer and Information Science and Engineering

Issue’s Table of Contents

Copyright © 2019 ACM.

© 2019 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 November 2019

Accepted: 01 August 2019

Revised: 01 June 2019

Received: 01 January 2019

Published in CSUR Volume 52, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Survey
Research
Refereed

Funding Sources

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
Fundação de Amparo à Pesquisa do Estado de São Paulo
Conselho Nacional de Desenvolvimento Científico e Tecnológico

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

270
Total Citations
View Citations
8,714
Total Downloads

Downloads (Last 12 months)1,005
Downloads (Last 6 weeks)111

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Nasir Qureshi JShoaib Farooq MAli UKhelifi AAtal Z(2025)Exploring the Integration of Blockchain and Distributed DevOps for Secure, Transparent, and Traceable Software DevelopmentIEEE Access10.1109/ACCESS.2024.350903613(15489-15502)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2024.3509036
Zhang XZhao PJaskolka J(2025)Navigating the DevOps landscapeJournal of Systems and Software10.1016/j.jss.2024.112331(112331)Online publication date: Jan-2025
https://doi.org/10.1016/j.jss.2024.112331
Amrit CNarayanappa A(2025)An analysis of the challenges in the adoption of MLOpsJournal of Innovation & Knowledge10.1016/j.jik.2024.10063710:1(100637)Online publication date: Jan-2025
https://doi.org/10.1016/j.jik.2024.100637
Olsson HBosch J(2025)Strategic digital product managementInformation and Software Technology10.1016/j.infsof.2024.107594177:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.infsof.2024.107594
Amaro RPereira Rda Silva M(2025)Mapping DevOps capabilities to the software life cycleInformation and Software Technology10.1016/j.infsof.2024.107583177:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.infsof.2024.107583
Pérez-Sánchez JRafi SCarrillo de Gea JNicolás Ros JFernández Alemán J(2025)A theory on human factors in DevOps adoptionComputer Standards & Interfaces10.1016/j.csi.2024.10390792(103907)Online publication date: Mar-2025
https://doi.org/10.1016/j.csi.2024.103907
Matsushima TNoguchi YArima JHarada KAoki TOkita YIkeda YIshimoto KTaniguchi SYamashita YSeto SGu SIwasawa YMatsuo Y(2024)Data-driven Development Process in Service Robot Systemsサービスロボットシステムにおけるデータドリブンな開発工程の検討Journal of the Robotics Society of Japan10.7210/jrsj.42.18942:2(189-192)Online publication date: 2024
https://doi.org/10.7210/jrsj.42.189
Ashraf H(2024)Security ConcernsGenerative AI for Web Engineering Models10.4018/979-8-3693-3703-5.ch012(241-268)Online publication date: 27-Sep-2024
https://doi.org/10.4018/979-8-3693-3703-5.ch012
Moreb M(2024)Software Process Model for WebsiteDesign and Implementation of Software Engineering for Modern Web Applications10.4018/978-1-6684-4844-1.ch002(53-71)Online publication date: 30-Jun-2024
https://doi.org/10.4018/978-1-6684-4844-1.ch002
Oyucu SAksöz A(2024)Integrating Machine Learning and MLOps for Wind Energy Forecasting: A Comparative Analysis and Optimization Study on Türkiye’s Wind DataApplied Sciences10.3390/app1409372514:9(3725)Online publication date: 27-Apr-2024
https://doi.org/10.3390/app14093725
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents