DevOps - A Business Perspective

1What is DevOps?

Methods of IT management do not stand still. Approaches to the development and operation of information systems nowadays are different from those several decades ago. Moreover, tomorrow will be the time of the next generation of refined methods and techniques, which will be based on new knowledge, experience and technology. Most of the time, management methods evolve gradually, by means of systematizing and honing of the models created earlier, based on certain basic principles and postulates. However, from time to time, discontinuities occur, allowing individual leader organizations to make a significant step forward with regards to effective and efficient use of information technology.

A good example is the transition of IT management from focus on IT systems to managing IT services. Having started around the year 2000, this change in the view of management enabled pioneers to gain significant competitive advantages. Successfully adopted by the leaders, emerging management practices became so-called best practices; and some of the best practices evolved further to generally accepted good practices, and even contributed to industry standards. Of course, some organizations did not use the best practices or standards in their work: not all spheres of economy were significantly IT-dependent in those days.

Fig. 1.1 Emergence and use of new practices

Let us look at IT service management, for example. In the 1980s, the idea to provide value from information technology in the form of services and to organize IT activities in the form of processes arose. Certain European companies became pioneers, developing new practices in organizing work and approaches to solving management problems. Some of the practices, such as introduction of a Service Desk; distinction between incidents and problems; managed and controlled processing of IT infrastructure changes, etc., were formulated in 2000-2001 in key publications such as ITIL® (it used to stand for IT Infrastructure library in those days)1. This allowed them to move into the category of best practices, and not only leading organizations, but also the ‘followers’ started using them. Eventually, in the year 2002 BS 15000-1:2002, the first standard for IT service management was published, which established a certain norm to be followed by those who seek to build a coherent IT service management system. That said, practices, publications and standards do not stop developing:

Fig. 1.2 Development of practices

Similar dynamics can be observed now in Agile software development. However, the revolution that is brewing here affects a larger area than software development alone and the scale of the consequences may be on the same level as that of ITSM.

New, emerging practices have been labelled ‘DevOps’ (Development + Operations), which is as far from the intended meaning as ITIL® is far from the concept of ‘library’, and today’s COBIT from control objectives.

While publishing COBIT 5 in 2012, the copyright holder pointed out that, even though originally COBIT was an abbreviation of ‘Control Objectives for Information and Related Technology’, now it is a just proper name2.

ITIL® custodian since 2013, AXELOS Limited has made similar comments about ITIL®.

DevOps experts, who were the originators of this movement, acknowledge the limited nature of the name, calling to use more accurate in their opinion ‘BizDevOps’, ‘DevSecOps’ and the like. However, the probability of changing the name is now insignificant.

So, the DevOps phenomenon is worth studying. To understand fully the essence of DevOps, it is necessary to consider the background of both the idea and the movement associated with it.

1.1 Origins

One could argue that DevOps appeared due to two factors: wide adoption of agile software development methods and of management of IT infrastructure as a program code. Let’s look at each of them.

1.1.1 Agile methods for software development

At the end of the 20th century, the dominant methodology of software development was the so-called ‘waterfall model’: sequential execution of predetermined stages, each of which takes significant time and ends with the achievement of previously agreed results; transition to the next stage in many cases occurs only after the previous stage is fully and formally completed. An additional distinguishing feature of this model is the functional specialization of the people involved at each stage: analysts, architects, developers, testers, and so on.

When developing large information systems of pre-defined functionality and with no or limited requirements for fast delivery of the product, this model enables creation of highquality products, combined with effective and detailed cost control.

However, at the end of the 1990s, with the rapid growth of Internet technologies and web programming, downsides of the waterfall model started to affect interaction and understanding between information systems customers (internal or external business) and providers (internal or external software developers). Indeed, emerging market opportunities available for business customers required rapid launches (within a few months) of new products to the market. However, a typical development cycle from the beginning of the project to the first working prototype could take from six to 18 months; up to 2-3 years in larger enterprises. In addition, with the emergence of previously unknown but potentially promising market opportunities, customer requirements could change in the course of the development, which was extremely difficult to take into account without extending the deadlines, or reducing the quality of the product.

Fig. 1.3 An example of a waterfall software development model

Fig. 1.4 Classical pyramid of the project management constraints

Thus, tension was building up between customers and providers; between the core business and software developers. Innovative approaches to programming were the answer to this challenge. Ken Schwaber published several books about Scrum3. Kent Beck published a book on extreme programming, or XP4. However, the effect of the application of these new ideas was moderate, mainly because it was focused on just one of the stages of the software development cycle — the actual programming, while the problem was wider. The end-to-end software development cycle needed to be simplified and speeded up.

In 2001, Schwaber and Beck, along with fifteen other experts, met up to discuss the existing problems and to work out a solution. The outcome of the meeting was the so-called Agile Manifesto. It was designed to bridge the gap between business and software developers. One of the manifesto’s authors, Robert C. Martin, explains5:

‘Trust between developers and business can emerge and develop when the right disciplines and the right minimum process are used. Business will start to trust the developers, instead of thinking that they are lazy, corrupt, nasty creatures, and the developers will start to pay attention to business and realize that they are reasonable and rational beings, rather than someone from another planet.’

The subsequent developing and adoption of agile methods by the community of programmers and project managers greatly accelerated and restructured software development.

Agile Manifesto6

We are uncovering better ways of developing software by doing it and helping others do it. Through this work, we have come to value:

That is, while there is value in the items on the right side, we value the items on the left more.

We follow these principles:

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

4. Business people and developers must work together daily throughout the project.

5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

7. Working software is the primary measure of progress.

8. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. Agile processes promote sustainable development.

9. Continuous attention to technical excellence and good design enhances agility.

10. Simplicity — the art of maximizing the amount of work not done — is essential.

11. The best architectures, requirements, and designs emerge from self-organizing teams.

12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

The key elements of agile development are: closer interaction between the customer and the developer, reduction of the batch size, products delivered at short intervals (cycles) and limited size of the teams.

Using an agile approach, the software development team releases a new viable product every two to four weeks. End users are closely involved in the development, thus ensuring fast feedback, which, in turn, inspires faster changes.

However, in many companies, abandoning the waterfall model in favour of agile development, effect was smaller than expected. Failure to benefit from agile observed in many companies, often has little to do with advantages of the waterfall model or the disadvantages of agile. The problem roots in the fact that development of the code is only one of the links in a long value chain.

Indeed, prior to the development there is still a significant group of steps aimed at identifying business needs, their elaboration, analysis, prioritization, and so on.

Furthermore, after development, applications need to be quickly deployed in the production environment, so that the customers received all the benefits they had been promised, and could provide feedback to the developers. However, IT infrastructure of almost every organization established before 2010 is based on rigid, expensive hardware procured a long time ago; budgets for it were obtained with great difficulty and the budgeting process for new procurement is lengthy.

Moreover, this infrastructure is in a rather fragile state in a large number of organizations. One of the factors contributing to such fragility is that the IT solutions used are extremely complex. There are many thousands of interconnected items in the infrastructure. Another contributor is the lack of IT systems documentation, as well as the rapid obsolescence of the documentation. The latter is continually enhanced by the loss of knowledge due to the staff turnover.

In many organizations it is unsafe to touch IT infrastructure. Change is the biggest evil for IT operations department, and a constant large flow of changes may lead to truly catastrophic consequences.

Thus, advanced methods of software development are held up by obstacles on the IT operations side, which decreases the possible positive effect of applying agile approaches.

To deal with IT infrastructure fragility, some organizations use formalized and automated change management process designed to structure the flow of changes and minimize the risks associated with their implementation,

1.1.2 Managing infrastructure as code

The emergence of management of IT infrastructure as code was preceded by development of two technologies: virtualization and cloud computing.

The history of virtualization of software and hardware environments began quite a long time ago, in 1964, with the beginning of the development of the IBM CP-40 operating system7. During the years of consistent development of this area, considerable progress has been made. First commercially available systems for mainframes appeared in the 1970s, and those for subsequently more common machines based on the Intel x86 architecture appeared in the 1980s8. The chart below shows the number of key events related to virtualization between 1964 and 2008 (the graph does not stop at this year by accident, as you will see further):

Fig. 1.5 Key virtualization events distributed