BEYOND YAGNI

How to Agilely Architect an Agile Architecture

AGILITY AND ARCHITECTURE team has a very narrow range of choices. There is in fact
The phrase “Agile architecture” evokes two concepts: little significant architectural work left to be done.

1. A system or software architecture that is versatile, Architectural design, when it is really needed because
easy to evolve, and easy to modify, while resilient of a project’s novelty, has an uneasy relationship with
enough not to degrade after a few changes. traditional Agile practices. Unlike the functionality of
a system, design cannot easily be decomposed into
2. An Agile way to define an architecture, using an small chunks of work, user stories, or “technical
iterative lifecycle, allowing the architectural design stories.” Most of the difficult aspects of architectural
to tactically evolve over time, as the problem and the design are driven by nonfunctional requirements,
constraints are better understood. or quality attributes: security, high availability, fault
The two concepts are not the same: you can have a tolerance, interoperability, scalability, and so on. Other
non-Agile development process that leads to a flexible, difficulties are driven by quality attributes related to
adaptable architecture, and vice versa, an Agile process development itself — such as testability, certification,
that leads to a rigid and inflexible architecture. One and maintainability — which cannot be parceled up
does not imply the other. In the best of worlds, though, and for which tests are difficult to produce up front.
we’d like to have an Agile process that leads to a flexi- Key architectural choices cannot be easily retrofitted
ble architecture. on an existing system by means of simple refactorings.
Some of the late decisions may gut out large chunks of
The eleventh principle behind the Agile Manifesto1 the code, and therefore many of the architectural deci-
— “The best architectures, requirements, and designs sions have to be made early, although not all at once.
emerge from self-organizing teams” — reinforces the
belief among some teams that an architecture will grad- Many practitioners have grappled with the issue
ually emerge out of applying Agile practices, such as of marrying an Agile approach to designing a solid
biweekly refactorings. This thinking was cemented by architecture, such as Cutter Senior Consultant
mantras such as YAGNI (You Ain’t Gonna Need It) and Alistair Cockburn and his “walking skeleton”3 or
No BDUF (No Big Design Up Front), as well as a belief Dean Leffingwell and his colleagues’ Scaled Agile
in deferring decisions to the last responsible moment. Framework® (SAFe™).4 Common thinking nowadays
Principle 11, however, is neither prescriptive nor is that architectural design and the gradual building of
testable.2 the system (that is, its user-visible functionality) must
go hand in hand, but there are several delicate issues:
This thinking about the spontaneous emergence of archi-
tecture is reinforced by experiences with IT software n How do we pace ourselves?
endeavors that do not require a significant amount n How do we make decisions over time in a way
of bold new architectural design because (1) the most that will lead to a flexible architecture and enable
important design decisions have been made months ear- developers to proceed?
lier, (2) they are fixed by current preexisting conditions,
or (3) they are the result of a de facto architectural setup n In which order do we pick the quality attribute
in a specific domain. These architectural decisions are aspects and address them?
already embodied within the choice of frameworks and The concept of Agile architecture is not new: evolvabil-
off-the-shelf software packages. The operating system, ity, software evolution, and reengineering of existing
servers, programming language, database, middleware, systems have been studied and understood for a long
and other choices are predetermined in the vast majority time. Indeed, Manny Lehman started this investigation
of these software development projects, or the project circa 1980.5 The novel challenge that Agile architecting

10 CUTTER IT JOURNAL February 2014 NOT FOR DISTRIBUTION • For authorized use, contact ©2014 Carnegie Mellon University.
Cutter Consortium: +1 781 648 8700 • service@cutter.com
brings to system evolvability is the practices that allow AGILE ARCHITECTING
architecting the system in smaller chunks. Successful Agile architecting is not only the process of allocating
teams are those that can take advantage of existing soft- architectural work to iterations. There is also a winding
ware engineering techniques with slight modifications, route in which development proceeds as the require-
in particular those that provide early feedback and ments, architecture, and design/implementation are
learning, such as prototyping.6 elaborated iteratively and often concurrently. Early
In this article, we present lessons learned about the understanding of requirements and architecture choices
characteristics of an Agile architecture that enabled is key to managing large-scale systems and projects;
an organization to develop its architecture in an Agile however, linkage between architecture and implementa-
manner and continue to rapidly deliver features when tion design choices is also crucial.7 While requirements
more stringent quality attribute requirements emerged. originating from the problem space inform architecture
and development, explorations originating from archi-
To investigate why some teams have fewer delays and
tecture and implementation investigations also assist
disruptions to continuous delivery of features than oth-
in eliciting and detailing requirements. The essence
ers, we interviewed project teams from several govern-
of Agile architecting is to conduct these activities
ment and commercial organizations. The examples we
concurrently with the right balance.
present here come from one such project team, which
was challenged when an unexpected performance qual- In Figure 1, we illustrate an example (captured during
ity attribute requirement surfaced in customer feedback our interviews with the team) of such an Agile archi-
during a user demo. Fortunately, the team was able to tecting process for prototyping with a quality attribute
react to the emerging requirements without disrupting focus.8 In this example, the team probes for quality
the project. When we examined the team’s practices, we concerns during a user demo, discovers an emerging
found they were doing Agile architecting that resulted performance requirement (the user expected faster
in an Agile architecture. rendering of data-intensive pages than anticipated),

Figure 1 — Agile architecting process example.

Get The Cutter Edge free: www.cutter.com NOT FOR DISTRIBUTION • For authorized use, contact Vol. 27, No. 2 CUTTER IT JOURNAL 11
Cutter Consortium: +1 781 648 8700 • service@cutter.com
 analyzes the requirement, architects a solution to the infrastructure improvements targeted at future phases,
performance problem, implements the solution in a the team explained that they often create an R&D proto-
demonstrable prototype, and gets feedback from the type to begin reasoning about foundational architecture
user at the next user demo. The user accepts the proto- that may be needed to support emerging functional and
typed implementation, and the team merges the imple- quality attribute requirements (e.g., clustering infra-
mented code into the project baseline for a future structure for the scalability quality attribute require-
release. ment). The team creates such prototypes in an entirely
separate lab environment, so there is no risk to the
The team operates in three cycles in which develop-
development environment.
ment is iterated through biweekly sprints; the archi-
tecture and product team manage the releases through
forward-looking architecture-focused investigations; AGILE ARCHITECTURE
and a separate R&D cycle investigates long-term strate-
gic technology goals. Figure 1 shows the process steps We found that the team has an Agile architecture that
as integration points (IPs) between requirements and they described as a “flexible architecture.” This allows
architecture or between architecture and design/ them to explore technical options rapidly with minimal
implementation: ripple effect. The architecture can be understood in
terms of patterns and tactics that influence the time
n IP-1* (Rqmts->Arch): Probe for emerging quality and cost to implement, test, and deploy changes.9
attribute requirements at user demo.
The overarching pattern the team uses to enhance modi-
n IP-2 (Rqmts->Arch): Analyze quality attribute fiability and control the time and cost of change is the
requirements. layered pattern. Other supporting patterns and tactics to
n IP-3 (Rqmts->Arch): Conduct deeper analysis separate interfaces, restrict dependencies, and separate
of emerging quality attribute. concerns are employed as well. The team described
their architecture choices as follows:
n IP-4 (Rqmts->Arch): Identify architectural
approach/patterns. n Layered architecture, client-server architecture, and
separation of the presentation from business/data
n IP-5 (Arch->Design/Imp): Elaborate architectural
layers. Over time, the architecture and vision have
patterns for specific system.
been implemented as various layers, and components
n IP-6 (Arch->Design/Imp): Implement portion can be replaced wholesale without significant disrup-
of prototyped solution as a spike. tion elsewhere. The layered architecture supports
adding new client interfaces and replacing presen-
n IP-7 (Rqmts->Arch): Get user feedback on prototype.
tation components with minimal ripple effect. For
n IP-8 (Rqmts->Arch): Analyze feedback on prototype. example, the presentation layer was modified to add
a new Web client to an existing smart client imple-
n IP-9: Get approval for prototyped changes (merge
mentation with no effect on the underlying architec-
with release).
ture and no down time in operations to introduce the
*Note that the team probes for emerging requirements dur- new interface. The smart client was later replaced
ing the user demo as part of the requirements elicitation with .NET C# WPF with no impact to the back end.
process, which explains why IP-1 starts there. All three presentation technologies existed simultane-
Two particularly important practices that helped the ously using the same back end.
team succeed are prototyping prior to the target sprint n Service orientation with separate interfaces and
and prototyping in a separate environment. This contin- restricted dependencies. The data layer was built
uous architecture exploration allowed the team to focus behind an API service layer to insulate the client
on making the architecture Agile enough to support from changes in the data layer. These tactics enable
upcoming needs, which helped strike a balance between the team to fence off third-party dependencies by
too much up-front architecture and not enough. creating a metadata definition layer (in C#). They
The example shown in Figure 1 illustrates the proto- also support replacing the underlying database and
typing process the team uses for a relatively small archi- access control without impact to the service-oriented
tectural change. The team handles these smaller changes data layer, thereby allowing the presentation layer to
at the sprint/release planning level. For larger-scale continue in production without modification.

12 CUTTER IT JOURNAL February 2014 NOT FOR DISTRIBUTION • For authorized use, contact ©2014 Carnegie Mellon University.
Cutter Consortium: +1 781 648 8700 • service@cutter.com
n Publish-subscribe pattern. This pattern is used to or learning from spikes (that is, prototyping activities
monitor the data layer schema for changes since the that are timeboxed10), because they have the architec-
database is owned and managed by a third party. tural knowledge to rapidly analyze the impact of
changes and conduct architecture tradeoff analysis.
The team uses the following patterns and tactics to
improve scalability:
n Clustered architecture with load balancing and INTEGRATING PRACTICES WITHIN INCREMENTAL
replicated copies. The cluster-based architecture DEVELOPMENT ENVIRONMENTS
allows rapid scaling, whether increasing for mission Understanding the desired state of development and
need or scaling back due to budgetary constraints. delivery helps tie together Agile architecting and Agile
It is possible to scale up by increasing the size of architecture. A desired software development state is
the virtual servers or to scale out by increasing the one that enables Agile teams to quickly deliver releases
number of clusters, the number of analysts using that are valuable for stakeholders.11 The teams them-
the App-V client, or the number of Citrix servers selves typically define the desired state. It is their vision
in the cluster. of the ideal development infrastructure that they would
n Encapsulation of algorithms. Computationally inten- like to work with. When product development starts,
sive algorithms, which have a significant impact on the desired state does not necessarily exist. Setting up
system performance, are encapsulated to allow them the initial architecture helps push the team and the
to evolve or be replaced. delivery into the desired state and enables Agile
architecting that results in an Agile architecture.
n Data caching. A local copy of a subset of data is kept
to optimize performance of data retrieval and data In this example, the team needs both Agile architecting
processing for frequently accessed data. and architecting for agility to stay within the desired
state. Agile architecting provides the team a regular
The team also focuses on flexibility in deployment and cadence for periodically considering whether they are
controlling the cost and time for testing by using several already out of bounds or about to get out of bounds.
patterns and tactics: The regular cadence of Agile architecting also helps the
n Virtualization, layering both the infrastructure and team manage architecture exploration and balance com-
the application. By adopting virtualization at a very peting concerns of too much or too little architecting.
early stage, the system may be hosted in any data Agilely architecting an Agile architecture, then, mini-
center with the appropriate network connections. For mally has four key requirements:
example, infrastructure (VMware) and the application
(App-V) are virtualized. This enabled the system to 1. Focusing on key quality attributes and incorporating
be consolidated from deployment in five regional these into technical explorations within prototyping
centers to two, allowing the customer to reduce costs. and spikes

n Standardized and configurable architecture: para- 2. Understanding that a successful product is a combi-
meterization and static and dynamic binding. Each nation of not only customer-visible features but also
cluster has a standard configuration and can be cus- the underlying infrastructure that enables those;
tomized. This standardization allows installations to hence, architectural requirements are incorporated
be accomplished at will; for example, the team can into sprint planning and demos
allocate new servers (in the cloud), install the soft- 3. Recognizing that an Agile architecture that enables
ware, and replicate joins to scale the system to meet ease of maintainability and evolvability is the result
new needs. Additional processors and memory can of ongoing, explicit attention given to the architec-
be added to a virtual machine to increase capacity. ture, not a natural byproduct of an Agile — or any —
Similarly, clusters can be removed at will to meet software development process
resource constraints.
4. Continuously managing and synchronizing depen-
n Executable, interface-driven code structure. dencies between functional and architectural require-
Executable code structure enables automated testing ments and ensuring that the architectural foundation
of business layer functionality through interfaces. is put in place in a just-in-time manner
A solid understanding of the overall architectural struc- At a high level, the process for Agilely developing an
ture helps the team to respond to stakeholder feedback Agile architecture can be seen as a zipper, as shown in

Get The Cutter Edge free: www.cutter.com NOT FOR DISTRIBUTION • For authorized use, contact Vol. 27, No. 2 CUTTER IT JOURNAL 13
Cutter Consortium: +1 781 648 8700 • service@cutter.com
 Figure 2 — The zipper model.

Figure 2. From the requirements, as they evolve, the the Department of Defense under Contract No. FA8721-
designers extract and separate functional requirements 05-C-0003 with Carnegie Mellon University for the
(mostly features seen by the end users) and architec- operation of the Software Engineering Institute, a
tural requirements (mostly derived from key quality federally funded research and development center.
attributes), which are already expressed or anticipated.
Dependencies between these two kinds of requirements
must be managed to ensure that necessary elements of ENDNOTES
the architecture are present (or at least “stubbed”) in 1
“Principles Behind the Agile Manifesto”
upcoming iterations with functional requirements that (http://agilemanifesto.org/principles.html).
depend on them. This skeletal foundation must be 2
Séguin, Normand, Guy Tremblay, and Houda Bagane.
woven into early iterations of architectural and func- “Agile Principles as Software Engineering Principles: An
tional increments. Analysis.” Lecture Notes in Business Information Processing,
Vol. 111, edited by Claes Wohlin. Springer, 2012.
This approach will facilitate a deliberate (not accidental)
3
emergence of an architecture, constantly validated by Cockburn, Alistair. “Walking Skeleton” (http://
the functionality developed on top of it. The develop- alistair.cockburn.us/Walking+skeleton).
4
ment of the architecture occurs over several iterations, Scaled Agile Framework® (http://scaledAgileframework.com).
without being stopped, blocked, or slowed down 5
Lehman, Meir M. “Programs, Lifecycles, and Laws of Software
by developers claiming “YAGNI” or “No BDUF.” Evolution.” Proceedings of the IEEE (Special Issue on Software
However, as when a zipper gets out of alignment, caus- Engineering), Vol. 68, No. 9, September 1980.
ing it to get stuck, teams that do not pay close attention 6
Ozkaya, Ipek, Robert L. Nord, Stephany Bellomo, and Heidi
to evolving dependencies can get caught off guard with- Brayer. “Beyond Scrum + XP: Agile Architecture Practice.”
out the architectural foundation they need to support Cutter IT Journal, Vol. 26, No. 6, 2013.
emerging requirements. Agility and architecture do 7
Nuseibeh, Bashar. “Weaving the Software Development
support each other very well. Process Between Requirements and Architectures.” IEEE
Computer, Vol. 34, No. 3, March 2001.
8
Bellomo, Stephany, Robert L. Nord, and Ipek Ozkaya.
ACKNOWLEDGMENTS “Elaboration on an Integrated Architecture and Requirement
We would like to acknowledge Sphere of Influence Practice: Prototyping with Quality Attribute Focus.” Proceedings
for their technical contributions to this article. This of the Second International Workshop on the Twin Peaks of
material is based upon work funded and supported by Requirements and Architecture. IEEE, 2013.

14 CUTTER IT JOURNAL February 2014 NOT FOR DISTRIBUTION • For authorized use, contact ©2014 Carnegie Mellon University.
Cutter Consortium: +1 781 648 8700 • service@cutter.com
9
Bass, Len, Paul Clements, and Rick Kazman. Software processes in large and globally distributed teams. Dr. Kruchten
Architecture in Practice. 3rd edition. Addison-Wesley teaches courses in entrepreneurship, software project management,
Professional, 2012. and design. He is a senior member of IEEE Computer Society; an
10
Leffingwell, Dean. Agile Software Requirements: Lean IEEE Certified Software Development Professional; a member of
Requirements Practices for Teams, Programs, and the Enterprise. ACM, INCOSE, and CEEA; the founder of Agile Vancouver; and a
Addison-Wesley Professional, 2011. professional engineer in British Columbia. Dr. Kruchten has a diploma
in mechanical engineering from Ecole Centrale de Lyon and a doctor-
11
Bachmann, Felix, Robert L. Nord, and Ipek Ozkaya. ate degree in information systems from Ecole Nationale Supérieure des
“Architectural Tactics to Support Rapid and Agile Stability.” Télécommunications in Paris. He can be reached at pbk@ece.ubc.ca.
CrossTalk, May/June 2012.
Robert L. Nord is a senior member of the technical staff at the SEI.
Stephany Bellomo is a senior member of the technical staff at Dr. Nord is engaged in activities focusing on Agile and architecting
the Carnegie Mellon University (CMU) Software Engineering at scale and works to develop and communicate effective methods
Institute (SEI). Ms. Bellomo is a member of the Architecture and practices for software architecture. His collaboration with
Practices group and an active member of the Value-Driven Philippe Kruchten and Ipek Ozkaya is helping shape the research
Incremental Development research team. She teaches SEI courses in agenda on technical debt. He is coauthor of the practitioner-oriented
Service-Oriented Architecture Migration of Legacy Components and books Applied Software Architecture and Documenting Software
Software Architecture Principles and Practice. Ms. Bellomo has over Architectures: Views and Beyond and lectures on architecture-
15 years’ experience in the software field. Prior to joining the SEI, she centric approaches. Dr. Nord is a member of the steering committee of
worked as a software engineer for several organizations, including the WICSA Conference series, in addition to organizing events at
Lockheed Martin, Intuit, and VeriSign Network Solutions. She served software engineering, Agile, and architecture venues. He earned a
as tutorial chair for SEI’s SATURN Conference and is currently a PhD in computer science from CMU and is a distinguished member
member of the organizing committee for the International Workshop of the ACM. He can be reached at rn@sei.cmu.edu.
on Release Engineering 2014 (hosted by Google). Ms. Bellomo Ipek Ozkaya is a senior member of the technical staff at the SEI. With
received an MS in software engineering from the George Mason her team, Dr. Ozkaya works to help organizations improve their soft-
University. She can be reached at sbellomo@sei.cmu.edu. ware development efficiency and system evolution. Her work focuses
Philippe Kruchten is a professor of software engineering at the on software architecture practices, software economics, and require-
University of British Columbia (UBC), in Vancouver, Canada, ments management, and her latest publications include articles on
where he holds an NSERC Chair in Design Engineering. Dr. Agile architecting, dependency management, and architectural techni-
Kruchten joined UBC in 2004 after a 30-plus-year career in industry, cal debt. Dr. Ozkaya also chairs the advisory board of IEEE Software
where he worked in large software-intensive systems design in the and serves as an adjunct faculty member for the Master of Software
domains of telecommunications, defense, aerospace, and transpor- Engineering Program at CMU. She also organizes different events
tation. Some of his experience is embodied in the Rational Unified (tutorials, workshops, and sessions) and is an invited speaker at
Process, whose development he directed from 1995 to 2003. His cur- software engineering, Agile, and architecture venues (e.g., ICSE,
rent research interests reside mostly with software architecture, in OOPSLA, SATURN, and WICSA). She holds a doctorate from
particular architectural decisions and the decision process, as well as CMU. She can be reached at ozkaya@sei.cmu.edu.
software engineering processes, especially the application of Agile

Get The Cutter Edge free: www.cutter.com NOT FOR DISTRIBUTION • For authorized use, contact Vol. 27, No. 2 CUTTER IT JOURNAL 15
Cutter Consortium: +1 781 648 8700 • service@cutter.com