Chapter Title: Background: The Capabilities-Development Process

Book Title: Portfolio-Analysis Methods for Assessing Capability Options

Book Author(s): Paul K. Davis, Russell D. Shaver and Justin Beck
Published by: RAND Corporation

Stable URL: https://www.jstor.org/stable/10.7249/mg662osd.11

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms
This content is licensed under a RAND Corporation License. To view a copy of this license, visit
https://www.rand.org/pubs/permissions.html.

RAND Corporation is collaborating with JSTOR to digitize, preserve and extend access to
Portfolio-Analysis Methods for Assessing Capability Options

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
Chapter Two

Background: The Capabilities-Development

Process

Context: DoD’s Capabilities-Development Process

In recent years, the Department of Defense has sought to create a pro-

cess suitable for the capabilities-based planning (CBP) mandated in
2001 and reinforced in the most recent Quadrennial Defense Reviews
(Rumsfeld, 2001, 2006). The Aldridge Report (Joint Defense Capabil-
ities Study Team, 2004) reviewed problems with the preexisting plan-
ning system and laid out objectives and schematics for an improved
process. The Joint Staff has identified both operational and functional
capability areas (see Appendix A) and has organized to address them
systematically and comprehensively. The military Services have re-
organized their planning systems accordingly. To a significant degree,
commonalities of vocabulary and categorizations have been achieved.
The Joint Staff has issued a draft instruction that summarizes the over-
all current process, including the primary activities, their relationships,
and the inputs and outputs for the various activities (Chairman of the
Joint Chiefs of Staff, 2007).

Problems with the Current Process

The overall process is described in the Aldridge Report. Figure 2.1,

taken from that report, shows an overview. In this idealized concept,

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
 Portfolio-Analysis Methods for Assessing Capability Options

Figure 2.1
End-State Planning Process

Enhanced Execution and

Strategy planning Resourcing accountability

Operational
planning

SecDef Capabilities SecDef

strategic planning Defense Execution
joint
planning resourcing and
programming
guidance process feedback
Resources guidance

Enterprise
(non-warfighting)
planning

Feedback during next cycle

What How well

to do How to do it did we do?

SecDef decision points Iterative SecDef engagement

SOURCE: Joint Defense Capabilities Study Team, 2004.

RAND MG662-2.1

strategy poses challenges to planners, who develop requirements and

capability options. Choices must then be made, programs and bud-
gets developed, and capabilities acquired. Iteration occurs (see bottom
of figure) as the result of time, experience, and changes in the envi-
ronment. The authors of the Aldridge Report acknowledged that the
boxed segment, enhanced planning, was where miracles must happen.
When viewed in detail, that component turns out to be complex, bur-
densome, and bureaucratic—a problem recognized by senior officials
and officers. DoD leaders have been trying to improve that process,
and such efforts are continuing. In the meantime, success depends on
high-level good will and collaborative efforts transcending organiza-
tional structure.

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
 Background: The Capabilities-Development Process 

A sense of the problems is given by Figure 2.2, a widely used offi-

cial depiction in 2005. Putting aside the busy nature of the diagram, a
striking feature is the sequentiality of the process, with separate steps
for defining operational concepts, conducting functional-area analy-
ses, conducting functional-needs analyses, and conducting functional-
solutions analyses, creating an initial capability document, followed by
another sequence that includes analysis of refined alternatives, system
engineering, and so on. This sequentiality is represented organization-
ally by a myriad of committees, boards, working groups, and studies.
Less evident but even more troublesome is an underlying concept of the
military developing “requirements” and potential solutions and then
passing those potential solutions over the transom to the acquisition
world of planning, programming, budgeting, and execution (PPBE),
where decisions must be made on resource allocation. The separation
was described rather starkly in classroom lectures on DoD’s acquisition
system until quite recently. Figure 2.3 was used in an internal Penta-
gon course on the JROC/JWCA processes as of 2001. The vertical line
separating the requirements side of the figure from the acquisition side
was referred to as a “firewall.”
This separation of supposedly military and civilian functions
dates back to an unfortunate consequence of the Goldwater-Nichols
Act (U.S. Congress, 1986). Superficially, it may seem logical for the
military operators to establish requirements and the civilians to figure
out how to meet those requirements and pay the bill. Upon reflection,
however, that concept breaks down. In the real world, strategy, require-
ments, technological issues, economics, and other factors need to be
addressed simultaneously—at least at the front end of the process.
“Requirements” are not handed down on tablets of stone but should
instead be seen as outputs of decisions reached after consideration of
challenges, desired capabilities, technical feasibility, economics, orga-
nizational realities, and other factors.4
A historical problem with the linear construct has been that
requirement-setters, when working in isolation, have often been either
too conservative (not adequately appreciating future needs, technologi-

Joint Requirements Oversight Council/Joint Warfare Capability Area.

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
 

Figure 2.2
A More Complex View of the Process

• Strategic • Refined concept • Affordable • Revise KPPs • LRIP

planning Capabilities-based assessment military-useful
• Analysis of • Detailed design • IOT&E
guidance alternatives increment
• Capabilities • Gaps • Nonmaterial • System integration
• Defense solutions • Technology • Technology
• Tasks • Shortfalls • DT&E
planning development demonstrated
scenarios • Attributes • Redundancies • Material
solutions strategy • Initial KPPs
• Family of • Metrics • Risk areas • SoS system • SEP
concepts • S&T initiatives
engineering
• Transformation • Experimentation
Concept MS A MS B MS C
Activity decision
Select
Functional Functional Functional
a Joint Develop ICD Analysis of Technology CDD System CPD
area needs solutions Production
integrating concept alternatives development development
analysis analysis analysis
concept

Capabilities-based assessment
Evolutionary or spiral
development
OSD/JCS COCOM

This content downloaded from

FCB

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC

OSD (AT&L)-led
capability roadmaps Services

All use subject to https://about.jstor.org/terms

Portfolio-Analysis Methods for Assessing Capability Options

Oversight

Joint Chiefs of Staff and

SecDef Joint Staff/OSD OSD (AT&L, PA&E), services and OSD (DOT&E)—Joint Staff (JROC)
Joint Requirements Oversight Council

Policy Capabilities definition Concept refinement Acquisition and test

RAND MG662-2.2
 Background: The Capabilities-Development Process 

Figure 2.3
Stark View of Requirements Versus Acquisition

Military responsibility Civilian responsibility

Combatant Requirements Acquisition

commands

Under
Joint Secretary
Mission Acquisition
Requirements of Defense
need Decision
Oversight (Acquisition,
statement Memorandum
Council Technology
and Logistics)

Military
departments VJCS Defense
Acquisition
Board

Linking military requirements with civilian acquisition

SOURCE: From a Joint Staff course on the JROC/JWCA process in 2001.

RAND MG662-2.3

cal opportunities, and the likelihood of different concepts of opera-

tions) or naively optimistic (postulating the successful rapid develop-
ment of capabilities beyond what industry could realistically gener-
ate on the timescale imagined).5 Another problem has been that the
linear construct defers serious thinking about economic considerations
until—rather late in the process—the favored concept is finally recog-
nized as unaffordable. In such cases, valuable time has been lost, time
that might have been spent on developing more appropriate concepts
or even modifying existing capabilities in lieu of developing something
new.

Historical Successes in Capability Development

It is useful in thinking about the way ahead for DoD’s capability-

development efforts to also look backward. This is particularly so

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
 Portfolio-Analysis Methods for Assessing Capability Options

because the department has been spectacularly successful over the

decades in many of its developments. What factors contributed to these
successes?
Table 2.1 lists some of those spectacular developments. An over-
arching observation is that almost all of them were the result of far-
sighted and technologically savvy people championing developments
outside the mainstream bureaucratic processes of the time, often
achieving their successes because they evaded or finessed the more tidy,
deliberate, and sequential processes of their time.6
Another observation is that most of the great developments included
early systems analysis and systems engineering that integrated considerations
of strategy, technological promise and feasibility, operational effectiveness,
risk, and economic considerations. There was also a good deal of creative
tension, because participants had different perspectives and experi-
ences. Recognition of such matters underlies part of DoD leadership’s
interest in the Concept Decision Reviews discussed below, which focus
on the “big-A” issues rather than, say, those of how best to purchase
equipment from industry (the “small-a” issues) (Krieg, 2007).
Although numerous examples could be given, we shall mention
only a few:

• The 1967 STRAT-X study examined a broad range of options and

greatly influenced development of U.S. strategic nuclear forces
over subsequent decades.7
• Top Secret studies on strategic nuclear command and control,
conducted by the Joint Chiefs’ Weapons Systems Evaluation
Group (WSEG), worked through the complex challenge of assur-
ing the ability to command and control the nuclear retaliatory
forces after even a bolt-from-the-blue attack that destroyed many
communication mechanisms.
• The multi-decade development of the Global Positioning System
(GPS) and its many applications to navigation and precision
fires were strongly supported by studies conducted by federally
funded research and development centers (FFRDCs) and indus-
try, including very early studies envisioning with remarkable

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
 Background: The Capabilities-Development Process 

Table 2.1
Some Spectacularly Successful Capability Developments

Capability System Comment

Sea-based airpower for Aircraft carriers Developed despite lack of

fleet defense, strike, interest by and opposition of
surveillance, etc. the “Battleship Navy”
Projection of ground Amphibious operations Conceived and developed by
forces from ships into visionary Marine colonel
defended land areas

Precision fires Laser-guided bomb (LGB) Resisted consistently by

and joint direct attack Service PPBE processes and
munition (JDAM) those adhering to a peculiarly
strict interpretation of
acquisition regulations
Maritime operations in Littoral combat ship Scorned initially but
littoral areas championed by Chief of Naval
Operations
Detection and tracking of J-STARS Moved directly into field from
moving ground targets R&D during first Gulf War
Unmanned surveillance Global Hawk and Resisted by Air Force
and targeting Predator
Tactical mobility Stryker Championed by Army Chief of
Staff amid controversy
Strike, penetration F-117, B-2 Championed by Air Force
Chief of Staff and DDR&E
Precision navigation, Global Positioning System Championed by DDR&E and
precision fires (GPS) Secretary of Defense and
a few Air Force and Navy
leaders
Early warning of ballistic- DSP (early warning Resisted by most, due
missile attack satellites) to legacy programs, but
supported by Air Force Chief
of Staff
Submarine-based nuclear SSBNs (nuclear-armed Imposed on Navy by president
retaliatory forces ballistic-missile
submarines)
Assured dissemination Strategic nuclear Involved largely separate
of emergency action command and control system, thereby avoiding
messages, even in surprise during Cold War many sources of friction
attack

NOTE: PPBE = planning, programming, budgeting, and execution; DDR&E = Director,

Defense Research and Engineering. In the period referred to, DDR&E had power
comparable to that of the current USD (AT&L).

prescience many of the radical changes that would occur in both

military and civilian operations.8

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
10 Portfolio-Analysis Methods for Assessing Capability Options

It is the history of broadly based, cross-cutting, front-end analytic

and decision efforts that we primarily wish to emphasize, because it
bears on what we suggest in subsequent chapters. However, it is also
appropriate to list some of the other characteristics common to the
great developments.
People. The great developments have been conceived, envisioned,
and managed by remarkable individuals. The consistent emphasis in
interviews with “old-timers” who were involved in those developments
is on people, not process. This is in contrast to the prevailing Pentagon
culture, which emphasizes consensus and process. Indeed, this culture
has now existed for so long that the very intuition of many people
in the system revolves around consensus and process, with resulting
“requirements” for bloated, exhausting activities and no one having the
freedom to act.
A Sense of Mission. Many of the great developments were strongly
motivated by a sense of mission, such as defending NATO in the event
of a massive armored invasion by the Warsaw Pact or deterring major
war generally through a combination of assured nuclear retaliation and
the credible ability to use limited nuclear options to “reestablish deter-
rence” if conventional war began in Europe.
The Role of Vision. The great developments have often had an
accompanying vision that provided coherence and direction over time,
a vision with both substance and legs. These coherent visions identified
“thrusts,” examples of which we shall show later. These were developed
and honed by small groups of top-notch mid-level “up-and-comers.” A
number of these young officers later became well-known flag officers.
Champions. The concepts that led to the great developments,
however, would have gone nowhere except for another “people issue,”
the championing of ideas by, e.g., the Service chief and senior officials
such as the DDR&E, who in earlier years had power comparable to
that of today’s USD (AT&L).9 Typically, the concepts were disruptive
and were therefore resisted by the existing organizations. Leaders had
to override this organizational tendency to resist. That they often did so
is perhaps remarkable to those familiar with the “innovator’s dilemma”
in industry (Christensen, 1997), but defense planning has objectives
very different from profit-making.

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
 Background: The Capabilities-Development Process 11

Strategic Thinking. Top leaders associated with the great develop-

ments thought about the near, mid-, and long term and across devel-
opment categories. They were not particularly focused on science and
technology (S&T); they did, however, value it greatly and pushed inno-
vation in diverse ways (e.g., Advanced Concept Development Tests
(ACDTs) and continuing DARPA activities). They thought in terms of
phased, evolutionary developments.
Mainstream Service Opposition. The great developments (e.g.,
the Global Positioning System, Global Hawk, Predator, the Non-
Lethal Capability Set (NLCS), F-16s) usually had to overcome a lack
of interest or even strong opposition within the eventually sponsoring
Service. Often, Service chiefs played a critical role by “reaching down”
and championing innovators who would otherwise have lost out in
the competition for funding and priority. Similarly, Service chiefs were
often the ones who faced reality about the need for tradeoffs. Sometimes
they did so in behind-closed-doors cooperation with the DDR&E and
the Secretary of Defense (SecDef), without consensus within the Ser-
vice. Such was the case when the Air Force agreed to cooperate in a
high-low-mix approach that included procurement of F-16s.
SecDef Intervention. Upon occasion, the SecDef had to inter-
vene, overriding the preferences of the Services and even the Service
chiefs. Sometimes it was as part of introducing new missions; some-
times it was to protect “national” programs such as SSBNs; sometimes
it was to reflect conclusions of economic analysis (e.g., increasing the
rate of procurement of precision weapons or unmanned aerial vehicles
(UAVs)).
Finessing or Evading Processes. Especially relevant is the obser-
vation that the great developments often succeeded despite, rather than
because of, normal processes. Nothing as complex as the Joint Capa-
bilities Integration Development System (JCIDS) process (see Figure
2.2) existed until recent years, but at any given time, the then-normal
process was often regarded as too burdensome and too perilous. The
champions of the new ideas found ways to avoid that process (e.g., by
black programs). In more-recent times, senior officials have again noted
that many important developments are dealt with outside the normal
process (e.g., the current task force on improvised explosive devices

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
12 Portfolio-Analysis Methods for Assessing Capability Options

(IEDs)). This should be a sobering cautionary for those who seek to

solve current problems by perfecting established processes.

Past Recommendations and Recent Changes

Past Recommendations for Change
Against this background, a number of prominent public studies have
called for major changes in the DoD acquisition system. These include
reports of the Defense Science Board (Defense Science Board, 2005,
2007a, 2007b), the Kadish Study (Kadish, 2006), and a Center for
Strategic and International Studies (CSIS) report, Beyond Goldwater-
Nichols (Murdock, Flournoy, Campbell, Chao, Smith, Witkowsky, and
Wormuth, 2005).
In 2004–2005, RAND provided a highly critical independent
review of DoD’s capabilities-development process, characterizing it as
bureaucratic and dysfunctional.10 It recommended major changes, par-
ticularly in the front end:

• Integration and iteration, rather than lengthy and sequential pro-

cesses (of policy-setting, assessment, requirement-setting, solution
specification, and handover to acquisition).
• Having a single group approach each problem area early, a group
comprising strategic worriers, operators, technologists, and pro-
gram analysts, although perhaps headed, in an individual case, by
an ecumenical “operator” working for the SecDef and the Chair-
man of the Joint Chiefs of Staff.
• More extensive early use of systems analysis and systems engineering to
filter by feasibility, define threshold capabilities and stretch goals,
provide a framework for assessment and tradeoffs, and provide
information with which to monitor and adjust.

An overarching recommendation was that capability assessments

should be accomplished on a DoD-wide basis, with close collaboration of
the USD (AT&L) and the VJCS .

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
 Background: The Capabilities-Development Process 13

This was in contrast to redundant and sometimes antagonistic

Joint Staff and OSD processes. We suggested the name Defense Capa-
bility Area Reviews (DCARs) and anticipated a strong role for AT&L,
particularly in technology-informed concept development, initial sys-
tems engineering, and systems analysis.
In its 2006 summer study, the Defense Science Board had a
number of recommendations consistent with this move toward more-
integrated planning. They included recommendations on assuring
early-in-the-process influence of technological expertise, more-creative
search for ways in which to exploit technological opportunities arising
in the civilian sector, and the creation of mission-oriented portfolio
managers (Defense Science Board, 2007b).

A Recent Change: Concept Decision Reviews

A consensus has emerged among DoD decisionmakers on many of the
items described above, including the need for integrated work in which
requirements, acquisition, and resourcing are all considered at the same
time (Figure 2.3) and the special need for rejuvenated efforts of this
type early in the development process. It is hoped that early top-level
decisions will both avoid the start of inappropriate programs and build
the intellectual and organizational consensus needed for rapid develop-
ment of those programs that are approved.
A potentially important aspect of the new thinking is having DoD
potentially move toward a new approach currently called the Concept
Decision Review, illustrated schematically in Figure 2.4 (Krieg, 2007).
In this new approach, important potential capability developments
will be reviewed and the concepts will be accepted or rejected by a
tri-chair group of decisionmakers, nominally the DepSecDef or USD
(AT&L), the VJCS, and the Director of OSD (PA&E). Their deci-
sion will be based on substantial analysis prior to milestone A (a new
concept), and following an evaluation of alternatives (EoA), which in
turn will build upon products of the JCIDS process. The EoAs that
are envisioned would be significantly different from the analyses of
alternatives (AoAs) of past years. They would be broader and more
creative in identifying options and would involve the integration of

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
 Portfolio-Analysis Methods for Assessing Capability Options

Figure 2.4
The Capability-Development Process

Planning, programming,
budgeting, and execution CBP integrates major
system DoD processes to
facilitate strategic
planning

Strategic
planning CBP is enabled
through a common
set of joint-capability
Requirements Acquisition
definitions

SOURCE: Adapted from a briefing by James Durham, OSD (AT&L), August 2006.
RAND MG662-2.4

need, technology, concepts, and economics, discussed earlier. Because

decisions would also occur earlier, they would likely favor planning for
sensible evolutionary development, with planned times for subsequent
decisions at branch points.
The intent is then to have intensive technology development of
an approved concept prior to a milestone B, which could be reached
more rapidly and with less Office of the Secretary of Defense (OSD)
management oversight than has typically been involved in going from
milestone 1 to milestone 2 of the current process. As indicated at the
bottom of Figure 2.5, the intention is to strongly inform early concept
development by technology and analysis.

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
 Background: The Capabilities-Development Process 15

Figure 2.5
The Envisioned Concept Decision Process

To be:
• Aligned
• Capability orientation
• Open aperture
Concept • Joint
decision
Future focus here • Broad range of options
modeling,
simulation,
and analysis MS B MS C
Full rate
production
DR
JCD EoA System
Strategic CDD CPD Production
Joint Technology development
planning FAA FNA EoA report and O&S
concepts development and
guidance deployment
demonstration

Incremental development
FCB
OSD/ COCOM
JCS

Stronger emphasis

Technology development—TRL/IRL

Science and technology base

SOURCE: Adapted from a briefing by James Durham, OSD (AT&L), August 2006.
RAND MG662-2.5

This content downloaded from

190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms
 This content downloaded from
190.107.66.66 on Thu, 19 Jan 2023 19:35:42 UTC
All use subject to https://about.jstor.org/terms