ROI of Agile vs.

Traditional Methods
Costs Benefits B/CR ROI% NPV BEP $/Person Risk ROA
PSPsm $105,600 $4,469,997 42:1 4,133% $3,764,950 $945 $26,400 6.44% $4,387,756
Inspections $82,073 $2,767,464 34:1 3,272% $2,314,261 $51,677 $20,518 26.78% $2,703,545
TSPsm $148,400 $4,341,496 29:1 2,826% $3,610,882 $5,760 $37,100 37.33% $4,225,923
Agile Methods $217,712 $4,292,285 20:1 1,872% $3,498,958 $11,043 $54,428 59.87% $4,125,209
SW-CMM® $311,433 $3,023,064 10:1 871% $2,306,224 $153,182 $77,858 83.51% $2,828,802
ISO 9001 $173,000 $569,841 3:1 229% $320,423 $1,196,206 $43,250 98.66% $503,345
CMMI® $1,108,233 $3,023,064 3:1 173% $1,509,424 $545,099 $277,058 100.00% $2,633,052
More Graphs -->

ROI of Agile vs. Traditional Methods C&B

4,500% 1.0
4,133%
4,000%

3,500% 0.8
3,272%

3,000% 2,826%
0.6
2,500%

2,000% 1,872%
0.4
1,500%

1,000% 871%
0.2
500% 229% 173%
0% 0.0
PSPsm Inspections TSPsm Agile Methods SW-CMM® ISO 9001 CMMI® PSPsm
More Graphs -->
sm
PSP and TSP are service marks of Carnegie Mellon University
® SW-CMM and CMMI are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University

More Graphs -->

ROI of Agile Methods

Costs Benefits B/CR ROI% NPV BEP $/Person Risk ROA
XP $127,125 $4,382,872 34:1 3,348% $3,667,983 $3,687 $31,781 22.99% $4,283,867
Agile Methods $217,712 $4,292,285 20:1 1,872% $3,498,958 $11,043 $54,428 65.80% $4,127,723
TDD $249,653 $4,260,344 17:1 1,607% $3,439,359 $14,629 $62,413 73.49% $4,079,401
PP $265,437 $4,244,560 16:1 1,499% $3,409,908 $16,599 $66,359 76.61% $4,056,842
Scrum $505,259 $4,004,738 8:1 693% $2,962,424 $63,746 $126,315 100.00% $3,758,304
More Graphs -->

ROI of Agile Methods

3,700% 1.0
3,348%

2,960% 0.8

2,220% 0.6
1,872%
1,607%
1,499%
1,480% 0.4
 3,700% 1.0
3,348%

2,960% 0.8

2,220% 0.6
1,872%
1,607%
1,499%
1,480% 0.4

693%
740% 0.2

0% 0.0
XP Agile Methods TDD PP Scrum XP
More Graphs -->

ROI of Agile vs. Trad. Methods (detailed)

Costs Benefits B/CR ROI% NPV BEP $/Person Risk ROA
PSPsm $105,600 $4,469,997 42:1 4,133% $3,764,950 $945 $26,400 6.44% $4,387,756
XP $127,125 $4,382,872 34:1 3,348% $3,667,983 $3,687 $31,781 25.00% $4,283,867
Inspections $82,073 $2,767,464 34:1 3,272% $2,314,261 $51,677 $20,518 26.78% $2,703,545
TSPsm $148,400 $4,341,496 29:1 2,826% $3,610,882 $5,760 $37,100 37.33% $4,225,923
Agile Methods $217,712 $4,292,285 20:1 1,872% $3,498,958 $11,043 $54,428 59.87% $4,125,209
TDD $249,653 $4,260,344 17:1 1,607% $3,439,359 $14,629 $62,413 66.13% $4,073,167
PP $265,437 $4,244,560 16:1 1,499% $3,409,908 $16,599 $66,359 68.67% $4,048,404
SW-CMM® $311,433 $3,023,064 10:1 871% $2,306,224 $153,182 $77,858 83.51% $2,828,802
Scrum $505,259 $4,004,738 8:1 693% $2,962,424 $63,746 $126,315 87.72% $3,715,411
ISO 9001 $173,000 $569,841 3:1 229% $320,423 $1,196,206 $43,250 98.66% $503,345
CMMI® $1,108,233 $3,023,064 3:1 173% $1,509,424 $545,099 $277,058 100.00% $2,633,052
More Graphs -->

ROI of Agile vs. Traditional Methods (detailed) C&B

4,500% 1.0
4,133%
4,000%

3,500% 3,348% 3,272% 0.8

3,000% 2,826%
0.6
2,500%

2,000% 1,872%
1,607% 1,499% 0.4
1,500%

1,000% 871%
693% 0.2
500% 229% 173%
0% 0.0
s s
m P
on m od
D PP ® m 01 I® m
Ps X
ti Ps th TD M ru 90 M Ps
PS ec TS e M Sc M PS
sp M -C IS
O C
In le SW
gi
A
More Graphs -->
sm
PSP and TSP are service marks of Carnegie Mellon University
® SW-CMM and CMMI are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University
 C&B of Agile vs. Trad. Methods (normalized)
Costs Benefits B/CR ROI% NPV BEP $/Person Risk ROA Average
PSPsm 0.9788 1.0000 1.0000 1.0000 1.0000 1.0000 0.9788 1.0000 1.0000 0.9953
Inspections 1.0000 0.6191 0.7966 0.7917 0.6147 0.9576 1.0000 0.7966 0.6162 0.7992
TSPsm 0.9402 0.9713 0.6911 0.6837 0.9591 0.9960 0.9402 0.6911 0.9631 0.8706
Agile Methods 0.8776 0.9602 0.4658 0.4528 0.9294 0.9916 0.8776 0.4658 0.9402 0.7734
SW-CMM® 0.7930 0.6763 0.2293 0.2107 0.6126 0.8727 0.7930 0.2293 0.6447 0.5624
ISO 9001 0.9180 0.1275 0.0778 0.0555 0.0851 0.0008 0.9180 0.0778 0.1147 0.2639
CMMI® 0.0741 0.6763 0.0644 0.0418 0.4009 0.5451 0.0741 0.0644 0.6001 0.2824
More Graphs -->

C&B of Agile vs. Traditional Methods (aggregated)

1.0 1.0

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0.0 0.0
PSPsm Inspections TSPsm Agile Methods SW-CMM® ISO 9001 CMMI® PS
More Graphs -->
sm
PSP and TSP are service marks of Carnegie Mellon University
® SW-CMM and CMMI are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University

More Graphs -->

C&B of Agile Methods (normalized)

Costs Benefits B/CR ROI% NPV BEP $/Person Risk ROA Average
XP 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
Agile Methods 0.8207 0.9793 0.5718 0.5591 0.9539 0.8846 0.8207 0.5718 0.9636 0.7917
TDD 0.7575 0.9720 0.4950 0.4799 0.9377 0.8283 0.7575 0.4950 0.9523 0.7417
PP 0.7263 0.9684 0.4638 0.4478 0.9296 0.7974 0.7263 0.4638 0.9470 0.7189
Scrum 0.2516 0.9137 0.2299 0.2069 0.8076 0.0578 0.2516 0.2299 0.8773 0.4252
More Graphs -->

C&B of Agile Methods (aggregated)

1.0 1.0

0.8 0.8

0.6 0.6

0.4 0.4
 1.0 1.0

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0.0 0.0
XP Agile Methods TDD PP Scrum
More Graphs -->

C&B of Agile vs. Trad. Methods (normalized)

Costs Benefits B/CR ROI% NPV BEP $/Person Risk ROA Average
PSPsm 0.9788 1.0000 1.0000 1.0000 1.0000 1.0000 0.9788 1.0000 1.0000 0.9953
XP 0.9593 0.9805 0.8145 0.8100 0.9742 0.9977 0.9593 0.8145 0.9763 0.9207
Inspections 1.0000 0.6191 0.7966 0.7917 0.6147 0.9576 1.0000 0.7966 0.6162 0.7992
TSPsm 0.9402 0.9713 0.6911 0.6837 0.9591 0.9960 0.9402 0.6911 0.9631 0.8706
Agile Methods 0.8776 0.9602 0.4658 0.4528 0.9294 0.9916 0.8776 0.4658 0.9402 0.7734
TDD 0.8488 0.9531 0.4031 0.3887 0.9135 0.9886 0.8488 0.4031 0.9283 0.7418
PP 0.8345 0.9496 0.3778 0.3627 0.9057 0.9869 0.8345 0.3778 0.9227 0.7280
SW-CMM® 0.7930 0.6763 0.2293 0.2107 0.6126 0.8727 0.7930 0.2293 0.6447 0.5624
Scrum 0.6181 0.8959 0.1872 0.1676 0.7868 0.9475 0.6181 0.1872 0.8468 0.5839
ISO 9001 0.9180 0.1275 0.0778 0.0555 0.0851 0.0008 0.9180 0.0778 0.1147 0.2639
CMMI® 0.0741 0.6763 0.0644 0.0418 0.4009 0.5451 0.0741 0.0644 0.6001 0.2824
More Graphs -->

C&B of Agile vs. Traditional Methods (aggregated)

1.0 1.0

0.8 0.8

0.6
0.6

0.4
0.4

0.2
0.2
0.0
m P ns sm d s D PP ® um 01 I® 0.0
Ps X
tio P ho TD M r 90 M
PS ec TS et M Sc M PSPsm
sp M -C IS
O C
In le SW
gi
A
More Graphs -->
sm
PSP and TSP are service marks of Carnegie Mellon University
® SW-CMM and CMMI are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University
 C&B of Agile vs. Traditional Methods (normalized)
1.0

0.8

0.6

0.4

0.2

0.0
PSPsm Inspections TSPsm Agile Methods SW-CMM® ISO 9001

sm
PSP and TSP are service marks of Carnegie Mellon University
® SW-CMM and CMMI are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University

C&B of Agile Methods (normalized)

1.0

0.8

0.6

0.4
 1.0

0.8

0.6

0.4

0.2

0.0
XP Agile Methods TDD PP

1.0
C&B of Agile vs. Traditional Methods (normalized)

0.8

0.6

0.4

0.2

0.0
PSPsm XP Inspections TSPsm Agile TDD PP SW-CMM® Scrum ISO 9
Methods

sm
PSP and TSP are service marks of Carnegie Mellon University
® SW-CMM and CMMI are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University
normalized)

Costs
Benefits
B/CR
ROI%
NPV
BEP
$/Person
Risk
ROA

® ISO 9001 CMMI®

zed)

Costs
Benefits
B/CR
ROI%
NPV
BEP
$/Person
Risk
 Costs
Benefits
B/CR
ROI%
NPV
BEP
$/Person
Risk
ROA

PP Scrum

normalized)

Costs
Benefits
B/CR
ROI%
NPV
BEP
$/Person
Risk
ROA

MM® Scrum ISO 9001 CMMI®

 Agile Methods — Costs & Benefits
Author(s) Year Tech Cost Sched Prod Quality Satis Method N
Abrahamsson 2003 XP 88% Case 4
Abrahamsson 2007 General 70% 700% 250% Case 1,800
Al-Kilidar et al. 2005 PP 13% Exp 121
Arisholm et al. 2007 PP 11% 23% Exp 295
Back, Hirkman, & Milovanov 2004 XP 87% Exp 8
Bhat & Nagappan 2006 TDD 71% Case 12
Bipp, Lepper, & Schmedding 2008 PP 62% Exp 95
Canfora et al. 2006 PP 14% 20% Exp 70
Canfora et al. 2007 PP 39% 39% Exp 18
Cohn 2008 Scrum 405% 71% Case 7
Dalcher, Benediktsson, & Thorbergsson 2005 XP 21% 384% Exp 55
Damm & Lundberg 2006 TDD 56% Case 100
Drobka, Noftz, & Raghu 2004 XP 289% 63% Case 29
Erdogmus, Morisio, & Torchiano 2005 TDD 28% Exp 24
Fitzgerald, Hartnett, & Conboy 2006 Scrum 700% Case 45
Flohr & Schneider 2006 TDD 27% Exp 18
George 2002 TDD 16% Exp 138
George & Williams 2003 TDD 18% Exp 24
George & Williams 2004 TDD 18% Exp 24
Heiberg et al. 2003 PP 16% Exp 100
Huang & Holcombe 2008 TDD 172% Exp 274
Hulkko & Abrahamsson 2005 PP 18% 46% Case 18
Ilieva, Ivanov, & Stefanova 2004 XP 12% 41% 13% Exp 8
Janzen & Saiedian 2008 TDD 34% Exp 64
Jensen 2003 PP 127% 1,000% Case 10
Jones 2008 Scrum 74% Case 5
Kaufmann & Janzen 2003 TDD 50% 50% Exp 8
Kuppuswami et al. 2003 XP 28% Sim n/a
Layman 2004 XP 61% 48% Case 21
Lui & Chan 2004 PP 24% Exp 3
Lui & Chan 2006 PP 23% Exp 40
Lui, Chan, & Nosek 2008 PP 70% Exp 15
Madeyski 2006 PP 14% Exp 188
Madeyski & Szala 2007 TDD 18% 45% Case 1
Mann 2004 TDD 81% Case 7
Maurer & Martel 2002 XP 66% Case 9
Maximilien & Williams 2003 TDD 50% Case 9
McDowell et al. 2003 PP 27% Exp 555
McDowell et al. 2006 PP 27% Case 486
Melis et al. 2006 TDD 36% Case 4
Mendes, Al-Fakhri, & Luxton-Reilly 2005 PP 10% Exp 300
Molokken-Ostvold & Jorgensen 2005 General 12% Survey 42
Muller 2005 PP 29% Exp 38
Muller 2006 PP 29% 11% Exp 18
Muller 2007 PP 50% Exp 21
Muller & Padberg 2003 XP 20% Sim n/a
Nawrocki & Wojciechowski 2001 PP 25% 15% Exp 21
Nosek 1998 PP 29% 36% Exp 15
 Pandey et al. 2003 PP 40% 20% 40% Exp 10
Phongpaibul & Boehm 2006 PP 24% 34% Exp 104
Reifer 2003 XP 10% 53% 20% Survey 18
Rico 2007 General 51% 65% 56% 63% 70% Survey 122
Saff & Ernst 2004 TDD 16% Exp 39
Sanchez, Williams, & Maximilien 2007 TDD 40% Case 17
Schatz & Abdelshafi 2005 Scrum 29% 30% Case 90
Schatz & Abdelshafi 2005 TDD 75% Case 90
Sutherland 2007 Scrum 712% Case 5
Talby et al. 2006 TDD 90% Case 60
Van Schooenderwoert 2006 XP 192% 89% Case 4
Vanhanen & Lassenius 2005 PP 42% Exp 20
Version One 2006 General 10% 18% 17% 17% Survey 722
Version One 2007 General 11% 16% 17% 17% Survey 1,681
Williams 2001 PP 47% 15% Exp 41
Williams et al. 2003 PP 16% Exp 575
Williams, Maximilien, & Vouk 2003 TDD 40% Case 14
Wilson, Hoskin, & Nosek 1993 PP 38% Exp 34
Wolf & Roock 2008 General 72% 78% 74% Survey 200
Xu & Rajlich 2006 PP 48% 201% 21% Exp 12
Ynchausti 2001 TDD 153% Case 5
 Agile Methods
Category Low Median High Points
Cost 10% 26% 70% 9
Schedule 11% 71% 700% 19
Productivity 14% 122% 712% 27
Quality 10% 75% 1,000% 53
Satisfaction 70% 70% 70% 1
109

PP
Category Low Median High Points
Cost n/a n/a n/a n/a
Schedule 11% 34% 70% 12
Productivity 14% 76% 201% 5
Quality 10% 69% 1,000% 24
Satisfaction n/a n/a n/a n/a
41

Scrum
Category Low Median High Points
Cost n/a n/a n/a n/a
Schedule n/a n/a n/a n/a
Productivity 29% 305% 712% 4
Quality 30% 267% 700% 3
Satisfaction n/a n/a n/a n/a
7

TDD
Category Low Median High Points
Cost n/a n/a n/a n/a
Schedule n/a n/a n/a n/a
Productivity 18% 64% 172% 6
Quality 16% 50% 153% 16
Satisfaction n/a n/a n/a n/a
22

XP
Category Low Median High Points
Cost 10% 18% 28% 5
Schedule 53% 53% 53% 1
Productivity 20% 143% 384% 8
Quality 13% 60% 89% 5
Satisfaction n/a n/a n/a n/a
19

General
Category Low Median High Points
Cost 10% 36% 70% 4
Schedule 12% 147% 700% 6
Productivity 17% 42% 78% 4
Quality 17% 84% 250% 5
Satisfaction 70% 70% 70% 1
20
 Agile Methods — Metrics Agile Meth
Author(s) Year Tech LOC/Hr Def/KLOC Method N Category
Abrahamsson 2003 XP 19.2550 2.1450 Case 4 LOC/Hour
Abrahamsson & Koskela 2004 XP 16.9000 1.4300 Case 4 Def/KLOC
Back, Hirkman, & Milovanov 2004 XP 8.0000 0.7000 Exp 8
Baheti, Gehringer, & Stotts 2002 PP 16.6370 Exp 132
Bowers et al. 2002 XP 18.1731 0.0325 Case ???
Cohn 2008 Scrum 5.9050 2.9000 Case 7 PP
Dalcher, Benediktsson, & Thorbergsson 2005 XP 14.8667 Exp 55 Category
Erdogmus & Williams 2003 PP 43.4780 5.8500 Case 41 LOC/Hour
Grewal & Maurer 2007 Scrum 6.1711 Case 36 Def/KLOC
Hashmi & Baik 2008 XP 16.8420 Case 19
Huang & Holcombe 2008 TDD 12.3800 Exp 274
Hulkko & Abrahamsson 2005 PP 15.6667 4.1500 Case 18
Ilieva, Ivanov, & Stefanova 2004 XP 20.2030 0.0032 Exp 8 Scrum
Jones 2008 Scrum 5.7400 8.5000 Case 5 Category
Layman 2004 XP 9.1154 0.6250 Case 21 LOC/Hour
Layman et al. 2006 XP 13.3846 1.6200 Case 8 Def/KLOC
Madeyski & Szala 2007 TDD 46.1800 Case 1
Manzo 2002 XP 43.0000 0.5000 Case 17
Maurer & Martel 2002 XP 17.0000 Case 9
Maximilien & Williams 2003 TDD 3.7000 Case 9 TDD
Nawrocki & Wojciechowski 2001 PP 49.2500 Exp 21 Category
Pandey et al. 2003 PP 22.4462 2.3900 Exp 10 LOC/Hour
Schatz & Abdelshafi 2005 Scrum 0.4350 Case 90 Def/KLOC
Sutherland 2007 Scrum 4.6858 Case 5
Sutherland et al. 2008 Scrum 4.6154 1.0000 Case 25
Van Schooenderwoert 2006 XP 3.5000 0.1700 Case 4
Vanhanen & Korpi 2007 PP 15.4667 0.5500 Case 4 XP
Vanhanen & Lassenius 2005 PP 17.8403 0.3250 Exp 20 Category
Williams, Layman, & Krebs 2004 XP 9.8077 0.2400 Case 19 LOC/Hour
Williams, Maximilien, & Vouk 2003 TDD 0.6100 Case 14 Def/KLOC
Xu 2005 XP 27.6785 0.0400 Case 10
Xu & Rajlich 2006 PP 86.4502 0.8651 Exp 12
Agile Methods
Low Median High Points
3.5000 20.3668 ### 29
0.0032 1.6861 8.5000 23
52

PP
Low Median High Points
### 33.4044 ### 8
0.3250 2.3550 5.8500 6
14

Scrum
Low Median High Points
4.6154 5.4235 6.1711 5
0.4350 3.2088 8.5000 4
9

TDD
Low Median High Points
### 29.2800 ### 2
0.6100 2.1550 3.7000 2
4

XP
Low Median High Points
3.5000 16.9804 ### 14
0.0032 0.6823 2.1450 11
25
 Agile Methods — ROI Analysis
Author(s) Year Tech Costs Benefits B/CR ROI NPV BEP
Abrahamsson 2003 XP $266,435 ### 16:1 1,493% ### $16,728
Abrahamsson & Koskela 2004 XP $202,172 ### 21:1 2,031% ### $9,488
Back, Hirkman, & Milovanov 2004 XP $195,000 ### 22:1 2,113% ### $8,812
Baheti, Gehringer, & Stotts 2002 PP $295,608 ### 14:1 1,326% ### $20,735
Bowers et al. 2002 XP $58,276 ### 76:1 7,539% ### $763
Cohn 2008 Scrum $459,348 ### 9:1 782% ### $52,091
Dalcher, Benediktsson, & Thorbergsson 2005 XP $135,498 ### 32:1 3,128% ### $4,197
Erdogmus & Williams 2003 PP $608,000 ### 6:1 542% ### $94,737
Grewal & Maurer 2007 Scrum $482,921 ### 8:1 734% ### $57,911
Hashmi & Baik 2008 XP $127,609 ### 34:1 3,334% ### $3,716
Huang & Holcombe 2008 TDD $296,275 ### 14:1 1,322% ### $20,832
Hulkko & Abrahamsson 2005 PP $478,830 ### 8:1 742% ### $56,876
Ilieva, Ivanov, & Stefanova 2004 XP $49,821 ### 90:1 8,852% ### $556
Jones 2008 Scrum ### ### 3:1 240% ### ###
Layman 2004 XP $172,205 ### 25:1 2,419% ### $6,836
Layman et al. 2006 XP $236,713 ### 18:1 1,705% ### $13,112
Madeyski & Szala 2007 TDD $237,154 ### 18:1 1,702% ### $13,163
Manzo 2002 XP $73,256 ### 61:1 5,957% ### $1,210
Maurer & Martel 2002 XP $127,057 ### 34:1 3,350% ### $3,683
Maximilien & Williams 2003 TDD $404,153 ### 10:1 916% ### $39,782
Nawrocki & Wojciechowski 2001 PP $255,805 ### 17:1 1,563% ### $15,382
Pandey et al. 2003 PP $283,551 ### 15:1 1,391% ### $19,023
Schatz & Abdelshafi 2005 Scrum $227,884 ### 19:1 1,779% ### $12,127
Sutherland 2007 Scrum $534,288 ### 7:1 644% ### $71,802
Sutherland et al. 2008 Scrum $316,666 ### 13:1 1,224% ### $23,914
Van Schooenderwoert 2006 XP $302,714 ### 14:1 1,290% ### $21,780
Vanhanen & Korpi 2007 PP $119,655 ### 37:1 3,569% ### $3,261
Vanhanen & Lassenius 2005 PP $88,553 ### 50:1 4,893% ### $1,774
Williams, Layman, & Krebs 2004 XP $125,961 ### 35:1 3,380% ### $3,619
Williams, Maximilien, & Vouk 2003 TDD $95,153 ### 46:1 4,540% ### $2,051
Xu 2005 XP $40,129 ### 111:1 11,039% ### $360
Xu & Rajlich 2006 PP $98,073 ### 45:1 4,399% ### $2,180
lysis Agile Methods
Cost/Per Risk ROA Method N Category Low Median High Points
$66,609 89% ### Case 4 Costs $40,129 $263,093 ### 32
$50,543 84% ### Case 4 Benefits ### ### ### 32
$48,750 83% ### Exp 8 B/CR 3:1 29:1 111:1 32
$73,902 90% ### Exp 132 ROI 240% 2,811% 11,039% 32
$14,569 34% ### Case ??? NPV ### ### ### 32
$114,837 95% ### Case 7 BEP $360 $28,233 $300,942 32
$33,875 74% ### Exp 55 Cost/Person $10,032 $65,773 $256,054 32
$152,000 97% ### Case 41 Risk 3% 77% 100% 32
$120,730 96% ### Case 36 ROA ### ### ### 32
$31,902 72% ### Case 19
$74,069 90% ### Exp 274
$119,707 95% ### Case 18
$12,455 23% ### Exp 8 PP
$256,054 100% ### Case 5 Category Low Median High Points
$43,051 80% ### Case 21 Costs $88,553 $278,509 $608,000 8
$59,178 87% ### Case 8 Benefits ### ### ### 8
$59,289 87% ### Case 1 B/CR 6:1 24:1 50:1 8
$18,314 49% ### Case 17 ROI 542% 2,303% 4,893% 8
$31,764 72% ### Case 9 NPV ### ### ### 8
$101,038 94% ### Case 9 BEP $1,774 $26,746 $94,737 8
$63,951 88% ### Exp 21 Cost/Person $22,138 $69,627 $152,000 8
$70,888 90% ### Exp 10 Risk 58% 81% 97% 8
$56,971 86% ### Case 90 ROA ### ### ### 8
$133,572 96% ### Case 5
$79,166 91% ### Case 25
$75,679 91% ### Case 4
$29,914 70% ### Case 4 Scrum
$22,138 58% ### Exp 20 Category Low Median High Points
$31,490 72% ### Case 19 Costs $227,884 $507,554 ### 6
$23,788 61% ### Case 14 Benefits ### ### ### 6
$10,032 3% ### Case 10 B/CR 3:1 10:1 19:1 6
$24,518 63% ### Exp 12 ROI 240% 901% 1,779% 6
NPV ### ### ### 6
BEP $12,127 $86,464 $300,942 6
Cost/Person $56,971 $126,888 $256,054 6
Risk 86% 94% 100% 6
ROA ### ### ### 6

TDD
Category Low Median High Points
Costs $95,153 $258,184 $404,153 4
Benefits ### ### ### 4
B/CR 10:1 22:1 46:1 4
ROI 916% 2,120% 4,540% 4
NPV ### ### ### 4
BEP $2,051 $18,957 $39,782 4
Cost/Person $23,788 $64,546 $101,038 4
Risk 61% 83% 94% 4
ROA ### ### ### 4

XP
Category Low Median High Points
Costs $40,129 $150,918 $302,714 14
Benefits ### ### ### 14
B/CR 14:1 42:1 111:1 14
ROI 1,290% 4,116% 11,039% 14
NPV ### ### ### 14
BEP $360 $6,776 $21,780 14
Cost/Person $10,032 $37,729 $75,679 14
Risk 3% 65% 91% 14
ROA ### ### ### 14
 Agile Methods — References
XP, Metrics,
ROI

General

XP, Metrics,
ROI

PP

PP

XP, Metrics,
ROI
PP, Metrics,
ROI

TDD

PP

Metrics, ROI

PP

PP

Scrum,
Metrics, ROI
XP, Metrics,
ROI

TDD

XP

PP, Metrics,
ROI

TDD

Scrum

TDD

TDD

TDD

TDD

Scrum,
Metrics, ROI
XP, Metrics,
ROI
 PP

TDD, Metrics,
ROI
PP, Metrics,
ROI
XP, Metrics,
ROI

TDD

PP

Scrum,
Metrics, ROI

TDD

XP

XP, Metrics,
ROI
XP, Metrics,
ROI

PP

PP

PP

PP

TDD, Metrics,
ROI

TDD

Metrics, ROI

XP, Metrics,
ROI
TDD, Metrics,
ROI

PP

PP

TDD

PP

General

PP
 PP

PP

XP

PP, Metrics,
ROI

PP

PP, Metrics,
ROI

PP

XP

General

TDD

TDD

Scrum, TDD,
Metrics, ROI
Scrum,
Metrics, ROI
Scrum,
Metrics, ROI

TDD

XP, Metrics,
ROI
PP, Metrics,
ROI
PP, Metrics,
ROI

General

General

PP

XP, Metrics,
ROI
TDD, Metrics,
ROI

PP

PP

General
XP, Metrics,
ROI
PP, Metrics,
ROI

TDD
 Agile Methods — References
Abrahamsson, P. (2003). Extreme programming: First results from a controlled case study. Proceedings of the 29th Euromicro Conference (EUROMICRO 2003), Belek-Anta

Abrahamsson, P. (2007). Speeding up embedded software development: Application of agile processes in complex systems development projects. Eindhoven, Netherlands: Inf

Abrahamsson, P., & Koskela, J. (2004). Extreme programming: A survey of empirical data from a controlled case study. Proceedings of the 2004 ACM/IEEE International Sy
Redondo Beach, California, USA, 73-82.

Al-Kilidar, H., Parkin, P., Aurum, A., & Jeffery, R. (2005). Evaluation of effects of pair work on quality of designs. Proceedings of the 2005 Australian Software Engineering

Arisholm, E., Gallis, H., Dyba, T., & Sjoberg, D. I. (2007). Evaluating pair programming with respect to system complexity and programmer expertise. IEEE Transactions on

Back, R. J., Hirkman, P., & Milovanov, L. (2004). Evaluating the XP customer model and design by contract. Proceedings of the 30th Annual Euromicro Conference (Eurom

Baheti, P., Gehringer, E., & Stotts, D. (2002). Exploring the efficacy of distributed pair programming. Proceedings of the Second XP Universe and First Agile Universe Confe
Universe 2002), Chicago, Illinois, USA, 208–220.

Bhat, T., & Nagappan, N. (2006). Evaluating the efficacy of test-driven development: Industrial case studies. Proceedings of the 2006 ACM/IEEE International Symposium o
Brazil, 356-363.

Bipp, T., Lepper, A., & Schmedding, D. (2008). Pair programming in software development teams: An empirical study of its benefits. Information and Software Technology,

Bowers, J., May, J., Melander, E., Baarman, M., & Ayoob, A. (2002). Tailoring XP for large system mission critical software development. Proceedings of the Second XP Un
Programming and Agile Methods (XP/Agile Universe 2002), Chicago, Illinois, USA, 100-111.

Canfora, G., Cimitile, A., Garcia, F., Piattini, M., & Visaggio, C. A. (2007). Evaluating performances of pair designing in industry. Journal of Systems and Software, 80(8), 1

Canfora, G., Cimitile, A., Visaggio, C. A., Garcia, F., & Piattini, M. (2006). Performances of pair designing on software evolution: A controlled experiment. Proceedings of t
Reengineering (CSMR 2006), Bari, Italy, 197-205.

Cohn, M. (2008). Agile and the seven deadly sins of project management. Proceedings of the Better Software Conference and EXPO (BSCE 2008), Las Vegas, Nevada, USA.

Dalcher, D., Benediktsson, O., & Thorbergsson, H. (2005). Development life cycle management: A multiproject experiment. Proceedings of the 12th IEEE International Con
Systems (ECBS 2005), Greenbelt, Maryland, USA, 289-296.

Damm, L. O., & Lundberg, L. (2006). Results from introducing component-level test automation and test-driven development. Journal of Systems and Software, 79(7), 1001-

Drobka, J., Noftz, D., & Raghu, R. (2004). Piloting XP on four mission-critical projects. IEEE Software, 21(6), 70-75.

Erdogmus, H., & Williams, L. (2003). The economics of software development by pair programmers. The Engineering Economist, 48(4), 283-319.

Erdogmus, H., Morisio, M., & Torchiano, M. (2005). On the effectiveness of the test-first approach to programming. IEEE Transactions on Software Engineering, 31(3), 226

Fitzgerald, B., Hartnett, G. & Conboy, K. (2006). Customising agile methods to software practices at intel shannon. European Journal of Information Systems, 15(2), 200-213

Flohr, T., & Schneider, T. (2006). Lessons learned from an XP experiment with students: Test-first needs more teachings. Proceedings of the Seventh International Conferenc
Improvement (PROFES 2006), Amsterdam, Netherlands, 305–318.

George, B. (2002). Analysis and quantification of test driven development approach. Unpublished master’s thesis, North Carolina State University, Raleigh, NC, United State

George, B., & Williams, L. (2003). An initial investigation of test driven development in industry. Proceedings of the 2003 ACM Symposium on Applied Computing (SAC 200

George, B., & Williams, L. (2004). A structured experiment of test-driven development. Information and Software Technology, 46(5), 337-342

Grewal, H., & Maurer, F. (2007). Scaling agile methodologies for developing a production accounting system for the oil and gas industry. Proceedings of the Agile 2007 Con

Hashmi, S. I., & Baik, J. (2008). Quantitative process improvement in XP using six sigma tools. Proceedings of the Seventh IEEE/ACIS International Conference on Compute
519-524.
Heiberg, S., Puus, U., Salumaa, P., & Seeba, A. (2003). Pair-programming effect on developers productivity. Proceedings of the Fourth International Conferences on Extrem

Huang, L., & Holcombe, M. (2008). Empirical investigation towards the effectiveness of test first programming. Information and Software Technology, 50(3).

Hulkko, H., & Abrahamsson, P. (2005). A multiple case study on the impact of pair programming on product quality. Proceedings of the 27th International Conference on So
504.

Ilieva, S., Ivanov, P., & Stefanova, E. (2004). Analysis of an agile methodology implementation. Proceedings of the 30th Annual Euromicro Conference (Euromicro 2004), R

Janzen, D. S., & Saiedian, H. (2008). Does test-driven development really improve software design quality? IEEE Software, 25(2), 77-84.

Jensen, R. (2003). A pair programming experience. CrossTalk, 16(3), 22-24.

Jones, C. (2008). Development practices for small software applications. Crosstalk, 21(2), 9-13.

Kaufmann, R., & Janzen, D. (2003). Implications of test-driven development: A pilot study. Proceedings of the 18th Annual ACM SIGPLAN Conference on Object-Oriented
2003), Anaheim, California, USA, 298-299.

Kuppuswami, S., Vivekanandan, K., Ramaswamy, P., & Rodrigues, P. (2003). The effects of individual XP practices on software development effort. ACM SIGSOFT Softwa

Layman, L. (2004). Empirical investigation of the impact of extreme programming practices on software projects. Proceedings of the 19th Annual ACM SIGPLAN Conferenc
Applications (OOPSLA 2004), Vancouver, BC, Canada, 328-329.

Layman, L., Williams, L., Damian, D., & Bures, H. (2006). Essential communication practices for extreme programming in a global software development team. Information

Lui, K. M., & Chan, K. C. (2004). A cognitive model for solo programming and pair programming. Proceedings of the Third IEEE International Conference on Cognitive Inf

Lui, K. M., & Chan, K. C. (2006). Pair programming productivity: Novice–novice vs. expert–expert. International Journal of Human-Computer Studies, 64(9), 915–925.

Lui, K. M., Chan, K. C., & Nosek, J. T. (2008). The effect of pairs in program design tasks. IEEE Transactions on Software Engineering, 34(2), 197-211.

Madeyski, L. (2006). The impact of pair programming and test-driven development on package dependencies in object-oriented design. Proceedings of the Seventh Internatio
Process Improvement (PROFES 2006), Amsterdam, Netherlands, 278-289.

Madeyski, L., & Szala, L. (2007). The impact of test-driven development on software development productivity: An empirical study. Proceedings of the 2007 European Syste
(EUROSPI 2007), Potsdam, Germany, 200-211.

Mann, C. (2004). An exploratory longitudinal case study of agile methods in a small software company. Unpublished master’s thesis, University of Calgary, Calgary, Alberta,

Manzo, J. (2002). Odyssey and other code science success stories. Crosstalk, 15(10), 19-30.

Maurer, F., & Martel, S. (2002). Extreme programming: Rapid development for web-based applications. IEEE Internet Computing, 6(1), 86-90.

Maximilien, E. M., & Williams, L. (2003). Assessing test-driven development at IBM. Proceedings of the 25th International Conference on Software Engineering (ICSE 200

McDowell, C., Werner, L., Bullock, H. E., & Fernald, J. (2003). The impact of pair programming on student performance, perception, and persistence. Proceedings of the 25t
Portland, Oregon, USA, 602-607.

McDowell, C., Werner, L., Bullock, H. E., & Fernald, J. (2006). Pair programming improves student retention, confidence, and program quality. Communications of the ACM

Melis, M., Turnu, I., Cau, A., & Concas, G. (2006). Evaluating the impact of test-first programming and pair programming through software process simulation. Journal of So

Mendes, E., Al-Fakhri, L. B., & Luxton-Reilly, A. (2005). Investigating pair programming in a 2nd-year software development and design computer science course. Proceed
Technology in Computer Science Education (ITICSE 2005), Monte da Caparica, Portugal, 296–300.

Molokken-Ostvald, K., & Jorgensen, M. (2005). A comparison of software project overruns: Flexible versus sequential development models. IEEE Transactions on Software

Muller, M. M. (2005). Two controlled experiments concerning the comparison of pair programming to peer review. Journal of Systems and Software, 78(2), 166-179.
Muller, M. M. (2006). A preliminary study on the impact of a pair design phase on pair programming and solo programming. Information and Software Technology, 48(5), 33

Muller, M. M. (2007). Do programmer pairs make different mistakes than solo programmers? Journal of Systems and Software, 80(9), 1460-1471.

Muller, M. M., & Padberg, F. (2003). On the economic evaluation of XP projects. Proceedings of the Joint Ninth European Software Engineering Conference (ESEC-2003) a
of Software Engineering (FSE 2003), Helsinki, Finland, 168-177.

Nawrocki, J., & Wojciechowski, A. (2001). Experimental evaluation of pair programming. Proceedings of the 12th European Software Control and Metrics Conference (ESC

Nosek, J. T. (1998). The case for collaborative programming. Communications of the ACM, 41(3), 105–108.

Pandey, A., Kameli, N., Eapen, A., Miklos, C., Boudigou, F., Sutedjo, I., et al. (2003). Application of tightly coupled engineering team for development of test automation sof
International Computer Software and Applications Conference (COMPSAC 2003), Dallas, Texas, USA, 56-63.

Phongpaibul, M., & Boehm, B. (2006). An empirical comparison between pair development and software inspection in thailand. Proceedings of the 2006 ACM/IEEE Internat
2006), Rio de Janeiro, Brazil, 85–94.

Reifer, D. J. (2003). The business case for agile methods/extreme programming (XP). Proceedings of the Seventh Annual Practical Software and Systems Measurement Users

Rico, D. F. (2007). Effects of agile methods on website quality for electronic commerce. Unpublished doctoral dissertation, University of Maryland University College, Adelp

Saff, D., & Ernst, M. D. (2004). An experimental evaluation of continuous testing during development. ACM SIGSOFT Software Engineering Notes, 29(4), 76-85.

Sanchez, J. C., Williams, L., & Maximilien, E. M. (2006). A longitudinal study of the use of a test-driven development practice in industry. Proceedings of the Agile 2007 Co

Schatz, B., & Abdelshafi, I. (2005). Primavera gets agile: A successful transition to agile development. IEEE Software, 22(3), 36-42.

Sutherland, J., Viktorov, A., Blount, J., & Puntikov, N. (2007). Distributed scrum: Agile project management with outsourced development teams. Proceedings of the 40th An
2007), Waikaloa, Big Island, Hawaii, 274-284.

Sutherland, J., Schoonheim, G., Rustenburg, E., & Rijk, M., (2008). Fully distributed scrum: The secret sauce for hyperproductive offshored development teams. Proceedings

Talby, D., Hazzan, O., Dubinsky, Y., & Keren, A. (2006). Agile software testing in a large-scale project. IEEE Software, 23(4), 30-37.

Van Schooenderwoert, N. (2006). Embedded agile project by the numbers with newbies. Proceedings of the Agile 2006 Conference (Agile 2006), Minneapolis, Minnesota, U

Vanhanen, J., & Korpi, H. (2007). Experiences of using pair programming in an agile project. Proceedings of the 40th Annual Hawaii International Conference on System Sc

Vanhanen, J., & Lassenius, C. (2005). Effects of pair programming at the development team level: An experiment. Proceedings of the Fourth International Symposium on Em
Australia, 336-345.

Version One. (2006). The state of agile development. Alpharetta, GA: Author.

Version One. (2007). The state of agile development: Second Annual Survey. Alpharetta, GA: Author.

Williams, L. (2001). Integrating pair programming into a software development process. Proceedings of the 14th Conference on Software Engineering Education and Trainin

Williams, L., Layman, L., & Krebs, W. (2004). Extreme programming evaluation framework for object-oriented languages: Version 1.4 (TR-2004-18). Raleigh, NC: North C

Williams, L., Maximilien, E. M., & Vouk, M. (2003). Test-driven development as a defect-reduction practice. Proceedings of the 14th International Symposium on Software R
45.

Williams, L., McDowell, C., Nagappan, N., Fernald, J., & Werner, L. (2003). Building pair programming knowledge through a family of experiments. Proceedings of the 200
Engineering (ISESE 2003), Rome, Italy, 143-152.

Wilson, J., Hoskin, N., & Nosek, J. T. (1993). The benefits of collaboration for student programmers. Proceedings 24th SIGCSE Technical Symposium on Computer Science

Wolf, H., & Roock, A. (2008). Agile becomes mainstream: Results of Online Survey 2008. Objectspektrum, 15(3), 10-13.
Xu, B. (2005). Extreme programming for distributed legacy system reengineering. Proceedings of the 29th Annual International Computer Software and Applications Confer

Xu, S., & Rajlich, V. (2006). Empirical validation of test-driven pair programming in game development. Proceedings of the Fifth IEEE/ACIS International Conference on C
International Workshop on Component-Based Software Engineering,Software Architecture and Reuse (ICIS-COMSAR 2006), Honolulu, Hawaii, USA, 500-505.

Ynchausti, R. A. (2001). Integrating unit testing into a software development team's process. Proceedings of the Second International Conference on Extreme Programming a
Cagliari, Sardinia, Italy, 79-83.