Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo
T H E G Q M A P P R O A C H T O
E N T E R P R I S E A G I L E M E T R I C S
Version 1.42
2
JOHN
TANNER
SVP &Enterprise Transformation
Consultant
John.Tanner@leadingagile.com
757.580.2078
www.linkedin.com/in/tannerjs
www.facebook.com/leadingagile
www.twitter.com/leadingagile
3
JOHN
TANNER
20+ years in Software Development
Roles in industry as Consultant,
Engineer, Architect, Product & Project
Manager
Implemented Agile in All the
environments:
Non-profit
Corporate
Government
D E F I N I N G T H E
P R O B L E M
5
We need maximum metrics, because we just don’t know what we don’t know
We need minimal metrics, because they just don’t add value
We need some metrics, because something is better than nothing
THREE VIEWS ON METRICS
6
“Metrics are how we measure success. So let’s capture metrics for everything.”
Metrics dashboards become prolific, and someone has to maintain them
Each Team winds up with a unique set of metrics to capture, but with no common
expectation for using them
The organization spends more time creating metrics roll-ups than they do creating
product
There is no cohesive strategy for tying the data together to make targeted improvements
WE NEED ALL OF THE METRICS!
7
“Metrics do not add value, so they are waste. Let’s eliminate them altogether.”
Assumes all teams are currently producing working, tested product at high levels of
efficiency
Only works when the organization doesn’t care if they ever improve the system…
… doesn’t care if they aren’t producing value
… doesn’t get hung up on details like making payroll
WE NEED NONE OF THE METRICS!
8
“Metrics might help us improve, so we need measures of some sort.”
Without clarity, we often capture metrics for metrics’ sake
When we gather metrics without clear expectations around their use they can be
wasteful, or even harmful
We focus on metrics in a way that ultimately drives “local optimization” at the expense
of the overall system
We forget that the “why” and the “how” matters
WE NEED SOME METRICS…
T H E “ W H Y ” A N D
T H E “ H O W ”
10
LET’S TALK ABOUT WHY, FIRST…
BACKLOGS TEAMS WORKING TESTED PRODUCT
11
In Agile, we feel pretty confident we know how to measure success because…
OUR APPROACH TO THE PROBLEM
... is Making and Meeting Commitments
... is Producing Quality Software
The Team
... has a Stable Velocity
12
OUR APPROACH TO THE PROBLEM
MAKES AND
MEETS
COMMITMENTS
HAS STABLE
VELOCITY
PRODUCES QUALITY
SOFTWARE
METRIC:
Story Point
Completion %
METRIC:
Velocity Variance
METRIC:
Escaped Defect Count
13
Yay! We did it team!
No, but seriously, many people think like that…
PROBLEM SOLVED?
14
HOW DOES THAT WORK AT SCALE?
PROGRAM
TEAMS
PORTFOLIO
TEAMS
DELIVERY
TEAMS
G O A L - Q U E S T I O N -
M E T R I C
16
The Goal-Question-Metric approach is a simple model where we approach measures
from three distinct levels:
Conceptual level (Goal)
Operational level (Question)
Quantitative level (Metric)
WHAT IS GQM?
17
Conceptual level (Goal)
A goal is defined for a team or system, to meet a specific purpose, with a focus on
demonstrating a meaningful outcome for the business.
WHAT IS GQM?
18
Operational level (Question)
A set of questions is crafted to define an objective model for assessing the achievement
of our goals.
WHAT IS GQM?
19
Quantitative level (Metric)
A set of metrics, based upon the objective model, is associated with every question in
order to answer it in a measurable way.
WHAT IS GQM?
20
Identify the right goals for a team or system
Clarify our goals by asking the right questions
Identify measures and metrics that can answer the questions
Find ways to collect the data necessary to realize our measures and metrics
Analyze the data to assess whether we are answering our questions and achieving our
goals
Use what we discover to improve our goals, questions, and metrics
HOW DO WE USE GQM?
21
EXAMPLE: MY GOAL FOR TODAY
Goal
My	audience	doesn't	fall	
asleep	in	the	next	half	
hour
Question
Did	they	even	show	up	in	the	first	place?
Have	they	been	thoroughly	caffeinated?
Are	they	paying	attention	so	far?
Metric
Empty	Seat	vs	Sold	Seat
(Actual	vs	Expected)
Snore	to	Silence	Ratio
(Snore	Complete	/	
Silence	Committed)
Eye	Contact	Variance
Heckling	Density
R E D E F I N I N G T H E
P R O B L E M
23
In Agile, we feel pretty confident we know how to measure success because…
REDEFINING THE PROBLEM
... is meeting Commitments
... is Producing Quality Software
The Team
... has a Stable Velocity
24
In	Enterprise	Agile,	we	feel	pretty	confident	we	know	how	to	measure	success	because…
Let’s figure it out, GQM style
REDEFINING THE PROBLEM
25
Delivery Team Health (Scrum)
Technical Quality
Program Health
Product Quality
Portfolio Health
Portfolio Financials
GQM AREAS TO DISCUSS
G Q M A T T H E
D E L I V E R Y T I E R
27
Teams	can	plan,	coordinate,	
and	deliver	predictably	
enough	to	meet	a	
release	level	commitment
Has	the	team	established	
a	stable	velocity?
Does	the	team	have	the	resources	
it	needs	to	be	successful?
Does	the	team	deliver	the	committed	
functionality	each	sprint?
Story	Point	Completion	%
(SPs	Complete	/	SPs	Committed)
Velocity	Variance
Team	Stability	Index
Is	anything	preventing	the	team	from	
delivering	on	commitments?
Blockers
Does	the	team	deliver	throughout	the	
sprint	or	at	the	“last	minute”?
Sprint	Completion	Ratio
(%SPs	Accepted	by	Sprint	End	- 2)
Goal MetricQuestion
User	Story	Completion	%
(USs	Complete	/	USs	Committed)
To	what	degree	is	the	team	encapsulated	
(i.e.	make/meet	commitments	on	its	own)?
%	User	Stories	w/	outside	
dependencies
DELIVERY TEAM HEALTH (SCRUM)
28
Teams	can	frequently	
deliver	working,	tested	
software	of	a	
high	level	of	quality
Is	the	team	able	to	maintain	the	
overall	technical	health	of	the	product?
Can	the	quality	of	the	product	being	shipped	
be	verified?
Open	Defects
Technical	Debt
Are	the	teams	able	to	address	technical	
problems	in	a	timely	manner?
Defect	Aging
Are	technical	problems	getting	
past	our	quality	checks?
Escaped	Defects
Are	technical	problems	being	
discovered	in	production?
Latent	Defects
Goal Question Metric
TECHNICAL QUALITY
G Q M A T T H E
P R O G R A M T I E R
30
Program	Management	can	
make	delivery	commitments	
and	successfully	manage	
incoming	requests
Is	the	Program	effectively	managing	
orchestration	/	mitigating	dependencies?
Is	the	program	delivering	committed	
functionality	each	release?
Is	the	roadmap	of	features	sufficient	to	
effectively	maintain	a	ready	backlog?
Feature	Roadmap	Visibility
%	Ready	Backlog	User	Stories	
w/	open	dependencies
Feature	Completion	Ratio
How	long	does	it	take	for	an	Opportunity	to	
move	from	Idea	to	Delivery?
Feature	Lead	Time
Goal MetricQuestion
Is	the	Ready	Backlog	sufficiently	groomed	to	
be	predictably	consumed	by	the	DT?
%	Ready	Backlog
Is	the	Program	(PO	Team)	focused	on	
delivering	value?
%	Value	in	Sprint
PROGRAM HEALTH
31
Goal Question Metric
The	organization	delivers	an	
intuitive	experience	and	
product	features	that	delight	
our	customers	and	produce	
the	highest	business	value	
possible
Did	the	delivered	solution	
meet	the	business	need?
Product/Market	Fit	
(Actual	vs	Expected)
Does	the	Overall	Product
Delight	the	Customer?	
Net	Promoter	Score
Did	the	Feature	Delight	the	Customer?	 Pirate	Metrics	
(Acquisition,	Activation,	
Retention,	Revenue,	Referral)
PRODUCT QUALITY
G Q M A T T H E
P O R T F O L I O T I E R
33
PORTFOLIO HEALTH
33
The	organization	delivers	
product	features	and	
capabilities	with	the	best	
ability	to	deliver	early	ROI
How	long	does	it	take	for	an	opportunity	to	
move	from	“good	idea”	to	”done”?
Is	the	system	delivering	consistently?
Epic	Lead	Time
Due	Date	Performance
Are	we	investing	in	the	right	type	of	work?	 Portfolio	Investment	Mix
Are	we	focusing	only	on	the	most	important	
things?
Value	Stream	WIP
Abandoned	WorkAre	we	starting	on	too	many	things?
Goal Question Metric
34
PORTFOLIO FINANCIALS
34
Goal Metric
The	Portfolio	is	meeting	its	
strategic,	financial	obligations	
and	objectives
Are	we	getting	the	right	return	
on	our	investments?	
Are	we	spending	too	much	just	to	
keep	the	lights	on?
Value	to	Cost	Ratio
CAPEX	to	OPEX	Ratio
Are	we	getting	the	expected	results	
from	this	Portfolio?
Variance	to	Results
($	or	%	Market	Share)
Question
Are	we	meeting	our	targeted	spend	rates? Variance	to	Spend	Target
Are	we	hitting	our	targeted	value
and	maintenance	blend?
Variance	to	Value/
Maintenance	Target
($	or	%	Spend)
R E D E F I N I N G T H E
P R O B L E M ( S O L V E D )
36
The Portfolio
... consistently delivers product features & capabilities with a focus on time to market and early
return on investment (Portfolio Health)
... consistently meets its strategic financial objects (Portfolio Financials)
The Program
... can make delivery commitments and successfully manage incoming requests (Program
Health)
... can ensure the organization delivers the most effective experience and product features to
delight our customers and produce the highest business value possible (Product Quality)
The Delivery Teams
... can plan, coordinate, and deliver predictably, in order to meet a release commitments (Team
Health)
... can frequently deliver working tested code of a high level of quality (Technical Quality)
MEASURE SUCCESS-ENTERPRISE LEVEL
S O N O W W H A T ?
38
Identify the right goals for a team or system
Clarify our goals by asking the right questions
Identify measures and metrics that can answer the questions
Find ways to collect the data necessary to realize our metrics and measures
Analyze the data to assess whether we are answering our questions and achieving our
goals
Use what we discover to improve our goals, questions, and metrics
NEXT STEPS
F I N A L T H O U G H T S

More Related Content

Enterprise Agile Metrics: A GQM Approach