Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)

•
•
Matthew Philip
Matthew Philip

This document summarizes Matt Philip's presentation on NoEstimates forecasting with less effort and more accuracy. Some key points from the presentation include: reducing sources of variation can improve forecasting more than improving estimating skills; data and probabilistic forecasts are preferable to intuition and deterministic estimates; and focusing on delivering value quickly can reduce the need for detailed estimates. The presentation provided examples of techniques to reduce variation, biases to consider, and recommendations for incorporating probabilistic forecasting into projects.

1 of 53
# a g i l e c a m p 2 0 1 8 | @ m a t t p h i l i p NO ESTIMATES Forecasting with Less Effort and More Accuracy
@mattphilip #agilecamp2018 ESTIMATES, PLEASE! ▫︎ Number of triples that Babe Ruth hit in his career ▫︎ Distance in km of Chile north to south ▫︎ Year in which Newark was founded ▫︎ Windows in the Gateway Arch in St. Louis Aim for 75% confidence
@mattphilip #agilecamp2018 ESTIMATES, PLEASE! ▫︎ Number of triples that Babe Ruth hit in his career (136) ▫︎ Distance in km of Chile north to south (4,270 km) ▫︎ Year in which Newark was founded (1666) ▫︎ Windows in the Gateway Arch in St. Louis (32; 16 per side)
@mattphilip #agilecamp2018 MANIFESTO RELEVANCE Planning Poker! Estimates Value? VUCA
@mattphilip #agilecamp2018 WHAT NOESTIMATES IS NOT SAYING ▫︎ You are evil if you estimate ▫︎ All estimates are totally useless ▫︎ Stop doing your successful estimating practice ▫︎ Stop having the conversations to understand/analyze/ break down work ▫︎ Work items must be the same size ▫︎ You must place your full faith and confidence in Monte Carlo forecasts
@mattphilip #agilecamp2018 WHAT NOESTIMATES IS SAYING ▫︎ Know why you are estimating ▫︎ Discover for yourself how good you are at estimating (measure) ▫︎ Keep doing the things that help you understand the work ▫︎ Upfront estimates need to be held loosely ▫︎ If you focus on delivering value quickly, you obviate the need for Iron Triangle considerations
@mattphilip #agilecamp2018 MY NOESTIMATES MANIFESTO … We have come to value: Reducing sources of variation over Improving estimating Data over Intuition* Probabilistic over Deterministic Minimal Viable Product over Full scope That is, while there is value in the items on the right, we value the items on the left more. *Neil Killick uses “empiricism over guesswork”
REDUCING SOURCES OF VARIATION
@mattphilip #agilecamp2018 HOFSTADTER’S LAW “It always takes longer than you expect, even when you take into account Hofstadter’s Law.”
@mattphilip #agilecamp2018 PARKINSON’S LAW Work expands so as to fill the time available for its completion.
@mattphilip #agilecamp2018 “POOR ESTIMATION SKILLS”
@mattphilip #agilecamp2018 WHAT DO WE ESTIMATE?
@mattphilip #agilecamp2018 WHAT WE ESTIMATE Code
@mattphilip #agilecamp2018 WHAT WE ESTIMATE Analyze+Design+Code+Test+Deploy
@mattphilip #agilecamp2018 WHAT ACTUALLY HAPPENS Analyze Code Test DeployDesign Wait for approval Wait for staff availability Rework Wait for API availability Switch to 
 higher-priority work Wait for approval Wait for 
 blocker 
 removal
@mattphilip #agilecamp2018 WHAT’S GOING ON? Value-added time _________________________ Total elapsed delivery time Flow efficiency =
@mattphilip #agilecamp2018 WHAT’S GOING ON? Low process efficiency (typically 5-15% in software delivery) means that even if we nailed the effort estimates … we would be accurately predicting 5-15% of elapsed delivery time!
 — Troy Magennis
@mattphilip #agilecamp2018 DO YOU ASSUME CORRELATION? Is the initial sizing a good predictor for when you can get your stuff? In our case, the surprising truth was ”no.” — Mattias Skarin, Real-World Kanban
@mattphilip #agilecamp2018 SOURCES OF VARIATION How many can you name?
@mattphilip #agilecamp2018 SOURCES OF VARIATION Waiting for 
 availability Rework Team 
 dependencies Stages in team development (Tuckman) Steps/
 handoffs Selection 
 policy Essential 
 complication Accidental 
 complication System 
 dependencies Work In 
 Progress Technology/
 domain/
 product Specialization Team 
 composition Multitasking/
 context 
 switching Collaboration
 policy Blockers
@mattphilip #agilecamp2018 WHAT YOU CAN DO ABOUT VARIATION How many remedies
 can you name?
@mattphilip #agilecamp2018 WHAT YOU CAN DO ABOUT VARIATION ▫︎Lower WIP/System WIP ▫︎Five Focusing Steps ▫︎Blocker clustering ▫︎Reduce workflow stages ▫︎Explicit policies ▫︎Cost of Delay scheduling, sequencing and selection ▫︎Reduce “expedites” Lean-Kanban
@mattphilip #agilecamp2018 WHAT YOU CAN DO ABOUT VARIATION ▫︎“Agile 101” (simple, decoupled design; thin vertical slices; pairing) ▫︎Identify/make visible/measure dependencies ▫︎Bigger teams (to reduce dependencies) ▫︎Collaborate/Share work (Dimitar Bakardzhiev) ▫︎Spike and stabilize (Dan North) ▫︎Reduce accidental complexity (Liz Keogh) Team Why?
@mattphilip #agilecamp2018 POLICIES TO REDUCE VARIATION ▫︎We will only start new work at about the same rate that we finish old work. ▫︎We will make every reasonable effort to finish all work that is started and minimize wasted effort due to discarded work items ▫︎If work becomes blocked, we will do everything we can do unblock that work as expeditiously as possible. ▫︎We will closely monitor our policies around the order in which we pull items through our system so that some work items do not sit and age unnecessarily. — Daniel Vacanti, When Will It Be Done?
DATA OVER INTUITION
@mattphilip #agilecamp2018 HUMAN BIAS How many biases can you name?
@mattphilip #agilecamp2018 HUMAN BIAS
@mattphilip #agilecamp2018 MANAGEMENT PARADIGMS
@mattphilip #agilecamp2018 FIRST, KNOW YOUR DOMAIN
@mattphilip #agilecamp2018 KEOGH’S “SCALE OF IGNORANCE” 1 Just about everyone in the world has done this. Lots of people have done this, including someone on our team.2 Someone in our company has done this, or we have access to expertise.3Someone in the world did this, but not in our organization (and probably at a competitor). 4 Nobody in the world has ever done this before. 5
@mattphilip #agilecamp2018 CHOOSE RELEVANT APPROACH Monte Carlo simulations Wisdom of crowds Planning poker Experts
@mattphilip #agilecamp2018 DATA OVER INTUITION
@mattphilip #agilecamp2018 LET’S PRACTICE! 1.At your table, choose someone to roll the die/dice. 2.Roll the die/dice. 3.Everyone else record each result (roll 10 times).
34 PROBABILISTIC OVER DETERMINISTIC
@mattphilip #agilecamp2018 PROBABILISTIC OVER DETERMINISTIC Dec 28 — graphic courtesy of Larry Maccherone, (maccherone.com/lumenize)
@mattphilip #agilecamp2018 VACANTI’S VERITY When making a forecast (predicting the future), you have to accept that there is more than one possible outcome. Therefore… A forecast is a calculation about the future that includes both a range and a probability of that range occurring.
@mattphilip #agilecamp2018 PROBABILISTIC OVER DETERMINISTIC
@mattphilip #agilecamp2018 PROBABILISTIC OVER DETERMINISTIC
@mattphilip #agilecamp2018 PROBABILISTIC OVER DETERMINISTIC
@mattphilip #agilecamp2018 PROBABILISTIC OVER DETERMINISTIC 2% 20% 21% 3% 16% 16% 9% 7% 2% 2% 2% Dec 28 Feb 17Nov 30 Jan 20
@mattphilip #agilecamp2018 PROBABILISTIC FORECASTING
@mattphilip #agilecamp2018 PROBABILISTIC FORECASTING: ONE ITEM
@mattphilip #agilecamp2018 BACK TO YOUR SCATTERPLOT
@mattphilip #agilecamp2018 STRAIGHT-LINE (AVERAGE) FORECASTS Team A Team B Work item #1 5 days 3 days Work item #2 1 day 3 days Work item #3 3 days 3 days Average 3 days 3 days 3-day delivery time 
 expectancy 66% 100%
@mattphilip #agilecamp2018 EXPECTATIONS, NOT COMMITMENTS
@mattphilip #agilecamp2018 SERVICE-DELIVERY EXPECTATIONS Expectation Performance Expedite 90% 3 days or less Standard Urgency 85% 10 days or less Fixed-Date 95% On-time
47 IN SUMMARY
@mattphilip #agilecamp2018 TO ESTIMATE OR NOT TO ESTIMATE? NeverEstimateAnything AlwaysEstimateEverything Estimate Default Behavior Don’t estimate Effort Factors All sources of variation Intuition Approach Data Deterministic Forecast Type Probabilistic Tasks in hours Level of Detail Work items in days High Effort Spent Low Estimating better Improvement Aim Reducing variation Commitment Management Service Expectations
@mattphilip #agilecamp2018 BETTER QUESTIONS TO ASK @mattphilip ▫︎In what context would estimates bring value, and what are we willing to do about it when they don’t? – Woody Zuill ▫︎How much time do we want to invest in this? – Matt Wynne ▫︎What can you do to maximize value and reduce risk in planning and delivery? – Vasco Duarte ▫︎Can we build a minimal set of functionality and then learn what else we must build? ▫︎Would we/you not invest in this work? If not, at what order-of-magnitude estimate would we/you take that action? ▫︎What actions would be different based on an estimate?
@mattphilip #agilecamp2018 HOW TO GET STARTED (PROJECT IN PROGRESS) 1. Continue to discuss the work to analyze it and break it into thin vertical slices. 2. Start tracking two pieces of data for each work item: commit date and delivery date. 3. After accumulating 10 data points, run a Monte Carlo simulation to provide a probabilistic forecast. 4. Begin answering the “When” question using data, either at the project level or individual work-item level. 5. Validate whether the probabilistic forecasts match what actually occurs. 6. If the forecasts are helpful, wean your team off estimating. 7. Focus improvement efforts on reducing the sources of variation.
@mattphilip #agilecamp2018 HOW TO GET STARTED (NEW PROJECT) 1. Ask the “better questions” above about the use and importance of estimates. 2. Use reference-class data (similar delivery times from other projects in similar tech stacks with similar teams) to provide an initial probabilistic forecast 3. As soon as you accumulate 10 data points, run a Monte Carlo simulation to provide an updated probabilistic forecast. 4. Continually reforecast when you get more information.
@mattphilip #agilecamp2018 REFERENCES AND FURTHER EXPLORATION ▫︎ noestimatesbook.com (Vasco Duarte) ▫︎ infoq.com/articles/noestimates-monte-carlo (Dimitar Bakardzhiev) ▫︎ priceonomics.com/why-are-projects-always-behind-schedule ▫︎ http://scrumandkanban.co.uk/estimation-meets-cynefin/ ▫︎ ronjeffries.com ▫︎ Lizkeogh.com ▫︎ https://neilkillick.wordpress.com/ ▫︎ http://zuill.us/WoodyZuill/ ▫︎ mattphilip.wordpress.com/noestimates-game ▫︎ When Will It Be Done? (Dan Vacanti) ▫︎ focusedobjective.com (Troy Magennis) ▫︎ actionableagile.com (Dan Vacanti) ▫︎ kanbanize.com ▫︎ scrum.org
@mattphilip #agilecamp2018 www.noestimat.es THANK YOU @mattphilip@mastodon.cloud gab.ai/mattphilip @mattphilip

More Related Content

Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)