Thomas Axen - Lean Kaizen Applied To Software Testing - EuroSTAR 2010

Lean Kaizen Applied To Software Testing
Thomas Axen

Agenda
Short about ATP -Test Organization
Why Test Kaizen -Blitz and Relationship to our test improvement areas
Test Kaizen -Blitz seen from the theory to practices
The Test Kaizen -Blitz process in ATP
Test improvement overall view
Summary
Questions
4

5
The purpose of this presentation
To share my experiences with others
We hope that you can use some of this information in your daily work
We hope that you come up with some good questions for me, that I can use in my daily work

6
Short about ATP -General
Employees in ATP
-800 FTE at Hillerød Campus
-1000 FTE in 5 call centers
Employees in IT
-210 FTE
-80 Consultants
Employees in Test
-27 FTE
-23 Consultants
Customers
–ATP livslang Pension (ATP)
–Arbejdsgivernes Elevrefusion (AER)
–Arbejdsmarkedets Erhvervssygdomssikring (AES)
–Barsel.dk
–DA-Barsel
–FerieKonto(FK)
–Finansieringsbidrag (FIB)
–Lønmodtagernes Dyrtidsfond (LD)
–Lønmodtagernes Garantifond (LG)
–Supplerende arbejdsmarkedspension for førtidspensionister (SUPP)
–Den Særlige Pensionsopsparing (SP)
–Finansieringsbidrag (FIB)
–Skattenedslag for seniorer (SFS)
–Pension Danmark (PD)
–Objektiv sagsbehandling –New customers…
-Folkepensionen
-Udbetaling og indtægtsregulering af fortidspension
-Boligstøtte
-Barselsdagspenge
-Forskudsvis udbetaling af børnebidrag

7
Short about ATP - Test Organization
 The test resources (50 FTE)
- 3 Test chiefs
- 6 Test Managers (RTM, TI and TM)
- 9 Test Coordinators
- 3 Test Specialists (automated test and test data)
- 6 IT-Testers and students workers (ITU and DTU)
- 23 Test consultants
 Responsibility areas
- Test in projects and Sustained Engineering
- Test methods and tools (QC, QTP, ATSA, TDM)

Why Test Kaizen -Blitz
We have used Kaizen on other areas with great success
Top management –see it as a way of ”tuning” the organization
A god way of interactingwith the stakeholders, to understand their needs and attitude towards test
An new positive wayof identifying improvements areas on test
All the testers can impact/influenceon what the improvement areas are and what we shall prioritize to work on
It verifies if we are working on the right improvement areas
8

Relationship to our test improvements areas9
Inspiration
Process improvementImplementationTest organisationImprovementsGovernancePrioritizingExecutionImprovement inspirationFeedback TMM assessmentKaizen -BlitzGuru NetworkBest Practice
Management decisionStrategy
ProjectSustained Engineering

Test Kaizen -Blitz seen from the theory to practices
Lean thinking:
Elimination of waste
Flow (production processes are made continuous)
Three central concepts:
Wasteis ’everything the customer won’t pay for’
Flowimplies elimination of batches and queues
Productionis ’mass production’
10

…but Isn’t All Testing Waste?
The answer is yes in the context of software production:
11
Customers will not pay you for makingerrors.
Consequently customer will not pay you for findingand correcting the errors you make.
But we are not looking at test in the context of software production. We are looking at test in the context of a customer checking the quality of software received from a vendor.
In this context test is not waste.

How Do We Identify Waste in Testing? 12
ATP chose Kaizen –Blitz from the Lean Toolbox.
Kaizen means ’continuous improvements’ and is “by the book”:
Dealing with a limited and thus manageable part of a production process (comparable to the work of 5 to 6 persons).
Being performed quickly but having ambitious goals for removing waste from the area under consideration.
Involving the 5 to 6 persons, who operate the process and are familiar with it.
Looking for improvements, which the persons involved are themselves in command of.
Being based on facts and measurements as far as possible.
Being repeated at regular intervals.

ATP chose a pragmatic approach to Kaizen
13
Dealing with a limited and thus manageable part of a production process. The areas chosen were somewhat big, since it included both Domain, Integration, and User Test.
Being performed quickly but having ambitious goals for removing waste from the area under consideration. 3 short workshops over 6 weeksinstead of 3 intensive days.
Involving exactly the 5 to 6 persons, who operate the process and are familiar with it.
Looking for improvements, which the persons involved are themselves in command of. Several useful improvements were “owned by other parties”.
Being based on facts and measurements as far as possible. Neither time nor money for measurement on a large scale. (Common sense)
Being repeated at regular intervals. One time must be the first time.
Next timeNext time

The Test Kaizen -Blitz process in ATP
1 Step -Define the reason for the investment in kaizen
-Business case with focus on cost/benefit and scope.
-Communication to sponsor ongoing on status and progress
2 Step -ATP Test Kaizen kick-off
-Selecting the tester group that shall work with the kaizen process
-Short introduction to kaizen, and the plan (30 min.)
3 Step -Interviewing stakeholder and collecting facts
-The VoxPop(Voice of the Customer) input.
4 Step -Workshop, Testers as active players
-Present Voxpopand collected facts
-Kaizen scope and definition of waste in testing
-Value Stream (mapping all test activities and waste pr. Test phase)
5 Step -Workshop, RCA and classification
-Root Cause Analysis (Fishbone and selected defects analysis)
6 Step -Workshop, Consultation
-Classificationand prioritization the improvement areas
-Finalize the proposal to management (the presentation)
7 Step -Summary (Presentation for managers and a execution plan)
14

3 Step -Interviewing stakeholder and collecting facts
Objective: To challenge the Kaizen team members attitude towards their work.
15
Examples:
Far to many errors do notsurface before full production.
The Voice of the Customer:
-”To little testing. To much testadministration. Spend more time testing!”
-”I have never been on a project with so many test phases...”
-...

4 Step -Workshop, Kaizen Scope and Definition of Waste
SIPOC -Scope of ATP Test Kaizen (Black outline in Kaizen scope)
Definition of waste in testing:
All activities not directly involved in finding errors or in proving the absence of errors are considered waste.
All repetition of test etc. are considered waste.
16ProgrammingComponenttest
DomaintestIntegrationstest
UsertestOperation

4 Step - Workshop, Value Stream (mapping all test activities and waste pr. Test phase)
Objective: To get a clear picture of the time and resources spend on
testing and to identify activities where time and resources are
wasted.
What was done?
 Detailed mapping of 3 subprocesses: Domain test,
Integration test and User test (3 value stream maps).
 Identification of waste pr. subprocess.
Fordeling af kundeværdi og spil - Best case
0
50
100
150
200
250
300
350
400
1 2 3 4 5 6a 6b 6c 7 8a 8b 9b 9c 10 11
Aktivitet
Ressourceforbrug/timer
NVA
BNVA
CVA
Example: Integration test

5 Step -Workshop, Root Cause Analysis
Objective: To prepare for elimination of waste by identifying the underlying reasons.
What was done?
Systematic root-cause-analysis of all activities involving waste.
A. Testing is not finished within time and budget
I.To few people with proper business and domain knowledge in the individual test phases
a.Different (new) people are testing a given piece of software in each of the test phases.
1.To many handovers between phases.

6 Step -Workshop, classificationand prioritization the improvement areas
What was done:
Brainstorm among participants on the basis of the Root Cause Analysis.
Identification of 38 proposalsfor elimination of waste.
Prioritisation of proposals: ’Resources vs. Effect’.
Anchoring: Who is doing what?
Objective: To formulate operational proposals, leading to elimination of waste in testing.
Low ConsiderDon't High DoConsiderLowHighResources needed for implementation Effect on waste elimination

6 Step -Workshop, Finalize the proposal to management
We identified 38 improvement areas
We grouped the improvement areas into 4 categories
-Go Fix –without any additional investments (9 areas)
-Investment –We need an investment based on a Business case and return of investment analysis (3 areas)
-In progress –investment already in place project up an running (3 areas)
-Relationship to other areas (4 areas)
Rewriting of all the improvement areas so management can understand the value of the investments
20

21
7 Step -Summary, “Go Fix” without any additional investments
Priority 1
-Tester should have more focus on progress, instead of roles
-Release management plan, must reflect dependencies so it is easy for the tester to plan their test
-With critical issues, it should be possible to gather a team across the normal teams
Priority 2
-Post-mortem on found defect in production and projects (Root course analysis)
-Optimize the defect process
-General education plan (Consultants, PM, Business, Testers)
Priority 3
-Test cases must follow the BPT standard
-Improvement of the code documentation on defect and design changes, so test can focus and minimizing the cost
-Quality assurance on deliverables in a structured way (Focus on static testing)

7 Step -Summary, “Investment” based on BC and ROI
Organization of the test pr. release
-All hours on test are used optimal –focus on planning, competences and the team work together
Quality assurance of all software deliverables collected systematically for reporting to management
-Systematic collection of data, for reporting to management so they can use it for negotiationswith vendors/teams on what the given quality should be for each of them (Project and SE)
Release regressions test coverage
-Well documented release regressions test coverage pr. customer/solution, so management can decide if we should “go live”
22

7 Step -Summary, “In progress”
Strategy for how to test letters
Test Data Management (Tools and reuse data)
More automated regression test (BPT, QTP)
23

7 Step -Summary, Relationship to other areas
Release management
-Overall plan, deliverables, dependencies, risk and issues.
-Requirements and change management
Deployment to test environments (36 applications using SOA)
-System test environments
-Integration test environments
Tester education with different focus
-Consultants, Business unit area, End users, Project management
Post-mortem on found defects In production and projects/Root cause analysis
24

25Test improvement overall view
Department vision/mission & overall goals
-Be proactive (defects prevention, Innovation, think ahead)
-Agreements attitude (Clear, on time, expectation, follow-up)
-Well-known ABP (Stable, optimized, functional)
-Ready for changes (New things, flexible, developing and managed)
Test vision & overall goals
-From known to respected in ATP
-Test in ATP should increase one level on TMM(i)
-Effective test with use of Best Practices and front-runners on selected areas
-Test process exists; it is always used and improved continually
-Competent/ professional test resources in the IT-organization
-Competent tests in the business-organization
Test leadership
-Updating the A4 plans for the whole test area
-Updating detailed plans for each sub test areas
-Ongoing prioritization and follow-up on tests areas across applications
-Focus on improvements to minimize the effort
TMM –improvements (Level 3)
-Continually optimizing/updating/support on test artifacts
-Continually optimizing all test processes
-Overview of all test tools and usages
-Extend the defect –root cause analysis
-More focus on non-functional requirement and test techniques
-More tester education to business, end users and consultants
-Focus on test data availability
-Use review for testing
-Metrics on test
Tester feedback loops
-The best test case practice in ATP
-Optimize the usages of our test environments
-Optimizing of the review process on important artifacts
Test Kaizen –Blitz Improvements area
-Quality assurance on deliverables in a structured way
-New test organization to handle releases
-Release regressions test coverage
-Tester should have more focus on progress, instead of roles
-Release management plan, must reflect dependencies so it is easy for the tester to plan their test
-With critical issues, it should be possible to gather a team across the normal teams, to analyse the issue
-Post-mortem on found defects in production and projects
-Optimize the defect process
-General education plan (Consultants, Business, End users, PM)
-Test cases must follow the BPT standard
-Improvement of the code documentation on defect and design changes, so test can focus and minimizing the cost
-Strategy for how to test letters
-Test Data Management (Tools and reuse data)
-More automated regression test (BPT, QTP)
”Guru sanity check” Recommendations
-All tester roles must be defined and communicated in ATP
-Test management meeting each 14 days
-PM gets a Tester education day, and understand each others roles
Governance projects
-Extend the amount of automated regression test
-Optimizing of our Test Data Management area
-Stabilising of all reg. test cases for automation
Project managers improvements areas
-Improve the test environment setup
-Test statistic (ex. defects in projects / production)
-Target groups on test artefacts (Who should read what)
-Move the test activities to the start of the project in the risk and QA
-Root cause analysis on project defects

Summary
We will definitely have a test Kaizen again, when we are ready
It is a good way of making sure that all stakeholders are being heard
Great way of getting a ”buy in”from all the testers, where the testers have an influence on the improvements areas to execute on
It improves the image and awarenessabout Test in ATP
Remove activities that don’t add value
All relevant improvements areas are identified and prioritized
This is a “new angle”on ways to identify improvement areas
It also spins of improvement areas to other partsof the IT organization
Adjust the communication with respect, so the stakeholders can see the value of each improvements area from their point of view.
26

Thomas Axen - Lean Kaizen Applied To Software Testing - EuroSTAR 2010

Related slideshows

More Related Content

Thomas Axen - Lean Kaizen Applied To Software Testing - EuroSTAR 2010