Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Henrik Andersson - Exploratory Testing Champions - EuroSTAR 2010

•
•
TEST Huddle
TEST Huddle

EuroSTAR Software Testing Conference 2010 presentation on Henrik Andersson by Exploratory Testing Champions. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/

1 of 14
Download to read offline
Exploratory Testing Champions Henrik Andersson henrik.andersson@houseoftest.se
The Client
Background
What is Exploratory Testing Exploratory testing is simultaneous learning, test design, and test execution. "...a way of thinking about testing“ Programs that pass certain tests tend to continue to pass the same tests and are more likely to fail other tests that are yet to be explored Diversity is good!
The ET pilot ”ET could also be a good complement when debugging or investigating failures in the platform. Now when we know the tools it should be quite easy to set up and explore the platform.” ”One shouldn’t forget that we are primarily test developers with diverse knowledge in different fields such as Signal Processing and programming.” ”Good with the scheduling, otherwise it's easy to stop after i.e. 30 or 60 min” ”It's a little bit hard to find good missions.” ”The testers really liked this way of working. They felt like it was really productive when actually performing test cases without writing extensive documentation beforehand.” ”When workiing in a more agile way, we have to look at ET and actually think about what kind of documentation we need and what we can skip doing.”
Statistics from One Pilot Errors found during ET •4 resources for 1 day •The purpose was to more extensively test areas with poor test coverage, stress the system and to find new bugs. Total: 3-4 platform ERs were found. These ERs wouldn't been found by the regression TCs. 6 issues to be further investigated Errors found by regression test •Approx four (4) times more time spent then during ET •The purpose of the regression test was to see if new labels are stable and to find degrades. Total: 3 platform ERs were found Disclamer: It is not possible to make any conclusions based on this limited number of data. Same application of the Sw Same build/sw version
The Decision
Schedule •Day 1 –First ET meetingwith ETC, go through the schedule –One ET session –Debriefing –Preparation •Day 2 –WorkshopDesigning the Process –Output draft of the ET process and reviewed templates for test design –Mini Workshopsession discipline –Output Session guidelines –One ET sessionwith the defined process –Debriefing •Day 3 –Workshop Heuristics Bug taxonomy -Output checklists –Two ET sessionsusing heuristics and Test design techniques –Debriefing •Day 4 –Workshop, Test environment –Output list on what changes needs to be done to the test tools and which areas can the tool be used for as is. –One ET session –Debriefing –Mini workshop–input to testing –Summary discussion •Day 5 –Hand over –Department meeting –Close down
Output •Hands on experience of Exploratory testing •A process •Check list -Exploratory Test Session •Input to testing •Tool Improvements •Real examples of ET Session reports •Session Report Template
Exploratory Testing Champions The ETC will spend one day/week within the projects. The ETC will coach, tutor and share their knowledge about ET, test techniques, tools and testing strategies to everyone in their teams. The ETC will drive, develop, build and explore the craftsmanship of testing. Everyone in the team is allowed to and are encouraged spend time together with the advisor and take usage of his knowledge. The ETCs will meet up every second week to exchange experience, discuss issues and try out new thoughts.
About the process •Suggestion is to run two (2) test sessions every 2nd week. •Each session about 90 minute long. •Setup and preparation shall have been done before the session ( Check list - Exploratory Test Session) •Alter testing alone and testing two and two (pair testing) •To start with only reporting done is session reports and defects.We do not use any tool or metrics to track the sessions. •Have a flexible approach to debriefings. Can be performed one on one, in group, after every session, after a day of sessions, tester to tester or tester to test manager. Important is that this does not become routine and the value from it decreases: That is one reason for variation. •Use a wikki accessible for the whole team to enter mission ideas •Common folder for each team to save session reports •Common folder for everyone to save test scripts the helps us to get to a certain state quick.
What was left to do?
Where are they today?
Thank you for your attention!

More Related Content

Henrik Andersson - Exploratory Testing Champions - EuroSTAR 2010

  • 1. Exploratory Testing Champions Henrik Andersson henrik.andersson@houseoftest.se
  • 4. What is Exploratory Testing Exploratory testing is simultaneous learning, test design, and test execution. "...a way of thinking about testing“ Programs that pass certain tests tend to continue to pass the same tests and are more likely to fail other tests that are yet to be explored Diversity is good!
  • 5. The ET pilot ”ET could also be a good complement when debugging or investigating failures in the platform. Now when we know the tools it should be quite easy to set up and explore the platform.” ”One shouldn’t forget that we are primarily test developers with diverse knowledge in different fields such as Signal Processing and programming.” ”Good with the scheduling, otherwise it's easy to stop after i.e. 30 or 60 min” ”It's a little bit hard to find good missions.” ”The testers really liked this way of working. They felt like it was really productive when actually performing test cases without writing extensive documentation beforehand.” ”When workiing in a more agile way, we have to look at ET and actually think about what kind of documentation we need and what we can skip doing.”
  • 6. Statistics from One Pilot Errors found during ET •4 resources for 1 day •The purpose was to more extensively test areas with poor test coverage, stress the system and to find new bugs. Total: 3-4 platform ERs were found. These ERs wouldn't been found by the regression TCs. 6 issues to be further investigated Errors found by regression test •Approx four (4) times more time spent then during ET •The purpose of the regression test was to see if new labels are stable and to find degrades. Total: 3 platform ERs were found Disclamer: It is not possible to make any conclusions based on this limited number of data. Same application of the Sw Same build/sw version
  • 8. Schedule •Day 1 –First ET meetingwith ETC, go through the schedule –One ET session –Debriefing –Preparation •Day 2 –WorkshopDesigning the Process –Output draft of the ET process and reviewed templates for test design –Mini Workshopsession discipline –Output Session guidelines –One ET sessionwith the defined process –Debriefing •Day 3 –Workshop Heuristics Bug taxonomy -Output checklists –Two ET sessionsusing heuristics and Test design techniques –Debriefing •Day 4 –Workshop, Test environment –Output list on what changes needs to be done to the test tools and which areas can the tool be used for as is. –One ET session –Debriefing –Mini workshop–input to testing –Summary discussion •Day 5 –Hand over –Department meeting –Close down
  • 9. Output •Hands on experience of Exploratory testing •A process •Check list -Exploratory Test Session •Input to testing •Tool Improvements •Real examples of ET Session reports •Session Report Template
  • 10. Exploratory Testing Champions The ETC will spend one day/week within the projects. The ETC will coach, tutor and share their knowledge about ET, test techniques, tools and testing strategies to everyone in their teams. The ETC will drive, develop, build and explore the craftsmanship of testing. Everyone in the team is allowed to and are encouraged spend time together with the advisor and take usage of his knowledge. The ETCs will meet up every second week to exchange experience, discuss issues and try out new thoughts.
  • 11. About the process •Suggestion is to run two (2) test sessions every 2nd week. •Each session about 90 minute long. •Setup and preparation shall have been done before the session ( Check list - Exploratory Test Session) •Alter testing alone and testing two and two (pair testing) •To start with only reporting done is session reports and defects.We do not use any tool or metrics to track the sessions. •Have a flexible approach to debriefings. Can be performed one on one, in group, after every session, after a day of sessions, tester to tester or tester to test manager. Important is that this does not become routine and the value from it decreases: That is one reason for variation. •Use a wikki accessible for the whole team to enter mission ideas •Common folder for each team to save session reports •Common folder for everyone to save test scripts the helps us to get to a certain state quick.
  • 12. What was left to do?
  • 13. Where are they today?
  • 14. Thank you for your attention!