Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Using JIRA for Risk Based Testing - QASymphony Webinar

•Download as PPTX, PDF•
•
QASymphony
QASymphony

Learn how to leverage risk based testing with JIRA to prioritize the testing of features and functions based on risk of failure, function of importance and likelihood or impact of failure.

1 of 37
September 25, 2015 Introduction to QASymphony for [INSERT COMPANY NAME] #AgileTransformation Agile Transformation: People, Process and Tools to Make Your Transformation Successful Using JIRA for Risk Based Testing #RiskBasedTesting
#JIRATesting @RyanYackAttack Test Case Management | Exploratory Testing | Test Analytics | BDD and more... ABOUT QASYMPHONY The #1 Enterprise Testing Platform for
#AgileTransformation OUR PRESENTER Ryan Yackel Senior Product Engineer QASymphony @ryanyackattack #RiskBasedTesting • Serves as the “customer voice” for all QASymphony products. • Comes to QASymphony from Macy’s, where he managed testing on large enterprise initiatives. • Certified scrum master from Scrum Alliance and holds a Bachelor of Arts degree from Covenant College. • Has a newborn baby, and doesn’t sleep a ton :-)
The Test Everything Mentality Traditional vs Risk Based Testing How can JIRA be used to track risk Use Case of JIRA + qTest Products Recommended Add-ons to try out WHY ARE WE HERE? 4 #RiskBasedTesting
#AgileTransformation SOFTWARE TESTING IN A RISKY BUSINESS 5 #RiskBasedTesting
#AgileTransformation PRESSURE IS ON TEST TEAMS 6 #RiskBasedTesting What’s the plan to test X new requirements? What’s the plan to test this one requirement, that affects 50 applications? I guess I could create X tests, multiplied by the 100 configurations I guess I could create 1,000 tests, multiplied by the 50 configurations
#AgileTransformation TIME IS OF THE ESSENCE 7 #RiskBasedTesting
#AgileTransformation TEST EVERYTHING 88 #RiskBasedTesting
#AgileTransformation WHAT DO I TEST? 99 #RiskBasedTesting Do we need to test everything?....If not, what can I cut from my test plans? When should my testing be complete? How do I get team convinced that we are limited in test? Where am I going to get all my resources for this?
#AgileTransformation TRADITIONAL vs RISK BASED TESTING 1010 #RiskBasedTesting TRADITIONALRISKBASED TESTING
September 25, 2015 Introduction to QASymphony for [INSERT COMPANY NAME] #AgileTransformation Agile Transformation: People, Process and Tools to Make Your Transformation Successful 11 RISK BASED TESTING (RBT) #RiskBasedTesting
#AgileTransformation WHAT IS RISK? 12 #RiskBasedTesting Risk = Damage * Frequency Risk = Impact * Probability of Failure A Risk is a problem or situation that has not happened yet and it may never happen in future as well. It is a possible issue or problem that may occur. DEFINITIONS Michael Scott poorly shoots a three pointer (Impact) * Everytime he gets the ball (Frequency) = EXTREMELY HIGH Risk
#AgileTransformation TWO TYPES OF RISK ANALYSIS 1313 Statistical Risk Analysis a) Weighted Metrics b) Color Coding c) P1 - P4 Areas Heuristic Risk Analysis a) Consider more possibilities and interesting aspects of the problem b) Prioritize a List of Risks DistributedPopular NewComplex Down Stream Strategic #RiskBasedTesting (Daniele Gagliardi – eclipsecon Europe 2015)
#AgileTransformation STATISTICAL RISK ANALYSIS 1414 #RiskBasedTesting (Sticky Minds Risk-Based Testing: Test Only What Matters) Marginal impact—1 Moderate impact— 2 Medium impact—3 High impact—4 Critical impact—5 A cosmetic error. The progress was disrupted with manageable extensions to the short-term schedule and cost; customers were not affected The progress was disrupted with large extensions to project schedule and cost; customers were not directly affected An application or transaction caused a significant disruption to production The problem caused all major and critical functioning of the application to fail. This resulted in a large loss of revenue. Marginal—1 Moderate—2 Medium—3 High—4 Critical—5 Low complex code written on low complicated system architectural design Moderate complex code written on moderate complicated system architectural design Medium complex codes written on medium complicated system architectural design There is a large amount of complex code written on a highly complicated system architectural design This suggests extreme complex code that has been written on top of on extremely complicated architectural design IMPACT PROBABILITY OF FAILURE
#AgileTransformation STATISTICAL RISK ANALYSIS 1515 #RiskBasedTesting - P1 – Critical Impact: Must be tested; ideal candidate for automation (candidate for automated smoke testing) - P2 – High Impact: Should be tested - P3 – Medium Impact: Can be tested if budget and schedule permits - P4 – Low Impact: May not be tested; no impact on application and no need of automation (Sticky Minds Risk-Based Testing: Test Only What Matters)
#AgileTransformation HEURISTIC RISK ANALYSIS: INSIDE OUT 1616 #RiskBasedTesting - Vulnerabilities: What weaknesses or possible failures are there in this component? - Threats: What inputs or situations could there be that might exploit a vulnerability and trigger a failure in this component? - Victims: Who or what would be impacted by potential failures and how bad would that be? Grab a developer and ask “What can go wrong here? and here? and …” :-) (James Bach Heuristic Risk-Based Testing)
#AgileTransformation HEURISTIC RISK ANALYSIS: OUTSIDE IN 1717 #RiskBasedTesting (James Bach Heuristic Risk-Based Testing) E D F A B C Complex: anything disproportionately large, intricate, or convoluted. New: anything that has no history in the product. Changed: anything that has been tampered with or "improved". Upstream Dependency: anything whose failure will cause cascading failure in the rest of the system. Downstream Dependency: anything that is especially sensitive to failures in the rest of the system. Critical: anything whose failure could cause substantial damage. Risk Lists
#AgileTransformation HEURISTIC RISK ANALYSIS: IMPACT 1818 #RiskBasedTesting Component Risk Matrix Component Risk Risk Heuristics Advanced Shipping Normal Complex, popular GUI Higher New, complex, third- party, critical Manifest Lower Legacy, changed, Point of Sale Lower Legacy, no change Purchasing Higher Upstream, Critical Receiving Normal Recent Failure Shipping Normal Downstream
#AgileTransformation PLAN OF ATTACK 1919 #RiskBasedTesting RISK IDENTIFICATION RISK ANALYSIS TEST PLANNING TEST EXECUTION TEST RESULT P1 P2 P3 P4 P5 Exploratory Testing
#AgileTransformation THE GOAL OF RBT 2020 #RiskBasedTesting GOAL Assurance of Quality CostofFailure CostofTesting (Sticky Minds Risk-Based Testing: Test Only What Matters) • Not Everything Needs to be Tested • Finding Balance the cost of testing and Failures with Quality Assurance • Taking Calculative Risk and Test
September 25, 2015 Introduction to QASymphony for [INSERT COMPANY NAME] #AgileTransformation Agile Transformation: People, Process and Tools to Make Your Transformation Successful 21 USING JIRA TO MANAGE RISK #RiskBasedTesting
#AgileTransformation FLEXIBLE AND CONFIGURABLE 2222 #RiskBasedTesting SOFTWARE DEVELOPMENT ISSUE TRACKING RISK MITIGATION SOFTWARE DEVELOPMENT MARKETING GO TO MARKET SOFTWARE DEVELOPMENT MARKETING GO TO MARKET PERSONAL TASK MANAGEMENT HR ON BOARDING
#AgileTransformation CONFIGURING JIRA ISSUES 23 #RiskBasedTesting Option 1: Create New “Risk” Issue
#AgileTransformation CONFIGURING JIRA ISSUES 24 #RiskBasedTesting Create Custom Statistical Fields a) Probability of Failure b) Impact c) Risk Analysis Custom Heuristic RISK Fields a) Risk Heuristics b) Heuristic Analysis Option 2: Custom Risk Fields
#AgileTransformation RISK WITH JIRA ISSUES 25 #RiskBasedTesting Issue Search Analysis Statistical HeuristicCommon
#AgileTransformation RISK WITH JIRA ISSUES 26 #RiskBasedTesting Board Analysis
#AgileTransformation CATAGORIZE RISK FOR TEST 27 #RiskBasedTesting Organize Risks for Test
#AgileTransformation PLAN AGAINST RISKS 28 #RiskBasedTesting Map High Priority Tests w/ High Risk
#AgileTransformation EXECUTE YOUR AGAINST RISKS 29 #RiskBasedTesting Coordinate Line of Priority for Execution
September 25, 2015 Introduction to QASymphony for [INSERT COMPANY NAME] #AgileTransformation Agile Transformation: People, Process and Tools to Make Your Transformation Successful 30 JIRA ADD-ONS #RiskBasedTesting
#AgileTransformation RISK REGISTER 31 #RiskBasedTesting
#AgileTransformation VISUALIZER FOR JIRA 32 #RiskBasedTesting
#AgileTransformation SCRIPT RUNNER FOR JIRA 33 #RiskBasedTesting JIRA Misc Custom Fields ScriptRunner for JIRA
September 25, 2015 Introduction to QASymphony for [INSERT COMPANY NAME] #AgileTransformation Agile Transformation: People, Process and Tools to Make Your Transformation Successful 34 WRAPPING IT UP #RiskBasedTesting
#AgileTransformation PROS AND CONS 35 #RiskBasedTesting Management might think you are lazy Miscalculate or mis-appropriate weight to a certain risk Coordinating risk identifiers is a challenge The impossible could happen High severity defects are flushed out Low values tests are kicked to curb Critical areas are tested first Confidence is boosted Deadline anxiety is removed
#AgileTransformation RESOURCES 36 #RiskBasedTesting JIRA Add-Ons • Risk Register - https://marketplace.atlassian.com/plugins/com.projectbalm.riskregister.riskregister-jira/server/overview • Visualizer for JIRA - https://marketplace.atlassian.com/plugins/vfjc/cloud/overview • ScriptRunner - https://marketplace.atlassian.com/plugins/com.onresolve.jira.groovy.groovyrunner/cloud/overview • JIRA MISC Custom Fields - https://marketplace.atlassian.com/plugins/com.innovalog.jmcf.jira-misc-custom- fields/server/overview References • Sticky Minds Risk-Based Testing: Test Only What Matters • Xoriant – An Approach for Risk Based Testing • James Bach - Heuristic Risk-Based Testing • Daniele Gagliardi – eclipsecon Europe 2015
Thank You #AgileTransformation Thank you #RiskBasedTesting

More Related Content

Using JIRA for Risk Based Testing - QASymphony Webinar

  • 1. September 25, 2015 Introduction to QASymphony for [INSERT COMPANY NAME] #AgileTransformation Agile Transformation: People, Process and Tools to Make Your Transformation Successful Using JIRA for Risk Based Testing #RiskBasedTesting
  • 2. #JIRATesting @RyanYackAttack Test Case Management | Exploratory Testing | Test Analytics | BDD and more... ABOUT QASYMPHONY The #1 Enterprise Testing Platform for
  • 3. #AgileTransformation OUR PRESENTER Ryan Yackel Senior Product Engineer QASymphony @ryanyackattack #RiskBasedTesting • Serves as the “customer voice” for all QASymphony products. • Comes to QASymphony from Macy’s, where he managed testing on large enterprise initiatives. • Certified scrum master from Scrum Alliance and holds a Bachelor of Arts degree from Covenant College. • Has a newborn baby, and doesn’t sleep a ton :-)
  • 4. The Test Everything Mentality Traditional vs Risk Based Testing How can JIRA be used to track risk Use Case of JIRA + qTest Products Recommended Add-ons to try out WHY ARE WE HERE? 4 #RiskBasedTesting
  • 5. #AgileTransformation SOFTWARE TESTING IN A RISKY BUSINESS 5 #RiskBasedTesting
  • 6. #AgileTransformation PRESSURE IS ON TEST TEAMS 6 #RiskBasedTesting What’s the plan to test X new requirements? What’s the plan to test this one requirement, that affects 50 applications? I guess I could create X tests, multiplied by the 100 configurations I guess I could create 1,000 tests, multiplied by the 50 configurations
  • 7. #AgileTransformation TIME IS OF THE ESSENCE 7 #RiskBasedTesting
  • 9. #AgileTransformation WHAT DO I TEST? 99 #RiskBasedTesting Do we need to test everything?....If not, what can I cut from my test plans? When should my testing be complete? How do I get team convinced that we are limited in test? Where am I going to get all my resources for this?
  • 10. #AgileTransformation TRADITIONAL vs RISK BASED TESTING 1010 #RiskBasedTesting TRADITIONALRISKBASED TESTING
  • 11. September 25, 2015 Introduction to QASymphony for [INSERT COMPANY NAME] #AgileTransformation Agile Transformation: People, Process and Tools to Make Your Transformation Successful 11 RISK BASED TESTING (RBT) #RiskBasedTesting
  • 12. #AgileTransformation WHAT IS RISK? 12 #RiskBasedTesting Risk = Damage * Frequency Risk = Impact * Probability of Failure A Risk is a problem or situation that has not happened yet and it may never happen in future as well. It is a possible issue or problem that may occur. DEFINITIONS Michael Scott poorly shoots a three pointer (Impact) * Everytime he gets the ball (Frequency) = EXTREMELY HIGH Risk
  • 13. #AgileTransformation TWO TYPES OF RISK ANALYSIS 1313 Statistical Risk Analysis a) Weighted Metrics b) Color Coding c) P1 - P4 Areas Heuristic Risk Analysis a) Consider more possibilities and interesting aspects of the problem b) Prioritize a List of Risks DistributedPopular NewComplex Down Stream Strategic #RiskBasedTesting (Daniele Gagliardi – eclipsecon Europe 2015)
  • 14. #AgileTransformation STATISTICAL RISK ANALYSIS 1414 #RiskBasedTesting (Sticky Minds Risk-Based Testing: Test Only What Matters) Marginal impact—1 Moderate impact— 2 Medium impact—3 High impact—4 Critical impact—5 A cosmetic error. The progress was disrupted with manageable extensions to the short-term schedule and cost; customers were not affected The progress was disrupted with large extensions to project schedule and cost; customers were not directly affected An application or transaction caused a significant disruption to production The problem caused all major and critical functioning of the application to fail. This resulted in a large loss of revenue. Marginal—1 Moderate—2 Medium—3 High—4 Critical—5 Low complex code written on low complicated system architectural design Moderate complex code written on moderate complicated system architectural design Medium complex codes written on medium complicated system architectural design There is a large amount of complex code written on a highly complicated system architectural design This suggests extreme complex code that has been written on top of on extremely complicated architectural design IMPACT PROBABILITY OF FAILURE
  • 15. #AgileTransformation STATISTICAL RISK ANALYSIS 1515 #RiskBasedTesting - P1 – Critical Impact: Must be tested; ideal candidate for automation (candidate for automated smoke testing) - P2 – High Impact: Should be tested - P3 – Medium Impact: Can be tested if budget and schedule permits - P4 – Low Impact: May not be tested; no impact on application and no need of automation (Sticky Minds Risk-Based Testing: Test Only What Matters)
  • 16. #AgileTransformation HEURISTIC RISK ANALYSIS: INSIDE OUT 1616 #RiskBasedTesting - Vulnerabilities: What weaknesses or possible failures are there in this component? - Threats: What inputs or situations could there be that might exploit a vulnerability and trigger a failure in this component? - Victims: Who or what would be impacted by potential failures and how bad would that be? Grab a developer and ask “What can go wrong here? and here? and …” :-) (James Bach Heuristic Risk-Based Testing)
  • 17. #AgileTransformation HEURISTIC RISK ANALYSIS: OUTSIDE IN 1717 #RiskBasedTesting (James Bach Heuristic Risk-Based Testing) E D F A B C Complex: anything disproportionately large, intricate, or convoluted. New: anything that has no history in the product. Changed: anything that has been tampered with or "improved". Upstream Dependency: anything whose failure will cause cascading failure in the rest of the system. Downstream Dependency: anything that is especially sensitive to failures in the rest of the system. Critical: anything whose failure could cause substantial damage. Risk Lists
  • 18. #AgileTransformation HEURISTIC RISK ANALYSIS: IMPACT 1818 #RiskBasedTesting Component Risk Matrix Component Risk Risk Heuristics Advanced Shipping Normal Complex, popular GUI Higher New, complex, third- party, critical Manifest Lower Legacy, changed, Point of Sale Lower Legacy, no change Purchasing Higher Upstream, Critical Receiving Normal Recent Failure Shipping Normal Downstream
  • 19. #AgileTransformation PLAN OF ATTACK 1919 #RiskBasedTesting RISK IDENTIFICATION RISK ANALYSIS TEST PLANNING TEST EXECUTION TEST RESULT P1 P2 P3 P4 P5 Exploratory Testing
  • 20. #AgileTransformation THE GOAL OF RBT 2020 #RiskBasedTesting GOAL Assurance of Quality CostofFailure CostofTesting (Sticky Minds Risk-Based Testing: Test Only What Matters) • Not Everything Needs to be Tested • Finding Balance the cost of testing and Failures with Quality Assurance • Taking Calculative Risk and Test
  • 21. September 25, 2015 Introduction to QASymphony for [INSERT COMPANY NAME] #AgileTransformation Agile Transformation: People, Process and Tools to Make Your Transformation Successful 21 USING JIRA TO MANAGE RISK #RiskBasedTesting
  • 22. #AgileTransformation FLEXIBLE AND CONFIGURABLE 2222 #RiskBasedTesting SOFTWARE DEVELOPMENT ISSUE TRACKING RISK MITIGATION SOFTWARE DEVELOPMENT MARKETING GO TO MARKET SOFTWARE DEVELOPMENT MARKETING GO TO MARKET PERSONAL TASK MANAGEMENT HR ON BOARDING
  • 23. #AgileTransformation CONFIGURING JIRA ISSUES 23 #RiskBasedTesting Option 1: Create New “Risk” Issue
  • 24. #AgileTransformation CONFIGURING JIRA ISSUES 24 #RiskBasedTesting Create Custom Statistical Fields a) Probability of Failure b) Impact c) Risk Analysis Custom Heuristic RISK Fields a) Risk Heuristics b) Heuristic Analysis Option 2: Custom Risk Fields
  • 25. #AgileTransformation RISK WITH JIRA ISSUES 25 #RiskBasedTesting Issue Search Analysis Statistical HeuristicCommon
  • 26. #AgileTransformation RISK WITH JIRA ISSUES 26 #RiskBasedTesting Board Analysis
  • 27. #AgileTransformation CATAGORIZE RISK FOR TEST 27 #RiskBasedTesting Organize Risks for Test
  • 28. #AgileTransformation PLAN AGAINST RISKS 28 #RiskBasedTesting Map High Priority Tests w/ High Risk
  • 29. #AgileTransformation EXECUTE YOUR AGAINST RISKS 29 #RiskBasedTesting Coordinate Line of Priority for Execution
  • 30. September 25, 2015 Introduction to QASymphony for [INSERT COMPANY NAME] #AgileTransformation Agile Transformation: People, Process and Tools to Make Your Transformation Successful 30 JIRA ADD-ONS #RiskBasedTesting
  • 33. #AgileTransformation SCRIPT RUNNER FOR JIRA 33 #RiskBasedTesting JIRA Misc Custom Fields ScriptRunner for JIRA
  • 34. September 25, 2015 Introduction to QASymphony for [INSERT COMPANY NAME] #AgileTransformation Agile Transformation: People, Process and Tools to Make Your Transformation Successful 34 WRAPPING IT UP #RiskBasedTesting
  • 35. #AgileTransformation PROS AND CONS 35 #RiskBasedTesting Management might think you are lazy Miscalculate or mis-appropriate weight to a certain risk Coordinating risk identifiers is a challenge The impossible could happen High severity defects are flushed out Low values tests are kicked to curb Critical areas are tested first Confidence is boosted Deadline anxiety is removed
  • 36. #AgileTransformation RESOURCES 36 #RiskBasedTesting JIRA Add-Ons • Risk Register - https://marketplace.atlassian.com/plugins/com.projectbalm.riskregister.riskregister-jira/server/overview • Visualizer for JIRA - https://marketplace.atlassian.com/plugins/vfjc/cloud/overview • ScriptRunner - https://marketplace.atlassian.com/plugins/com.onresolve.jira.groovy.groovyrunner/cloud/overview • JIRA MISC Custom Fields - https://marketplace.atlassian.com/plugins/com.innovalog.jmcf.jira-misc-custom- fields/server/overview References • Sticky Minds Risk-Based Testing: Test Only What Matters • Xoriant – An Approach for Risk Based Testing • James Bach - Heuristic Risk-Based Testing • Daniele Gagliardi – eclipsecon Europe 2015