Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Testing 1 - the Basics

•Download as PPT, PDF•
•
ArleneAndrews2
ArleneAndrews2

Testing is done throughout development to minimize risks. Testers evaluate the product, create test conditions, and identify potential issues to improve quality. Effective testing considers coverage of the product's structure, functions, data, interfaces, platform, and operations through accurate models and test procedures. Testers communicate risks and results so informed decisions can be made.

1 of 29
Explore! Software Testing In An Exploratory Style: Learning and Discoveries
Why Test? Minimizes the risk gap Outside perspective Edge cases User comfort Does it work Financial costs Build The Right Thing Does it meet the needs of a user? Build It Correctly Does it function as it should?
Why Test? Software is a relationship between people – who are both rational and emotional. Projects are completed, with both uncertainty and time pressures. We give it our best, but there will always be errors, inexperience, and carelessness. We focus on not misleading either colleagues or clients, knowingly or by neglect.
Why Test? Testing is an activity – and performance. Not a remainder. It discovers the status and threats of a product, so correct decisions can be made about it. Testers accept responsibility for their work, not overall product quality – but can contribute to it. Credible, cost-effective testing is the goal, and communication of anything that might interfere with it.
Costs?  NIST report estimates that business loses between $22 and $59 billion annually (in 2002 dollars) from non-functional products  Stecklein, J. et al. (2004) conducted a study for NASA type of products and found that if finding an error in requirements phase costs 1 unit then it increases so significantly that the same defect if undetected until operations phase can cost to fix anywhere from 29 units to more than 1,500 units. So if a unit costs $1000 then at requirements phase a defect would cost a $1,000 to fix while in the operations it can cost from $29,000 to $150,000 to fix.
A good tester is always learning how to Be comfortable with uncertainty and change Design tests Learn from other people To effectively use testing tools Envision what the product can be Identify potential bugs in the product Construct and share stories of the tests
The Testing Mentality  You are an agent first, but a curious one  And one who takes pride in their work.  You can provide information, and analyze the results. These skills allow you to be an effective, and efficient critic of a check, and explore options to write them better, or find a new and different way to check.
Standards and Models  IEEE Std 1012-2012 IEEE Standard for System and Software Verification and Validation  SEI Capability Maturity Model (CMM)  ISO SPICE  ISO 12207  European Cooperation for Space Standardization (ECSS)  GDPR
Criteria are chosen by both risk/safety level and organization or products. Testing is done under scrutiny, with uncertainty, in a timely, ethical manner.
Testing Activities
Testing is not just finding out what a specific activity does. You need to be able to evaluate and model the product - and question this model - then create the conditions where you can model it, and gain confidence, credibility, and motivation. This way, risks are communicated, so informed decisions can be made. If this is done early and often, the quality of the end product can be improved.
Testing is in your head! Specifications Product Report Coverage Use models (technical, domain, and product ones), think both deep and wide. And don't discount your instincts. Good testing doesn't just happen in physical space!
Testing throughout development is vital Inspection and analysis of user requirements Test system requirements and setup Tasks, schedules, test scope division of works Test check: inspection and analysis Test check: design and implementation Test execution and documentation
Check versus Suite  A test check examines one aspect of a system: for example, using various inputs on one page to test the reaction of the system.  This is testing a specific behavior.  A test suite, like its musical namesake, covers all important aspects of what needs to be tested.  This is testing the entire system.
Test Plans Include  Manual testing: A human goes in, and does the tests  Automatic testing: a program is made to run tests  The test plan will include such things as who is responsible, what is to be tested, how it should be tested, and milestones to meet
Common Information for Tests  Unique Test Case name and/or ID  Test Scenario and test summary or description  Prerequisites & setup  Test data & inputs  Execution Steps
Common Information for Tests  Expected behavior  Assumptions – Any assumptions that are made about the system or the test case.  Actual results  Status
Writing a good test check Test one thing only Make it exact and accurate Make it clear and easy to understand Make it small Make it independent No extra steps or words Make it traceable to requirements Make it repeatable Make it consistent, internally and over the test suite
Learn from your checks: they can be improved, they can be wrong, OR they can show a bug that isn't important. Checks show something can work, not that it will.
Elements of Testing Make accurate models of the product and the risks Consider what to cover Determine which procedures cover the product, and apply the oracles (quality checks) Configure and operate the testing system(s) Observe, evaluate and report the ACTUAL test results
___ coverage is how completely you have examined the product, with a model of ___ What aspects of the product were tested? What risks did you look for? What requirements have you tested for?
Test Factors Determine which of these test factors are the most important for each test. Model Specific Factors Specific procedure Test Incidental Factors
SFDIPOT (San Francisco Depot) Coverage Includes: Structure Platform Function Operations Data Time Interfaces
Product Elements Structural Coverage Test what it's made of Example: Print Testing Files associated with Printing Code modules that trigger printing Code statements and branches in the printing modules
Product Elements Functional Coverage Test what it does Example: Print Testing Print, page setup, and print preview Print range, print copies, zoom Print all, print current page, print specific page
Product Elements Data Coverage Test what it does it to Example: Print Testing Types of documents Item in document, size and structure Data about how to print (zoom, copy count)
Product Elements Interface Coverage Test how we interact with it Example: Print Testing Menus, windows, dialogs, and controls Launch and control interfaces Print to or from files USB printer interface protocol(s) Physical connections between printer & computer or network
Product Elements Platform Coverage Test what it depends on Example: Print Testing Printers, spoolers, network behavior Computers Operating Systems Printer drivers
Product Elements Operations Coverage Test how it's actually used Example: Print Testing Use defaults Use realistic environments Use realistic scenarios Use complex flow

More Related Content

Testing 1 - the Basics

  • 1. Explore! Software Testing In An Exploratory Style: Learning and Discoveries
  • 2. Why Test? Minimizes the risk gap Outside perspective Edge cases User comfort Does it work Financial costs Build The Right Thing Does it meet the needs of a user? Build It Correctly Does it function as it should?
  • 3. Why Test? Software is a relationship between people – who are both rational and emotional. Projects are completed, with both uncertainty and time pressures. We give it our best, but there will always be errors, inexperience, and carelessness. We focus on not misleading either colleagues or clients, knowingly or by neglect.
  • 4. Why Test? Testing is an activity – and performance. Not a remainder. It discovers the status and threats of a product, so correct decisions can be made about it. Testers accept responsibility for their work, not overall product quality – but can contribute to it. Credible, cost-effective testing is the goal, and communication of anything that might interfere with it.
  • 5. Costs?  NIST report estimates that business loses between $22 and $59 billion annually (in 2002 dollars) from non-functional products  Stecklein, J. et al. (2004) conducted a study for NASA type of products and found that if finding an error in requirements phase costs 1 unit then it increases so significantly that the same defect if undetected until operations phase can cost to fix anywhere from 29 units to more than 1,500 units. So if a unit costs $1000 then at requirements phase a defect would cost a $1,000 to fix while in the operations it can cost from $29,000 to $150,000 to fix.
  • 6. A good tester is always learning how to Be comfortable with uncertainty and change Design tests Learn from other people To effectively use testing tools Envision what the product can be Identify potential bugs in the product Construct and share stories of the tests
  • 7. The Testing Mentality  You are an agent first, but a curious one  And one who takes pride in their work.  You can provide information, and analyze the results. These skills allow you to be an effective, and efficient critic of a check, and explore options to write them better, or find a new and different way to check.
  • 8. Standards and Models  IEEE Std 1012-2012 IEEE Standard for System and Software Verification and Validation  SEI Capability Maturity Model (CMM)  ISO SPICE  ISO 12207  European Cooperation for Space Standardization (ECSS)  GDPR
  • 9. Criteria are chosen by both risk/safety level and organization or products. Testing is done under scrutiny, with uncertainty, in a timely, ethical manner.
  • 11. Testing is not just finding out what a specific activity does. You need to be able to evaluate and model the product - and question this model - then create the conditions where you can model it, and gain confidence, credibility, and motivation. This way, risks are communicated, so informed decisions can be made. If this is done early and often, the quality of the end product can be improved.
  • 12. Testing is in your head! Specifications Product Report Coverage Use models (technical, domain, and product ones), think both deep and wide. And don't discount your instincts. Good testing doesn't just happen in physical space!
  • 13. Testing throughout development is vital Inspection and analysis of user requirements Test system requirements and setup Tasks, schedules, test scope division of works Test check: inspection and analysis Test check: design and implementation Test execution and documentation
  • 14. Check versus Suite  A test check examines one aspect of a system: for example, using various inputs on one page to test the reaction of the system.  This is testing a specific behavior.  A test suite, like its musical namesake, covers all important aspects of what needs to be tested.  This is testing the entire system.
  • 15. Test Plans Include  Manual testing: A human goes in, and does the tests  Automatic testing: a program is made to run tests  The test plan will include such things as who is responsible, what is to be tested, how it should be tested, and milestones to meet
  • 16. Common Information for Tests  Unique Test Case name and/or ID  Test Scenario and test summary or description  Prerequisites & setup  Test data & inputs  Execution Steps
  • 17. Common Information for Tests  Expected behavior  Assumptions – Any assumptions that are made about the system or the test case.  Actual results  Status
  • 18. Writing a good test check Test one thing only Make it exact and accurate Make it clear and easy to understand Make it small Make it independent No extra steps or words Make it traceable to requirements Make it repeatable Make it consistent, internally and over the test suite
  • 19. Learn from your checks: they can be improved, they can be wrong, OR they can show a bug that isn't important. Checks show something can work, not that it will.
  • 20. Elements of Testing Make accurate models of the product and the risks Consider what to cover Determine which procedures cover the product, and apply the oracles (quality checks) Configure and operate the testing system(s) Observe, evaluate and report the ACTUAL test results
  • 21. ___ coverage is how completely you have examined the product, with a model of ___ What aspects of the product were tested? What risks did you look for? What requirements have you tested for?
  • 22. Test Factors Determine which of these test factors are the most important for each test. Model Specific Factors Specific procedure Test Incidental Factors
  • 23. SFDIPOT (San Francisco Depot) Coverage Includes: Structure Platform Function Operations Data Time Interfaces
  • 24. Product Elements Structural Coverage Test what it's made of Example: Print Testing Files associated with Printing Code modules that trigger printing Code statements and branches in the printing modules
  • 25. Product Elements Functional Coverage Test what it does Example: Print Testing Print, page setup, and print preview Print range, print copies, zoom Print all, print current page, print specific page
  • 26. Product Elements Data Coverage Test what it does it to Example: Print Testing Types of documents Item in document, size and structure Data about how to print (zoom, copy count)
  • 27. Product Elements Interface Coverage Test how we interact with it Example: Print Testing Menus, windows, dialogs, and controls Launch and control interfaces Print to or from files USB printer interface protocol(s) Physical connections between printer & computer or network
  • 28. Product Elements Platform Coverage Test what it depends on Example: Print Testing Printers, spoolers, network behavior Computers Operating Systems Printer drivers
  • 29. Product Elements Operations Coverage Test how it's actually used Example: Print Testing Use defaults Use realistic environments Use realistic scenarios Use complex flow