Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 37 views

    Black-Box (Functional) White-Box (Structural) : Testing Methods

    Uploaded by

    postscript
    The document discusses various software testing methods, including black-box testing techniques like equivalence partitioning and boundary value analysis, as well as white-box testing techniques like statement coverage, decision coverage, condition coverage, and dataflow coverage. It defines these terms and provides examples to illustrate how to achieve different levels of coverage when testing code.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PS, PDF, TXT or read online from Scribd

    Black-Box (Functional) White-Box (Structural) : Testing Methods

    Uploaded by

    postscript
    37 views16 pages
    The document discusses various software testing methods, including black-box testing techniques like equivalence partitioning and boundary value analysis, as well as white-box testing techniques like statement coverage, decision coverage, condition coverage, and dataflow coverage. It defines these terms and provides examples to illustrate how to achieve different levels of coverage when testing code.

    Original Title

    bTestingMethods

    Copyright

    © Attribution Non-Commercial (BY-NC)

    Available Formats

    PS, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document discusses various software testing methods, including black-box testing techniques like equivalence partitioning and boundary value analysis, as well as white-box testing techniques like statement coverage, decision coverage, condition coverage, and dataflow coverage. It defines these terms and provides examples to illustrate how to achieve different levels of coverage when testing code.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PS, PDF, TXT or read online from Scribd
    Download as ps, pdf, or txt
    37 views16 pages

    Black-Box (Functional) White-Box (Structural) : Testing Methods

    Uploaded by

    postscript
    The document discusses various software testing methods, including black-box testing techniques like equivalence partitioning and boundary value analysis, as well as white-box testing techniques like statement coverage, decision coverage, condition coverage, and dataflow coverage. It defines these terms and provides examples to illustrate how to achieve different levels of coverage when testing code.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PS, PDF, TXT or read online from Scribd
    Download as ps, pdf, or txt
    of 16

    Testing Methods

     Black-box (functional)
     White-box (structural)
    – Background
    – Statement coverage
    – Decision coverage
    – (Short circuit) condition coverage
    – Dataflow coverage
     Coverage terminology
     Coverage tools

    1
    Black-Box Testing

     Also called “functional testing”


     Testing based on requirements
     Major methods:
    – Equivalence partitioning on inputs
    – Equivalence partitioning on outputs
    – Boundary value analysis
    – Extreme value analysis
    – Syntax testing

    2
    Equivalence Partitioning

     On inputs:
    – Identify input variables
    e.g. side lengths, number of sides
    – Identify valid and invalid value sets for each variable
    e.g. fl j l > 0g valid; fl j l  0g invalid; fl j l not a numberg invalid
    – Build complete test cases from those sets
    e.g. (3, 4, 5), (-1, 3, 3), (3, a, 3), (3), (3, 4, 5, 6, 7)
     On outputs:
    – Identify output events, output variables, (sets of) variable values
    e.g. triangle types, angle types
    – Build test cases that generate each one
    e.g. (3, 3, 3), (2, 2, 3), (3, 4, 5), (2, 10, 2)

    3
    Boundary Value Analysis

     Find some n such that as n either increases or decreases,


    we go from one EC to another (e.g. valid to invalid)
     Identify values of n that are
    – Close to boundary on each side
    – On boundary (if possible)
    EC 1 EC 2  Generate test cases for each
    n
     n can be integer or real; input value, output value, number
    of repetitions, etc.
     For number representing month: 0, 1, 12, 13
     For TRIANGLE example:
    – On outputs: (3, 3, 2.99), (3, 3, 3), (3, 3, 3.01), (3, 4,
    4.99), (3, 4, 5), (3, 4, 5.01), . . .
    – Number of occurrences: (3, 3), (3, 3, 3), (3, 3, 3, 3), . . .

    4
    Extreme Value Analysis

     Find an n again
     Choose extremely large/small values for n
     Generate test cases for each
     n can be integer or real
     n can be input value, output value, number of repetitions/occurrences,
    etc.
     For TRIANGLE:
    – On inputs/outputs: (0.000004, 0.000002, 0.000003), (43728947,
    43928410, 54386880)
    – On number of repetitions: () (empty test case), (3, 3, 3, 3, 3, 3, 3,
    3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3)

    5
    Syntax Testing

    -f
    put -g filename
    -m filename

    get filename

     Basic steps:
    – Identify target language and make explicit if not already
    – Define syntax formally with state machine
    – Write test cases to cover:
    – All transitions
    – All incomplete inputs
    – Each change of a valid token to an invalid one
     For above grammar: (put -f foo), (put -g foo), (put -m foo bar), (get foo), (put -f), (get),
    (put -f -g), . . .

    6
    White-Box Testing

     Also called “structural testing”


     Basic idea: if 2 test cases execute code differently, cannot
    be test-equivalent
    while (p!=null) {
    if (p->n==y) {
    return p;  What criteria do we use to decide “executing the code dif-
    }
    p = p->next;
    ferently”?
    } ...
    – “Coverage criteria”
     Hierarchies of coverage criteria
     Levels of coverage

    7
    Statement Coverage

     A test suite achieves (100%) statement coverage on code if:


    For every statement in the code, there is at least one test case which causes the program to
    execute that statement
     Example: consider the insurance code

    premium = 500;
    if ((age<25) && (sex==male) && (!married)) {
    premium += 1500;
    } else {
    if (married || (sex==female)) {
    premium -= 200;
    }
    if ((age>45) && (age<65)) {
    premium -= 100;
    }
    }

    (from Software Testing in the Real World, Edward Kit)

    8
    Statement Coverage

     In the insurance code:


    – 3 “if” statements, 4 assignment statements
     Two test cases can execute all statements:
    – One with ((age<25) && (sex==male) && (!married)) true
    – One with ((age<25) && (sex==male) && (!married)) false,
    but (married || (sex==female))
    and ((age>45) && (age<65)) both true
     Statement coverage also called block coverage
    – (to be precise: block coverage is just very closely related)

    9
    Decision Coverage

     Decision: entire conditional expression after an if, while, etc.


     A test suite achieves (100%) decision coverage on code if:
    – For every decision in the code, there is at least one test case that evaluates the decision
    to true
    – For every decision in the code, there is at least one test case that evaluates the decision
    to false
     Also called “branch coverage”
     Three test cases achieve 100% decision coverage on insurance code:
    – One with ((age<25) && (sex==male) && (!married)) true
    – One with (age<25) false, but (married || (sex==female))
    and ((age>45) && (age<65)) both true
    – One with (age<25) false, but (married || (sex==female))
    and ((age>45) && (age<65)) both false

    10
    (Short-Circuit) Condition Coverage

     Condition: atomic expression with no logical operators, within decision


    – Logical operator: &&, ||, !, etc.
    – Every decision contains one or more conditions
    – e.g. ((age<25) && (sex==male) && (!married)) : three conditions
     A test suite achieves (100%) condition coverage on code if:
    – For every condition in the code, there is at least one test case that evaluates the condi-
    tion to true
    – For every condition in the code, there is at least one test case that evaluates the condi-
    tion to false
     For many languages we need to assume short-circuit evaluation: e.g.
    when evaluating B && C, if B false, don’t evaluate C
     Use phrase “short-circuit (SC) condition coverage” for this
     Need at least 5 test cases for SC condition coverage on insurance code

    11
    Comparing Coverage Criteria

     Statement, decision, SC condition = “coverage criteria”


     If a test suite achieves decision coverage, it achieves statement coverage
    – (There are unimportant, rare exceptions)
    – Decision coverage “stronger than” statement coverage
     If a test suite achieves SC condition coverage, it achieves decision coverage
    – SC condition coverage “stronger than” decision coverage
     US Military Standard DO-178B defines criteria for testing software at different safety lev-
    els
    – Highest level: needs SC condition coverage to satisfy

    12
    Dataflow Coverage

     Definition statement (“def”): statement which assigns a value to a variable; e.g.


    – “x = 3;” is a definition of x
    – “add(&tree, i);” is a definition of tree
     Use statement: statement which uses the value of a variable
    – Predicate use (p-use): statement which uses value in a decision; e.g.
    “if (x == y) {...}” is a predicate use of x and y
    – Calculation use (c-use): all other uses; e.g.
    – “x = y+3;” is a calculation use of y
    – “add(&tree, i);” is a calculation use of i
     Def-use pair: def of a variable x + a later use (in same method) of x, that can be reached
    from the def by a path q with no other defs of x in-between
    – Parameter decls considered “virtual defs” here

    13
    Def-Use Coverage

     A test suite achieves (100%) def-use coverage on code if:


    – For every def-use pair, there is at least one test case that executes the path q between
    the def and the use
     C-use coverage: def-use coverage for c-uses
     P-use coverage: def-use coverage for p-uses

     Statement, decision, condition coverage: controlflow criteria


     Def-use, C-use, P-use coverage: dataflow criteria
     Some studies have shown that test suites achieving dataflow criteria are more effective than
    those achieving controlflow criteria

    14
    Coverage Terminology

     A test suite achieves (90%) statement coverage on code if 90% of the statements are exe-
    cuted
    – Similarly 90%, 80%, etc., decision coverage, def-use coverage, etc.
     Some statements cannot be executed (“dead code”)
    – Consider these coverage elements infeasible for statement coverage
    – Similarly, infeasible decisions, conditions, etc.
     Generally only count feasible coverage elements
    – Thus say “100% feasible statement coverage”, etc.
     Minimal test suite for a criterion: one that achieves the criterion in the smallest number of
    test cases

    15
    Coverage Tools

     Step 1: run program that either


    – Compiles your software under test
    (SUT) in a special way
    – Adds statements to your SUT and
    compiles it
    – Instruments your SUT bytecode
     Step 2: run your compiled program
    (your SUT)
     Step 3: run coverage reporting pro-
    gram
    – Can tell you which statements,
    etc. in your SUT were executed
     Most tools measure only statement
    coverage
     For C: gcc / gcov
     For Java: Cobertura

    16

    You might also like