1

Software Testing is a method to check whether the actual

software product matches expected requirements and to ensure
that software product is Defect free.

2
How do you test a system?

 Input test data to the system.

 Observe the output:
Check if the system behaved as expected.

3
How usually you test something?
How do you test a system?

 Ifthe program does not behave as

expected:
note the conditions under which it failed.
later debug and correct.

5
 Errors and Failures
 A failure is a manifestation of an error.
 presence of an error may not lead to a failure.

6
Test cases and Test suite
A TEST CASE is a set of actions executed to
verify a particular feature or functionality of
your software application. A Test Case
contains test steps, test data, precondition,
postcondition

A TEST SUITE is a collection of test cases

7
1. Check system behavior when valid email id and password is entered.
2. Check system behavior when invalid email id and valid password is
entered.
3. Check system behavior when valid email id and invalid password is
entered.
4. Check system behavior when invalid email id and invalid password is
entered.
5. Check system behavior when email id and password are left blank and
Sign in entered.
6. Check Forgot your password is working as expected
7. Check system behavior when valid/invalid phone number and
password is entered.
1. Check system behavior when "Keep me signed" is checked

8
9
Test cases and Test suite

A test case is a triplet [I,S,O]:

I is the data to be input to the
system,
S is the state of the system at
which the data is input,
O is the expected output from
the system.
10
Test Case Document Template
Examples

11
12
13
14
15
Verification versus Validation
 Verificationis the process of
determining:
 whether output of one phase of development
conforms to its previous phase.
 Validation is the process of determining
whether a fully developed system conforms
to its SRS document.
16
Verification versus Validation

Aim of Verification:
phase containment of errors
Aim of validation:
final product is error free.
17
Verification versus Validation

Verification:
are we doing right?
Validation:
have we done right?
18
Design of Test Cases
Exhaustive testing of any system is
impractical:
input data domain is extremely large.
Design an optimal test suite:
of reasonable size
to uncover as many errors as possible.

19
Design of Test Cases
 If test cases are selected randomly:
 many test cases do not contribute to the
significance of the test suite,
 do not detect errors not already detected by
other test cases in the suite.
 The number of test cases in a randomly
selected test suite:
 not an indication of the effectiveness of the
testing.

20
TYPES/APPROACHES
OF
SOFTWARE TESTING

21
 Two main approaches to design test cases:
 Black-box approach
 White-box (or glass-box) approach

22
Black Box Testing

23
Black-box Testing
 Testcases are designed using only
functional specification of the software:
 without any knowledge of the internal
structure of the software.
 Forthis reason, black-box testing is also
known as functional testing.

24
25
White-box Testing
Designing white-box test cases:
requires knowledge about the internal
structure of software.
white-box testing is also called
structural testing.

26
27
Black-box Testing
 There are essentially two main approaches to
design black box test cases:
 Equivalence class partitioning
 Boundary value analysis

28
29
Equivalence Class
Partitioning
 Input values to a program are
partitioned into equivalence classes
.
 Partitioning is done such that:
program behaves in similar ways to
every input value belonging to an
equivalence class.
30
Why define equivalence classes?

 Test the code with just one representative value from each equivalence class:

 as good as testing using any other values from the equivalence

classes.

31
Equivalence Class Partitioning
 How do you determine the equivalence classes?
 examine the input data.
 few general guidelines for determining the equivalence classes can
be given

32
Equivalence Class Partitioning
 Ifthe input data to the program is
specified by a range of values:
e.g.numbers between 1 to 5000.
one valid and two invalid equivalence
classes are defined.

1 5000

33
Equivalence Class Partitioning

 If input is an enumerated set of values:

e.g.{a,b,c}
one equivalence class for valid input
values
another equivalence class for invalid input
values should be defined.

34
Example (cont.)
 There are three equivalence classes:

the set of negative integers,

set of integers in the range of 1 and 5000,

integers larger than 5000.

35
Example (cont.)
The test suite must include:
representatives from each of the three
equivalence classes:
a possible test suite can be:
{-5,500,6000}.
1 5000

36
Boundary Value Analysis
 Some typical programming errors occur:

atboundaries of equivalence classes

might be purely due to psychological
factors.
 Programmers often fail to see:
special processing required at the
boundaries of equivalence classes.

37
Boundary Value
Analysis
 Programmers may improperly use < instead of <=
 Boundary value analysis:
 select test cases at the boundaries of different equivalence classes.

38
Example
For a function that computes the
square root of an integer in the
range of 1 and 5000:
testcases must include the values:
{0,1,5000,5001}.
1 5000

39
White-Box Testing
 There exist several popular white-box
testing methodologies:
 Statement coverage
 branch coverage
 path coverage
 condition coverage
 mutation testing
 data flow-based testing
40
Statement Coverage
Statement coverage
methodology:
design test cases so that
every statement in a program
is executed at least once.

41
Statement Coverage
 The principal idea:
 unless a statement is executed,
 we have no way of knowing if an error exists in that statement.

42
 Example
 int f1(int x, int y){
 1 while (x != y){
 2 if (x>y) then
3 x=x-y;
 4 else y=y-x;
5 }
 6 return x; }

Test suit is: (3,3), (4,3) and (3,4) which will do

100% statement coverage 43
Branch Coverage
Test cases are designed
such that:
differentbranch
conditions
given true and false
values in turn.
44
Branch Coverage
Branch testing guarantees
statement coverage:
a stronger testing
compared to the
statement coverage-based
testing.
45
Example
 intf1(int x,int y){
 1 while (x != y){
 2 if (x>y) then
3 x=x-y;
 4 else y=y-x;
5 }
 6 return x; }
46
Example
 Test cases for branch coverage can be:
 {(x=3,y=3),(x=3,y=2), (x=4,y=3), (x=3,y=4)}

47
Condition Coverage
 Test cases are designed such that:
 each component of a composite conditional expression is
executed at least once.
 given both true and false values.

48
Example
 Consider the conditional expression
 ((c1.and.c2).or.c3):
 Each of c1, c2, and c3 are exercised at least once,
 i.e. given true and false values.

49
Branch testing
 Condition testing
 stronger testing than branch testing:
 Branch testing
 stronger than statement coverage testing.

50
Condition coverage

Consider a boolean
expression having n
components:
for condition coverage
n
we require 2 test cases.
(n is no. of conditions)
51
Path Coverage

Design test cases such

that:
all linearly independent
paths in the program are
executed at least once.
52
Linearly independent paths

Defined in terms of
controlflow graph
(CFG) of a program.

53
Path coverage-based testing
 To understand the path coverage-based testing:
 we need to learn how to draw control flow graph of a program.

54
Control flow graph (CFG)

 A control flow graph (CFG) describes:

 the sequence in which different instructions
of a program get executed.
 the way control flows through the program.

55
How to draw Control flow graph?
 Number all the statements of a program.
 Numbered statements:
 represent nodes of the control flow graph.

56
How to draw Control flow
graph?
 Sequence:
1
 1 a=5;
 2 b=a*b-1;
2

57
How to draw Control flow
graph?
 Selection:
 1 if(a>b) then 1
 2 c=3;
 3 else c=5;
2 3
 4 c=c*c; 4

58
How to draw Control flow
graph?
1
 Iteration:
 1 while(a>b){
 2 b=b*a; 2
 3 b=b-1;}
 4 c=b+d;
3
4

59
 Example
 intf1(int x,int y){
 1 while (x != y){
 2 if (x>y) then
3 x=x-y;
 4 else y=y-x;
5 }
 6 return x; }

60
Example Control Flow Graph

1
2
3 4
5
6
Path
 A path through a program:
 a node and edge sequence from the starting node to a terminal
node of the control flow graph.
 There may be several terminal nodes for program.

62
Independent path
 Any path through the program:
 introducing at least one new node:
 that is not included in any other independent paths.

63
Independent path
 It is straight forward:
to identify linearly independent
paths of simple programs.
 For complicated programs:
it is not so easy to determine the
number of independent paths.
64
McCabe's cyclomatic metric

 An upper bound:
for the number of linearly
independent paths of a program
 Provides a practical way of
determining:
the maximum number of
linearly independent paths in a
program.
65
McCabe's cyclomatic metric

 Given a control flow graph G,

cyclomatic complexity V(G):
 V(G)= E-N+2
 N is the number of nodes in G
 E is the number of edges in G

66
Example Control Flow Graph
1
2
3 4
5
6

67
Example
Cyclomatic complexity
=
7-6+2 = 3.

68
Cyclomatic complexity

 Anotherway of computing
cyclomatic complexity:
inspect control flow graph
determine number of bounded
areas in the graph
 V(G)= Total number of
bounded areas + 1
69
Bounded area
Any region enclosed
by a nodes and edge
sequence.

70
Example Control Flow Graph
1
2
3 4
5
6

71
Example
 From a visual examination of the CFG:
 the number of bounded areas is 2.
 cyclomatic complexity = 2+1=3.

72
 IF A = 10 THEN
 IF B > C
 THEN A = B
 ELSE A = C
 ENDIF
 ENDIF
 Print A Print B Print C

73
Cyclomatic complexity

 McCabe's metric provides:

a quantitative measure of
testing difficulty and the
ultimate reliability
 Intuitively,
number of bounded areas
increases with the number of
decision nodes and loops.
74
Cyclomatic complexity

 Thefirst method of computing

V(G) is amenable to
automation:
you can write a program which
determines the number of nodes
and edges of a graph
applies the formula to find
V(G).
75
Cyclomatic complexity

The cyclomatic complexity

of a program provides:
a lower bound on the number
of test cases to be designed
to guarantee coverage of all
linearly independent paths.
76
Cyclomatic complexity

 Defines the number of independent paths in a program.

 Provides a lower bound:
 for the number of test cases for path coverage.

77
Cyclomatic complexity

 Knowing the number of test

cases required:
does not make it any easier to
derive the test cases,
only gives an indication of the
minimum number of test cases
required.
78
Complexity No. Corresponding Meaning of V(G)

1-10 1) Well-written code,

2) Testability is high,
3) Cost / effort to maintain is low

10-20 1) Moderately complex code,

2) Testability is medium,
3) Cost / effort to maintain is medium.

20-40 1) Very complex code,

2) Testability is low,
3) Cost / effort to maintain is high.

> 40 1) Not testable,

2) Any amount of money / effort to maintain may
not be enough.
79
Derivation of Test Cases

Let us discuss the steps:

to derive path coverage-
based test cases of a
program.

80
Derivation of Test Cases

 Draw control flow graph.

 Determine V(G).
 Determine the set of linearly
independent paths.
 Prepare test cases:
to force execution along each
path.
81
Example
 int f1(int x,int y){
 1 while (x != y){
 2 if (x>y) then
3 x=x-y;
 4 else y=y-x;
5 }
 6 return x; }
82
 Derivation of Test Cases

1
2
3 4
5
6
 Number of independent paths: 3
 1,6 test case (x=1, y=1)
 1,2,3,5,1,6 test case(x=1, y=2)
 1,2,4,5,1,6 test case(x=2, y=1) 83
Exercise: 2

IF A = 354
THEN IF B >C
THEN A = B
ELSE A= C
ENDIF
ENDIF
Print A
84
Exercise:3

i = 0;
while (i<n-1)
do j = i + 1;
while (j<n)
do
if A[i]<A[j] then
swap (A[i], A[j]);
end do;
i=i+1;
85
end do;
 Data Flow-Based Testing

 Data flow testing is a family of test strategies based on selecting

paths through the program's control flow in order to explore
sequences of events related to the status of variables or data
objects.
 Dataflow Testing focuses on the points at which variables receive
values and the points at which these values are used.

86
 Advantages of Data Flow Testing:

 Data Flow testing helps us to pinpoint any of the

following issues:
1. A variable that is declared but never used within
the program.
2. A variable that is used but never declared.
3. A variable that is defined multiple times before it
is used.
4. Deallocating a variable before it is used.

87
Mutation Testing

88
 Definition

 Mutation Testing is a type of software testing in which

certain statements of the source code are
changed/mutated to check if the test cases are able to find
errors in source code.
 The goal of Mutation Testing is ensuring the quality
of test cases in terms of robustness that it should fail
the mutated source code.

89
90
Mutation Testing

Each time the program is

changed,
itis called a mutated
program
the change is called a
mutant.
91
Mutation Testing

The primitive changes can be:

alteringan arithmetic operator,
changing the value of a
constant,
changing a data type, etc.

92
Mutation Testing

A major disadvantage of
mutation testing:
computationally very expensive,
a large number of possible
mutants can be generated.

93