SOFTWARE TESTING

• Software Testing and Maintenance-UNIT 4

• Software project Management-UNIT 5-5.1 AND 5.3
• Software project Management

TOPICS • Software Risk Management

 • 1. Software Engineering: A practitioner’s Approach,
Roger S Pressman, Seventh Edition, McGrawHill
BOOKS FOR International Edition, 2010.
REFERENCE • Software Engineering, Ian Sommerville, Eighth Edition,
Pearson Education, 2017.
 • What?

SOFTWARE • Why?
TESTING • When?
• Who?
 What is Software Testing?
• To find out the errors in the software
• How?

Testing Strategy- But Why?

• Testing begins “in the small” and progresses “to the large.”
• Give the answer to the Who and when

Work Product?
• Test Specification documents
 VERIFICATION AND VALIDATION

• Testing is also referred as verification and validation

• Verification: “Are we building the product right?”
• Validation: “Are we building the right product?”
• Software developer
• Independent Testing Team
TESTING STRATEGY
 Unit Testing- Integration
FOUR STEPS FOR coverage of each
path
Testing- Test cases
for Input/output
TESTING

Validation Testing- System Testing-

Requirements system performance
UNIT TESTING
CATEGORIES

Interface • test data flow across the component

Local Data • Analysis of Local data structure and impact on

Structures Global Variable

Boundary • test the boundary Values

Conditions
• Identify due to erroneous computations,
Independent Paths incorrect comparisons, or improper control flow.

Error-Handling • To test the error handling paths

Paths
UNIT TESTING PROCEDURE

Stub- Invoked by the component to be tested

Drivers- “main program” that accepts test case data and

invoked the component to be tested
 • Need of Integration Testing?
• -Interfacing

• systematic technique for constructing the software

architecture while at the same time conducting tests to
uncover errors associated with interfacing.
INTEGRATION • How?
TESTING • Big Bang Approach
• Incremental approach-
• Top Down Approach
• Bottom Up Approach
TOP-DOWN APPROACH

• either a depth-first or breadth-first manner

• Steps to follow
• main control module is used as a test driver
and stubs are substituted for all components
directly subordinate
• subordinate stubs are replaced one at a time
with actual components
• Test is conducted
• On completion, another stub is replaced
• Regression testing
LIMITATION WITH TOP-DOWN APPROACH

Solutions
• delay many tests until
stubs are replaced
Stubs are used in place with actual modules,
of subordinates of main • develop stubs that
perform limited
module functions
• integrate the software
from the bottom of
the hierarchy upward.
BOTTOM-UP INTEGRATION

Need for stubs is eliminated

Procedure
• Low-level components are combined into clusters
• A driver is written to coordinate test case input and
output.
• The cluster is tested.
• Drivers are removed and clusters are combined
moving upward
REGRESSION TESTING

Re-execution of some subset of tests that have already been

conducted to ensure that changes have not propagated unintended
side effects.

The Regression test suit includes these categories:

• A representative sample of tests that will exercise all software functions.
• Additional tests that focus on software functions that can be affected by
the change.
• Tests that focus on the software components that have been changed.
SMOKE TESTING

A build includes all data files, libraries, reusable modules, and engineered
components that are required to implement one or more product
functions.

tests is designed to expose errors that will keep the build from properly
performing its function.

The build is integrated with other builds, and the entire product is smoke
tested daily.
 VALIDATION TESTING
The function or performance characteristic conforms to specification and is accepted deviation from
specification is uncovered and a deficiency list is created.

Configuration Alpha Testing Beta Testing Acceptance Testing

Review
to ensure that all elements The alpha test is The beta test is conducted Customer performs a
of the software conducted at the at one or more end-user series of specific tests
configuration have been developer’s site by a sites
properly developed, are representative group of
cataloged, and have the end users.
necessary detail.
 RECOVERY TESTING SECURITY TESTING STRESS TESTING

SYSTEM TESTING

PERFORMANCE TESTING DEPLOYMENT TESTING

DEBUGGING PROCESS

• When a test case uncovers an error, debugging is the

process that results in the removal of the error.
• The debugging process will usually have one of two
outcomes:
• the cause will be found and corrected
• the cause will not be found.
DEBUGGING STRATEGIES

manually searching through stack-traces, memory-dumps, log

Brute Force files, and so on, for traces of the error.
Extra output statements, in addition to break points, are often

backtracks through the execution path, looking for the

Backtracking cause.

Hypotheses constructed as to why the bug has occurred.

Cause Elimination The code can either be directly examined for the bug, or
data to test the hypothesis can be constructed.

Automated Debugging Automated Debugging tool Available

Previous versions of the software are examined until a

Bisect version which does not have the error is located.
SOFTWARE TEST CASES DESIGN
 • Test-case design focuses on a set of techniques for the
SOFTWARE creation of test cases that meet overall testing
TEST CASES objectives and the testing strategies.

DESIGN • Goal- series of test cases that have a high likelihood of

finding errors
TESTABILITY

• Operability. “The better it works, the more efficiently it can be tested.”

• Observability. “What you see is what you test.”
• Controllability. “The better we can control the software, the more the testing can be automated and
optimized.”
• Decomposability. “quickly isolate problems and perform smarter retesting.”
• Simplicity. “The less there is to test, the more quickly we can test it.
• functional
• structural
• code
• Stability. “The fewer the changes, the fewer the disruptions to testing’
• Understandability. “The more information we have, the smarter we will test.”
INTERNAL AND EXTERNAL VIEWS OF TESTING

• White box testing

examines some fundamental aspect of a system with little regard for the internal logical structure of the
software.

• Black box testing

Logical paths through the software and collaborations between components are tested
 • glass-box testing
• all independent paths within a module have been
exercised
• exercise all logical decisions on their true and false
WHITE BOX sides
TESTING
• Execute all loops at their boundaries and within their
operational bounds
• Exercise internal data structures to ensure their
validity
BASIS PATH TESTING

• enables the test-case designer to derive a logical

complexity measure of a procedural design and use this
measure as a guide for defining a basis set of execution
paths.
• Test cases derived to exercise the basis set are
guaranteed to execute every statement in the program
at least one time during testing.
• Flow Graph Notation
CONTROL STRUCTURES
COMPOUND CONDITION STATEMENT

• Predicate Node
• Region
INDEPENDENT PROGRAM PATHS

• An independent path is any path through the

program that introduces at least one new set of
processing statements or a new condition.

Path 1: 1-11

Path 2: 1-2-3-4-5-10-1-11

Path 3: 1-2-3-6-8-9-10-1-11

Path 4: 1-2-3-6-7-9-10-1-11

1-2-3-4-5-10-1-2-3-6-8-9-10-1-11 ?????
How do you know how many paths to look for?
 • software metric that provides a quantitative measure
CYCLOMATIC of the logical complexity of a program.

COMPLEXITY • provides an upper bound for the number of tests

WAYS TO CALCULATE THE CYLCOMATIC
COMPLEXITY
• The number of regions of the flow graph corresponds to the cyclomatic complexity.
• Cyclomatic complexity V(G) for a flow graph G is defined as
V(G) = E – N + 2
where E is the number of flow graph edges and N is the number of flow graph nodes.

• Cyclomatic complexity V(G) for a flow graph G is also defined as

V(G) = P + 1
where P is the number of predicate nodes contained in the flow graph G.
EXAMPLE

• The flow graph has four regions.

• V(G) = 11 edges - 9 nodes + 2 = 4.
• V(G) = 3 predicate nodes + 1 = 4.
CYCLOMATIC COMPLEXITY?

If (a<b)
F1 () ;
else
If (a<c)
F2 () ;
else
F3 () ;
CYCLOMATIC COMPLEXITY?

If (a<b)
F1 () ;
else
If (a<c)
F2 () ;
else
F3 () ;
1. begin int x, y, power;
2. float z;
3. input(x, y);
4. if(y<0)
5. power = -y;
6. else power = y;
7. z=1;
8. while(power!=0)
9. { z=z*x;
10. power=power-1;
11. } if(y<0)
12. z=1/z;
13. output(z);
14. end
int main()
{
1 int n, index;
2 cout << "Enter a number: " <> n;
3 index = 2;
4 while (index <= n - 1)
5 {
6 if (n % index == 0)
7 {
8 cout << "It is not a prime number" << endl;
9 break;
10 }
11 index++;
12 }
13 if (index == n)
14 cout << "It is a prime number" << endl;
15 } // end main
int main()
{
1 int n, index;
2 cout << "Enter a number: " <> n;
3 index = 2;
4 while (index <= n - 1)
5 {
6 if (n % index == 0)
7 {
8 cout << "It is not a prime number" << endl;
9 break;
10 }
11 index++;
12 }
13 if (index == n)
14 cout << "It is a prime number" << endl;
15 } // end main
CYCLOMATIC COMPLEXITY?

i = 0;
n=4; //N-Number of nodes present in the graph
while (i<n-1) do
j = i + 1;
while (j<n) do
if A[i]<A[j] then
swap(A[i], A[j]);
end do;
j=j+1;
end do;
CYCLOMATIC COMPLEXITY?

i = 0;
n=4; //N-Number of nodes present in the graph
while (i<n-1) do
j = i + 1;
while (j<n) do
if A[i]<A[j] then V(G) = 9 – 7 + 2 = 4
V(G) = 3 + 1 = 4 (Condition nodes are 1,2 and 3 nodes)
swap(A[i], A[j]); Basis Set – A set of possible execution path of a program
1, 7
end do; 1, 2, 6, 1, 7
j=j+1; 1, 2, 3, 4, 5, 2, 6, 1, 7
1, 2, 3, 5, 2, 6, 1, 7
end do;
GRAPH MATRICES

A graph matrix is a square matrix whose size

(i.e., number of rows and columns) is equal to
the number of nodes on the flow graph.

Each row and column corresponds to an

identified node, and matrix entries correspond
to connections (an edge) between nodes.
 • link weight : 1 - a connection exists
: 0 - a connection does not exist
• Properties of link weight:
LINK WEIGHT • probability that a link (edge) will be execute.
• processing time expended during traversal of a link
• memory required during traversal of a link
• resources required during traversal of a link.
CONTROL STRUCTURE TESTING- CONDITION TESTING

• focuses on testing each condition in the program to ensure that it does not contain errors

• A simple condition is a Boolean variable or a relational expression, possibly preceded with

one NOT (¬) operator. A relational expression takes the form
E1 <relational-operator> E2
where E1 and E2 are arithmetic expressions and <relational-operator>
DATA FLOW TESTING

• The data flow testing method [Fra93] selects test paths of a program according to the locations of
definitions and uses of variables in the program.
• DEF(S) = {X | statement S contains a definition of X}

• USE(S) = {X | statement S contains a use of X}

• It checks following a
• If the variables are used without initialization.

• If the initialized variables are not used at least once.

1. read x, y;
2. if(x>y) variable Defined used
3. a = x+1 x 1 2, 3
else y 1 2, 4
4. a = y-1
a 3,4 5
5. print a;

• Definition-Use chain of variable x [x, s, s’]

• One simple data flow testing strategy is to require that every DU chain be covered at least
once.
LOOP TESTING

• The objective of Loop Testing is:

• To fix the infinite loop repetition problem.
• To know the performance.
• To identify the loop initialization problems.
• To determine the uninitialized variables.
SIMPLE LOOPS

• Skip the loop entirely.

• Only one pass through the loop.
• Two passes through the loop.
• m passes through the loop where m n.
• n - 1, n, n + 1 passes through the loop.
 NESTED LOOPS

• Start at the innermost loop. Set all other loops to minimum values.
• Conduct simple loop tests for the innermost loop while holding the outer
• loops at their minimum iteration parameter (e.g., loop counter) values. Add other tests
for out-of-range or excluded values.
• Work outward, conducting tests for the next loop, but keeping all other outer loops at
minimum values and other nested loops to “typical” values.
• Continue until all loops have been tested.
CONCATENATED LOOPS

• Use same approach for simple loop if loops are independent

• Use nested loop approach if loops are dependent
BLACK BOX TESTING
BLACK-BOX TESTING

• Incorrect or missing functions

• interface errors

• errors in data structures or external database access

• behavior or performance errors
• initialization and termination errors
GRAPH-BASED TESTING
BEHAVIORAL TESTING METHODS

• Transaction flow modelling

• Finite state modelling

• Data flow modelling

• Timing modeling
EQUIVALENCE PARTIONING

• It divides the input domain of a program into classes of data from which test cases can be derived.
• Guidelines
• Range- 1 valid and 2 invalid classes
• Specific values- 1 valid and 2 invalid classes
• Member of set, 1 valid and 1 invalid class
• Boolean, 1 valid and 1 invalid class
BOUNDARY VALUE ANALYSIS

• leads to a selection of test cases that exercise bounding values.

• Guidelines
• i/p specified as range bounded by values a and b- just above and below
• Specified a number of values-Test cases should be designed to create an output report that
produces the maximum and minimum allowable number of table entries.
• Prescribed boundary, exercise that data structure at its boundary.
OTRTHOGONAL ARRAY TETSING

• Black box testing techniques

• Test large numbers of possible input combinations
• Pair wise testing
• Any two columns give pair wise combinations
• 2-D array of numbers
 Top Left Right

TS1 Displayed Displayed Displayed

TS2 Displayed Hidden Hidden

TS3 Hidden Displayed Hidden

TS4 Hidden Hidden Displayed

• No of variables-column
• No of possible values-factors
• Find OA with Smallest no of rows
• Place the variable name in column
• Given by Taguchi Designs (orthogonal array)
L9 ORTHOGONAL ARRAY
EXAMPLE

• Consider the following “calculateMembershipDiscount" function for black box testing.

Generate the test cases needed for the complete BBT for above function.

The function should take MembershipDuration as a input. The function should calculate the
discount percentage based on the MembershipDuration. The discount percentages for
different membership durations are predefined and stored in a lookup table or configuration
file. The function should retrieve the discount percentage corresponding to the
MembershipDuration from the lookup table or configuration file. The predefined discount
rate for membership duration 5>x>=1 is 10%, for membership duration 10 >x >= 5 year is
12%, for more than or equal to 10 year is 15%.
TEST CASES

Criteria Invalid Valid Invalid Class Boundary condition

Class class
1. MembershipDuration x =0 x =4 x =6 x =0,1,2
5>x>=1 x =4,5,3
2. MembershipDuration x=13 x =7 x=4 x=5, 4, 6
10 >x >= 5 x=10,9,8
3. Valid duration x=-3 x=8 x=20000 x=0, -1, 1
4. MembershipDuration x=7 x=15 x=1000 x=11,10,9
10 =<x