Software Testing Techniques

Deepanjal Shrestha (PhD)

Associate Professor

1
 Software Testing

Testing is the process of exercising a

program with the specific intent of finding
errors prior to delivery to the end user.

2
 Testability
 Operability—it operates cleanly
 Observability—the results of each test case are
readily observed
 Controllability—the degree to which testing can be
automated and optimized
 Decomposability—testing can be targeted
 Simplicity—reduce complex architecture and logic
to simplify tests
 Stability—few changes are requested during testing
 Understandability—of the design

3
 What Testing Shows
errors
requirements conformance

performance

an indication
of quality

4
 Who Tests the Software?

developer independent tester

Understands the system Must learn about the system,
but, will test "gently" but, will attempt to break it
and, is driven by "delivery" and, is driven by quality

5
 Exhaustive Testing

loop < 20 X

14
There are 10 possible paths! If we execute one
test per millisecond, it would take 3,170 years to
test this program!!

6
 Selective Testing

Selected path

loop < 20 X

7
 Test Case Design
"Bugs lurk in corners
and congregate at
boundaries ..."
Boris Beizer

OBJECTIVE to uncover errors

CRITERIA in a complete manner

CONSTRAINT with a minimum of effort and time

8
 White-Box Testing

... our goal is to ensure that all

statements and conditions have
been executed at least once ...

9
 Why Cover?
logic errors and incorrect assumptions
are inversely proportional to a path's
execution probability

we often believe that a path is not

likely to be executed; in fact, reality is
often counter intuitive

typographical errors are random; it's

likely that untested paths will contain
some

10
Basis Path Testing
First, we compute the cyclomatic
complexity:

number of simple decisions + 1

or

number of enclosed areas + 1

In this case, V(G) = 4

11
 Cyclomatic Complexity
A number of industry studies have indicated
that the higher V(G), the higher the probability
or errors.

modules

V(G)

modules in this range are

more error prone

12
 Basis Path Testing
Next, we derive the
independent paths:
1

Since V(G) = 4,
2 there are four paths

3 Path 1: 1,2,3,6,7,8
4
5 6 Path 2: 1,2,3,5,7,8
Path 3: 1,2,4,7,8
Path 4: 1,2,4,7,2,4,...7,8
7
Finally, we derive test
cases to exercise these
8
paths.

13
Basis Path Testing Notes

you don't need a flow chart,

but the picture will help when
you trace program paths

count each simple logical test,

compound tests count as 2 or
more

basis path testing should be

applied to critical modules

14
 Loop Testing

Simple
loop
Nested
Loops
Concatenated
Loops Unstructured
Loops
15
 Loop Testing: Simple Loops
Minimum conditions—Simple Loops
• Skip the loop entirely: Test the scenario where the loop's body
does not execute at all.
• Only one pass through the loop: Test the scenario where the
loop's body executes exactly once.
• Two passes through the loop: Test the scenario where the loop's
body executes exactly twice.
• m passes through the loop (m < n): Test the scenario where the
loop executes m times, which is fewer than the maximum
allowable passes.
• (n-1), n, and (n+1) passes through the loop: Test scenarios where
the loop executes one less than the maximum (n-1), exactly the
maximum (n), and one more than the maximum allowable passes
(n+1).
16
 Loop Testing: Nested Loops
Nested Loops
Start at the innermost loop. Set all outer loops to their
minimum iteration parameter values.
Test the min+1, typical, max-1 and max for the
innermost loop, while holding the outer loops at their
minimum values.
Move out one loop and set it up as in step 2, holding all
other loops at typical values. Continue this step until
the outermost loop has been tested.
Concatenated Loops
If the loops are independent of one another
then treat each as a simple loop
else* treat as nested loops
endif*
for example, the final loop counter value of loop 1 is
used to initialize loop 2.

17
 Black-Box Testing

requirements

output

input events

18
Equivalence Partitioning:
Sample Equivalence
Classes
Valid data
user supplied commands
responses to system prompts
file names
computational data
physical parameters
bounding values
initiation values
output data formatting
responses to error messages
graphical data (e.g., mouse picks)
Invalid data
data outside bounds of the program
physically impossible data
proper value supplied in wrong place

19
Software Testing Strategies

20
 Testing Strategy
unit test integration
test

system validation
test test

21
 Unit Testing

module
to be
tested

results
software
engineer test cases

22
 Unit Testing
module
to be
tested
interface
local data structures
boundary conditions
independent paths
error handling paths

test cases

23
 Unit Test Environment

driver
interface
local data structures
Module boundary conditions
independent paths
error handling paths

stub stub

test cases
RESULTS

24
Integration Testing Strategies
Options:
• the “big bang” approach
• an incremental construction strategy

25
 Top Down Integration

A
top module is tested with
stubs

B F G

stubs are replaced one at

a time, "depth first"
C
as new modules are integrated,
some subset of tests is re-run
D E

26
Bottom-Up Integration
A

B F G

drivers are replaced one at a

time, "depth first"
C

worker modules are grouped into

builds and integrated
D E

cluster

27
 Sandwich Testing
A
Top modules are
tested with stubs

B F G

Worker modules are grouped into

builds and integrated
D E

cluster

28
 High Order Testing
validation test

system test

alpha and beta test

other specialized testing

• User Acceptance Testing (UAT)
• Regression Testing (Testing after change)
• Compliance Testing: Ensures that the software
meets regulatory and legal requirements
• Performance Testing

29
 Debugging:
A Diagnostic Process

30
 The Debugging Process
test cases

new test results

regression cases
tests suspected
causes
corrections
Debugging
identified
causes

31
 Debugging Effort

time required
to diagnose the
time required symptom and
to correct the error determine the
and conduct cause
regression tests

32
 Symptoms & Causes
symptom and cause may be
geographically separated

symptom may disappear when

another problem is fixed

cause may be due to a

combination of non-errors
cause may be due to a system
or compiler error

cause may be due to

symptom assumptions that everyone
cause believes
symptom may be intermittent

33
 Consequences of Bugs

infectious
damage
catastrophic
extreme
serious
disturbing
annoying
mild
Bug Type
Bug Categories: function-related bugs,
system-related bugs, data bugs, coding bugs,
design bugs, documentation bugs, standards
violations, etc.

34
Debugging Techniques

Brute force / testing

Exhaustive Testing and Trial and Error
backtracking
cause elimination

35
 Debugging: Final Thoughts

1. Don't run off half-cocked, think about the

symptom you're seeing.

2. Use tools (e.g., dynamic debugger) to gain

more insight.

3. If at an impasse, get help from someone else.

4. Be absolutely sure to conduct regression tests

when you do "fix" the bug.

36
Thank You

37