Department of Computing

Adama Science and Technology University

Course Number --- CSE5310

Course Title ------- Software Testing and Quality
Assurance
Degree Program ---
CSE 5th Year II sem
Instructor ---------
E-mail --------------
Dr. T.Gopi Krishna
gktiruveedula@gmail.com
 Chapter 3 & 4
Specification / Black Box Testing
 Specification based test construction techniques
 Black box testing [
Equivalence portioning,
Boundary Value Analysis] ,
White-box testing and
Grey-box testing
Its Advantages and Disadvantages
How do you test a system?

3
 Specification Based Testing

Classification based on the source of information to derive test cases:

 Black-box testing (functional, specification-based) 
Equivalence portioning,
Boundary Value Analysis]
 White-box testing (structural, program-based)
 Gray box testing(Classes, Objects, DESIGN)

Input
determined Result
by... Actual output
compared
… requirements* Black box with
required
* from previous phase
 Black, Gray and White-box Testing
Result
Input determined by... Actual output
compared
… requirements Black box or with
Glass-box required output

Gray box As for black- and

… requirements &
white box testing
key design elements

White box
Confirmation
…design of expected
elements behavior
 Characteristics of Testable Software
 Operable
– The better it works (i.e., better quality), the easier it is to test
 Observable
– Incorrect output is easily identified; internal errors are automatically detected
 Controllable
– The states and variables of the software can be controlled directly by the tester
 Decomposable
The software is built from independent modules that can be tested independently
 Simple
– The program should exhibit functional, structural, and code simplicity
 Stable
– Changes to the software during testing are infrequent and do not invalidate
existing tests
 Understandable
– The architectural design is well understood; documentation is available and
organized
6
 Test Characteristics

 A good test has a high probability of finding an error

– The tester must understand the software and how it might fail

 A good test is not redundant

– Testing time is limited; one test should not serve the same purpose as another test.

 A good test should be “best of breed”

– Tests that have the highest likelihood of uncovering a whole class of errors should
be used

 A good test should be neither too simple nor too complex

– Each test should be executed separately; combining a series of tests could cause
side effects and mask certain errors

7
 Black box Testing
 In a black box, the test item is
treated as "black" whose logic
is Unknown.
 All that's known is what goes in
and what comes out, the input
and output
 Focuses on the functional
requirements specifications of
the software.
Ex:
Unit,integration,system,acceptanc
e testing etc…
 Also called behavioral testing
 Black-box Testing
Test cases are designed using only functional specification of the
software:
– without any knowledge of the internal structure of the software.

For this reason, black-box testing is also known as functional

testing.

Black-box testing: ignores the internal logic of the software, and

looks at what happens at the interface (e.g., given this inputs, was
the produced output correct?)

Designing test cases for black box testing:

– Does not require any knowledge of how the functions have been
designed and implemented.
– Test cases can be designed by examining only SRS document. 9
 Black-box Testing..Cont
 Complements white-box testing by uncovering different classes of errors.

 Unlike white-box testing, here we don’t use any knowledge about the internals of
the code

 Focuses on the functional requirements and the information domain of the

software

 Used during the later stages of testing after white box testing has been performed

 The tester identifies a set of input conditions that will fully exercise all functional
requirements for a program

 The test cases satisfy the following:

– Reduce, by a count greater than one, the number of additional test cases that
must be designed to achieve reasonable testing
– Test cases are designed based on specifications

10
 Questions answered by Black-box Testing
 How is functional validity tested?

 How are system behavior and performance tested?

 What classes of input will make good test cases?

 Is the system particularly sensitive to certain input values?

 How are the boundary values of a data class isolated?

 What data rates and data volume can the system tolerate?

 What effect will specific combinations of data have on system

operation?

11
 Black box Testing Techniques
 Boundary Value Testing
 Equivalence class Testing
 Error Guessing
 Decision Table based Testing
 Combinatorial Testing
 Grammar Based Testing
 Module Base Testing
 Orthogonal matrix
 Equivalence Partitioning
 A black-box testing method that divides the input domain of a
program into classes of data from which test cases are derived.

 An ideal test case single-handedly uncovers a complete class of

errors, thereby reducing the total number of test cases that must
be developed.

 Test case design is based on an evaluation of equivalence classes

for an input condition.

 An equivalence class represents a set of valid or invalid states for

input conditions.

 From each equivalence class, test cases are selected so that the
largest number of attributes of an equivalence class are exercise at
once
13
 Equivalence Partitioning
Equivalence testing is a black-box testing method that divides the space of all possible

inputs into equivalence groups such that the program “behaves the same” on each

group

 Two steps:
1. Partitioning the values of input parameters into equivalence groups

2. Choosing the test input values

 Guidelines for Defining Equivalence Classes
 If an input condition specifies a range, one valid and two invalid
equivalence classes are defined
– Input range: 1 – 10 Eq classes: {1..10}, {x < 1}, {x > 10}

 If an input condition requires a specific value, one valid and two invalid
equivalence classes are defined
– Input value: 250 Eq classes: {250}, {x < 250}, {x > 250}

 If an input condition specifies a member of a set, one valid and one

invalid equivalence class are defined
– Input set: {-2.5, 7.3, 8.4} Eq classes: {-2.5, 7.3, 8.4}, {any other x}

 If an input condition is a Boolean value, one valid and one invalid class
are define
– Input: {true condition} Eq classes: {true condition}, {false
condition}
15
 Equivalence Partitioning
 Basic idea: consider input/output domains and partition them into equiv. classes
– For different values from the same class, the software should behave
equivalently

 Use test values from each class

– Example: if the range for input x is 2..5,
Eg: there are three classes: “<2”, “between 2..5”, “5<”

 Input x in a certain range [a..b]: this defines three classes “x<a”, “a<=x<=b”,
“b<x”

 Choosing test values

– Choose a typical value in the middle of the class(es) that represent valid input
– Also choose values at the boundaries of all classes: e.g., if the range is [a..b],
use a-1,a, a+1, b-1,b,b+1

16
 Example: Searching
 Search for a value in an array
– Return value is the index of some occurrence of the
value, or -1 if the value does not occur in the array

 One partition: size of the array

– Since people often make errors for arrays of size 1,
we decide to create a separate equivalence class
– Classes are “empty arrays”, array with one element”,
“array with many elements”

17
 Example: Searching
Another partition: location of the value
– Four classes: “first element”, “last element”, “middle element”, “not
found”
Array Value Output
Empty 5 -1
[7] 7 0
[7] 2 -1
[1,6,4,7,2] 1 0
[1,6,4,7,2] 4 2
[1,6,4,7,2] 2 4
[1,6,4,7,2] 3 -1

18
 Boundary Value Analysis

 A greater number of errors occur at the boundaries of

the input domain rather than in the "center”

 Boundary value analysis is a test case design method

that complements equivalence partitioning
– It selects test cases at the edges of a class
– It derives test cases from both the input domain
and output domain

19
 Boundary Value Analysis
Boundary Value Analysis (BVA) technique is a test case design technique that

allows to identify appropriate number of input values to be supplied to test the

system.

 Human brain is evolved in emotional side and less evolved in logical side.

 Hence during software development, people always get confused in using

< and <= ; > and >= relational operators.

 As such large number of errors tends to occur at boundaries of the input

domain.

 BVA leads to selection of test cases that exercise boundary values.

BVA based test case design helps to write test cases that exercise
 Boundary Value Analysis

“Bugs hide in corners and congregate at

boundaries...”

---
The test cases developed based on equivalence class partitioning can be strengthened
Boris Beizer
by use of a technique called boundary value analysis.

Many defects occur directly on, and above and below, the edges of

equivalence classes.

BVA is a software design technique to determine test cases off-by-one errors

 Boundary Value Analysis
 If an input condition for the software-under-test is specified as a range of
values, develop valid test cases for the ends of the range, and invalid test

cases for possibilities just above and below the ends of the range.

For example if a specification states that an input value for a module must lie

in the range between -1.0 and +1.0,

Valid tests that include values -1.0, +1.0 for ends of the range, as well as

Invalid test cases for values just above and below the ends -1.1, +1.1, should

be included.
 Boundary Value Analysis
 If an input condition for the software-under-test is specified as a

number of values, develop valid test cases for the minimum and

maximum numbers as well as invalid test cases that include one lesser

than minimum and one greater than the maximum.

For example, for the real-estate module, a house can have four owners,

Valid Tests that includes tests for minimum and maximum values 1, 4

owners
 Boundary Value Analysis
If the input or output of the software-under-test is an ordered set, such as a

table or a linear list, develop tests that focus on the first and last elements of

the set
 Boundary Value Analysis
Using BVA- Range

When the programming element is a range type, we can arrive at test

cases using BVA as follows:

For a range of values bounded by a and b, then test: (Minimum

Value - precision)

Minimum Value

(Minimum Value + precision), (Maximum Value - precision) Maximum

 BVA Illustration
Write Test Cases using BVA for a requirement that is stated as follows:

“In the examination grading system, if the student scores 0 to less than 40 then

assign E Grade, if the student scores between 40 to 49 then assign D Grade, if

the student scores between 50 to 69 then assign C Grade, if the student scores

between 70 to 84 then assign B Grade, and if the student scores 85 to 100 then

assign A Grade.”

In the above problem definition, after analysis, we identify following Boundary

Values:

0 to 39 -E

40 to 49-D
 BVA-An Illustration
Based on BVA, we identify following input values to test each boundary: For

Boundary Values in range 0 to 39:

For 0 to 39 boundary values, Minimum Value= 0, Maximum Value = 39,

Precision is 1.

Thus, input values for testing for this boundary values are: (Minimum Value-

precision)= -1 Minimum Value= 0

(Minimum Value+ precision)= 1 (Maximum Value- precision)= 38 Maximum

Value= 39 (Maximum Value + precision)= 40

Thus, input values for Boundary Values 0 and 39 are: -1, 0, 1, 38, 39, 40.

On the similar lines of analysis, we arrive at following input values for other

Boundary values:
 Guidelines for Boundary Value Analysis
 If an input condition specifies a range bounded by values a and b,
test cases should be designed with values a and b as well as values
just above and just below a and b

 If an input condition specifies a number of values, test case

should be developed that exercise the minimum and maximum
numbers. Values just above and just below the minimum and
maximum are also tested

 Apply guidelines 1 and 2 to output conditions; produce output that

reflects the minimum and the maximum values expected; also test
the values just below and just above

 If internal program data structures have prescribed boundaries

(e.g., an array), design a test case to exercise the data structure at
its minimum and maximum boundaries
28
 Error Guessing
 Tester guesses from his /her experience on the error-prone area
and concentrates around that area.

Example : When the bike stops, first thing you do is — check the
petrol.

You probably wont

go to mechanic or
read the manual or
check the spark plug
You know from your experience or
usage that petrol could be over.
 Decision Tables - General
Decision tables are a precise yet compact way to model
complicated logic.

Decision tables, like if-then-else and switch-case statements,

associate conditions with actions to perform. But, unlike
the control structures found in traditional programming
languages.

Decision tables can associate many independent conditions

with several actions in an elegant way.

30
 Decision Tables - Usage
 Decision tables make it easy to observe that all possible
conditions are accounted for.

 Decision tables can be used for:

 Specifying complex program logic
 Generating test cases (Also known as logic-based testing)

 Logic-based testing is considered as:

 Structural testing when applied to structure (i.e. control flowgraph of
an implementation).
 Functional testing when applied to a specification.

31
 Decision Tables - Structure
Conditions - (Condition stub) Condition Alternatives –
(Condition Entry)
Actions – (Action Stub) Action Entries

 Each condition corresponds to a variable, relation or predicate

 Possible values for conditions are listed among the condition
 alternatives
 Boolean values (True / False) – Limited Entry Decision Tables
 Several values – Extended Entry Decision Tables
 Don’t care value

 Each action is a procedure or operation to perform

 The entries specify whether (or in what order) the action is to be
performed
32
 To express the program logic we can use a limited-entry
decision table consisting of 4 areas called the condition
stub, condition entry, action stub and the action entry:
Condition entry

Rule1 Rule2 Rule3 Rule4

Condition Yes Yes No No
1
Condition
Condition Yes X No X
stub 2
Condition No Yes No X
3
Condition No Yes No Yes
4
Action stub
Action1 Yes Yes No No
Action2 No No Yes No
Action Entry
Action3 No No No Yes 33
 We can specify default rules to indicate the action to be
taken when none of the other rules apply.
 When using decision tables as a test tool, default rules and
their associated predicates must be explicitly provided.

Rule5 Rule6 Rule7 Rule8

Condition1 X No Yes Yes

Condition2 X Yes X No

Condition3 Yes X No No

Condition4 No No Yes X

Default Yes Yes Yes Yes

action

34
 Black box testing
Advantages:

 Well suited and efficient for large code segments.

Code access is not required.
 Clearly separates user's perspective from the developer's perspective through visibly
defined roles.

 Large numbers of moderately skilled testers can test the application with no

knowledge of implementation, programming language, or operating systems.

Disadvantages:

 Limited coverage, since only a selected number of test scenarios is actually

performed.

 Inefficient testing, due to the fact that the tester only has limited knowledge about

an application.

Blind coverage, since the tester cannot target specific code segments or error-
Black Box Testing
 White-Box Testing
 White-box testing: uses knowledge of the internal
structure of the software
E.g: write tests to “cover” internal paths
Note: white-box testing is also called structural testing/Glass
box/open box testing.
 White-box testing Techniques:
 Control-flow-based testing
 Loop testing
 Data-flow-based testing
 Statement coverage
 Path coverage
 Branch testing
Unit Testing Tools/Frameworks:
 Junit
 TestNG
 Control-flow-based Testing

 A traditional form of white-box testing

 Step 1: From the source, create a graph describing the
flow of control
 Called the control flow graph
 The graph is created (extracted from the source code)
manually or automatically
 Step 2: Design test cases to cover certain elements of
this graph
 Nodes, edges, paths

38
 Example of a Control Flow Graph (CFG)

1
s:=0;
d:=0; 2
while (x<y) {
x:=x+3; 3
y:=y+2;
if (x+y < 100) 4
s:=s+x+y; 5 6
else 7
d:=d+x-y;
} 8

39
 Elements of a CFG

 Three kinds of nodes:

– Statement nodes: represent single-entry-single-exit sequences
of statements
– Predicate nodes: represent conditions for branching
– Auxiliary nodes: (optional) for easier understanding (e.g.,
“join points” for IF, etc.)
 Edges: represents possible flow of control.
 It is relatively easy to map standard constructs from
programming languages to elements of CFGs

40
 IF-THEN, IF-THEN-ELSE, SWITCH

if (c) if (c) switch (c)

then then case 1:
// join point else case 2:
// join point // join
point

…
… … …..
 Example

switch (position)
case CASHIER
if (empl_yrs > 5)
bonus := 1; .
else .
bonus := 0.7; .
case MANAGER .
bonus := 1.5;
if (retiring_soon)
bonus := 1.2 * bonus
case …
end switch
42
 Mapping for Loops

while (c) {

} …

Note: other loops (e.g., FOR, DO-WHILE,…) are mapped similarly.

Figure out how this is done.

43
 Statement Coverage
 Basic idea: given the control flow graph define a
“coverage target” and write test cases to achieve it.

Traditional target: statement coverage

– Need to write test cases that cover all nodes in the
control flow graph

Intuition: code that has never been executed during

testing may contain errors
– Often this is the “low-probability” code

44
 Example
 Suppose that we write and
execute two test cases 1
 Test case #1: follows path 1-
2-exit (e.g., we never take the 2
loop)
 Test case #2: 1-2-3-4-5-7-8- 3
2-3-4-5-7-8-2-exit (loop T 4 F
twice, and both times take the
true branch) 5 6
 Do we have 100% statement 7
coverage?
8

45
 Branch Coverage

Target: write test cases that cover all branches

of predicate nodes
– True and false branches of each IF
– The two branches corresponding to the condition
of a loop
– All alternatives in a SWITCH statement
In modern languages, branch coverage implies
statement coverage

46
 Branch Coverage
Statement coverage does not imply branch
coverage
– Can you think of an example?
Motivation for branch coverage: experience
shows that many errors occur in “decision
making” (i.e., branching)
– Plus, it subsumes statement coverage.

47
 Example
Same example as before
1
Test case #1: follows path 1-
2
2-exit
Test case #2: 1-2-3-4-5-7-8- 3
2-3-4-5-7-8-2-exit T 4 F
What is the branch 5 6
coverage? 7

48
 Achieving Branch Coverage
For decades, branch coverage has been
considered a necessary testing minimum
To achieve it: pick a set of start-to-end paths
(in the CFG) that cover all branches, and then
write test cases to execute these paths
It can be proven that branch coverage can be
achieved with at most E-N+2 paths

49
 White-Box Testing
Advantages:
As the tester has knowledge of the source code, it becomes very easy to find out
which type of data can help in testing the application effectively.
 It helps in optimizing the code.
 Extra lines of code can be removed which can bring in hidden defects.
 Due to the tester's knowledge about the code, maximum coverage is attained
during test scenario writing.

Disadvantages:
 Due to the fact that a skilled tester is needed to perform white box testing, the
costs are increased.
 Sometimes it is impossible to look into every nook and corner to find out hidden
errors that may create problems as many paths will go untested.

 It is difficult to maintain white box testing as the use of specialized tools like code
analyzers and debugging tools are required.
White Box Testing
 Grey Box Testing

 Grey Box testing is a technique to test the application with limited

knowledge of the internal workings of an application.

 In software testing, the term “the more you know the better”
carries a lot of weight when testing an application.

 Mastering the domain of a system always gives the tester an edge

over someone with limited domain knowledge.

 Unlike black box testing, where the tester only tests the
application’s user interface, in grey box testing, the tester has
access to design documents and the database. Having this
knowledge, the tester is able to better prepare test data and test
scenarios when making the test plan.
 Grey Box Testing

Advantages:
 Offers combined benefits of black box and white box testing wherever possible.
 Grey box testers don’t rely on the source code; instead they rely on interface
definition and functional specifications.
 Based on the limited information available, a grey box tester can design
excellent test scenarios especially around communication protocols and data
type handling.
 The test is done from the point of view of the user and not the designer.
Disadvantages:
 Since the access to source code is not available, the ability to go over the code
and test coverage is limited.
 The tests can be redundant if the software designer has already run a test case.
 Testing every possible input stream is unrealistic because it would take an
unreasonable amount of time; therefore, many program paths will go untested.
Gray Box Testing
Black Box Testing Grey Box Testing White Box Testing
The Internal Workings of an Somewhat knowledge of the Tester has full knowledge of the
application are not required to internal workings are known. Internal workings of the application .
be known .
Also known as closed box Another term for grey box Also known as clear box testing,
testing, data driven testing testing is translucent testing as structural testing or code based
and functional testing . the tester has limited testing .
knowledge of the insides of the
application .
Performed by end users and Performed by end users and Normally done by testers and
also by testers and also by testers and developers . developers .
developers .
Testing is based on external Testing is done on the basis of Internal workings are fully known
expectations -Internal high level database diagrams and the tester can design test data
behavior of the application is and data flow diagrams . accordingly .
unknown.
This is the least time Partly time consuming and The most exhaustive and time
consuming and exhaustive . exhaustive . consuming type of testing .

Not suited to algorithm Partially suited to algorithm Suited for algorithm testing.
testing, but suitable for End- testing.
to-End testing.
This can only be done by trial Data domains and Internal Data domains and Internal
and error method . boundaries can be tested, if boundaries can be better tested .
known.
 Questions Please!!!!

29/08/2022 ASTU 3rd Year SE Chapter-3

 Assignement-1

Briefly explain following topics

 What is cleanroom?
 Cleanroom process model.
 Clean room development process.
 Clean-room approach to quality assurance.
 Software quality achievement through
cleanroom.
 Assignement-2

Define and explain with neat diagrams about the

Data flow machines, Data flow graphs, Data flow
anomalies, state graph, Data Flow model Testing,
Maintenance and debugging in the Data flow
context.
 Topics to be covered

1.Defect life cycle- with priority and severity definitions

2.Different testing techniques
3.Unit testing- junit live demo
4. Functional testing – Selenium live demo