Software Testing Fundamentals: 1.1 Characteristics of Testable Software
Software Testing Fundamentals: 1.1 Characteristics of Testable Software
CML
Module 5
1. Software Testing Fundamentals
2. Test Case Design
3. White-Box Testing-Basis Path Testing-Control Structure Testing
4. Black-Box Testing
5. Various levels of Testing: Modules to System.
6. Structured Programming
7. Implementation Techniques
8. Programming principles and guidelines
9. Case study : Test case design and Testlog preparation
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:1 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:2 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:3 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
Flow graph of a flow chart is described in Figure 17.2 . To illustrate the use of a flow graph, we consider the
procedural design representation in Figure 17.2A. Here, a flowchart is used to depict program control structure.
Figure 17.2B maps the flowchart into a corresponding flow graph (assuming that no compound conditions are
contained in the decision diamonds of the flowchart). Referring to Figure 17.2B, each circle, called a flow graph
node, represents one or more procedural statements. A sequence of process boxes and a decision diamond can
map into a single node. The arrows on the flow graph, called edges or links, represent flow of control and are
analogous to flowchart arrows. An edge must terminate at a node, even if the node does not represent any procedural
statements (e.g., see the symbol for the if-then-else construct). Areas bounded by edges and nodes are
called regions. When counting regions, we include the area outside the graph as a region.
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:4 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:5 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:6 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
V(G) = 6 regions
V(G) = 17 edges -13 nodes + 2 = 6
V(G) = 5 predicate nodes + 1 = 6
3) Determine a basis set of linearly independent paths
The value of V(G) provides the number of linearly independent paths through the program control
structure. In the case of procedure average, we expect to specify six paths:
path 1: 1-2-10-11-13
path 2: 1-2-10-12-13
path 3: 1-2-3-10-11-13
path 4: 1-2-3-4-5-8-9-2-. . .
path 5: 1-2-3-4-5-6-8-9-2-. . .
path 6: 1-2-3-4-5-6-7-8-9-2-. . .
The ellipsis (. . .) following paths 4, 5, and 6 indicates that any path through the remainder of the control
structure is acceptable. It is often worthwhile to identify predicate nodes as an aid in the derivation of
test cases. In this case, nodes 2, 3, 5, 6, and 10 are predicate nodes.
4) Prepare test cases that will force execution of each path in the basis set
Data should be chosen so that conditions at the predicate nodes are appropriately set as each path is
tested.
4. Graph Matrics
Graph matrix
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:7 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
A graph matrix is a square matrix whose size (i.e., number of rows and columns) is equal to the number
of nodes on the flow graph.
It is a data structure useful for developing a software tool that assists in basis path testing.
Each row and column corresponds to an identified node, and matrix entries correspond to connections
(an edge) between nodes.
Adding a link weight to each matrix entry, the graph matrix can become a powerful tool for evaluating
program control structure during testing.
Link weight is 1 (a connection exists) or 0 (a connection does not exist).
Link weights are assigned with following properties:
o The probability that a link (edge) will be executed.
o The processing time expended during traversal of a link.
o The memory required during traversal of a link.
o The resources required during traversal of a link.
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:8 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:9 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
d. unstructured loops
a. Simple loops
The following set of tests can be applied to simple loops, where n is the maximum number of allowable passes
through the loop.
1. Skip the loop entirely.
2. Only one pass through the loop.
3. Two passes through the loop.
4. m passes through the loop where m < n.
5. n 1, n, n + 1 passes through the loop.
b. Nested loops
The following set of tests can be applied to nested loops.
1. Start at the innermost loop. Set all other loops to minimum values.
2. Conduct simple loop tests for the innermost loop while holding the outer loops at their minimum
iteration parameter (e.g., loop counter) values. Add other tests for out-of-range or excluded values.
3. Work outward, conducting tests for the next loop, but keeping all other outer loops at minimum values
and other nested loops to "typical" values.
4. Continue until all loops have been tested.
c. Concantenated loops
Concatenated loops can be tested using the approach defined for simple loops, if each of the loops is independent
of the other. However, if two loops are concatenated and the loop counter for loop 1 is used as the initial value for
loop 2, then the loops are not independent. When the loops are not independent, the approach applied to nested
loops is recommended.
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:10 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
d. Unstructured loops
Whenever possible, this class of loops should be redesigned to reflect the use of the structured
programmingconstructs.
4. Black-Box Testing
Black-box testing, also called behavioral testing, focuses on the functional requirements of the
software.Complements white-box testing by uncovering different classes of errors. Used during the later stages
of testing after white box testing has been performed. Black-box testing attempts to find errors in the following
categories:
(1) incorrect or missing functions
(2) interface errors
(3) errors in data structures or external data base access
(4) behavior or performance errors
(5) initialization and termination errors.
Questions answered by Black-box Testing are:-
How is functional validity tested?
How are system behavior and performance tested?
What classes of input will make good test cases?
Is the system particularly sensitive to certain input values?
How are the boundary values of a data class isolated?
What data rates and data volume can the system tolerate?
What effect will specific combinations of data have on system operation?
Software testing begins by creating a graph of important objects and their relationships and then devising a
series of tests that will cover the graph so that each object and relationship is exercised and errors are
uncovered. A collection of nodes that represent objects; links that represent the relationships between objects.
Node weights that describe the properties of a node (e.g., a specific data value or state behavior) . Link
weights that describe some characteristic of a link.
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:11 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
Kinds of link:-
1. Directed link (represented by an arrow) indicates that a relationship moves in only one direction.
2. Bidirectional link, also called a symmetric link, implies that the relationship applies in both
directions.
3. Parallel links are used when a number of different relationships are established between graph
nodes.
As a simple example, consider a portion of a graph for a word-processing application
Object #1 = new file menu select
Object #2 = document window
Object #3 = document text
The software engineer then derives test cases by traversing the graph and covering each of the
relationships shown. These test cases are designed in an attempt to find errors in any of the
relationships.
The nodes are data objects and the links are the transformations that occur to translate one data object into
another.
4. Timing modeling
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:12 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
The nodes are program objects and the links are the sequential connections between those objects. Link weights
are used to specify the required execution times as the program executes.
2. Equivalence Partitioning
A black-box testing method that divides the input domain of a program into classes of data from which test
cases are derived.Test case design is based on an evaluation of equivalence classes for an input condition. An
equivalence class represents a set of valid or invalid states for input conditions. Guidelines for Defining
Equivalence Classes:-
If an input condition specifies a range, one valid and two invalid equivalence classes are defined
Input range: 1 10 Eq classes: {1..10}, {x < 1}, {x > 10}
If an input condition requires a specific value, one valid and two invalid equivalence classes are
defined
Input value: 250 Eq classes: {250}, {x < 250}, {x > 250}
If an input condition specifies a member of a set, one valid and one invalid equivalence class are
defined
Input set: {-2.5, 7.3, 8.4} Eq classes: {-2.5, 7.3, 8.4}, {any other x}
If an input condition is a Boolean value, one valid and one invalid class are define
Input: {true condition} Eq classes: {true condition}, {false condition}
A greater number of errors occur at the boundaries of the input domain rather than in the "center". Boundary
value analysis is a test case design method that complements equivalence partitioning. It selects test cases at the
edges of a class. It derives test cases from both the input domain and output domain. Guidelines for Boundary
Value Analysis are:-
1. If an input condition specifies a range bounded by values a and b
test cases should be designed with (values a and b ), (values just above a and b) and (just below a and b).
2. If an input condition specifies a number of values,
test case should be developed that exercise (minimum and maximum numbers), (Values just above and just
below the minimum and maximum ).
Orthogonal array testing can be applied to problems in which the input domain is relatively small but too
large to accommodate exhaustive testing.
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:13 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
Orthogonal array testing enables you to design test cases that provide maximum test coverage with a
reasonable number of test cases.
To illustrate the difference between orthogonal array testing and more conventional one input item at a
time approaches, consider a system that has three input items, X, Y, and Z. Each of these input items has
three discrete values associated with it. There are 33 = 27 possible test cases.
a geometric view of the possible test cases associated with X, Y, and Z illustrated in Figure 17.10.
Referring to the figure, one input item at a time may be varied in sequence along each input axis.
This results in relatively limited coverage of the input domain (represented by the left-hand cube in the
figure).
1. Unit Testing
Unit testing focuses verification effort on the smallest unit of software designthe software component or
module.The unit test is white-box oriented, and the step can be conducted in parallel for multiple
components.
Unit testing is discussed under these topics:-
1.1 Unit Test Considerations
1.2 Unit Test Procedures
1.1 Unit Test Considerations
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:14 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
Module interface is tested to ensure that information properly flows into and out of the program unit under test.
Local data structure is examined to ensure that data stored temporarily maintains its integrity during all steps in
an algorithm's execution. Boundary conditions are tested to ensure that module operates properly at boundaries
established to limit or restrict processing. All independent paths (basis paths) through the control structure are
exercised to ensure that all statements in a module have been executed at least once. All error handling paths
are tested.
Many unit tested modules are combined into subsystems , which are then tested. Goal of integration testing is
to see if modules can be inegrated properly. Incremental integration strategies are:-
a. Top-down integration
b. Bottom-up integration
c. Sandwich integration
d. Regression testing
e. Smoke testing
a. Top-down Integration
Modules are integrated by moving downward through the control hierarchy, beginning with the main
module. Subordinate modules are incorporated in either a depth-first or breadth-first fashion.
Depth-First: All modules on a major control path are integrated
Breadth -First: All modules directly subordinate at each level are integrated
Advantages
This approach verifies major control or decision points early in the test process
Disadvantages
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:15 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
Stubs need to be created to substitute for modules that have not been built or tested yet; this
code is later discarded
Because stubs are used to replace lower level modules, no significant data flow can occur
until much later in the integration/testing process
b. Bottom-up integration
Integration and testing starts with the most atomic modules in the control hierarchy
Advantages
This approach verifies low-level data processing early in the testing process
Need for stubs is eliminated
Disadvantages
Driver modules need to be built to test the lower-level modules; this code is later discarded or expanded into
a full-featured version. Drivers inherently do not contain the complete algorithms that will eventually use the
services of the lower-level modules; consequently, testing may be incomplete or more testing may be needed
later when the upper level modules are available.
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:16 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
c. Sandwich integration
Consists of a combination of both top-down and bottom-up integration.
Reaps the advantages of both types of integration while minimizing the need for drivers and stubs.
d. Regression testing
Each new addition or change to baselined software may cause problems with functions that previously
worked flawlessly.
Regression testing re-executes a small subset of tests that have already been conducted.
Regression test suite contains three different classes of test cases
i. A representative sample of tests that will exercise all software functions
ii. Additional tests that focus on software functions that are likely to be affected by the change
iii. Tests that focus on the actual software components that have been changed
e. Smoke testing
Smoke testing is an integration testing approach that is commonly used when shrinkwrapped software
products are being developed.Benefits of Smoke Testing
Integration risk is minimized
The quality of the end-product is improved
Error diagnosis and correction are simplified
Progress is easier to assess
5. Validation testing
Validation testing follows after integration testing
Designed to ensure that
All functional requirements are satisfied
All behavioral characteristics are achieved
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:17 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
1. Alpha testing
Conducted at the developers site by end users.
Software is used in a natural setting with developers watching intently
Testing is conducted in a controlled environment
Errors and usage problems are recorded and corrected.
2. Beta testing
Conducted at one or more end-user sites
Developer is generally not present
It serves as a live application of the software in an environment that cannot be controlled by
the developer
The end-user records all problems that are encountered and reports these to the developers
at regular intervals
After beta testing is complete, software engineers make software modifications and prepare
for release of the software product to the entire customer base.
4.System Testing
During system testing entire software system is tested.
Kinds of System testing:-
1. Recovery testing
2. Security testing
3. Stress testing
4. Performance testing
5. Deployment Testing
1. Recovery testing
Forces the software to fail in a variety of ways and verifies that recovery is properly performed.
2. Security testing
Verifies that protection mechanisms built into a system and protect it from improper access.
3. Stress testing
It determines the robustness of software by testing beyond the limits of normal operation such as
Executing a system in a manner that demands resources in abnormal quantity, frequency, or volume.It
involves testing beyond normal operational capacity, often to a breaking point, in order to determine
breaking points or safe usage limits.
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:18 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
4. Performance testing
Tests the run-time performance of software within the context of an integrated system.Often coupled with
stress testing and usually requires both hardware and software instrumentation.
5. Deployment Testing
It is also called configuration testing , exercises the software in each environment in which it is to be
operated.It examines all installation procedures and specialised installation software that will be used by
customers.
6. Structured programming
Structured Programming is a design technique that constrains logic flow to three constructs:
1. Sequence
2. condition
3. repetition
Sequence implements processing steps that are essential in the specification of any algorithm.
Condition provides the facility for selected processing based on some logical occurrence.
Repetition allows for looping.
Various notations depict the use of these constructs
1. Graphical design notation
Sequence, if-then-else, selection, repetition (see next slide)
2. Tabular design notation
3. Program design language
Similar to a programming language; however, it uses narrative text embedded
directly within the program statements
1. Graphical design notation
1. A box is used to indicate a processing step.
2. A diamond represents a logical condition, and arrows show the flow of control.
3. Sequence is represented as two processing boxes connected by an line (arrow) of control.
4. Condition, also called if then-else, is depicted as a decision diamond that if true, causes then-part
processing to occur, and if false, invokes else-part processing.
5. Repetition is represented using two slightly different forms.
6. The do while tests a condition and executes a loop task repetitively as long as the condition holds true.
7. A repeat until executes the loop task first, then tests a condition and repeats the task until the condition
fails.
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:19 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:20 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
It uses the vocabulary of one language (i.e., English) and the overall syntax of another (i.e., a
structured programming language).
The difference between PDL and a real programming language lies in the use of narrative text (e.g.,
English) embedded directly within PDL statements.
Given the use of narrative text embedded directly into a syntactical structure, PDL cannot be
compiled.
PDL tools currently exist to translate PDL into a programming language skeleton and/or a
graphical representation (e.g., a flowchart) of design. These tools also produce nesting maps, a
design operation index, cross-reference tables, and a variety of other information.
PDL Example
To illustrate the use of PDL, we present an example of a procedural design for the SafeHome
security system software.
The system monitors alarms for fire, smoke, burglar, water, and temperature (e.g., furnace breaks
while homeowner is away during winter) and produces an alarm bell and calls a monitoring
service , generating a voice-synthesized message. In the PDL that follows, we illustrate some of the
important constructs.
The following PDL defines an elaboration of the procedural design for the security monitor
component.
PROCEDURE security.monitor;
INTERFACE RETURNS system.status;
TYPE signal IS STRUCTURE DEFINED
name IS STRING LENGTH VAR;
address IS HEX device location;
bound.value IS upper bound SCALAR;
message IS STRING LENGTH VAR;
END signal TYPE;
TYPE system.status IS BIT (4);
TYPE alarm.type DEFINED
smoke.alarm IS INSTANCE OF signal;
fire.alarm IS INSTANCE OF signal;
water.alarm IS INSTANCE OF signal;
temp.alarm IS INSTANCE OF signal;
burglar.alarm IS INSTANCE OF signal;
TYPE phone.number IS area code + 7-digit number;
initialize all system ports and reset all hardware;
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:21 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:22 Prepared By: Merin
Mary Philip
Dept of Computer Science and Engineering
CML
As we have seen, the objective of software testing is to uncover errors. It follows that the most severe
defects (from the customers point of view) are those that cause the program to fail to meet its
requirements.
Test planning can begin as soon as the requirements model is complete. Detailed definition of test cases
can begin as soon as the design model has been solidified. Therefore, all tests can be planned and
designed before any code has been generated.
Stated simply, the Pareto principle implies that 80 percent of all errors uncovered during testing
will likely be traceable to 20 percent of all program components. The problem, of course, is to isolate
these suspect components and to thoroughly test them.
4. Testing should begin in the small and progress toward testing in the large.
The first tests planned and executed generally focus on individual components. As testing progresses,
focus shifts in an attempt to find errors in integrated clusters of components and ultimately in the entire
system.
__________________________________________________________________________________________
CS010 605 /SOFTWARE ENGINEERING Page No:23 Prepared By: Merin
Mary Philip