Fundamentals of Software Engineering
Fundamentals of Software Engineering
Fundamentals of Software Engineering
IT3203:Fundamentals of Software
Engineering
Testing
Duration : 7 hours
UCSC-2007
IT3203-Testing
Instructional Objectives
• Design test cases and write test programs for a given simple
software problem
UCSC-2007
IT3203-Testing
Detailed Syllabus
UCSC-2007
IT3203-Testing
UCSC-2007
IT3203-Testing
• Validation and verification ( V & V ) is the name given to the checking and
analysis processes that ensure that software conforms to its specification and
meets the needs of the customers who are paying for the software.
UCSC-2007
IT3203-Testing
Within the V & V process, two techniques of system checking and analysis may be
used:
Software Inspections - Analyse and check system representations such as the
requirements documents, design diagrams and program source code. They may be
applied at all stages of the development process. Inspections may be supplemented
by some automated analysis of the source text of a system or associated documents.
Software inspections and automated analysis are static V & V techniques as they
do not require the system to be executed.
Software Testing - Involves executing an implementation of the software with test
data and examining the outputs of the software and its operational behaviour to
check that it is performing as required. Testing is a dynamic technique of V & V
because it works with an executable representation of the system.
UCSC-2007
IT3203-Testing
The outcome of any tests should be recorded in a test results document that
include whether the test succeeded or failed and a description of the failure.
Test results for all passes through the test plan must be recorded to allow
accurate records to be kept of where problems occur and when they were
identified and corrected.
UCSC-2007
IT3203-Testing
Testing Process
UCSC-2007
IT3203-Testing
UCSC-2007
IT3203-Testing
Unit
testing
Sub-system
testing
System
testing
Individual Acceptance
components
testing
collections of
components
(sub-systems) The whole
finished system The finished
- developers system - users
UCSC-2007
IT3203-Testing
Acceptance Test
Functional
Integration Test
Specification
Code
UCSC-2007
IT3203-Testing
UCSC-2007
IT3203-Testing
Black-box testing
• Approach to testing where the program is
considered as a ‘black-box’
• The program test cases are based on the system
specification
• Inputs from test data may reveal anomalous outputs
i.e. defects
• Test planning can begin early in the software
process
• Main problem - selection of inputs
– equivalence partitioning
UCSC-2007
IT3203-Testing
Black-box testing
Inputs causing
anomalous
Input test data I behaviour
e
e
System
UCSC-2007
IT3203-Testing
White-box
Code Code Code Code Code Code Code Code Code Code Co
de Code Code
Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code
Code Code Code Code Code Code Code Code Code Code Code Code Code
UCSC-2007
IT3203-Testing
Structural testing
Test data
Tests Derives
Component Test
code outputs
UCSC-2007
IT3203-Testing
Software Inspection
UCSC-2007
IT3203-Testing
UCSC-2007
IT3203-Testing
Program Inspections
UCSC-2007
IT3203-Testing
Inspection Roles
Author or owner The programmer or designer responsible for
producing the program or document. Responsible
for fixing defects discovered during the inspection
process.
Inspector Finds errors, omissions and inconsistencies in
programs and documents. May also identify
broader issues that are outside the scope of the
inspection team.
Reader Presents the code or document at an inspection
meeting.
Scribe Records the results of the inspection meeting.
Chairman or moderator Manages the process and facilitates the inspection.
Reports process results to the Chief moderator.
Chief moderator Responsible for inspection process improvements,
checklist updating, standards development etc.
UCSC-2007
IT3203-Testing
Inspection Pre-conditions
UCSC-2007
IT3203-Testing
Planning
Overview Follow-up
Individual
Rework
preparation
Inspection
meeting
UCSC-2007
IT3203-Testing
UCSC-2007
IT3203-Testing
Inspection Checks
Data faults Are all program variables initialised before their values are used?
Have all constants been named? Should the upper bound of
arrays be equal to the size of the array or Size -1?If character
strings are used, is a delimiter explicitly assigned? Is there any
possibility of buffer overflow?
Input/output Are all input variables used? Are all output variables assigned a
faults value before they are output? Can unexpected inputs cause
corruption?
UCSC-2007
IT3203-Testing
Interface Do all function and method calls have the correct number of
faults parameters? Do formal and actual parameter types match? Are the
parameters in the right order? If components access shared
memory, do they have the same model of the shared memory
structure?
Storage If a linked structure is modified, have all links been correctly
management reassigned? If dynamic storage is used, has space been allocated
faults correctly? Is space explicitly de-allocated after it is no longer
required?
Exception Have all possible error conditions been taken into account?
management
faults
UCSC-2007
IT3203-Testing
UCSC-2007
IT3203-Testing
Test Phases
Test Phase Test Plan Author Technique Run by
Unit Test Code design Designer White Box, Black Programmer
box,
static
Integration Functional Author of Black box, white box, Programming
Test specification specification Top-down, bottom-up team
Interface Test System Authors of the White box Programmers
Specification system
specification
System Test Requirements Analyst Black box, stress System test team
testing,performance
testing
Acceptance Requirements Analyst/custo Black box Analyst/customer
Test mer
Alpha Test No test plan Black box Selected set of
users
Beta Test No test plan Black box Any user
Unit Testing
Unit Testing is carried out as a part of the coding task. This phase is based on the
design of the software for a piece of code. Unit testing should prove the following
about the code
• Robust – the code should not fail under any circumstances.
• Functionally correct – the code should carry out the task defined by the code design
• Correct interface – the inputs and outputs from the code should be as defined in the
design
UCSC-2007
IT3203-Testing
Integration/Sub-systems Testing
Integration Testing is carried out after the separate software modules have been unit
testing. Integration testing is based on the functional specification of the software.
Integration testing should prove the following about the software:
Integration - the modules of the system should interact as designed.
Functionally correct - the system should behave as defined in the functional
specification
UCSC-2007
IT3203-Testing
Initially, integrate a minimal system configuration and test the system. Then add
components to this minimal configuration and test after each added increment.
T1
A
T1
A
T2
T1
A T2
B
T3
T2 B
T3 C T4
B T3
C T4
D T5
T1,T2 and T3 tests are carried out after integration of components A and B. If the tests are
successful, add C and carry out tests T1,T2,T3 and T4. If new errors introduced, that is due
to the integration of C.
UCSC-2007
IT3203-Testing
In top-down integration, the high-level components of a system are integrated and tested
before design and implementation of some of the high-level components has been
completed.
Level 2
components
Level 3 components
UCSC-2007
IT3203-Testing
Low-level components are integrated and tested before the higher-level components have
been developed.
Test
drives
Testing
Level N Level N Level N Level N Level N
sequence
Test
drives
UCSC-2007
IT3203-Testing
Interface Testing
• Interface testing takes place when modules or sub-
systems are integrated to create large systems.
• Each module or sub-system has a defined interface
which is called by other program components.
• Objectives are to detect faults due to interface
errors or invalid assumptions about interfaces.
• Particularly important for object-oriented
development as objects are defined by their
interfaces.
• Cannot be detected by testing the individual
objects.
– The errors will occur due to the interaction
between objects.
UCSC-2007
IT3203-Testing
Types of Interfaces
UCSC-2007
IT3203-Testing
System Testing
System testing is carried out at the completion of the integration testing. The
purpose of system testing is to prove that the software meets the agreed user
requirements and works in the target environment. System testing covers both
functional and non-functional requirements.
• Security – Attempt to access the system without the correct authority, or attempt
to carry out restricted functions.
• Stress - Attempt to break the system by overloading it.
Acceptance Testing
UCSC-2007
IT3203-Testing
Alpha, Beta and Regression Testing
Alpha Testing –
Alpha testing is the first real use of the software product. Having completed
system testing the product will be given to a small number of selected customers
or possible just distributed within the company itself. The software will be used
‘in anger’ and errors reported to the development and maintenance team.
Beta Testing –
After alpha testing has been completed and any changes made a new version of
the product will be released to much wider audience. The objective of Beta
testing is to iron out the remaining problems with the product before it is put on
the general release.
Regression Testing –
Regression testing carried out after any changes are made to a software
application. The purpose of regression test is to prove that the change has been
made correctly and that the change has not introduced any new errors.
Regression test plans are usually a sub-set of the integration or systems test
plans. It is designed to cover enough functionality to give confidence that the
change has not effected any other part of the system.
UCSC-2007
IT3203-Testing
Back-to-back testing
Back-to-back testing means that multiple versions of the software
are executed on the same test data. If they all agree on the answer,
it is assumed that they are all correct.
Thread testing
Based on testing an operation which involves a sequence of processing steps
which thread their way through the system
Start with single event threads then go on to multiple event threads
Advantages
Suitable for real-time and object-oriented systems
Disadvantage
Difficult to assess test adequacy, because of large number of event
combinations
UCSC-2007
IT3203-Testing
UCSC-2007
IT3203-Testing
UCSC-2007
IT3203-Testing
UCSC-2007
IT3203-Testing
Object Integration
In object oriented systems, there is no obvious ‘top’ that provides for the
integration nor is there a clear hierarchy of objects that can be created. Clusters
therefore have to be created using knowledge of there operation and the
features of the system that are implemented by these clusters. There are three
possible approaches to integration testing that may be used:
UCSC-2007
IT3203-Testing
UCSC-2007
IT3203-Testing
UCSC-2007
IT3203-Testing
UCSC-2007