Software Testing

Contact Session - 1
VIVEKANANDA . M. R
BITS Pilani vivekanandamr@wilp.bits-pilani.ac.in
Pilani|Dubai|Goa|Hyderabad
Software Testing (SEWI ZC328)

Contact Session : 01
Introduction to Software Testing &
Techniques
 BITS Pilani
Pilani|Dubai|Goa|Hyderabad

Introduction and Fundamentals of Testing

• FAQ about Software Testing

• What is Software Testing?

• Why testing is necessary?
• Who does the testing?
• What has to be tested?
• When is testing done?
• How often to test?
• Some of the most common software
problems
 Incorrect calculation
 Incorrect data edits & ineffective data
edits
 Data searches that yields incorrect
results
 Incorrect processing of data relationship
 Incorrect coding / implementation
 Inadequate software performance
• Some of the objectives of testing
• Executing a program with the intent of finding an
error.
• To check if the system meets the requirements and
be executed successfully in the Intended
environment.
• To check if the system is “ Fit for purpose”.
• To check if the system does what it is expected to
do.
• Objective of a Software Tester
• Find bugs as early as possible and make sure they
get fixed.
• To understand the application well.
• Study the functionality in detail to find where the
bugs are likely to occur.
• Study the code to ensure that each and every line
of code is tested.
• Create test cases in such a way that testing is done
to uncover the hidden bugs and also ensure that the
software is usable and reliable
Basic Terminologies
• Error:
• A mistake . When people make mistakes while coding we
call those mistakes as bugs/errors.
• Errors tend to propagate during coding
• Refers to difference between Actual Output and Expected
output.
• Error is terminology of Developer.
• Bug is terminology of Tester
• Fault:
• It is a condition that causes the software to fail to perform
its required function.
• Result of an error is a fault or representation of an error
• Failure occurs when faulty piece of code is executed
leading to an incorrect state.
• Errors:
• Are part of daily life.
• Humans make error in their thoughts, in their actions,
and in the products that results from their actions.
• Errors occur in various fields
• Hearing – Spoken vs Heard
• Medicine – Incorrect prescription
• Music – Incorrect note played
• Sports – Incorrect call by referee in match
• Writing – Written vs Intent
• Software – Mistakes in programming statements
• Failure:
• Failure occurs when fault executes
• The inability of a system or component to perform its
required functions within specified performance
requirements.
• Incident:
• Failure occurrence may or may not be visible to user.
• Incident is associated with a failure that alerts user the
occurrence of failure
• Test:
• Act of exercising software with test cases
• It has two goals: to find failure and to demonstrate correct
execution.
• Test Automation
• What is Automation Testing?
• Manual Testing is performed by a human sitting in front of a
computer carefully executing the test steps
• Automation Testing means using an automation tool to execute
your test case suite.
• The automation software can also enter test data into the System
Under Test, compare expected and actual results and generate
detailed test reports.
• Why Automated Testing?
• Manual Testing of all workflows, all fields, all negative scenarios is
time and money consuming
• It is difficult to test for multilingual sites manually
• Automation does not require Human intervention. You can run
automated test unattended (overnight)
• Automation increases the speed of test execution
• Automation helps increase Test Coverage
• Developer and Tester as Two Roles
• Developer :
• One who writes the code.
• Individual who designs application and writes code.
• Developer assumes two roles(developer and tester)
• Tester :
• One who tests the code
 BITS Pilani
Pilani|Dubai|Goa|Hyderabad

Software Quality
• SOFTWARE QUALITY
• Quality:
• The degree to which a component, system or process meets
specified requirements and/or user/customer needs and
expectations.
• Software quality:
• The total functionality and features of a software product that bear
on its ability to satisfy stated or implied needs.
• Quality Attributes
• Static QA :
• It refers to the actual code and related documentation
• Dynamic QA:
• It refers to the behavior of the application while in use
• SOFTWARE QUALITY
• Quality Attributes
• Reliability
• Measure if product is reliable enough to sustain in any condition.
• Should give consistently correct results.
• Product reliability is measured in terms of working of project under
different working environment and different conditions.
• Maintainability
• Different versions of the product should be easy to maintain.
• For development it should be easy to add code to existing system,
should be easy to upgrade for new features and new technologies
time to time.
• System be easy to maintain and correcting defects or making a
change in the software
• Quality Attributes
• Usability
• This can be measured in terms of ease of use. Application should
be user friendly.
• Should be easy to learn. Navigation should be simple.
• Easy for new or infrequent users to learn to use the system.
• Portability
• This can be measured in terms of Costing issues related to porting,
Technical issues related to porting, Behavioral issues related to
porting.
• Correctness
• Application should be correct in terms of its functionality,
calculations used internally and the navigation should be correct.
• Flexibility
• Should be flexible enough to modify. Adaptable to other products
with which it needs interaction.
• Quality Attributes
• Reusability
• Software reuse is a good cost efficient and time saving
development way.
• Different code libraries classes should be generic enough to use
easily in different application modules.
• Dividing application into different modules so that modules can be
reused across the application.
• Interoperability
• Interoperability of one system to another should be easy for product
to exchange data or services with other systems.
• Different system modules should work on different operating
system platforms, different databases
• Efficiency
• To Major system quality attribute. Measured in terms of time
required to complete any task given to the system.
• Quality Attributes
• Completeness
• Refers to availability of all features listed in the requirements or in
user manual.
• An incomplete software is one that does not fully implement all
features required.
• Consistency
• Refers to adherence to a common set of conventions and
assumptions.
• For example, all buttons in the user interface might follow a
common color coding convention.
 BITS Pilani
Pilani|Dubai|Goa|Hyderabad

Requirements, Behavior and Correctness

Requirement Analysis
Analysis Objectives
• Identify customer’s needs.
• Evaluate system for feasibility.
• Perform economic and technical analysis.
• Allocate functions to system elements.
• Establish schedule and constraints.
• Create system definitions.
Requirement Analysis
• Software Products are designed in response to
requirements.
• Requirements specify the functions that a
product is expected to perform.
• Requirements leading to two different programs:
• Requirement 1: It is required to write a program that
inputs two integers and outputs the maximum of
these.
• Requirement 2: It is required to write a program that
inputs a sequence of integers and outputs the sorted
version of this sequence.
Requirements:Incompleteness
• Suppose that program max is developed to satisfy
Requirement 1. The expected output of max when
the input integers are 13 and 19 can be easily
determined to be 19.
• Suppose now that the tester wants to know if the
two integers are to be input to the program on one
line followed by a carriage return, or on two
separate lines with a carriage return typed in after
each number. The requirement as stated above fails
to provide an answer to this question.
 Input domain (Input space)

The set of all possible inputs to a program P is known as the input

domain or input space, of P.

Using Requirement 1 above we find the input domain of max

to be the set of all pairs of integers where each element in the pair
integers is in the range -32,768 till 32,767.

Using Requirement 2 it is not possible to find the input domain

for the sort program.
Requirements: Ambiguity
• Requirement 2 is ambiguous. It is not clear whether the
input sequence is to sorted in ascending or in descending
order. The behavior of sort program, written to satisfy this
requirement, will depend on the decision taken by the
programmer while writing sort.
• Modified Requirement 2:
• It is required to write a program that inputs a sequence of
integers and outputs the integers in this sequence sorted in
either ascending or descending order.
• The order of the output sequence is determined by an input
request character which should be ``A'' when an ascending
sequence is desired, and ``D'' otherwise
• Based on the above modified requirement, the input domain
for sort is a set of pairs. The first element of the pair is a
character. The second element of the pair is a sequence of
zero or more integers ending with a period.

• Input Domain:
• A : 3 15 12 55
• D : 23 78

• Correctness: A program is said to be correct if it behaves as

expected on each element of its input domain.
 Valid/Invalid Inputs
• The modified requirement for sort mentions that the request
characters can be ``A'' and ``D'', but fails to answer the
question ``What if the user types a different character ?’’

• When using sort it is certainly possible for the user to type a

character other than ``A'' and ``D‘’.
• Any character other than “A” and “D” is considered as invalid
input to sort.
• The requirement for sort does not specify what action it should
take when an invalid input is encountered.
 BITS Pilani
Pilani|Dubai|Goa|Hyderabad

Testing and Debugging

• Testing is the process of determining if a program
has any errors.
• When testing reveals an error, the process used to
determine the cause of this error and to remove it, is
known as debugging.
• A test cycle is often guided by a test plan.
• Example: The sort program is to be tested to meet the
requirements given earlier. Specifically, the following
needs to be done.
• Execute sort on at least two input sequences, one with
``A'' and the other with ``D'' as request characters.
• Execute the program on an empty input sequence.
• Test the program for robustness against erroneous inputs
such as ``R'' typed in as the .
• All failures of the test program should be recorded in a
suitable file using the Company Failure Report Form.
request character
 Test case/data

A test case is a pair consisting of test data to be input to

the program and the expected output. The test data is a
set of values, one for each input variable.

A test set is a collection of zero or more test cases.

Sample test case for sort:

Test data: <''A'’ 12 -29 32 >

Expected output: -29 12 32
 Testing Life Cycle
Project Initiation
Summary Reports
System Study
Analysis
Test Plan
Regression Test

Design Test Cases

Report Defects

Execute Test Cases

( manual /automated )
 Implementation and Testing
• V-Model: constructive and analytical part of software life
cycle

Requirements Analysis System Deployment

Requirements Testing Acceptance Testing

Architecture Design System Integration

Design Testing System Testing

Module Design Module Integration

Module Testing Integration Testing

Unit Design & Implementation

Unit Testing
Types of Requirements
• Functional requirements
• Performance requirements
• Speed, accuracy, frequency, throughput
• External interface requirements
• Design constraints
• Requirements are usually about “what”, this is a “how”.
• Quality attributes
• i.e. reliability, portability, maintainability, supportability
Test Plan and Test Case
• A test plan is a systematic approach to testing a system i.e.
software.
• The plan typically contains a detailed understanding of what
the eventual testing workflow will be.
• A test case is a specific procedure of testing a particular
requirement.
• It will include:
• Identification of specific requirement tested
• Test case success/failure criteria
• Specific steps to execute test
• Test Data
• Most important part of software testing is to determine a set
of test cases for the item to be tested.
• TEST CASE TEMPLATE
Test Suite ID The ID of the test suite to which this test case belongs.
Test Case ID The ID of the test case.
Test Case Summary The summary / objective of the test case.
Related
The ID of the requirement this test case relates/traces to.
Requirement
Any prerequisites or preconditions that must be fulfilled prior to
Prerequisites
executing the test.
Test Procedure Step-by-step procedure to execute the test.
The test data, or links to the test data, that are to be used while
Test Data
conducting the test.
Expected Result The expected result of the test.
Actual Result The actual result of the test; to be filled after executing the test.
Pass or Fail. Other statuses can be ‘Not Executed’ if testing is not
Status
performed and ‘Blocked’ if testing is blocked.
Remarks Any comments on the test case or test execution.
Created By The name of the author of the test case.
Date of Creation The date of creation of the test case.
Executed By The name of the person who executed the test.
Date of Execution The date of execution of the test.
The environment (Hardware/Software/Network) in which the test
Test Environment
was executed.
• Test case ID and Test case description – these are the
generic ones.
• The other fields can be explained as follows:
• Precondition – Circumstances that hold prior to test case
execution.
• Input - Data entry steps.
• For these steps it is important to note what kind of input info is
required – Test data
• Validation point/trigger/action:
• What is causing the validation to happen?
• Click of a button or toggle or the link access
• Output - expected result
• Post condition: What should be the state of the system after
executing this test case?
• Test Designed Date: Date when wrote
• Test Executed By: Name of tester who executed this test. To
be filled after test execution.
• Test Execution Date: Date when test executed.
• Test Title/Name: Test case title. E.g. verify login page with
valid username and password
• Test Summary/Description: Describe test objective in brief.
• Status (Pass/Fail): If actual result is not as per the expected
result mark this test as failed. Otherwise update as passed.