Software Validation and Verification

Software inspections
 These involve people examining the source representation with the aim of discovering
anomalies and defects.
 Inspections not require execution of a system so may be used before implementation.
 They may be applied to any representation of the system (requirements, design,
configuration data, test data, etc.).
 They have been shown to be an effective technique for discovering program errors.

Advantages of inspections
 During testing, errors can mask (hide) other errors. Because inspection is a static process,
you don’t have to be concerned with interactions between errors.
 Incomplete versions of a system can be inspected without additional costs. If a program is
incomplete, then you need to develop specialized test harnesses to test the parts that are
available.
 As well as searching for program defects, an inspection can also consider broader quality
attributes of a program, such as compliance with standards, portability and
maintainability.
 Inspections and testing are complementary and not opposing verification techniques.
 Both should be used during the V & V process.
 Inspections can check conformance with a specification but not conformance with the
customer’s real requirements.
 Inspections cannot check non-functional characteristics such as performance, usability,
etc.

Inspections and testing

 Inspections and testing are complementary and not opposing verification techniques.
 Both should be used during the V & V process.
 Inspections can check conformance with a specification but not conformance with the
customer’s real requirements.
 Inspections cannot check non-functional characteristics such as performance, usability,
etc.
 Software inspections Concerned with analysis of the static system representation to discover
problems (static verification)
 May be supplement by tool-based document and code analysis.
 Software testing Concerned with exercising and observing product behaviour (dynamic
verification)
 The system is executed with test data and its operational behaviour is observed.
 Figure: Relation between the software Inspection and testing

Figure: A model of the software testing process

As well as software testing, the verification and validation process may involve software
inspections and reviews. Inspections and reviews analyze and check the system requirements,
design models, the program source code, and even proposed system tests. These are so-called
‘static’ V & V techniques in which you don’t need to execute the software to verify it. Figure 8.2
shows those software inspections and testing support V & V at different stages in the software
process. The arrows indicate the stages in the process where the techniques may be used.

Inspections mostly focus on the source code of a system but any readable representation of the
software, such as its requirements or a design model, can be inspected. When you inspect a
system, you use knowledge of the system, its application domain, and the programming or
modeling language to discover errors.

Program inspections are an old idea and there have been several studies and experiments that
have demonstrated that inspections are more effective for defect discovery than program testing.
Fagan (1986) reported that more than 60% of the errors in a program can be detected using
informal program inspections. In the Cleanroom process (Prowell et al., 1999), it is claimed that
more than 90% of defects can be discovered in program inspections.

However, inspections cannot replace software testing. Inspections are not good for discovering
defects that arise because of unexpected interactions between different parts of a program, timing
problems, or problems with system performance.
Testing strategies
 Partition testing, where you identify groups of inputs that have common characteristics
and should be processed in the same way.
 You should choose tests from within each of these groups.
 Guideline-based testing, where you use testing guidelines to choose test cases.
 These guidelines reflect previous experience of the kinds of errors that
programmers often make when developing components.

Partition testing
 Input data and output results often fall into different classes where all members of a class
are related.
 Each of these classes is an equivalence partition or domain where the program behaves
in an equivalent way for each class member.
 Test cases should be chosen from each partition.

Figure: Equivalence partitioning

Figure: Equivalence partitioning

What is Software Testing?
As per ANSI/IEEE 1059, Testing in Software Engineering is a process of evaluating a software
product to find whether the current software product meets the required conditions or not. The
testing process involves evaluating the features of the software product for requirements in terms
of any missing requirements, bugs or errors, security, reliability and performance.

Two different components constitute together as Software Testing. They are Software Validation
and Software Verification.

What are the benefits of Software Testing?

Here are the benefits of using software testing:
 Cost-Effective: It is one of the important advantages of software testing. Testing any IT
project on time helps you to save your money for the long term. In case if the bugs caught
in the earlier stage of software testing, it costs less to fix.
 Security: It is the most vulnerable and sensitive benefit of software testing. People are
looking for trusted products. It helps in removing risks and problems earlier.
 Product quality: It is an essential requirement of any software product. Testing ensures a
quality product is delivered to customers.
 Customer Satisfaction: The main aim of any product is to give satisfaction to their
customers. UI/UX Testing ensures the best user experience.

Software Validation
The software is validated against the requirements of the user to verify that the requirements of
the user are being satisfied by the software. The software is always validated at the end of SDLC.
The software is considered for validation, only when it matches with the user requirements.
 You expect the system to perform correctly using a given set of test cases that reflect the
system’s expected use. .
 You expect the system to perform correctly using a given set of test cases that reflect the
system’s expected use.
 Validation emphasizes on user requirements.
 Methods of Validation: Dynamic Testing

Software Verification
The process of evaluating work-products (not the actual final product) of a development phase to
determine whether they meet the specified requirements for that phase. Verification is a static
practice of verifying documents, design, code and program. This process of ensuring is known as
verification.
 Verification process checks whether all the design specifications are followed while
developing the product.
  Verification is concerned with whether the system is well-engineered and error-free.
 Verification will help to determine whether the software is of high quality, but it will not
ensure that the system is useful.
 Methods of Verification: Static Testing

Barry Boehm described verification and validation as the following:

Validation: Are we building the right product? - The software should do what the user really
requires.
Verification: Are we building the product right? - The software should conform to its
specification.

Software Tester Job Description Template

The Software Tester will conduct tests, analyze the results, and report observations to the design
team. You may also interact with clients in order to understand the requirements of the product.
To be successful as a Software Tester, you should have a working knowledge of software and test
design, the capability to run through tests, and the ability to analyze the results. Ultimately, the
Software Tester should be result-driven, have good communication skills, and up-to-date
knowledge of software programming and software test design.
Software Tester Responsibilities:
1) Analyzing the Requirements from the client
2) Participating in preparing Test Plans
3) Preparing Test Scenarios, test cases
4) Defect Tracking
5) Preparing Suggestion Documents to improve the quality of the application
6) Communication with the Test Lead / Test Manager
7) Conducting Review Meetings within the Team

Dynamic Testing Technique:

 It is done during Validation Process evaluating the finished product.
 In dynamic testing technique, software is executed using a set of input values and its output
is then examined and compared to what is expected. Dynamic execution is applied as a
technique to detect defects and to determine the quality attributes of the code.
 But as it does not start early in the Software Development Life Cycle hence it definitely
increases the cost of fixing defects.

Types of Dynamic Testing

Dynamic Testing is classified into two categories
  White Box Testing
 Black Box Testing

The below pictorial representation gives us an idea about types of Dynamic Testing, Levels of
Testing, etc.

White-box testing
White Box Testing is a software testing method in which the internal structure/ design is known
to the tester. The main aim of White Box testing is to check on how System is performing based
on the code. It is mainly performed by the Developers or White Box Testers who has knowledge
on the programming.
Here, the tester is made available with the code and the design. This test is conducted directly on
the code by the programmer.

Some of the techniques of White-box testing are as follows -

 Control-flow testing – Considering all the statements and conditions, test cases are set up
for incorporating control-flow testing. By covering all the statements, the conditions are
tested for true and false.
 Data-flow testing – The program data variables are considered for testing. The variables are
tested for their declaration and definition and their usage.

Black-box testing
Black Box Testing is a method of testing in which the internal structure/ code/design
is NOT known to the tester. The main aim of this testing to verify the functionality of the system
under test and this type of testing requires to execute the complete test suite and is mainly
performed by the Testers, and there is no need of any programming knowledge.

This test is conducted by the test engineers and users and hence they are not known with the code
and the structure of the program.
Different techniques of Black-box testing are as follows:
 Equivalence class – The input is split into sub classes having different elements. The
complete class is considered to pass the test when any one element of the class passes the
test.
 Boundary values – Higher and lower end values of the input are identified. All the values
are considered to pass the test when both of these values pass the test.
 Cause-effect graphing – A combination of input values are tested at a time under cause-
effect graphing unlike the other two methods of testing.
 Pair-wise Testing – Different parameters are considered and are paired and tested. This pair
is again tested for different values under pair-wise testing.
 State-based testing – On the basis of the input state, the software is tested.

The Black Box Testing is again classified into two types. They are:

 Functional Testing
 Non-Functional Testing
Functional Testing:
Functional testing is performed to verify that all the features developed are according to the
functional specifications, and it is performed by executing the functional test cases written by the
QA team, in functional testing phase, system is tested by providing input, verifying the output
and comparing the actual results with the expected results.

There are different Levels of Functional Testing out of which the most important are

 Unit Testing – Generally Unit is a small piece of code which is testable, Unit Testing is
performed at individual unit of software and is performed by developers
 Integration Testing - Integration Testing is the testing which is performed after Unit
Testing and is performed by combining all the individual units which are testable and is
performed either by developers or testers
 System Testing - System Testing is a performed to ensure whether the system performs
as per the requirements and is generally performed when the complete system is ready, it
is performed by testers when the Build or code is released to QA team
 Acceptance Testing - Acceptance testing is performed to verify whether the system has
met the business requirements and is ready to use or ready for deployment and is
generally performed by the end users.
 Alpha testing -The software is made to appear as it appears to the user and the software is
tested by the developers themselves. This testing helps in identifying the reactions and
experience of the user when the software is being used and how the software is responding
and reacting to the inputs provided by the user.
 Beta testing -Once Alpha testing is done, there is one more level of testing, which is done
by involving the users. The software is handed over to the user and is tested but still under
the control of the developers and under the production environment. This testing enables to
identify and figure out the problems that are very minute and hence are being missed out at
previous levels of testing.

Regression Testing
Regression testing is done at each time the software is being updated with a new code, added with a
new feature or function. Regression testing is done to identify if there is any negative impact of the
software by the addition of the code.

Non- Functional Testing: Non-Functional testing is a testing technique which does not focus on
functional aspects and mainly concentrates on the nonfunctional attributes of the system such as
memory leaks, performance or robustness of the system. Non-Functional testing is performed at
all test levels.
There are many Non-Functional Testing Techniques out of which the most important are

 Performance Testing – Performance Testing is performed to check whether the response

time of the system is normal as per the requirements under the desired network load.
 Recovery Testing - Recovery testing is a method to verify on how well a system is able
to recover from crashes and hardware failures.
 Compatibility Testing – Compatibility testing is performed to verify how the system
behaves across different environments.
 Security testing – Security testing is performed to verify the robustness of the
application, i.e to ensure that only the authorizes users/roles are accessing the system
 Usability testing – Usability testing is a method to verify the usability of the system by
the end users to verify on how comfortable the users are with the system.

Static Testing Techniques

It provides a powerful way to improve the quality and productivity of software development by
assisting engineers to recognize and fix their own defects early in the software development
process. In this software is tested without executing the code by doing Review, Walk Through,
Inspection or Analysis etc.
Static Testing may be conducted manually or through the use of various software testing tools. It
starts early in the Software Development Life Cycle and so it is done during the Verification
Process.
Static testing is not magic and it should not be considered a replacement for Dynamic Testing,
but all software organizations should consider using reviews in all major aspects of their work
including requirements, design, implementation, testing, and maintenance.
Types of defects that are easier to find during static testing are: deviations from standards,
missing requirements, design defects, non-maintainable code and inconsistent interface
specifications.

Techniques of Static Testing

 Review – Typically used to find and eliminate errors or ambiguities in documents such as
requirements, design, test cases, etc.
 Informal Review
 WalkThrough
 Technical Review
 Inspection
What are the different types of Software Testing?
Software testing can be of two types – Manual testing or automated testing.
 Manual Testing: Test Cases executed manually.
 Automation Testing: Testing performed with the help of automation tools.

Manual Testing:
According to the sections and levels of the code, the test cases are generated by the software
tester. On the basis of these test cases the tests are carried out and the manager is informed about
the results.
It is a known fact that anything done manually takes more time and so the testing that is done
manually also takes more time and thus most of the resources are consumed.

Automated Testing:
Some of the specific automated testing tools are used for software testing. These are developed
to overcome the disadvantages and to reduce the consumption of time, effort and the resources.
For instance, manual testing can take up and test the opening of the web page on any web
browser. But it cannot take up and test for checking load of 1 million users by a web-server.
Testing related to load, stress and regression are conducted and checked by using some of the
hardware and software tools provided thus simplifying the work of the tester.

Automated testing tool : TestComplete

TestComplete is a open test platform that helps you create reusable, robust and automated tests
across desktop, web, mobile, and multiple devices easily and effectively. Silverlight applications
can also be tested using TestComplete.It allows testers to create automated tests for Microsoft
Windows, Web, Android, and iOS apps.

How to perform manual testing?

A complete manual testing process consists of the following steps:
 Let’s understand the details of all these steps:
Step 1: First, gather the requirements using the requirement analysis step. Once we gather and
understand the requirements, we know what the expected behavior is and what we need to test,
and when we say we have found the defect.
Step 2: Secondly, once we understand the requirements, we identify and draft the test cases that
will cover all the requirements contained in the project documentation. Additionally, the test
cases help us follow a sequence to test functionality and various test scenarios such that we cover
the entire application and check expected results.
Step 3: Once test cases are ready, the tester has to review the test cases with the team leader and
with the client if need be. By examining the test cases, we will find glitches, if any, and correct
them before executing the test cases.
Step 4: Once test cases are ready, and the test environment sets, we execute the test cases one by
one. Each test case will have one of the following states:
 Passed: If the scenario under test works as expected.
 Failed: If the working is not as expected.
Step 5: As the test cases execute, we have to report the identified bugs and defects to the
concerned developer and submit a bug report.
Step 6: Finally, we create a detailed test report that will include detailed information on how
many defects or bugs we found, how many test cases need to be rerun, how many test cases
failed, and how many we skipped. Once we fix the bugs and defects, execute the test cases that
could not verify the fixed bugs.
What is the difference between Manual and Automated testing?
Below are a few of the significant differences between manual and automation testing:

Comparison Parameter Manual Testing Automated Testing

Testers manually execute test Uses tools to schedule and
Execution
cases. execute the test cases.
The manual test takes up lots of Automated testing: since the
Time and cost time and implies a high cost. test cases are automated, it
saves time and is very low.
The process is such that it is Since the automation tool
Nature
repetitive and boring. handles the execution, the
tester skips the boring part.
Low reliability as manual High accuracy since all test
Reliability and Accuracy verification is prone to human cases are automated and
error executed by tools
Does not guarantee user-
More user-friendly and guarantees
User interface friendliness or good
improved customer experience
customer experience.

Testing Process

Testing is a complete process. For testing we need two types of inputs.First is software
configuration. It includes software requirement specification, design specifications and source
code of program.Second is test configuration. It is basically test plan and procedure.

Software configuration is required so that the testers know what is to be expected and tested
whereas test configuration is testing plan that is, the way how the testing will be conducted on
the system. It specifies the test cases and their expected value. It also specifies if any tools for
testing are to be used. Test cases are required to know what specific situations need to be tested.
When tests are evaluated, test results are compared with actual results and if there is some error,
then debugging is done to correct the error. Testing is a way to know about quality and
reliability. Error rate that is the occurrence of errors is evaluated. This data can be used to predict
the occurrence of errors in future.
 Fig . Testing Process

Test Case design

We now know, test cases are integral part of testing. So we need to know more about test cases
and how these test cases are designed. The most desired or obvious expectation from a test case
is that it should be able to find most errors with the least amount of time and effort.

A software product can be tested in two ways. In the first approach only the overall functioning
of the product is tested. Inputs are given and outputs are checked. This approach is called black
box testing. It does not care about the internal functioning of the product.

The other approach is called white box testing. Here the internal functioning of the product is
tested. Each procedure is tested for its accuracy. It is more intensive than black box testing. But
for the overall product both these techniques are crucial. There should be sufficient number of
tests in both categories to test the overall product.

Basic Path Testing

Basic path testing a white box testing technique .It was proposed by Tom McCabe.These tests
guarantee to execute every statement in the program at least one time during testing. Basis set is
the set of all the execution path of a procedure.

Flow graph Notation

Before basic path procedure is discussed, it is important to know the simple notation used for the
representation of control flow. This notation is known as flow graph. Flow graph depicts control
flow and uses the following constructs.

These individual constructs combine together to produce the flow graph for a particular
procedure
Sequence - Until -

If - While - Case -

Basic terminology associated with the flow graph

Node: Each flow graph node represents one or more procedural statements. Each node that
contains a condition is called a predicate node.

Edge: Edge is the connection between two nodes. The edges between nodes represent flow of
control. An edge must terminate at a node, even if the node does not represent any useful
procedural statements.

Region: A region in a flow graph is an area bounded by edges and nodes.

Cyclomatic complexity: Independent path is an execution flow from the start point to the end
point. Since a procedure contains control statements, there are various execution paths depending
upon decision taken on the control statement. So Cyclomatic complexity provides the number of
such execution independent paths. Thus it provides a upper bound for number of tests that must
be produced because for each independent path, a test should be conducted to see if it is actually
reaching the end point of the procedure or not.

Cyclomatic Complexity

Cyclomatic Complexity for a flow graph is computed in one of three ways:

1. The numbers of regions of the flow graph correspond to the Cyclomatic complexity.
2. Cyclomatic complexity, V(G), for a flow graph G is defined as
V(G) = E – N + 2
where E is the number of flow graph edges and N is the number of flow graph nodes.
3. Cyclomatic complexity, V(G), for a graph flow G is also defined as
V(G) = P + 1
Where P is the number of predicate nodes contained in the flow graph G.

Example: Consider the following flow graph

Region, R= 6
Number of Nodes = 13
Number of edges = 17
Number of Predicate Nodes = 5

Cyclomatic Complexity, V( C) :

V( C ) = R = 6;
Or

V(C) = Predicate Nodes + 1

=5+1 =6
Or

V( C)= E-N+2
= 17-13+2

Deriving Test Cases

The main objective of basic path testing is to derive the test cases for the procedure under test.
The process of deriving test cases is following

1. From the design or source code, derive a flow graph.

2. Determine the Cyclomatic complexity, V(G) of this flow graph using any of the formula
discussed above.
· Even without a flow graph, V(G) can be determined by counting the number of conditional
statements in the code and adding one to it.
3. Prepare test cases that will force execution of each path in the basis set.
· Each test case is executed and compared to the expected results.

Graph Matrices

Graph matrix is a two dimensional matrix that helps in determining the basic set. It has rows and
columns each equal to number of nodes in flow graph. Entry corresponding to each node-node
pair represents an edge in flow graph. Each edge is represented by some letter (as given in the
flow chart) to distinguish it from other edges. Then each edge is provided with some link weight,
0 if there is no connection and 1 if there is connection.

For providing weights each letter is replaced by 1 indicating a connection. Now the graph matrix
is called connection matrix.Each row with two entries represents a predicate node. Then for each
row sum of the entries is obtained and 1 is subtracted from it. Now the value so obtained for each
row is added and 1 is again added to get the cyclomatic complexity.

Once the internal working of the different procedure are tested, then the testing for the overall
functionality of program structure is tested. For this black box testing techniques are used which
are discussed in the following pages.

Some key information:

 Testing can only show the presence of errors in a program. It cannot demonstrate that there
are no remaining faults.
 Development testing is the responsibility of the software development team. A separate team
should be responsible for testing a system before it is released to customers. In the user
testing process, customers or system users provide test data and check that tests are
successful.
 Development testing includes unit testing, in which you test individual objects and methods;
component testing, in which you test related groups of objects; and system testing, in which
you test partial or complete systems.
 When testing software, you should try to ‘break’ the software by using experience and
guidelines to choose types of test cases that have been effective in discovering defects in
other systems.
 Wherever possible, you should write automated tests. The tests are embedded in a program
that can be run every time a change is made to a system.
 Test-first development is an approach to development where tests are written before the code
to be tested. Small code changes are made and the code is refactored until all tests execute
successfully.
 Scenario testing is useful because it replicates the practical use of the system. It involves
inventing a typical usage scenario and using this to derive test cases.
 Acceptance testing is a user testing process where the aim is to decide if the software is good
enough to be deployed and used in its operational environment.

How software testing is differentiated from Quality Control, Quality

Assurance and Audit?
 There is a significant difference between software testing and software quality assurance,
software auditing and software quality control.
Software quality assurance
 The software quality assurance is concerned with the aspect that the measures taken by the
process of software development are in accordance with the set standards. The methods adopted
for software development are ensured to be the best by this quality assurance.
Software quality control
 The software product quality is maintained by the software quality control measure. By
improving the software product quality, the goodwill of the organization can be developed. The
main aim of the software quality control is to ensure that the best quality product is developed for
the customer.
Software audit
 The organization methods that are implemented for developing the software are being reviewed
by the software audit. The process of the software, the software procedures and all other
components of SDLC are reviewed by a team of auditors who are no way related to the
development team. Software audit is done to ensure that the process of the software development
is in accordance with the set standards, rules and regulations.