Software Testing

Software Testing is evaluation of the software against requirements gathered from users and system
specifications. Testing is conducted at the phase level in software development life cycle or at
module level in program code. Software testing comprises of Validation and Verification.
Software Validation
Validation is process of examining whether or not the software satisfies the user requirements. It is
carried out at the end of the SDLC. If the software matches requirements for which it was made, it
is validated.
 Validation ensures the product under development is as per the user requirements.
 Validation answers the question – "Are we developing the product which attempts all
that user needs from this software ?".
 Validation emphasizes on user requirements.
Software Verification
Verification is the process of confirming if the software is meeting the business requirements, and is
developed adhering to the proper specifications and methodologies.
 Verification ensures the product being developed is according to design specifications.
 Verification answers the question– "Are we developing this product by firmly following
all design specifications ?"
 Verifications concentrates on the design and system specifications.
Target of the test are -
 Errors - These are actual coding mistakes made by developers. In addition, there is a
difference in output of software and desired output, is considered as an error.
 Fault - When error exists fault occurs. A fault, also known as a bug, is a result of an
error which can cause system to fail.
 Failure - failure is said to be the inability of the system to perform the desired task.
Failure occurs when fault exists in the system.
Manual Vs Automated Testing
Testing can either be done manually or using an automated testing tool:
 Manual - This testing is performed without taking help of automated testing tools. The
software tester prepares test cases for different sections and levels of the code, executes
the tests and reports the result to the manager. Manual testing is time and resource
consuming. The tester needs to confirm whether or not right test cases are used. Major
portion of testing involves manual testing.
 Automated This testing is a testing procedure done with aid of automated testing tools.
The limitations with manual testing can be overcome using automated test tools.
A test needs to check if a webpage can be opened in Internet Explorer. This can be easily
done with manual testing. But to check if the web-server can take the load of 1 million
users, it is quite impossible to test manually. There are software and hardware tools
which helps tester in conducting load testing, stress testing, regression testing.
Testing Approaches
Tests can be conducted based on two approaches –
 Functionality testing
 Implementation testing
When functionality is being tested without taking the actual implementation in concern it is
known as black-box testing. The other side is known as white-box testing where not only
functionality is tested but the way it is implemented is also analyzed.
Exhaustive tests are the best-desired method for a perfect testing. Every single possible value in the
range of the input and output values is tested. It is not possible to test each and every value in real
world scenario if the range of values is large.
Black-box testing
It is carried out to test functionality of the program. It is also called ‘Behavioral’ testing. The
tester in this case, has a set of input values and respective desired results. On providing input, if the
output matches with the desired results, the program is tested ‘ok’, and problematic otherwise.

In this testing method, the design and structure of the code are not known to the tester, and
testing engineers and end users conduct this test on the software.
Black-box testing techniques:
 Equivalence class - The input is divided into similar classes. If one element of a class
passes the test, it is assumed that all the class is passed.
 Boundary values - The input is divided into higher and lower end values. If these
values pass the test, it is assumed that all values in between may pass too.
 Cause-effect graphing - In both previous methods, only one input value at a time is
tested. Cause (input) – Effect (output) is a testing technique where combinations of
input values are tested in a systematic way.
 Pair-wise Testing - The behavior of software depends on multiple parameters. In
pairwise testing, the multiple parameters are tested pair-wise for their different values.
 State-based testing - The system changes state on provision of input. These systems
are tested based on their states and input.
White-box testing
It is conducted to test program and its implementation, in order to improve code efficiency
or structure. It is also known as ‘Structural’ testing.

In this testing method, the design and structure of the code are known to the tester. Programmers of
the code conduct this test on the code.
The below are some White-box testing techniques:
 Control-flow testing - The purpose of the control-flow testing to set up test cases
which covers all statements and branch conditions. The branch conditions are tested for
both being true and false, so that all statements can be covered.
  Data-flow testing - This testing technique emphasis to cover all the data variables
included in the program. It tests where the variables were declared and defined and
where they were used or changed.
White Box Testing Techniques:
 Statement Coverage - This technique is aimed at exercising all programming statements
with minimal tests.
 Branch Coverage - This technique is running a series of tests to ensure that all branches
are tested at least once.
 Path Coverage - This technique corresponds to testing all possible paths which means that
each statement and branch is covered.
Calculating Structural Testing Effectiveness:
Statement Testing = (Number of Statements Exercised / Total Number of Statements) x 100 %
Branch Testing = (Number of decisions outcomes tested / Total Number of decision Outcomes) x
100 %
Path Coverage = (Number paths exercised / Total Number of paths in the program) x 100 %

Advantages of White Box Testing:

 Forces test developer to reason carefully about implementation.
 Reveals errors in "hidden" code.
 Spots the Dead Code or other issues with respect to best programming practices.
Disadvantages of White Box Testing:
 Expensive as one has to spend both time and money to perform white box testing.
 Every possibility that few lines of code are missed accidentally.
 In-depth knowledge about the programming language is necessary to perform white box
testing.
Testing Levels
Testing itself may be defined at various levels of SDLC. The testing process runs parallel to
software development. Before jumping on the next stage, a stage is tested, validated and verified.
Testing separately is done just to make sure that there are no hidden bugs or issues left in the
software. Software is tested on various levels –

Unit Testing
While coding, the programmer performs some tests on that unit of program to know if it is
error free. Testing is performed under white-box testing approach. Unit testing helps developers
decide that individual units of the program are working as per requirement and are error free.
Integration Testing
Even if the units of software are working fine individually, there is a need to find out if the
units if integrated together would also work without errors. For example, argument passing and data
updation etc.
System Testing
The software is compiled as product and then it is tested as a whole. This can be
accomplished using one or more of the following tests:
 Functionality testing - Tests all functionalities of the software against the requirement.
 Performance testing - This test proves how efficient the software is. It tests the
effectiveness and average time taken by the software to do desired task. Performance
testing is done by means of load testing and stress testing where the software is put
under high user and data load under various environment conditions.
 Security & Portability - These tests are done when the software is meant to work on
various platforms and accessed by number of persons.
Acceptance Testing
When the software is ready to hand over to the customer it has to go through last phase of
testing where it is tested for user-interaction and response. This is important because even if the
software matches all user requirements and if user does not like the way it appears or works, it may
be rejected.
 Alpha testing - The team of developer themselves perform alpha testing by using the
system as if it is being used in work environment. They try to find out how user would
react to some action in software and how the system should respond to inputs.
 Beta testing - After the software is tested internally, it is handed over to the users to use
it under their production environment only for testing purpose. This is not as yet the
delivered product. Developers expect that users at this stage will bring minute problems,
which were skipped to attend.
Regression Testing
Whenever a software product is updated with new code, feature or functionality, it is tested
thoroughly to detect if there is any negative impact of the added code. This is known as regression
testing.
Testing Documentation
Testing documents are prepared at different stages -
Before Testing
Testing starts with test cases generation. Following documents are needed for reference –
 SRS document - Functional Requirements document
 Test Policy document - This describes how far testing should take place before
releasing the product.
 Test Strategy document - This mentions detail aspects of test team, responsibility
matrix and rights/responsibility of test manager and test engineer.
 Traceability Matrix document - This is SDLC document, which is related to
requirement gathering process. As new requirements come, they are added to this
matrix. These matrices help testers know the source of requirement. They can be
traced forward and backward.
While Being Tested
The following documents may be required while testing is started and is being done:
 Test Case document - This document contains list of tests required to be conducted.
It includes Unit test plan, Integration test plan, System test plan and Acceptance test
plan.
 Test description - This document is a detailed description of all test cases and
procedures to execute them.
 Test case report - This document contains test case report as a result of the test.
  Test logs - This document contains test logs for every test case report.
After Testing
The following documents may be generated after testing :
 Test summary - This test summary is collective analysis of all test reports and logs. It
summarizes and concludes if the software is ready to be launched. The software is
released under version control system if it is ready to launch.
Testing vs. Quality Control, Quality Assurance and Audit
We need to understand that software testing is different from software quality assurance, software
quality control and software auditing.
 Software quality assurance - These are software development process monitoring
means, by which it is assured that all the measures are taken as per the standards of
organization. This monitoring is done to make sure that proper software development
methods were followed.
 Software quality control - This is a system to maintain the quality of software product.
It may include functional and non-functional aspects of software product, which
enhance the goodwill of the organization. This system makes sure that the customer is
receiving quality product for their requirement and the product certified as ‘fit for use’.
 Software audit - This is a review of procedure used by the organization to develop the
software. A team of auditors, independent of development team examines the software
process, procedure, requirements and other aspects of SDLC. The purpose of software
audit is to check that software and its development process, both conform standards,
rules and regulations.

Integration Testing Unit Testing

The impression behind Integration Testing is to
The impression behind Unit Testing is to test
integrate all modules into the application and test
each part of the individual module and observe
them as a group to observe that they are working
that the individual parts are working as expected.
as expected.
It is also known as black box testing and internal
It is also known as white box testing as it
code of the application module is not visible and
requires the knowledge of the code and the
focus is on the given input and the expected
control flow within the program.
output.
Integration testing is carried out after unit testing Unit testing can be performed anytime but
but before system testing. always before integration testing.
Unit Testing tests only the functionality of the
Defects in Integrating testing are detected after modules at unit level and can not catch
modules are integrated to build the overall system integration level defects, or any other system-
wide issues.
It surfaces from interface specifications. It surfaces from module specification.
It is mainly focused on the functionality of an
It is mainly focused on module integration.
individual module.
Integration testing is conducted by testers. Unit testing is usually conducted by developers.
Error detection’s are usually difficult at integration Error detection’s are usually simple at unit
testing. testing.
Unit test maintenance cost is very less as it is
Integration test maintenance is quite expensive as
maintained and unit level with the help of Junit,
it requires the separate environment setup.
Nunit, TestNG, etc.
Integration tests helps to verify that our code Unit test does not verify if our code works as
works as expected with the external dependencies. expected with the external dependencies.

VALIDATION AND VERIFICATION TESTING

Difference between software Verification and Validation:

Verification Validation
Are we building the system right? Are we building the right system?
Verification is the process of evaluating Validation is the process of evaluating software
products of a development phase to find out at the end of the development process to
whether they meet the specified requirements. determine whether software meets the customer
expectations and requirements.
The objective of Verification is to make sure The objective of Validation is to make sure that
that the product being develop is as per the the product actually meet up the user’s
requirements and design specifications. requirements, and check whether the
specifications were correct in the first place.
Following activities are involved in Following activities are involved in Validation:
Verification: Reviews, Meetings and Testing like black box testing, white box testing,
Inspections. gray box testing etc.
Verification is carried out by QA team to check Validation is carried out by testing team.
whether implementation software is as per
specification document or not.
Execution of code is not comes under Execution of code is comes under Validation.
Verification.
Verification process explains whether the Validation process describes whether the
outputs are according to inputs or not. software is accepted by the user or not.
Verification is carried out before the Validation activity is carried out just after the
Validation. Verification.
Following items are evaluated during Following item is evaluated during Validation:
Verification: Plans, Requirement Actual product or Software under test.
Specifications, Design Specifications, Code,
Test Cases etc,
Cost of errors caught in Verification is less Cost of errors caught in Validation is more than
than errors found in Validation. errors found in Verification.
It is basically manually checking the of It is basically checking of developed program
documents and files like requirement based on the requirement specifications
specifications etc. documents & files.

Other Testing Types are,

Agile Testing
A software testing practice that follows the principles of agile software development is called Agile
Testing. Agile is an iterative development methodology, where requirements evolve through
collaboration between the customer and self-organizing teams and agile aligns development with
customer needs.
Advantages of Agile Testing
 Agile Testing Saves Time and Money
 Less Documentation
 Regular feedback from the end user
  Daily meetings can help to determine the issues well in advance
Principles of Agile Testing
 Testing is NOT a Phase: Agile team tests continuously and continuous testing is the only
way to ensure continuous progress.
 Testing Moves the project Forward: When following conventional methods, testing is
considered as quality gate but agile testing provide feedback on an ongoing basis and the
product meets the business demands.
 Everyone Tests: In conventional SDLC, only test team tests while in agile including
developers and BA's test the application.
 Shortening Feedback Response Time: In conventional SDLC, only during the acceptance
testing, the Business team will get to know the product development, while in agile for
each and every iteration, they are involved and continuous feedback shortens the feedback
response time and cost involved in fixing is also less.
 Clean Code: Raised defects are fixed within the same iteration and thereby keeping the
code clean.
 Reduce Test Documentation: Instead of very lengthy documentation, agile testers use
reusable checklist, focus on the essence of the test rather than the incidental details.
 Test Driven: In conventional methods, testing is performed after implementation while in
agile testing, testing is done while implementation.

The best practices for agile testing are,

1. Automated Unit Tests
2. Test Driven Development
3. Automated Regression Tests
4. Exploratory Testing
Adhoc Testing
When a software testing performed without proper planning and documentation, it is said to
be Adhoc Testing. Such kind of tests are executed only once unless we uncover the defects.
The tester has to find defects without any proper planning and documentation, solely based
on tester's intuition.
Adhoc testing can be performed when there is limited time to do exhaustive testing and usually
performed after the formal test execution. Adhoc testing will be effective only if the tester has in-
depth understanding about the System Under Test.
Forms of Adhoc Testing :
1. Buddy Testing: Two buddies, one from development team and one from test team mutually
work on identifying defects in the same module. Buddy testing helps the testers develop
better test cases while development team can also make design changes early. This kind of
testing happens usually after completing the unit testing.
2. Pair Testing: Two testers are assigned the same modules and they share ideas and work on
the same systems to find defects. One tester executes the tests while another tester records
the notes on their findings.
3. Monkey Testing: Testing is performed randomly without any test cases in order to break
the system.
Various ways to make Adhoc Testing More Effective
1. Preparation: By getting the defect details of a similar application, the probability of
finding defects in the application is more.
2. Creating a Rough Idea: By creating a rough idea in place the tester will have a focussed
approach. It is NOT required to document a detailed plan as what to test and how to test.
3. Divide and Rule: By testing the application part by part, we will have a better focus and
better understanding of the problems if any.
4. Targeting Critical Functionalities: A tester should target those areas that are NOT covered
while designing test cases.
 5. Using Tools: Defects can also be brought to the lime light by using profilers, debuggers and
even task monitors. Hence being proficient in using these tools one can uncover several
defects.
6. Documenting the findings: Though testing is performed randomly, it is better to document
the tests if time permits and note down the deviations if any. If defects are found,
corresponding test cases are created so that it helps the testers to retest the scenario

GREY BOX TESTING

Grey Box testing is testing technique performed with limited information about the internal
functionality of the system. Grey Box testers have access to the detailed design documents along
with information about requirements.
Grey Box tests are generated based on the state-based models, UML Diagrams or
architecture diagrams of the target system.

Gray-box testing Techniques:

 Regression testing
 Pattern Testing
 Orthogonal array testing
 Matrix testing
Benefits:
 Grey-box testing provides combined benefits of both white-box and black-box testing
 It is based on functional specification, UML Diagrams, Database Diagrams or architectural
view
 Grey-box tester handles can design complex test scenario more intelligently
 The added advantage of grey-box testing is that it maintains the boundary between
independent testers and developers
Drawbacks:
 In grey-box testing, complete white box testing cannot be done due to inaccessible source
code/binaries.
 It is difficult to associate defects when we perform Grey-box testing for a distributed
system.
Best Suited Applications:
Grey-box testing is a perfect fit for Web-based applications.
Grey-box testing is also a best approach for functional or domain testing.