Chapter 5 TESTING TOOLS AND MEASUREMENTS

1. List and explain advantages of using software testing tools

Advantages of using tools.
There are many benefits of using the testing tools for supporting the software testing
process. They are :
1. It Saves Time Most of the testers find problems with the time required to write long
test scripts to perform testing especially when it is regression testing.
This takes a lot of time of the testers and the delivery of the bug-free application is
delayed.(avoid late delivery)
A delayed product delivery is not good for any business.
2. Checks Quality
Once manual testing procedures are finished, if one applies automated testing processes,
it helps to cross-check the test results. Thus, improves the quality of the manual test
scripts.
3. Early Bug Detection
While developing a software, bugs are easily found when software testing is performed
via automated software testing tools. This can save a lot of time and efforts on the SDLC.
4. Performing Tests 24/7
The tests are exerted without stoppage day and night. The machines and tools never get
tired. Whatever may be the quantity if testing, can be performed without any kind of
human intervention and continuously. The test results are also produced automatically
after the tests are performed.
5. Reusability
This goes perfectly very easy to understand with automated testing. When the have test
scripts are already prepared using test automation tools, they are saved for the future
requirements. So, they can be utilized as many times as the testers want especially
for automating regression testing.
6. Distributed Test Execution
Automated testing comes with distributed testing feature. One can easily execute the test
scripts on more than one computer on a shared network or server simultaneously. So,
more than one tool is not needed, but only one automated testing tool will be needed.
7. Easy & Robust Reporting
Automation testing tools have this amazing benefit of tracking each and every test script.
Each and every test script executed can be seen in visual logs. These reports can clearly
show the no. of test scripts already executed, scheduled, their reported bugs and how they
had been fixed.
8. Testing Capabilities
When it comes to capabilities, automated testing tools can test the web applications on
the various browsers available in the market via browser testing automation.
Also, when it comes to mobile application testing, one can test them on various devices.
This is next to impossible to achieve with manual testing.
9. Improves Test Coverage
Many a time, there are test cases with more than thousand lines of code and to write it
and test it would be very difficult with manual testing. This can be easily done with
automated testing. Also, these tools will make sure about the in and out of the application
like the databases, UI, web services, etc. works according to the requirements. Thus,
improving the overall test coverage.

10. Manpower Utilization

This is relative to the above benefits. Implementing test automation in testing processes
will require less manual efforts. Thus, it will decrease the number of people on just one
particular project and can utilize them in the various testing projects.

11. Improves Team Motivation

This is a big challenge in the testing industry. When the manpower is distributed among
various testing projects, the team can learn about new challenges, new bugs, new skill
sets,
Etc.

12. Testing Flexibility

The automated testing tools are developed by the teams who have been into manual
testing
for years. So, these tools are going to be flexible to match the future testing
specifications.
These testing tools can be utilized for longer periods of time — say for years.
2. Differentiate between static testing tools and dynamic testing tools.

3.
State the need of software measurement and describe different types of metrics.
->Need of Software measurement
Software measurement is essential in testing for several reasons, as it provides
objective data to help guide, assess, and improve the testing process. Here’s why
software measurement is needed:
1. Improves Quality and Reliability
• Purpose: Measurements help ensure the software meets quality standards and is
reliable.
• How: By measuring metrics like defect density, code coverage, and test effectiveness,
testers can identify and address weaknesses in the software.
2. Enables Informed Decision-Making
• Purpose: Provides data-driven insights for making key decisions.
• How: Metrics like defect trends, test case pass/fail rates, and requirement coverage
help stakeholders decide on release readiness and areas that need further testing.
3. Tracks and Manages Progress
• Purpose: Helps monitor testing progress and track project timelines.
• How: Test case execution, defect discovery, and test completion percentages give
insight into testing progress and whether the team is on schedule.
 4. Identifies Areas for Improvement
• Purpose: Pinpoints inefficient processes and improves testing efficiency.
• How: Measurements of test case effectiveness and defect distribution allow teams to
focus on high-risk areas, optimize test cases, and eliminate redundant tests.
5. Enhances Resource Allocation
• Purpose: Ensures efficient use of testing resources.
• How: By understanding metrics like test execution time, effort required per test case,
and defect density, teams can allocate time and resources more effectively.
6. Facilitates Benchmarking and Continuous Improvement
• Purpose: Allows teams to measure progress over time and improve continuously.
• How: Metrics enable comparisons against past performance and industry standards,
helping teams improve future projects.
Common Software Testing Metrics
• Defect Density: Number of defects per module or lines of code.
• Code Coverage: Percentage of code executed during testing.
• Test Case Effectiveness: Ratio of passed test cases versus the total.
• Defect Removal Efficiency (DRE): Percentage of defects found and removed before
release.

1. Product Metrics in Testing

Definition: These metrics assess the quality of the software product being tested.
Purpose: To evaluate how well the software meets quality standards and requirements.
Examples:
• Defect Density: Measures the number of defects found in a specific size of the
software (e.g., per 1,000 lines of code). This helps determine the overall quality of
the product.
• Test Coverage: Indicates the percentage of code or functionalities tested, showing
how thoroughly the software has been evaluated.
• Defect Severity: Categorizes defects based on their impact (e.g., critical, major,
minor), helping prioritize fixes and assess product stability.
User Satisfaction: Gathered through feedback and surveys after testing phases,
reflecting how well the software meets user expectations
2. Project Metrics in Testing
Definition: These metrics evaluate the testing phase within the broader context of the
software development project.
Purpose: To monitor the efficiency and effectiveness of the testing efforts in relation to
project timelines and resources.
Examples:
• Test Execution Rate: The percentage of planned test cases that have been executed
within a certain timeframe, providing insight into testing progress.
 • Defect Discovery Rate: Tracks how many defects are identified during testing phases,
helping to assess the effectiveness of testing efforts and identifying if further testing
is needed.
• Test Case Pass Rate: The percentage of test cases that pass successfully, which helps
gauge the stability of the software at various stages.
• Schedule Variance: Compares planned testing timelines with actual timelines,
helping identify delays and potential issues in resource allocation.
3. Process Metrics in Testing
Definition: These metrics evaluate the testing processes and methodologies used during
software testing.
Purpose: To optimize testing practices and improve overall efficiency.
Examples:
• Mean Time to Detect (MTTD): The average time taken to identify defects after they
are introduced. A shorter MTTD indicates a more effective testing process.
• Mean Time to Repair (MTTR): Measures how long it takes to fix identified defects,
helping assess the responsiveness of the development team.
• Test Automation Rate: The percentage of test cases that are automated, which can
influence the speed and consistency of testing.
• Defect Resolution Rate: The percentage of reported defects that are resolved within
a given timeframe, providing insights into the efficiency of the defect management
process.

4. State and explain limitations of manual testing.

The limitations/ disadvantages of using the manual testing techniques are as follows:
1 Manual Test Case scope is very limited.
2 Comparing large amount of data is impractical.
3 Checking relevance of search of operation is difficult
4 Processing change requests during software maintenance takes more time.
5 Manual testing is slow and costly.
6 It is very labour intensive, it takes a long time to complete tests.
8 Manual testing is not consistent or repeatable.
Variations in how the tests are performed as inevitable, for various reasons. One tester may
approach and perform a certain test differently from another, resulting in different results on
the same test, because the tests are not being performed identically.
9 Lack of training is the common problem, although not unique to manual software
testing.
10 GUI objects size difference and color combinations are not easy to find in manual
testing.
11 Not suitable for large scale projects and time bound projects.
12 Batch testing is not possible, for each and every test execution Human user interaction is
mandatory.
13 Comparing large amount of data is impractical.
14 Processing change requests during software maintenance takes more time.

5. Define metrics and measurements.

The terms "metrics" and "measurement" are often used interchangeably, but they have
distinct meanings in the context of software testing and other fields. Here's a breakdown of
the differences:
Measurement
• Definition: Measurement refers to the process of quantifying a specific attribute or
characteristic. It involves collecting data to determine the value of that attribute.
• Example: Measuring the response time of a web application (e.g., 2 seconds for a
specific request) is a measurement. It provides a direct numerical value indicating
performance.
Metrics
• Definition: Metrics are calculated values derived from measurements that provide
insights into performance, quality, or efficiency. Metrics often involve aggregating,
analyzing, or comparing measurements to gain a broader understanding.
• Example: The average response time across multiple requests (e.g., the average
response time of all requests over a week) is a metric. It helps in evaluating overall
performance trends.

6. Describe all factors for selecting a testing tool.

Automation testing success largely depends on the selection of right testing tools.
It takes a lot of time to evaluate relevant automation tools available in the market.
But this is a must one- time exercise that will benefit your project in long run.
Here are the criteria you need to consider before selecting any testing tool:
1) Do you have the necessary skilled resource to allocate for automation tasks?
2) What is your budget?
3) Does the tool satisfy your testing needs? Is it suitable for the project environment and
technology you are using? Does it support all tools and objects used in the code? Sometime
you may get stuck for small tests due to inabilities of the tool to identify the objects used in
the application.
above three factors are considered as most important for selecting any tool. The factors
include :
4) Does the tool provide you the free trial version so that you can evaluate it before making a
decision?
Also, does the tool have all features available in the trial version?
5) Is the current tool version stable? Is the vendor company established with good customer
support as well as online help resources and user manual?
6) How is the tool learning curve? Is the learning time acceptable for your goals?
7) Do you want automation tool for only your project needs or you are looking for a
common tool for all projects in your company? It would be a good choice if you select a
tool that supports most of the coding languages on your projects.
8 ) Which testing types does it support? A tool which supports maximum testing types
(Unit, functional, regression etc.) is always a better choice. Don‘t go for a tool just because
it is supporting all testing types. It‘s also important that the tool should be powerful enough
to automate your complex requirements.
9) Does the tool support easy interface to create and maintain test scripts? Record and
playback tool with abilities to edit recorded scripts could be a good solution.
10) Does it provide simple interface yet powerful features to accomplish complex tasks?
11) How easy is it to provide input test data for complex or load tests? A tool supporting
test data input from various data files such as Excel, XML, text file etc. would be a big
relief for the automation the testers.
12) Does it provide the powerful reporting with graphical interface? Clear and concise
reports will always help you to conclude the test results quickly.
13) Does it integrate well with your other testing tools like project planning and test
management tools?
You may also want to consider other criteria like:
14) Tool vendor refund policy
15) Existing customer reviews for the tool
16) Is the vendor providing initial training?
i. While introducing the tool in the organization it must match a need within the
organization, and solve that need in a way that is both effective and efficient.
ii. The tool should help in building the strengths of the organization and should also address
its weaknesses.