Testing Interview Questions

Define Bug Life Cycle? What is Metrics?

When we find out the bug, we will put into the “open” status. After fixing the bug developer change the
status as “fixed”. Again we will test the fixed part, if there is no bug, change the bug status as
“Closed” other wise change the bug status as “Reopen”.
A s/w metric defines the a standard method of measuring certain attributes of the process or the
product or the service.

What is the difference between System testing and end to end testing?
System testing is testing conducted on a complete, integrated system to evaluate the system’s
compliance with its specified requirements.
end to end testing: here we give real time data…
System Testing: One module or part of the system is tested
end-to-end Testing: The entire project, i.e which has many integrated systems, are tested.

How would you test a fast lazer printer?

-Test for power supply
-pc connection test
-printer sample test
-buffer test
-alignment test
-test for clarity
-speed of printing
-performance

Difference between stress testing and load testing?

STRESS TESTING:
for all types of applications deny the resources it needs, like: application is developed for 256MB
Ram or higher. you test on 64Mb Ram see it fails and fails safely
LOAD TESTING:
for client/server applications { 2-tier or higher }
Your request:
5000 users at a time. Check throughput of application, if achieved, load testing for reqd load is
done. If not tune the application developed and/or servers
STRESS TESTING:
estimating the performance of a application under different load levels.
LOAD TESTING:
To estimate the performance of a application when number of concurrent users are accessing at a
time.
STRESS TESTING:
Estimating the performance of server or CPU by applying max load or peak load.
LOAD TESTING:
Estimating the performance of server or CPU under customer expected configuration load is called
load testing.
STRESS TESTING:
apart from the above information there is more of hardware dependency kind of testing (like: less
CPU, less RAM rtc..)
LOAD TESTING:
focuses on number of user over a certain peroid of time (like normal users, more than normal user,
huge number of users, unbearable number of users etc…)

How to write test cases for a search web page like google?
Test Case No.1
Test Data: Enter the URL of the website and press “Enter button”
Exp Result: Home page of the Website should appears.
Test Case No.2
Test Data: Check all the sub links are enable or disable.
Exp Result: All the sub links must be in enabled state.
Test Case No.3
Test Data: Check whether the Search Button is Enabled or disabled.
Exp Result: Search Button should be in Enabled state.
Test Case No.2
Test Data: Check whether the “search” Text box takes all data or not.
Exp Result: It should take all types of data (like, Numeric,Characters,special Characters etc…).

What is the difference between use case, test case and test plan?
Use Case:

It is prepared by Business analyst in the Functional Requirement Specification (FRS), which is

nothing but steps which are given by the customer.
Test cases: It is prepared by test engineer based on the use cases from FRS to check the
functionality of an application thoroughly.
Test Plan: Team lead prepares test plan, in it he represents the scope of the test, what to test and
what not to test, scheduling, what to test using automation etc.

How can we design the test cases from requirements? Do the requirements, represent
exact functionality of AUT?
Yes, requirements should represent exact functionality of AUT.
First of all you have to analyze the requirements very thoroughly in terms of functionality. Then we
have to thing about suitable test case design technique [Black Box design techniques like
Equivalence Class Partitioning (ECP), Boundary Valve Analysis (BVA),Error guessing and Cause Effect
Graphing] for writing the test cases.
By these concepts you should design a test case, which should have the capability of finding the
absence of defects.
How to launch the test cases in Test Director and where it is saved?
You create the test cases in the test plan tab and link them to the requirements in the requirement
tab. Once the test cases are ready we change the status to ready and go to the “Test Lab” Tab and
create a test set and add the test cases to the test set and you can run from there.
For automation, in test plan, create a new automated test and launch the tool and create the script
and save it and you can run from the test lab the same way as you did for the manual test cases.
The test cases are sorted in test plan tab or more precisely in the test director, lets say quality
centers database test director is now referred to as quality center.

What is the deference between a bug and a defect?

When tester verifies the test cases, all failed test cases are recorded as bugs directed for necessary
action and recorded in defected reports. As a testing point of view all fail test cases are defects as
well as found bugs. While development point of view if product doesn’t meet the software
requirement specifications or any other features that is to be required, it is defect in the system.
Who found this feature is not meeting his requirements, he call it is bug in that product.

How can we explain a bug, which may arrive at the time of testing. Explain?
First check the status of the bug, then check whether the bug is valid or not then forward the same
bug to the team leader and then after confirmation forward it to the concern developer.

What do you mean by reproducing the bug? If the bug was not reproducible, what is the
next step?
Reproducing a bug is as simple as reproducing a defect. If you find a defect, for example click the
button and the corresponding action didn’t happen, it is a bug. If the developer is unable to find this
behavior he will ask us to reproduce the bug.
In another scenario, if the client complaints a defect in the production we will have to reproduce it in
test environment.

How can you know bug is reproducible or not?

A bug is reproducible if we can reproduce it, If we cannot reproduce it, it is not reproducible in which
case we will do further testing around it and if we cannot see it we will close it, and just hope it
would never come back ever again.

On what basis we give priority and severity for a bug and give one example for high
priority and low severity and high severity and low priority?
Always the priority is given by our team leader. Tester will never give the priority.
For example,
High severity: hardware bugs application crash
Low severity: User interface bugs.
High priority: Error message is not coming on time, calculation bugs etc.
Low priority: Wrong alignment, final output wrong.
How is traceability of bug follow?
The traceability of bug can be followed in so many ways.
1. Mapping the functional requirement scenarios (FS Doc) - test cases (ID) - Failed test cases (Bugs)
2. Mapping between requirements (RS Doc) - Test case (ID) - Failed test cases.
3. Mapping between test plans (TP Doc) - test case (ID) - failed test cases.
4. Mapping between business requirements (BR Doc) - test cases (ID) - Failed test cases.
5. Mapping between high level design (Design Doc) - test cases (ID) - Failed test cases.
Usually the traceability matrix is mapping between the requirements, client requirements, function
specification, test plan and test cases.

What will be the role of tester if bug is reproduced?

When ever the bug is reproduced, tester can send it back to the developer and ask him to fix it
again. If the developer cannot fix the bug once again and if the tester sends the bug back to the
developer, the third time the tester can make the bug as deferred i.e. he can reject the build (.exe)

How many functional testing tools are available? What is the easiest scripting language
used?
There might be a lot but mostly used are Win runner , silk Test, Rational Robot, QTP These are the
functional Testing tools.

What are the types of functional tests available?

There are four types of function to test a system.
1. Functional testing
2. Performance testing
3. integration testing
4. System testing

What is the major difference between Web services & client server environment?
The major difference between them are:
Web Services:
Its more towards the internet side. When we talk about web services it could mean from the java side
(deployed on Apache) or Windows side (deployed on IIS). Testing web services is totally a different
topic here.
Client Server: The system here involves a client system or a GUI (wherein a user see the front end
by which he can input to the system) and a Server ( a backend usually) where in the data gets
saved via the GUI.

What is Test Procedure?

Test procedure:-document specifying a sequence of actions for the execution of a particular test.
Test Procedure is a part of Test Case document. It comprises of following three steps:
1. To verify the look and feel of AUT through GUI test cases.
2. To verify the positive behavior of AUT through positive test cases.
3. To verify the negative behavior of the AUT through negative test cases.
Key elements of test plan?
The TestPlan consists of so many things. These are Test scope, Test Strategy, Testing Schedule,
Resource planning, What to test and What not to test, What can do in manual and What can do in
Automation, If Automation then which tool is preferred etc.
Test plan is the plan where, the test lead will select how many testers are required for testing for
given application. How to divide the team and test cases. How much time it takes to wind up testing
process.

What is SQA testing and steps of SQA testing?

Software QA involves the entire software development PROCESS, monitoring and improving the
process. Making sure that any agreed-upon standards and procedures are followed and ensuring that
problems are found and dealt with. It is oriented to ‘prevention’. The life cycle begins when an
application is first conceived and ends when it is no longer in use. It includes aspects such as initial
concept, requirements analysis, functional design, internal design, documentation planning, test
planning, coding, document preparation, integration, testing, maintenance, updates, retesting,
phase-out, and other specification.

What is the difference between Use Case and Test Case?

Use Case is written in Business Design Document (BDD)by the Business Analyst. It is for
understanding the functionality for the person who is involved in writing the test cases.
USE CASE EXAMPLE Action Response
when OK button is clicked Screen 1 appears

Testcase is different perceptions for a functionality to be tested, usually written by a Test Engineer.
The same person who has written the testcase may execute them or the other person
Above Usecase is converted into TestCase keeping in mind different perceptions (-ve and +ve)
Action Expected Value Actual Value Result
click on Ok screen 1 should appear(+ve perception) screen1 appeared pass
click on ok screen 2 should appear(-ve perception) screen 1 appeared fail
click on ok screen 2 should appear(-ve perception screen 2 appeared pass
Difference between test case and use case is
use case is prepared by High Level Management team but test case is prepared by
Test engineers.
Use case is prepared for validating the applications in terms of Actors, actions and responses but
test case is used to test a specific functionality of an app.

Use case is description of series of events that occur between user and system.
i.e. how system will response when a particular action is taken by user.
Test case:-in use case a generalized scenario is given but in testcase we are more thoroughly testing
the various aspect of that generalized scenario.

What is the difference between System testing and end to end testing?
System testing is testing conducted on a complete, integrated system to evaluate the system’s
 compliance with its specified requirements.
end to end testing: here we give real time data…
System Testing: One module or part of the system is tested
end-to-end Testing: The entire project, i.e which has many integrated systems, are tested.

1. What kinds of testing should be considered?

o Black box testing - not based on any knowledge of internal design or code. Tests are
based on requirements and functionality.
o White box testing - based on knowledge of the internal logic of an application's code.
Tests are based on coverage of code statements, branches, paths, conditions.
o unit testing - the most 'micro' scale of testing; to test particular functions or code
modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of
the internal program design and code. Not always easily done unless the application has a well-
designed architecture with tight code; may require developing test driver modules or test
harnesses.
o incremental integration testing - continuous testing of an application as new
functionality is added; requires that various aspects of an application's functionality be independent
enough to work separately before all parts of the program are completed, or that test drivers be
developed as needed; done by programmers or by testers.
o integration testing - testing of combined parts of an application to determine if they
function together correctly. The 'parts' can be code modules, individual applications, client and
server applications on a network, etc. This type of testing is especially relevant to client/server and
distributed systems.
o functional testing - black-box type testing geared to functional requirements of an
application; this type of testing should be done by testers. This doesn't mean that the programmers
shouldn't check that their code works before releasing it (which of course applies to any stage of
testing.)
o system testing - black-box type testing that is based on overall requirements
specifications; covers all combined parts of a system.
o end-to-end testing - similar to system testing; the 'macro' end of the test scale;
involves testing of a complete application environment in a situation that mimics real-world use,
such as interacting with a database, using network communications, or interacting with other
hardware, applications, or systems if appropriate.
o sanity testing or smoke testing - typically an initial testing effort to determine if a new
software version is performing well enough to accept it for a major testing effort. For example, if
the new software is crashing systems every 5 minutes, bogging down systems to a crawl, or
corrupting databases, the software may not be in a 'sane' enough condition to warrant further
testing in its current state.
o regression testing - re-testing after fixes or modifications of the software or its
environment. It can be difficult to determine how much re-testing is needed, especially near the
end of the development cycle. Automated testing tools can be especially useful for this type of
testing.
o acceptance testing - final testing based on specifications of the end-user or customer,
or based on use by end-users/customers over some limited period of time.
 o load testing - testing an application under heavy loads, such as testing of a web site
under a range of loads to determine at what point the system's response time degrades or fails.
o stress testing - term often used interchangeably with 'load' and 'performance' testing.
Also used to describe such tests as system functional testing while under unusually heavy loads,
heavy repetition of certain actions or inputs, input of large numerical values, large complex queries
to a database system, etc.
o performance testing - term often used interchangeably with 'stress' and 'load' testing.
Ideally 'performance' testing (and any other 'type' of testing) is defined in requirements
documentation or QA or Test Plans.
o usability testing - testing for 'user-friendliness'. Clearly this is subjective, and will
depend on the targeted end-user or customer. User interviews, surveys, video recording of user
sessions, and other techniques can be used. Programmers and testers are usually not appropriate
as usability testers.
o install/uninstall testing - testing of full, partial, or upgrade install/uninstall processes.
o recovery testing - testing how well a system recovers from crashes, hardware failures,
or other catastrophic problems.
o failover testing - typically used interchangeably with 'recovery testing'
o security testing - testing how well the system protects against unauthorized internal or
external access, willful damage, etc; may require sophisticated testing techniques.
o compatability testing - testing how well software performs in a particular
hardware/software/operating system/network/etc. environment.
o exploratory testing - often taken to mean a creative, informal software test that is not
based on formal test plans or test cases; testers may be learning the software as they test it.
o ad-hoc testing - similar to exploratory testing, but often taken to mean that the testers
have significant understanding of the software before testing it.
o context-driven testing - testing driven by an understanding of the environment, culture,
and intended use of software. For example, the testing approach for life-critical medical equipment
software would be completely different than that for a low-cost computer game.
o user acceptance testing - determining if software is satisfactory to an end-user or
customer.
o comparison testing - comparing software weaknesses and strengths to competing
products.
o alpha testing - testing of an application when development is nearing completion; minor
design changes may still be made as a result of such testing. Typically done by end-users or others,
not by programmers or testers.
o beta testing - testing when development and testing are essentially completed and final
bugs and problems need to be found before final release. Typically done by end-users or others,
not by programmers or testers.
o mutation testing - a method for determining if a set of test data or test cases is useful,
by deliberately introducing various code changes ('bugs') and retesting with the original test
data/cases to determine if the 'bugs' are detected. Proper implementation requires large
computational resources.

2. What are 5 common problems in the software development process?

o poor requirements - if requirements are unclear, incomplete, too general, and not
testable, there will be problems.
o unrealistic schedule - if too much work is crammed in too little time, problems are
inevitable.
 o inadequate testing - no one will know whether or not the program is any good until the
customer complains or systems crash.
o featuritis - requests to pile on new features after development is underway; extremely
common.
o miscommunication - if developers don't know what's needed or customer's have
erroneous expectations, problems are guaranteed.

3. What are 5 common solutions to software development problems?

o solid requirements - clear, complete, detailed, cohesive, attainable, testable

requirements that are agreed to by all players. Use prototypes to help nail down requirements. In
'agile'-type environments, continuous coordination with customers/end-users is necessary.
o realistic schedules - allow adequate time for planning, design, testing, bug fixing, re-
testing, changes, and documentation; personnel should be able to complete the project without
burning out.
o adequate testing - start testing early on, re-test after fixes or changes, plan for
adequate time for testing and bug-fixing. 'Early' testing ideally includes unit testing by developers
and built-in testing and diagnostic capabilities.
o stick to initial requirements as much as possible - be prepared to defend against
excessive changes and additions once development has begun, and be prepared to explain
consequences. If changes are necessary, they should be adequately reflected in related schedule
changes. If possible, work closely with customers/end-users to manage expectations. This will
provide them a higher comfort level with their requirements decisions and minimize excessive
changes later on.
o communication - require walkthroughs and inspections when appropriate; make
extensive use of group communication tools - e-mail, groupware, networked bug-tracking tools and
change management tools, intranet capabilities, etc.; insure that information/documentation is
available and up-to-date - preferably electronic, not paper; promote teamwork and cooperation;
use protoypes if possible to clarify customers' expectations.

4. What is software 'quality'?

Quality software is reasonably bug-free, delivered on time and within budget, meets requirements
and/or expectations, and is maintainable.
However, quality is obviously a subjective term. It will depend on who the 'customer' is and their
overall influence in the scheme of things. A wide-angle view of the 'customers' of a software
development project might include end-users, customer acceptance testers, customer contract officers,
customer management, the development organization's management/accountants/testers/salespeople,
future software maintenance engineers, stockholders, magazine columnists, etc. Each type of
'customer' will have their own slant on 'quality' - the accounting department might define quality in
terms of profits while an end-user might define quality as user-friendly and bug-free.

5. What is 'good code'?

 'Good code' is code that works, is bug free, and is readable and maintainable. Some organizations
have coding 'standards' that all developers are supposed to adhere to, but everyone has different
ideas about what's best, or what is too many or too few rules. There are also various theories and
metrics, such as McCabe Complexity metrics. It should be kept in mind that excessive use of standards
and rules can stifle productivity and creativity. 'Peer reviews', 'buddy checks' code analysis tools, etc.
can be used to check for problems and enforce standards.
For C and C++ coding, here are some typical ideas to consider in setting rules/standards; these may
or may not apply to a particular situation:

o minimize or eliminate use of global variables.

o use descriptive function and method names - use both upper and lower case, avoid
abbreviations, use as many characters as necessary to be adequately descriptive (use of more than
20 characters is not out of line); be consistent in naming conventions.
o use descriptive variable names - use both upper and lower case, avoid abbreviations,
use as many characters as necessary to be adequately descriptive (use of more than 20 characters
is not out of line); be consistent in naming conventions.
o function and method sizes should be minimized; less than 100 lines of code is good,
less than 50 lines is preferable.
o function descriptions should be clearly spelled out in comments preceding a function's
code.
o organize code for readability.
o use whitespace generously - vertically and horizontally
o each line of code should contain 70 characters max.
o one code statement per line.
o coding style should be consistent throught a program (eg, use of brackets,
indentations, naming conventions, etc.)
o in adding comments, err on the side of too many rather than too few comments; a
common rule of thumb is that there should be at least as many lines of comments (including header
blocks) as lines of code.
o no matter how small, an application should include documentaion of the overall
program function and flow (even a few paragraphs is better than nothing); or if possible a separate
flow chart and detailed program documentation.
o make extensive use of error handling procedures and status and error logging.
o for C++, to minimize complexity and increase maintainability, avoid too many levels of
inheritance in class heirarchies (relative to the size and complexity of the application). Minimize use
of multiple inheritance, and minimize use of operator overloading (note that the Java programming
language eliminates multiple inheritance and operator overloading.)
o for C++, keep class methods small, less than 50 lines of code per method is preferable.
o for C++, make liberal use of exception handlers

6. What is 'good design'?

'Design' could refer to many things, but often refers to 'functional design' or 'internal design'. Good
internal design is indicated by software code whose overall structure is clear, understandable, easily
modifiable, and maintainable; is robust with sufficient error-handling and status logging capability; and
works correctly when implemented. Good functional design is indicated by an application whose
functionality can be traced back to customer and end-user requirements.
For programs that have a user interface, it's often a good idea to assume that the end user will have
little computer knowledge and may not read a user manual or even the on-line help; some common
rules-of-thumb include:
 o the program should act in a way that least surprises the user
o it should always be evident to the user what can be done next and how to exit
o the program shouldn't let the users do something stupid without warning them.

7. What is SEI? CMM? CMMI? ISO? IEEE? ANSI? Will it help?

o SEI = 'Software Engineering Institute' at Carnegie-Mellon University; initiated by the

U.S. Defense Department to help improve software development processes.
o CMM = 'Capability Maturity Model', now called the CMMI ('Capability Maturity Model
Integration'), developed by the SEI. It's a model of 5 levels of process 'maturity' that determine
effectiveness in delivering quality software. It is geared to large organizations such as large U.S.
Defense Department contractors. However, many of the QA processes involved are appropriate to
any organization, and if reasonably applied can be helpful. Organizations can receive CMMI ratings
by undergoing assessments by qualified auditors.

Level 1 - characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Few if
any processes in place; successes may not be repeatable. Level 2 - software project tracking, requirements management, realistic
planning, and configuration management processes are in place; successful practices can be repeated. Level 3 - standard software
development and maintenance processes are integrated throughout an organization; a Software Engineering Process Group is is in
place to oversee software processes, and training programs are used to ensure understanding and compliance. Level 4 - metrics are
used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high. Level 5 - the
focus is on continouous process improvement. The impact of new processes and technologies can be predicted and effectively
implemented when required. Perspective on CMM ratings: During 1997-2001, 1018 organizations were assessed. Of those, 27%
were rated at Level 1, 39% at 2, 23% at 3, 6% at 4, and 5% at 5. (For ratings during the period 1992-96, 62% were at Level 1, 23%
at 2, 13% at 3, 2% at 4, and 0.4% at 5.) The median size of organizations was 100 software engineering/maintenance personnel;
32% of organizations were U.S. federal contractors or agencies. For those rated at Level 1, the most problematical key process area
was in Software Quality Assurance.

o ISO = 'International Organisation for Standardization' - The ISO 9001:2000 standard

(which replaces the previous standard of 1994) concerns quality systems that are assessed by
outside auditors, and it applies to many kinds of production and manufacturing organizations, not
just software. It covers documentation, design, development, production, testing, installation,
servicing, and other processes. The full set of standards consists of: (a)Q9001-2000 - Quality
Management Systems: Requirements; (b)Q9000-2000 - Quality Management Systems:
Fundamentals and Vocabulary; (c)Q9004-2000 - Quality Management Systems: Guidelines for
Performance Improvements. To be ISO 9001 certified, a third-party auditor assesses an
organization, and certification is typically good for about 3 years, after which a complete
reassessment is required. Note that ISO certification does not necessarily indicate quality products -
it indicates only that documented processes are followed.
Also see http://www.iso.ch/ for the latest information. In the U.S. the standards can be purchased
via the ASQ web site at http://e-standards.asq.org/
o IEEE = 'Institute of Electrical and Electronics Engineers' - among other things, creates
standards such as 'IEEE Standard for Software Test Documentation' (IEEE/ANSI Standard 829),
'IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008), 'IEEE Standard for Software
Quality Assurance Plans' (IEEE/ANSI Standard 730), and others.
 o ANSI = 'American National Standards Institute', the primary industrial standards body in
the U.S.; publishes some software-related standards in conjunction with the IEEE and ASQ
(American Society for Quality).
o Other software development/IT management process assessment methods besides
CMMI and ISO 9000 include SPICE, Trillium, TickIT, Bootstrap, ITIL, MOF, and CobiT.

8. What is the 'software life cycle'?

The life cycle begins when an application is first conceived and ends when it is no longer in use. It
includes aspects such as initial concept, requirements analysis, functional design, internal design,
documentation planning, test planning, coding, document preparation, integration, testing,
maintenance, updates, retesting, phase-out, and other aspects.

9. Will automated testing tools make testing easier?

o Possibly. For small projects, the time needed to learn and implement them may not be
worth it. For larger projects, or on-going long-term projects they can be valuable.
o A common type of automated tool is the 'record/playback' type. For example, a tester
could click through all combinations of menu choices, dialog box choices, buttons, etc. in an
application GUI and have them 'recorded' and the results logged by a tool. The 'recording' is
typically in the form of text based on a scripting language that is interpretable by the testing tool.
If new buttons are added, or some underlying code in the application is changed, etc. the
application might then be retested by just 'playing back' the 'recorded' actions, and comparing the
logging results to check effects of the changes. The problem with such tools is that if there are
continual changes to the system being tested, the 'recordings' may have to be changed so much
that it becomes very time-consuming to continuously update the scripts. Additionally, interpretation
and analysis of results (screens, data, logs, etc.) can be a difficult task. Note that there are
record/playback tools for text-based interfaces also, and for all types of platforms.
o Another common type of approach for automation of functional testing is 'data-driven'
or 'keyword-driven' automated testing, in which the test drivers are separated from the data and/or
actions utilized in testing (an 'action' would be something like 'enter a value in a text box'). Test
drivers can be in the form of automated test tools or custom-written testing software. The data and
actions can be more easily maintained - such as via a spreadsheet - since they are separate from
the test drivers. The test drivers 'read' the data/action information to perform specified tests. This
approach can enable more efficient control, development, documentation, and maintenance of
automated tests/test cases.
o Other automated tools can include:

  code analyzers - monitor code complexity, adherence to standards, etc. coverage analyzers - these
tools check which parts of the code have been exercised by a test, and may be oriented to code
statement coverage, condition coverage, path coverage, etc. memory analyzers - such as bounds-
checkers and leak detectors. load/performance test tools - for testing client/server and web
applications under various load levels. web test tools - to check that links are valid, HTML code
usage is correct, client-side and server-side programs work, a web site's interactions are secure.
other tools - for test case management, documentation management, bug reporting, and
configuration management.
What is Acceptance Testing?
Testing conducted to enable a user/customer to determine whether to accept a software product.
Normally performed to validate the software meets a set of agreed acceptance criteria.

What is Accessibility Testing?

Verifying a product is accessible to the people having disabilities (deaf, blind, mentally disabled etc.).

What is Ad Hoc Testing?

A testing phase where the tester tries to 'break' the system by randomly trying the system's
functionality. Can include negative testing as well. See also Monkey Testing.

What is Agile Testing?

Testing practice for projects using agile methodologies, treating development as the customer of
testing and emphasizing a test-first design paradigm. See also Test Driven Development.

What is Application Binary Interface (ABI)?

A specification defining requirements for portability of applications in binary forms across defferent
system platforms and environments.

What is Application Programming Interface (API)?

A formalized set of software calls and routines that can be referenced by an application program in
order to access supporting system or network services.

What is Automated Software Quality (ASQ)?

The use of software tools, such as automated testing tools, to improve software quality.

What is Automated Testing?

Testing employing software tools which execute tests without manual intervention. Can be applied in
GUI, performance, API, etc. testing.
The use of software to control the execution of tests, the comparison of actual outcomes to predicted
outcomes, the setting up of test preconditions, and other test control and test reporting functions.

What is Backus-Naur Form?

A metalanguage used to formally describe the syntax of a language.

What is Basic Block?

A sequence of one or more consecutive, executable statements containing no branches.

What is Basis Path Testing?

A white box test case design technique that uses the algorithmic flow of the program to design tests.
What is Basis Set?
The set of tests derived using basis path testing.

What is Baseline?
The point at which some deliverable produced during the software engineering process is put under
formal change control.

What is Beta Testing?

Testing of a rerelease of a software product conducted by customers.

What is Binary Portability Testing?

Testing an executable application for portability across system platforms and environments, usually for
conformation to an ABI specification.

What is Black Box Testing?

Testing based on an analysis of the specification of a piece of software without reference to its internal
workings. The goal is to test how well the component conforms to the published requirements for the
component.

What is Bottom Up Testing?

An approach to integration testing where the lowest level components are tested first, then used to
facilitate the testing of higher level components. The process is repeated until the component at the
top of the hierarchy is tested.

What is Boundary Testing?

Test which focus on the boundary or limit conditions of the software being tested. (Some of these tests
are stress tests).

What is Bug?
A fault in a program which causes the program to perform in an unintended or unanticipated manner.

What is Boundary Value Analysis?

BVA is similar to Equivalence Partitioning but focuses on "corner cases" or values that are usually out of
range as defined by the specification. his means that if a function expects all values in range of
negative 100 to positive 1000, test inputs would include negative 101 and positive 1001.

What is Branch Testing?

Testing in which all branches in the program source code are tested at least once.

What is Breadth Testing?

A test suite that exercises the full functionality of a product but does not test features in detail.
What is CAST?
Computer Aided Software Testing.

What is Capture/Replay Tool?

A test tool that records test input as it is sent to the software under test. The input cases stored can
then be used to reproduce the test at a later time. Most commonly applied to GUI test tools.

What is CMM?
The Capability Maturity Model for Software (CMM or SW-CMM) is a model for judging the maturity of the
software processes of an organization and for identifying the key practices that are required to increase
the maturity of these processes.

What is Cause Effect Graph?

A graphical representation of inputs and the associated outputs effects which can be used to design test
cases.

What is Code Complete?

Phase of development where functionality is implemented in entirety; bug fixes are all that are left. All
functions found in the Functional Specifications have been implemented.

What is Code Coverage?

An analysis method that determines which parts of the software have been executed (covered) by the
test case suite and which parts have not been executed and therefore may require additional attention.

What is Code Inspection?

A formal testing technique where the programmer reviews source code with a group who ask questions
analyzing the program logic, analyzing the code with respect to a checklist of historically common
programming errors, and analyzing its compliance with coding standards.

What is Code Walkthrough?

A formal testing technique where source code is traced by a group with a small set of test cases, while
the state of program variables is manually monitored, to analyze the programmer's logic and
assumptions.

What is Coding?
The generation of source code.

What is Compatibility Testing?

Testing whether software is compatible with other elements of a system with which it should operate,
e.g. browsers, Operating Systems, or hardware.
What is Component?
A minimal software item for which a separate specification is available.

What is Component Testing?

See the question what is Unit Testing.

What is Concurrency Testing?

Multi-user testing geared towards determining the effects of accessing the same application code,
module or database records. Identifies and measures the level of locking, deadlocking and use of single-
threaded code and locking semaphores.

What is Conformance Testing?

The process of testing that an implementation conforms to the specification on which it is based.
Usually applied to testing conformance to a formal standard.

What is Context Driven Testing?

The context-driven school of software testing is flavor of Agile Testing that advocates continuous and
creative evaluation of testing opportunities in light of the potential information revealed and the value
of that information to the organization right now.

What is Conversion Testing?

Testing of programs or procedures used to convert data from existing systems for use in replacement
systems.

What is Cyclomatic Complexity?

A measure of the logical complexity of an algorithm, used in white-box testing.

What is Data Dictionary?

A database that contains definitions of all data items defined during analysis.

What is Data Flow Diagram?

A modeling notation that represents a functional decomposition of a system.

What is Data Driven Testing?

Testing in which the action of a test case is parameterized by externally defined data values,
maintained as a file or spreadsheet. A common technique in Automated Testing.

What is Debugging?
The process of finding and removing the causes of software failures.

What is Defect?
Nonconformance to requirements or functional / program specification

What is Dependency Testing?

Examines an application's requirements for pre-existing software, initial states and configuration in
order to maintain proper functionality.

What is Depth Testing?

A test that exercises a feature of a product in full detail.

What is Dynamic Testing?

Testing software through executing it. See also Static Testing.

What is Emulator?
A device, computer program, or system that accepts the same inputs and produces the same outputs as
a given system.
What is Endurance Testing?
Checks for memory leaks or other problems that may occur with prolonged execution.

What is End-to-End testing?

Testing a complete application environment in a situation that mimics real-world use, such as
interacting with a database, using network communications, or interacting with other hardware,
applications, or systems if appropriate.

What is Equivalence Class?

A portion of a component's input or output domains for which the component's behaviour is assumed to
be the same from the component's specification.

What is Equivalence Partitioning?

A test case design technique for a component in which test cases are designed to execute
representatives from equivalence classes.

What is Exhaustive Testing?

Testing which covers all combinations of input values and preconditions for an element of the software
under test.

What is Functional Decomposition?

A technique used during planning, analysis and design; creates a functional hierarchy for the software.

What is Functional Specification?

A document that describes in detail the characteristics of the product with regard to its intended
features.
What is Functional Testing?
Testing the features and operational behavior of a product to ensure they correspond to its
specifications.
Testing that ignores the internal mechanism of a system or component and focuses solely on the outputs
generated in response to selected inputs and execution conditions.
See also What is Black Box Testing.

What is Glass Box Testing?

A synonym for White Box Testing.

What is Gorilla Testing?

Testing one particular module, functionality heavily.

What is Gray Box Testing?

A combination of Black Box and White Box testing methodologies? testing a piece of software against its
specification but using some knowledge of its internal workings.

What is High Order Tests?

Black-box tests conducted once the software has been integrated.

What is Independent Test Group (ITG)?

A group of people whose primary responsibility is software testing,

What is Inspection?
A group review quality improvement process for written material. It consists of two aspects; product
(document itself) improvement and process improvement (of both document production and
inspection).

What is Integration Testing?

Testing of combined parts of an application to determine if they function together correctly. Usually
performed after unit and functional testing. This type of testing is especially relevant to client/server
and distributed systems.

What is Installation Testing?

Confirms that the application under test recovers from expected or unexpected events without loss of
data or functionality. Events can include shortage of disk space, unexpected loss of communication, or
power out conditions.

What is Load Testing?

See Performance Testing.
What is Localization Testing?
This term refers to making software specifically designed for a specific locality.

What is Loop Testing?

A white box testing technique that exercises program loops.

What is Metric?
A standard of measurement. Software metrics are the statistics describing the structure or content of a
program. A metric should be a real objective measurement of something such as number of bugs per
lines of code.

What is Monkey Testing?

Testing a system or an Application on the fly, i.e just few tests here and there to ensure the system or
an application does not crash out.

What is Negative Testing?

Testing aimed at showing software does not work. Also known as "test to fail". See also Positive Testing.

What is Path Testing?

Testing in which all paths in the program source code are tested at least once.

What is Performance Testing?

Testing conducted to evaluate the compliance of a system or component with specified performance
requirements. Often this is performed using an automated test tool to simulate large number of users.
Also know as "Load Testing".

What is Positive Testing?

Testing aimed at showing software works. Also known as "test to pass". See also Negative Testing.

What is Quality Assurance?

All those planned or systematic actions necessary to provide adequate confidence that a product or
service is of the type and quality needed and expected by the customer.

What is Quality Audit?

A systematic and independent examination to determine whether quality activities and related results
comply with planned arrangements and whether these arrangements are implemented effectively and
are suitable to achieve objectives.

What is Quality Circle?

A group of individuals with related interests that meet at regular intervals to consider problems or
other matters related to the quality of outputs of a process and to the correction of problems or to the
improvement of quality.
What is Quality Control?
The operational techniques and the activities used to fulfill and verify requirements of quality.

What is Quality Management?

That aspect of the overall management function that determines and implements the quality policy.

What is Quality Policy?

The overall intentions and direction of an organization as regards quality as formally expressed by top
management.

What is Quality System?

The organizational structure, responsibilities, procedures, processes, and resources for implementing
quality management.

What is Race Condition?

A cause of concurrency problems. Multiple accesses to a shared resource, at least one of which is a
write, with no mechanism used by either to moderate simultaneous access.

What is Ramp Testing?

Continuously raising an input signal until the system breaks down.

What is Recovery Testing?

Confirms that the program recovers from expected or unexpected events without loss of data or
functionality. Events can include shortage of disk space, unexpected loss of communication, or power
out conditions.

What is Regression Testing?

Retesting a previously tested program following modification to ensure that faults have not been
introduced or uncovered as a result of the changes made.

What is Release Candidate?

A pre-release version, which contains the desired functionality of the final version, but which needs to
be tested for bugs (which ideally should be removed before the final version is released).

What is Sanity Testing?

Brief test of major functional elements of a piece of software to determine if its basically operational.
See also Smoke Testing.
What is Scalability Testing?
Performance testing focused on ensuring the application under test gracefully handles increases in work
load.

What is Security Testing?

Testing which confirms that the program can restrict access to authorized personnel and that the
authorized personnel can access the functions available to their security level.

What is Smoke Testing?

A quick-and-dirty test that the major functions of a piece of software work. Originated in the hardware
testing practice of turning on a new piece of hardware for the first time and considering it a success if
it does not catch on fire.

What is Soak Testing?

Running a system at high load for a prolonged period of time. For example, running several times more
transactions in an entire day (or night) than would be expected in a busy day, to identify and
performance problems that appear after a large number of transactions have been executed.

What is Software Requirements Specification?

A deliverable that describes all data, functional and behavioral requirements, all constraints, and all
validation requirements for software/

What is Software Testing?

A set of activities conducted with the intent of finding errors in software.

What is Static Analysis?

Analysis of a program carried out without executing the program.

What is Static Analyzer?

A tool that carries out static analysis.

What is Static Testing?

Analysis of a program carried out without executing the program.

What is Storage Testing?

Testing that verifies the program under test stores data files in the correct directories and that it
reserves sufficient space to prevent unexpected termination resulting from lack of space. This is
external storage as opposed to internal storage.

What is Stress Testing?

Testing conducted to evaluate a system or component at or beyond the limits of its specified
requirements to determine the load under which it fails and how. Often this is performance testing
using a very high level of simulated load.

What is Structural Testing?

Testing based on an analysis of internal workings and structure of a piece of software. See also White
Box Testing.

What is System Testing?

Testing that attempts to discover defects that are properties of the entire system rather than of its
individual components.

What is Testability?
The degree to which a system or component facilitates the establishment of test criteria and the
performance of tests to determine whether those criteria have been met.

What is Testing?
The process of exercising software to verify that it satisfies specified requirements and to detect
errors.
The process of analyzing a software item to detect the differences between existing and required
conditions (that is, bugs), and to evaluate the features of the software item (Ref. IEEE Std 829).
The process of operating a system or component under specified conditions, observing or recording the
results, and making an evaluation of some aspect of the system or component.
What is Test Automation? It is the same as Automated Testing.

What is Test Bed?

An execution environment configured for testing. May consist of specific hardware, OS, network
topology, configuration of the product under test, other application or system software, etc. The Test
Plan for a project should enumerated the test beds(s) to be used.

What is Test Case?

Test Case is a commonly used term for a specific test. This is usually the smallest unit of testing. A Test
Case will consist of information such as requirements testing, test steps, verification steps,
prerequisites, outputs, test environment, etc.
A set of inputs, execution preconditions, and expected outcomes developed for a particular objective,
such as to exercise a particular program path or to verify compliance with a specific requirement.
Test Driven Development? Testing methodology associated with Agile Programming in which every chunk
of code is covered by unit tests, which must all pass all the time, in an effort to eliminate unit-level
and regression bugs during development. Practitioners of TDD write a lot of tests, i.e. an equal number
of lines of test code to the size of the production code.

What is Test Driver?

A program or test tool used to execute a tests. Also known as a Test Harness.

What is Test Environment?

The hardware and software environment in which tests will be run, and any other software with which
the software under test interacts when under test including stubs and test drivers.

What is Test First Design?

Test-first design is one of the mandatory practices of Extreme Programming (XP).It requires that
programmers do not write any production code until they have first written a unit test.

What is a "Good Tester"?

Could you tell me two things you did in your previous assignment (QA/Testing related hopefully) that
you are proud of?

List 5 words that best describe your strengths.

What are two of your weaknesses?

What methodologies have you used to develop test cases?

In an application currently in production, one module of code is being modified. Is it necessary to re-
test the whole application or is it enough to just test functionality associated with that module?

How do you go about going into a new organization? How do you assimilate?

Define the following and explain their usefulness: Change Management, Configuration Management,
Version Control, and Defect Tracking.

What is ISO 9000? Have you ever been in an ISO shop?

When are you done testing?

What is the difference between a test strategy and a test plan?

What is ISO 9003? Why is it important?

What is Test Harness?

A program or test tool used to execute a tests. Also known as a Test Driver.

What is Test Plan?

A document describing the scope, approach, resources, and schedule of intended testing activities. It
identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks
requiring contingency planning. Ref IEEE Std 829.

What is Test Procedure?

A document providing detailed instructions for the execution of one or more test cases.

What is Test Script?

Commonly used to refer to the instructions for a particular test that will be carried out by an
automated test tool.

What is Test Specification?

A document specifying the test approach for a software feature or combination or features and the
inputs, predicted results and execution conditions for the associated tests.

What is Test Suite?

A collection of tests used to validate the behavior of a product. The scope of a Test Suite varies from
organization to organization. There may be several Test Suites for a particular product for example. In
most cases however a Test Suite is a high level concept, grouping together hundreds or thousands of
tests related by what they are intended to test.

What is Test Tools?

Computer programs used in the testing of a system, a component of the system, or its documentation.

What is Thread Testing?

A variation of top-down testing where the progressive integration of components follows the
implementation of subsets of the requirements, as opposed to the integration of components by
successively lower levels.

What is Top Down Testing?

An approach to integration testing where the component at the top of the component hierarchy is
tested first, with lower level components being simulated by stubs. Tested components are then used
to test lower level components. The process is repeated until the lowest level components have been
tested.

What is Total Quality Management?

A company commitment to develop a process that achieves high quality product and customer
satisfaction.

What is Traceability Matrix?

A document showing the relationship between Test Requirements and Test Cases.
What is Usability Testing?
Testing the ease with which users can learn and use a product.

What is Use Case?

The specification of tests that are conducted from the end-user perspective. Use cases tend to focus on
operating software as an end-user would conduct their day-to-day activities.

What is Unit Testing?

Testing of individual software components.

 
What is Validation?
The process of evaluating software at the end of the software development process to ensure
compliance with software requirements. The techniques for validation is testing, inspection and
reviewing

What is Verification?
The process of determining whether of not the products of a given phase of the software development
cycle meet the implementation steps and can be traced to the incoming objectives established during
the previous phase. The techniques for verification are testing, inspection and reviewing.

What is Volume Testing?

Testing which confirms that any values that may become large over time (such as accumulated counts,
logs, and data files), can be accommodated by the program and will not cause the program to stop
working or degrade its operation in any manner.

What is Walkthrough?
A review of requirements, designs or code characterized by the author of the material under review
guiding the progression of the review.

What is White Box Testing?

Testing based on an analysis of internal workings and structure of a piece of software. Includes
techniques such as Branch Testing and Path Testing. Also known as Structural Testing and Glass Box
Testing. Contrast with Black Box Testing.

What is Workflow Testing?

Scripted end-to-end testing which duplicates specific workflows which are expected to be utilized by
the end-user.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Testing Questions and Answers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. What makes a good test engineer? 2

2. What makes a good software QA engineer? 2
3. What makes a good QA or Test manager? 2
4. What's the role of documentation in QA? 3
5. What's the big deal about 'requirements'? 3
6. What steps are needed to develop and run software tests? 3
7. What's a 'test plan'? 4
8. What's a 'test case'? 6
9. What should be done after a bug is found? 6
10. What is 'configuration management'? 8
11. What if the software is so buggy it can't really be tested at all? 8
12. How can it be known when to stop testing? 8
13. What if there isn't enough time for thorough testing? 8
14. What if the project isn't big enough to justify extensive testing? 9
15. What can be done if requirements are changing continuously? 9
16. What if the application has functionality that wasn't in the
requirements? 10
17. How can Software QA processes be implemented without stifling
productivity? 11
18. What if an organization is growing so fast that fixed QA processes
are impossible? 11
19. How does a client'server environment affect testing? 11
20. How can World Wide Web sites be tested? 11
21. How is testing affected by object-oriented designs? 13
22. What is Extreme Programming and what's it got to do with testing ? 13

1. What makes a good test engineer?

? A good test engineer has a 'test to break' attitude, an ability to
take the point of view of the customer, a strong desire for quality,
and an attention to detail.
? Tact and diplomacy are useful in maintaining a cooperative
relationship with developers, and an ability to communicate with both
technical (developers) and non-technical (customers, management)
people is useful.
? Previous software development experience can be helpful as it
provides a deeper understanding of the software development process,
gives the tester an appreciation for the developers' point of view,
and reduce the learning curve in automated test tool programming.
? Judgement skills are needed to assess high-risk areas of an
application on which to focus testing efforts when time is limited.
2. What makes a good software QA engineer?
? The same qualities a good tester has are useful for a QA engineer.
? Additionally, they must be able to understand the entire software
development process and how it can fit into the business approach and
goals of the organization.
? Communication skills and the ability to understand various sides of
issues are important.
? In organizations in the early stages of implementing QA processes,
patience and diplomacy are especially needed.
? An ability to find problems as well as to see 'what's missing' is
important for inspections and reviews.
3. What makes a good QA or Test manager?
? A good QA, test, or QA/Test (combined) manager should:
? be familiar with the software development process
? be able to maintain enthusiasm of their team and promote a positive
atmosphere, despite what is a somewhat 'negative' process (e.g.,
looking for or preventing problems)
? be able to promote teamwork to increase productivity
? be able to promote cooperation between software, test, and QA engineers
? have the diplomatic skills needed to promote improvements in QA
processes
? have the ability to withstand pressures and say 'no' to other
managers when quality is insufficient or QA processes are not being
adhered to
? have people judgement skills for hiring and keeping skilled personnel
? be able to communicate with technical and non-technical people,
engineers, managers, and customers.
? be able to run meetings and keep them focused
4. What's the role of documentation in QA?

? Critical. (Note that documentation can be electronic, not

necessarily paper.)
? QA practices should be documented such that they are repeatable.
? Specifications, designs, business rules, inspection reports,
configurations, code changes, test plans, test cases, bug reports,
user manuals, etc. should all be documented.
? There should ideally be a system for easily finding and obtaining
documents and determining what documentation will have a particular
piece of information.
? Change management for documentation should be used if possible.

5. What's the big deal about 'requirements'?

? One of the most reliable methods of insuring problems, or failure,

in a complex software project is to have poorly documented
requirements specifications.
? Requirements are the details describing an application's
externally-perceived functionality and properties.
? Requirements should be clear, complete, reasonably detailed,
cohesive, attainable, and testable. A non-testable requirement would
be, for example, 'user-friendly' (too subjective).
? A testable requirement would be something like 'the user must enter
their previously-assigned password to access the application'.
? Determining and organizing requirements details in a useful and
efficient way can be a difficult effort; different methods are
available depending on the particular project.
? Many books are available that describe various approaches to this task.
? Care should be taken to involve ALL of a project's significant
'customers' in the requirements process.
? 'Customers' could be in-house personnel or out, and could include
end-users, customer acceptance testers, customer contract officers,
customer management, future software maintenance engineers,
salespeople, etc.
? Anyone who could later derail the project if their expectations
aren't met should be included if possible.
? Organizations vary considerably in their handling of requirements
specifications.
? Ideally, the requirements are spelled out in a document with
statements such as 'The product shall.'.
? 'Design' specifications should not be confused with 'requirements';
design specifications should be traceable back to the requirements.
? In some organizations requirements may end up in high level project
plans, functional specification documents, in design documents, or in
other documents at various levels of detail.
? No matter what they are called, some type of documentation with
detailed requirements will be needed by testers in order to properly
plan and execute tests.
? Without such documentation, there will be no clear-cut way to
determine if a software application is performing correctly.

6. What steps are needed to develop and run software tests?

The following are some of the steps to consider:

? Obtain requirements, functional design, and internal design
specifications and other necessary documents
? Obtain budget and schedule requirements
? Determine project-related personnel and their responsibilities,
reporting requirements, required standards and processes (such as
release processes, change processes, etc.)
? Identify application's higher-risk aspects, set priorities, and
determine scope and limitations of tests
? Determine test approaches and methods - unit, integration,
functional, system, load, usability tests, etc.
? Determine test environment requirements (hardware, software,
communications, etc.)
? Determine testware requirements (record/playback tools, coverage
analyzers, test tracking, problem/bug tracking, etc.)
? Determine test input data requirements
? Identify tasks, those responsible for tasks, and labor requirements
? Set schedule estimates, timelines, milestones
? Determine input equivalence classes, boundary value analyses, error
classes
? Prepare test plan document and have needed reviews/approvals
? Write test cases
? Have needed reviews/inspections/approvals of test cases
? Prepare test environment and testware, obtain needed user
manuals/reference documents/configuration guides/installation guides,
set up test tracking processes, set up logging and archiving
processes, set up or obtain test input data
? Obtain and install software releases
? Perform tests
? Evaluate and report results
? Track problems/bugs and fixes
? Retest as needed
? Maintain and update test plans, test cases, test environment, and
testware through life cycle
7. What's a 'test plan'?

A software project test plan is a document that describes the

objectives, scope, approach, and focus of a software testing effort.
The process of preparing a test plan is a useful way to think through
the efforts needed to validate the acceptability of a software
product. The completed document will help people outside the test
group understand the 'why' and 'how' of product validation. It should
be thorough enough to be useful but not so thorough that no one
outside the test group will read it. The following are some of the
items that might be included in a test plan, depending on the
particular project:
? Title
? Identification of software including version/release numbers
? Revision history of document including authors, dates, approvals
? Table of Contents
? Purpose of document, intended audience
? Objective of testing effort
? Software product overview
? Relevant related document list, such as requirements, design
documents, other test plans, etc.
? Relevant standards or legal requirements
? Traceability requirements
? Relevant naming conventions and identifier conventions
? Overall software project organization and
personnel/contact-info/responsibilties
? Test organization and personnel/contact-info/responsibilities
? Assumptions and dependencies
? Project risk analysis
? Testing priorities and focus
? Scope and limitations of testing
? Test outline - a decomposition of the test approach by test type,
feature, functionality, process, system, module, etc. as applicable
? Outline of data input equivalence classes, boundary value analysis,
error classes
? Test environment - hardware, operating systems, other required
software, data configurations, interfaces to other systems
? Test environment setup and configuration issues
? Test data setup requirements
? Database setup requirements
? Outline of system-logging/error-logging/other capabilities, and
tools such as screen capture software, that will be used to help
describe and report bugs
? Discussion of any specialized software or hardware tools that will
be used by testers to help track the cause or source of bugs
? Test automation - justification and overview
? Test tools to be used, including versions, patches, etc.
? Test script/test code maintenance processes and version control
? Problem tracking and resolution - tools and processes
? Project test metrics to be used
? Reporting requirements and testing deliverables
? Software entrance and exit criteria
? Initial sanity testing period and criteria
? Test suspension and restart criteria
? Personnel allocation
? Personnel pre-training needs
? Test site/location
? Outside test organizations to be utilized and their purpose,
responsibilties, deliverables, contact persons, and coordination issues
? Relevant proprietary, classified, security, and licensing issues.
? Open issues
? Appendix - glossary, acronyms, etc.
8. What's a 'test case'?

? A test case is a document that describes an input, action, or event

and an expected response, to determine if a feature of an application
is working correctly.
? A test case should contain particulars such as test case identifier,
test case name, objective, test conditions/setup, input data
requirements, steps, and expected results.
? Note that the process of developing test cases can help find
problems in the requirements or design of an application, since it
requires completely thinking through the operation of the application.
For this reason, it's useful to prepare test cases early in the
development cycle if possible.
9. What should be done after a bug is found?

The bug needs to be communicated and assigned to developers who can

fix it.
After the problem is resolved, fixes should be re-tested, and
determinations made regarding requirements for regression testing to
check that fixes didn't create problems elsewhere. If a
problem-tracking system is in place, it should encapsulate these
processes. A variety of commercial problem-tracking/management
software tools are available . The following are items to be
considered in the tracking process:
? Complete information such that developers can understand the bug,
get an idea of it's severity, and reproduce it if necessary.
? Bug identifier (number, ID, etc.)
? Current bug status (e.g., 'Released for Retest', 'New', etc.)
? The application name or identifier and version
? The function, module, feature, object, screen, etc. where the bug
occurred
? Environment specifics, system, platform, relevant hardware specifics
? Test case name/number/identifier
? One-line bug description
? Full bug description
? Description of steps needed to reproduce the bug if not covered by a
test case or if the developer doesn't have easy access to the test
case/test script/test tool
? Names and/or descriptions of file/data/messages/etc. used in test
? File excerpts/error messages/log file excerpts/screen shots/test
tool logs that would be helpful in finding the cause of the problem
? Severity estimate (a 5-level range such as 1-5 or
'critical'-to-'low' is common)
? Was the bug reproducible?
? Tester name
? Test date
? Bug reporting date
? Name of developer/group/organization the problem is assigned to
? Description of problem cause
? Description of fix
? Code section/file/module/class/method that was fixed
? Date of fix
? Application version that contains the fix
? Tester responsible for retest
? Retest date
? Retest results
? Regression testing requirements
? Tester responsible for regression tests
? Regression testing results
A reporting or tracking process should enable notification of
appropriate personnel at various stages.
For instance, testers need to know when retesting is needed,
developers need to know when bugs are found and how to get the needed
information, and reporting/summary capabilities are needed for managers.

10. What is 'configuration management'?

Configuration management covers the processes used to control,

coordinate, and track: code, requirements, documentation, problems,
change requests, designs, tools/compilers/libraries/patches, changes
made to them, and who makes the changes.

11. What if the software is so buggy it can't really be tested at all?

The best bet in this situation is for the testers to go through the
process of reporting whatever bugs or blocking-type problems initially
show up, with the focus being on critical bugs. Since this type of
problem can severely affect schedules, and indicates deeper problems
in the software development process (such as insufficient unit
testing or insufficient integration testing, poor design, improper
build or release procedures, etc.) managers should be notified, and
provided with some documentation as evidence of the problem.

12. How can it be known when to stop testing?

This can be difficult to determine. Many modern software applications

are so complex, and run in such an interdependent environment, that
complete testing can never be done. Common factors in deciding when to
stop are:
? Deadlines (release deadlines, testing deadlines, etc.)
? Test cases completed with certain percentage passed
? Test budget depleted
? Coverage of code/functionality/requirements reaches a specified point
? Bug rate falls below a certain level
? Beta or alpha testing period ends
13. What if there isn't enough time for thorough testing?

Use risk analysis to determine where testing should be focused.

Since it's rarely possible to test every possible aspect of an

application, every possible combination of events, every dependency,
or everything that could go wrong, risk analysis is appropriate to
most software development projects.
This requires judgement skills, common sense, and experience. (If
warranted, formal methods are also available.)

Considerations can include:

? Which functionality is most important to the project's intended purpose?

? Which functionality is most visible to the user?
? Which functionality has the largest safety impact?
? Which functionality has the largest financial impact on users?
? Which aspects of the application are most important to the customer?
? Which aspects of the application can be tested early in the
development cycle?
? Which parts of the code are most complex, and thus most subject to
errors?
? Which parts of the application were developed in rush or panic mode?
? Which aspects of similar/related previous projects caused problems?
? Which aspects of similar/related previous projects had large
maintenance expenses?
? Which parts of the requirements and design are unclear or poorly
thought out?
? What do the developers think are the highest-risk aspects of the
application?
? What kinds of problems would cause the worst publicity?
? What kinds of problems would cause the most customer service complaints?
? What kinds of tests could easily cover multiple functionalities?
? Which tests will have the best high-risk-coverage to time-required
ratio?

14. What if the project isn't big enough to justify extensive testing?

Consider the impact of project errors, not the size of the project.
However, if extensive testing is still not justified, risk analysis is
again needed and the same considerations as described previously in.
The tester might then do ad hoc testing, or write up a limited test
plan based on the risk analysis.

15. What can be done if requirements are changing continuously?

A common problem and a major headache.
? Work with the project's stakeholders early on to understand how
requirements might change so that alternate test plans and strategies
can be worked out in advance, if possible.
? It's helpful if the application's initial design allows for some
adaptability so that later changes do not require redoing the
application from scratch.
? If the code is well-commented and well-documented this makes changes
easier for the developers.
? Use rapid prototyping whenever possible to help customers feel sure
of their requirements and minimize changes.
? The project's initial schedule should allow for some extra time
commensurate with the possibility of changes.
? Try to move new requirements to a 'Phase 2' version of an
application, while using the original requirements for the 'Phase 1'
version.
? Negotiate to allow only easily-implemented new requirements into the
project, while moving more difficult new requirements into future
versions of the application.
? Be sure that customers and management understand the scheduling
impacts, inherent risks, and costs of significant requirements
changes. Then let management or the customers (not the developers or
testers) decide if the changes are warranted - after all, that's their
job.
? Balance the effort put into setting up automated testing with the
expected effort required to re-do them to deal with changes.
? Try to design some flexibility into automated test scripts.
? Focus initial automated testing on application aspects that are most
likely to remain unchanged.
? Devote appropriate effort to risk analysis of changes to minimize
regression testing needs.
? Design some flexibility into test cases (this is not easily done;
the best bet might be to minimize the detail in the test cases, or set
up only higher-level generic-type test plans)
? Focus less on detailed test plans and test cases and more on ad hoc
testing (with an understanding of the added risk that this entails).

16. What if the application has functionality that wasn't in the

requirements?

It may take serious effort to determine if an application has

significant unexpected or hidden functionality, and it would indicate
deeper problems in the software development process.

If the functionality isn't necessary to the purpose of the

application, it should be removed, as it may have unknown impacts or
dependencies that were not taken into account by the designer or the
customer.
If not removed, design information will be needed to determine added
testing needs or regression testing needs.
Management should be made aware of any significant added risks as a
result of the unexpected functionality.
If the functionality only effects areas such as minor improvements in
the user interface, for example, it may not be a significant risk.

17. How can Software QA processes be implemented without stifling

productivity?

By implementing QA processes slowly over time, using consensus to

reach agreement on processes, and adjusting and experimenting as an
organization grows and matures, productivity will be improved instead
of stifled. Problem prevention will lessen the need for problem
detection, panics and burn-out will decrease, and there will be
improved focus and less wasted effort. At the same time, attempts
should be made to keep processes simple and efficient, minimize
paperwork, promote computer-based processes and automated tracking and
reporting, minimize time required in meetings, and promote training as
part of the QA process. However, no one - especially talented
technical types - likes rules or bureacracy, and in the short run
things may slow down a bit. A typical scenario would be that more days
of planning and development will be needed, but less time will be
required for late-night bug-fixing and calming of irate customers.

18. What if an organization is growing so fast that fixed QA processes

are impossible?

This is a common problem in the software industry, especially in new

technology areas. There is no easy solution in this situation, other than:
? Hire good people
? Management should 'ruthlessly prioritize' quality issues and
maintain focus on the customer
? Everyone in the organization should be clear on what 'quality' means
to the customer

19. How does a client'server environment affect testing?

Client/server applications can be quite complex due to the multiple

dependencies among clients, data communications, hardware, and servers.
Thus testing requirements can be extensive. When time is limited (as
it usually is) the focus should be on integration and system testing.
Additionally, load/stress/performance testing may be useful in
determining client/server application limitations and capabilities.
There are commercial tools to assist with such testing.

20. How can World Wide Web sites be tested?

Web sites are essentially client/server applications - with web

servers and 'browser' clients. Consideration should be given to the
interactions between html pages, TCP/IP communications, Internet
connections, firewalls, applications that run in web pages (such as
applets, javascript, plug-in applications), and applications that run
on the server side (such as cgi scripts, database interfaces, logging
applications, dynamic page generators, asp, etc.).
Additionally, there are a wide variety of servers and browsers,
various versions of each, small but sometimes significant differences
between them, variations in connection speeds, rapidly changing
technologies, and multiple standards and protocols.
The end result is that testing for web sites can become a major
ongoing effort.
Other considerations might include:
? What are the expected loads on the server (e.g., number of hits per
unit time?), and what kind of performance is required under such loads
(such as web server response time, database query response times).
? What kinds of tools will be needed for performance testing (such as
web load testing tools, other tools already in house that can be
adapted, web robot downloading tools, etc.)?
? Who is the target audience? What kind of browsers will they be
using? What kind of connection speeds will they by using? Are they
intra- organization (thus with likely high connection speeds and
similar browsers) or Internet-wide (thus with a wide variety of
connection speeds and browser types)?
? What kind of performance is expected on the client side (e.g., how
fast should pages appear, how fast should animations, applets, etc.
load and run)?
? Will down time for server and content maintenance/upgrades be
allowed? how much?
? What kinds of security (firewalls, encryptions, passwords, etc.)
will be required and what is it expected to do? How can it be tested?
? How reliable are the site's Internet connections required to be? And
how does that affect backup system or redundant connection
requirements and testing?
? What processes will be required to manage updates to the web site's
content, and what are the requirements for maintaining, tracking, and
controlling page content, graphics, links, etc.?
? Which HTML specification will be adhered to? How strictly? What
variations will be allowed for targeted browsers?
? Will there be any standards or requirements for page appearance
and/or graphics throughout a site or parts of a site??
? How will internal and external links be validated and updated? how
often?
? Can testing be done on the production system, or will a separate
test system be required? How are browser caching, variations in
browser option settings, dial-up connection variabilities, and
real-world internet 'traffic congestion' problems to be accounted for
in testing?
? How extensive or customized are the server logging and reporting
requirements; are they considered an integral part of the system and
do they require testing?
? How are cgi programs, applets, javascripts, ActiveX components, etc.
to be maintained, tracked, controlled, and tested?

? Pages should be 3-5 screens max unless content is tightly focused on

a single topic. If larger, provide internal links within the page.
? The page layouts and design elements should be consistent throughout
a site, so that it's clear to the user that they're still within a site.
? Pages should be as browser-independent as possible, or pages should
be provided or generated based on the browser-type.
? All pages should have links external to the page; there should be no
dead-end pages.
? The page owner, revision date, and a link to a contact person or
organization should be included on each page.
21. How is testing affected by object-oriented designs?

Well-engineered object-oriented design can make it easier to trace

from code to internal design to functional design to requirements.
While there will be little affect on black box testing (where an
understanding of the internal design of the application is
unnecessary), white-box testing can be oriented to the application's
objects. If the application was well-designed this can simplify test
design.

22. What is Extreme Programming and what's it got to do with testing ?

Extreme Programming (XP) is a software development approach for small

teams on risk-prone projects with unstable requirements.
It was created by Kent Beck who described the approach in his book
'Extreme Programming Explained' .
Testing ('extreme testing') is a core aspect of Extreme Programming.
Programmers are expected to write unit and functional test code first
- before the application is developed.
Test code is under source control along with the rest of the code.
Customers are expected to be an integral part of the project team and
to help develop scenarios for acceptance/black box testing.
Acceptance tests are preferably automated, and are modified and rerun
for each of the frequent development iterations.

QA and test personnel are also required to be an integral part of the

project team.

Detailed requirements documentation is not used, and frequent

re-scheduling, re-estimating, and re-prioritizing is expected

functional requirement: A requirement that specifies a function that a component or system must
perform.

functional testing: Testing based on an analysis of the specification of the functionality of a component
or system.

specification: A document that specifies...the requirements design behavior or other characteristics of a

component or system and often the procedures for determining whether these provisions have been
satisfied. [IEEE 610]

functionality: The capabilty of the software product to provide functions which meet stated and implied
needs when the software is used under specified conditions.
functionality testing: The process of testing to determine the functionality of a software product.