divine QA Testing

1 TESTING CONCEPTS AND TERMINOLOGY

1.1 Software Testing Strategy

Testing is a set of activities that can be planned in advance and conducted

systematically. Software testing strategy has the following characteristics:
• Testing begins at the component (unit) level and works "outward" towards the
integration of the entire software system.
• Different testing techniques are appropriate at different points of time
• Testing is conducted by the developer of the software and an independent test
group (QA testing group)
• Testing and debugging are different activities, but debugging must be
accommodated in any testing strategy.

A strategy for software testing must accommodate low level tests that are
necessary to verify that a small source code segment has been correctly
implemented as well as high level tests that validate major system functions
against customer requirements.

Test granularity refers to the finesse or coarseness of a test's focus. A highly

granular test allows the tester to check low - level details, often internal to the
system; a less granular test provides the tester with information about general
system behavior. Test granularity can be though of as running along a spectrum
ranging from structural (white box) to behavioral (black box and live).

Structural test (also known as white box tests) find bugs in low level operations
such as those that occur at the level of line of code / object level. Structural testing
involves a detailed knowledge of the system.

Behavioral tests (also known as black-box tests) are often used to find bugs in
high-level operations, at the levels of features, operational profiles and customer
scenarios.

Testing strategy for testing a software system should encompass both structural
and behavioral testing.

1.2 Testing Phases

A sequential view of software development and testing is shown in Fig 1.

Considering the process from a procedural point of view, testing within the
context of software engineering is actually the series of four steps that are
implemented sequentially. Initially test focus is on each component individually
ensuring that it functions properly as a unit. This phase of testing is known as unit
testing. Unit testing makes heavy use of white box testing techniques, exercising

Page 1 of 17
divine QA Testing

specific paths in a modules control structure to ensure complete coverage and

maximum error detection.

Requirements Design Acceptance Test Design

High level Design System Test Design

Implementation Design Integration Test Design

Code Unit Test Design

Unit Testing

Integration Testing

System Testing

Acceptance Testing

Fig. 1: Sequential view of software development and testing

Components, Classes are packaged and integrated to form the complete software
system. Integration testing addresses the issues associated with the dual problems
of verification and program construction. Black-box test case design techniques

Page 2 of 17
divine QA Testing

are the most prevalent during integration, although a limited amount of white -
box testing may be used to ensure coverage of major control paths.

Acceptance and validation criteria (Established during requirement analysis) must

be tested. Validation testing provides the final assurance that software meets all
functional, behavioral and performance requirement. Black-box testing techniques
are used exclusively during validation. Validations testing can be combined with
integration or system testing.

Software once validated, must be combined with other system elements

(hardware, network, databases, etc) system testing verifies that all elements mesh
properly and that overall system function/performance is achieved.

Acceptance testing is conducted by the customer to validate requirements.

Acceptance testing normally covers functionality and other aspects of the
software system to verify and validate against client requirements.

During the various phases of testing different types of test can be conducted to
test a specific view or aspect of the system. The following tests can be conducted
during unit, integration, system and acceptance testing.

Unit Testing: Functionality, User interface, Regression

Integration Testing: Functionality, User interface, Dependency (API testing),

Performance, Smoke, Regression, Forward backward
compatibility, Capacity and volume, Error/Disaster
handling and recover, Concurrent Execution

System and Acceptance Testing: Functionality, User interface, Capacity and

Volume, Error/Disaster handling and recovery, Stress,
Performance, Localization, Networks and disturbed
environments, Standards conformance, Security,
Installation and upgrade testing, Multiple configuration
testing, User Documentation testing, Beta, Alpha

1.3 Testing Types

Functionality Testing:

During unit testing each component or object is tested for completeness and
correctness of implementation of a specific feature of functionality. In integration
testing we focus on functionality that requires the correct operation of two or
more components or a flow of data between them.

Page 3 of 17
divine QA Testing

During system testing we should consider functionality in terms of whole

sequence of end user operations or an entire area of functionality. In functionality
testing we should consider "right action", "wrong side effect" risks and also test
with both valid and invalid data sets.

User Interface Testing

User interface of a project is the first thing a user encounters in a product. While
the product may perform its intended function correctly, if the UI behaves
incorrectly, does not refresh properly or over writes meaningful text on the screen
inadvertently the user can be seriously annoyed.

Testing the user interface of a product usually consist of the operation or

simulation of keyboard, typing and mouse clicking and movement to do menu
selection, field entry, field motion, control operation, request confirmation and so
on.

User interface testing during the unit testing phase often involves only a select
few UI objects or specific screen or page. During integration testing we need to
focus on navigation/page flow across several screens/pages. In system testing we
need to test the complete navigation required to meet client requirements. In
addition to testing the normal navigation with reference to various types of users,
it is necessary also to test for negative behavior.

Concurrent Execution Testing

Although a product may appear to function properly when tested by a single user,
when multiple instances of a product run concurrently undesirable results may
occur: they may show up in the application or in the operating system. A product
that can have many instances of it running at the same time in parallel should
undergo these concurrent execution tests:
• Simple usage (Two users)
• Standard usage (many users)
• Boundary situations (maximum number of users plus or minus one) if this
limit exists

Multiple Configuration Testing

A product may be usually tested under a default configuration, it is important to

test the non-default configurations also. Sometimes new tests may have to be
written to test under a different configuration. It is possible that there could be
infinite possible configurations that can be set up. It is necessary to identify key
configurations that are important for the customer to operate the product and test
the product for these configurations

Page 4 of 17
divine QA Testing

Dependency Testing

Dependency testing is recommended to test any API calls made to other products
and ensure that these calls act as promised. Such tests also provide a good
regression mechanism when the product being developed is integrated with the
new version of products it depends on. We need to test all interactions between
products, including error cases. In general, any where data or control is
transferred from one component to another component (or components), either
immediately or in delayed fashion, an interface exists that can cause trouble.

Forward/Backward Compatibility Testing:

The product must be compatible (to the published extent) with any previous
release and supported dependencies. Backward compatibility testing ensures that
no problem will arise from a user trying to use the new version of a product along
with an old version and verifies integration between products.

Installation and Upgrade Testing

If a user cannot install or upgrade the product whether it is through an inability to

understand documentation, a failure of the installation/upgrade program, or lack
of resource, the product has failed the user.

We need to check the installation/upgrade documentation for accuracy and follow

all instructions form the point of view of a naïve user. Test the entire installation
and upgrade procedure, both with correct and incorrect response to all prompts. If
verification scripts are provided to test the installation/upgrade, run the script as
an additional check. Generate possible system error to check the response of the
installation/upgrade program.

User Documentation Testing

The user documentation is part of the product. If a new user is unable to use the
software because of poor documentation, the product as a whole is unusable. All
sections of the user documentation should be tested for accuracy against the
reality of the product. Ideally user documentation should be held under source
control and made part of regression testing of the product. Verify as corrected and
complete all demos, tutorials and exercise described in the user documentation.

Standards Conformance Testing

Specific set of industry standards has to be tested. Tests need to be planned and
conducted to detect bugs in standards conformance that could seriously affect the
prospects of the product in the market.

Page 5 of 17
divine QA Testing

Capacity and Volume Testing

During unit testing, from a structural test perspective, every buffer, occurrence,
storage resource, processor, bus, and I/O channel in the system has to be tested. In
integration testing aspects related to capacity of network to handle traffic and its
behavior under various volumes are tested. Similarly the data storage capability is
tested. During system testing the capacity and volume limitations are tested from
a user's point of view.

Localization

Localization typically refers to problems associated with different languages.

Languages other than English may require different character sets in which data
can be represented. Testing character set conversion for data conversion between
different localization is a very important consideration for planning localization
testing. Another important item is the provision of different message files in
various languages for error messages and other system information. Testing for
localization may require setting up special hardware and software.

Security Testing

Security testing attempts to verify that protection mechanism built into a system
will, in fact protect it from improper penetration. During security testing the tester
plays the role of the individual who desires to penetrate the system. The tester
would attempt to acquire passwords, access the system through some
indirect/direct means, overwhelm the system so that it is not available to others,
cause system errors, etc.

Performance Testing

Any product, no matter how well constructed and reliable, is worthless if its
performance does not meet the demands of its users. Testing must be done in
realistic setting to ensure adequate performance. However, we have to be cost
effective.

We need to test how the product performs during operations normally executed by
the users regularly. If performance goals are set for various operations of the
product, it is necessary to verify that these goals are met. Performance is not only
"How many per second" but also" how long".

Smoke Testing

Smoke testing is an integration testing approach that is commonly used when

"Shrink wrapped" software products are being developed. It is designed as a

Page 6 of 17
divine QA Testing

pacing mechanism for time-critical projects, allowing the software team to assess
its projects on a frequent basis.
Software components produced are integrated into a "build". A series of tests are
designed to expose errors that will keep the build from properly performing its
function. The intent is to uncover "Show stopper" errors that have the highest
likelihood of throwing the software project behind schedule. Sometimes daily
builds are made and subjected to smoke testing.

The smoke test should exercise the entire system from end to end. It does not have
to be exhaustive, but it should be capable of exposing major problems. The smoke
test should be thorough enough that if the build passes you can assume that it is
stable enough to be tested more thoroughly.

Regression Testing

Regression testing is the re-execution of some subset or all tests that have been
previously conducted to ensure that changes have not propagated unintended side
effects. Regression testing is the activity that helps to ensure that changes do not
introduce unintended behavior or additional errors. When sub sets of test are
selected for regression testing, they should contain:
• A representative sample of test that will exercise all software functions
• Additional tests that focus on software functions that are likely to be affected
by the change
• Tests that focus on the software components that have been changed.

Networks and Distributed Environments

If a product works in a network or distributed environment we need to test for

• Date and time zone handling
• Communication and telephony standards
Environment factors (temperature, pressure, etc) and their impact on the product
should also be tested if required.

Error / Disaster Handling and Recovery

A software system must recover from faults and resume processing within a
prescribed time. In some cases, a system must be fault tolerant: that is, processing
faults must not cause overall system function to cease. In other cases, a system
failure must be corrected within a specific period of time.

Recovery testing forces the software to fail in a variety of ways and verifies that
recovery is properly performed. If recovery is automatic then re-initialization,
check pointing mechanisms, data recovery and restart are evaluated for
correctness. If recovery requires human intervention, the mean-time -to -repair
(MTTR) is evaluated to determine whether it is with in acceptable limits.

Page 7 of 17
divine QA Testing

Quality risks in this area include unacceptable failure rates, unacceptable recovery
times and the inability of the system to function under legitimate conditions
without failure.

Alpha and Beta Testing

If software is developed as a product to be used by many customers, it is

impractical to perform formal acceptance tests with each one. Most software
product builders use a process called alpha and beta testing to uncover errors that
only the end user seem to be able to find.

A customer conducts the alpha test at the developer site. The software is used in a
natural setting with the developer recording the errors and usage problems. Alpha
tests are conducted in a controlled environment.

The beta test is conducted at one or more customer sites by the end-user of the
software unlike alpha testing the developer is not present. Beta test is conducted at
live sites not controlled by the developer. The customer records all problems
encountered during beta and reports them to the developer

Stress Testing

Stress testing is the search for load dependent bugs that occur only when the
system is stressed and the specification of stress tests is a key activity. Stress
Testing increases the overall quality and reliability of software. Stress-related
bugs are the most difficult to find and exhume.

In stress testing, the system is subjected to unrealistically harsh inputs or loads

with inadequate resources, with the intention of breaking it. We test to find out
weather the system can operate reliably at the limit of its resources or not.

Stress testing with high background load, to a point where one or more or all
resources are simultaneously saturated. It is to system testing what destructive
testing is to physical objects.

Stress testing often finds bugs for the following reasons:

• It forces race conditions

(Race conditions reflect temporal boundaries. Normally event A occurs before
event B; By chance if event B occurs before A, then the system may fail,
giving a race condition bug)
• It totally distorts the normal order of processing, especially processing that
occurs at different priority levels.
• It forces the exercise of all system limits, thresholds, or other controls
designed to deal with overload situations.

Page 8 of 17
divine QA Testing

• It greatly increases the number of simultaneous actions.

• It depletes resource pools in extraordinary and unexpected sequences.

Some of the reasons why stress-related bugs are encountered in system are:

• Assumptions that there will be no interrupts.

• Failure to block or unblock interrupts.
• Assumptions that code is reentrant or not reentrant.
• By passing data interlock.
• Failure to close or open interlock.
• Assumptions that a called routine is resident or non-resident.
• Assumption that the calling program is resident or non-resident.
• Assumption that registers or memory were initialized or not initialized.
• Assumptions that the registers or memory location content did not change.
• Local setting of global parameter.
• Global setting of local parameter.

1.4 Testing Web Applications

The various types of tests that one can apply to web application testing are:

• Unit Testing - testing of the individual modules and pages that make up the
application.
• Page Flow Testing - ensures that pages that have a preceding page
requirement cannot be visited without having first visited the preceding page.
• Usability Testing – ensures that all pages present cohesive information to the
user. It is recommended that both end users and UI designers perform this
testing, and that this testing be done very early during the development cycle
so as to gain early user acceptance of the user interfaces and/or visual
semantics presented.
• Performance Testing – check that both the hardware (including firewalls,
routers, etc.,) and software scale to meet variant demands in user load.
• Data Volume Testing – Moving large amounts of data through the application,
either directly via input/output to a web page, or directly as input/output from
database that is referenced by a web page.
• Security Testing – ensuring that sensitive information is not presented in clear
text form, unauthorized access is not permitted, etc.,
• Regression Testing – re-testing specific parts or all of the application to ensure
that addition of new features has not affected other parts of the application.
• External Testing – deals with checking the effect of external factors on the
application. Example of external factors could be the web server, the database
server, the browser, network connectivity issues, etc.,

Page 9 of 17
divine QA Testing

• Business Logic Testing – checks that the applications' business functions work
accurately and perform according to constraints detailed in the requirements
document.

Some of the categories of testing suggested might not be applicable for certain
types of web enabled applications; the development/test teams(s) should make a
determination as to whether a test is or is not strictly required. If not, then the test
should be defined, but simply marked as optional or not required .The rationale
for stating a test is not required is simply to provide a record of the fact that the
condition requiring the test was considered, but deemed inappropriate for this
application.

Unit Testing

Unit testing involves testing of the individual modules and pages that make up the
application. Typical tests that fall in to this category are: Input tests (good input,
bad input, out of range input, excessive input, multiple clicks on buttons, reload of
pages prior to results being returned, etc.,). The deliberate attempt to confuse the
application can also be referred to as edge testing.

Some of the types of checking would include:

• Invalid input (Missing output, out of bound input, entering an integer when
float expected, and vice versa, control characters in strings etc.,)
• Alternate Input Format (e.g., 0 instead of 0.0, 0.00000001 instead of 0 etc.,)
• Button click testing e.g., multiple clicking with and without pauses between
clicks.
• Immediate reload after button click prior to response having been received.
• Multiple reloads in the same manner as above.

In general, unit tests check the behavior of a given page (i.e. does the application
behave correctly and consistently given either good or bad input). Random input
and random click testing also falls under the domain of unit testing. This testing
involves a user randomly pressing buttons (including multiple clicks on "hrefs")
and randomly picking checkboxes and selecting them. There are two forms of
output screen expected:

• An error page indicating the type of error encountered.

• A normal page showing either the results of the operation or the normal next
page where more options may be selected.

In no event should a catastrophic error occur.

Page Flow Testing

Page 10 of 17
divine QA Testing

Page flow testing deals with ensuring that jumping to random pages does not
confuse the application. Each page should typically check to ensure that it can
only be viewed via specific previous pages, and if the referring page was not one
of that set, then an error page should be displayed. A page flow diagram is a very
useful aid for the tester to use when checking for correct page flow within the
application.

Other aspects of page flow testing cross into other areas of testing, such as
security. Some simple checks to consider are forcing the application to move in an
unnatural path, the application must resist, and display appropriate error message.

Page flow testing involves logging into the system and then attempting to jump to
any page in any order once a session has been established. Security testing
encompasses several areas discussed later in this section as well as:

• Resilience to external attacks (e.g., denial of service)

• Resilience to accidental corrupt user input
• Resilience to deliberate corrupt user input

The use of bookmarks, temporary web pages setup to redirect into the middle of
an application using faked session information, etc., are all valid ways of testing
page flow and session security.

Usability Testing

Usability testing ensures that all pages present a cohesive look to the user,
including spelling, graphics, page size, response time, etc. This testing also
encompasses performance testing to some degree, as to load and regression
testing. Examples of usability testing include:

• Spelling checks
• Graphics checks (colors, dithering, aliasing, size, etc.,)
• Meaningful error messages
• Accuracy of data displayed
• Accuracy of data in the database as a result of user input
• Accuracy of data in the database as a result of external factors (e.g. imported
data)
• Meaningful help pages including context sensitive help

Load and Performance Testing

Load testing the application involves generation of varying loads against not only
the web server but also the databases supporting the web server and the middle
ware/application server logic connecting those pages to the databases. Load

Page 11 of 17
divine QA Testing

testing also includes verification of data integrity on the web pages, within the
back end database and also the load ramping or surges in activity against the
application. In particular, attention should be paid to pages that include large
amounts of data and what happens if multiple users hit these pages concurrently.
Some of the questions to be asked are: "Does the site scale ", "Is the site's
response time deterministic, etc.
Examples of load testing would include:
• Sustained low load test (50 users for around 48 hours).
• Sustained high load test (300+ users for 12 hours).
• Surge test (e.g. run 50 users, then surge to 500 users and then return to 50,no
memory leaks, lost users, orphaned processes, etc., should be seen). The
system should continue running with multiple surges at various times during
the day. This test should run for 48 hour.

Another very important facet of load testing is to discover at what load the
application would fail and what are the saturation point. All applications
eventually degrade given sufficient load. The collection of this degradation point
data is crucial since it may be used as a control set for monitoring data during live
use of the application. Predictive analysis made on live monitored data can
provide indications of when application stress points may appear on the horizon,
and proactive steps may be taken to ensure that the application and/or hardware
on which it runs scale up to meet new demand.

In case if major architectural changes occur in the application, (for example a total
face lift which makes it more usable), then the load tests must be re-run and a new
control set of performance data should be gathered, since the original control set
of data becomes invalid due to the architectural changes.

Table below illustrates loading estimates. This table is important in that it gives
the tester an indication of what is considered low, medium and high load. The test
scripts to generate these loads can then be constructed accordingly. The table
should accurately reflect the real load requirements. For example the high load
number could be as high as 5000 concurrent users or as low as only a dozen or so
concurrent users.

Server Load Description

Low 10 -500 concurrent users
Medium 100 -500 concurrent users
High 500 -5000 concurrent users

Data Volume Testing

Data volume testing involves testing the application under data load, where large
quantities of data are passed through the system. (e.g. large number of items in
dropdown/combo boxes, large amount of data in text boxes). Performance of the

Page 12 of 17
divine QA Testing

application should be monitored during this testing, since a slow database could
significantly affect response time.

A key point to be noted is that one should also monitor all the systems involved in
data movement. The data collected from the monitoring is valuable indicator of
system performance when coupled with the test load being delivered. This data
can be used as a control set for contrasting monitoring data from a live system and
providing predictive information indicating when major application stress points
may be encountered.
No errors should be seen on application pages or in error logs for pages that are
data intensive.

Security Testing

Security testing involves verifying weather both the servers and the application
are managing security correctly. All these tests must pass with no exceptions. A
partial pass must be recorded as a " Fail" since unauthorized access to data is a
major breach of security. Other forms of security testing involve checking not
only the electronic access security but also the physical security of the servers.
Some of the items that may be checked from a server perspective during this
testing are
• Attempt to penetrate system security both internally and externally to ensure
the system that houses the application is secure from bother internal and
external attacks.
• Attempt to cause things like buffer overflow to result in root access being
given accidentally, (such code does exist, but explaining it is beyond the scope
of this document).
• Attempt to cause the application to crash by giving it false or random
information.
• Ensure that the server OS is up to correct patch levels from security
viewpoint.
• Ensure that the server is physically secure.

Application level security testing involves testing some or all the following
• Unauthenticated access to the application
• Unauthorized access to the application
• Unencrypted data passing
• Protection of the data
• Log files should be checked to ensure they do not contain sensitive
information
• Faked sessions. Sessions information must be valid and secure. (e.g. a URL
containing a session identifier cannot be copied from one system to another
and then the application be continued from the different system without being
detected)
• Multiple login testing by a single user from several clients.

Page 13 of 17
divine QA Testing

• Automatic logout after N minutes of inactivity with positive feedback to the

user.
• Attempt to break into the application by running username/password checks
using password-cracking program.
Security audit, e.g. examine log files, etc., no sensitive information should be left
in raw text/human readable form in any log file.

Regression Testing

Regression testing ensures that during the lifetime of the application, any fixes do
not break other parts of the application. This type of testing typically involves
running all the other tests, or a relevant subset of those tests. A multidisciplinary
team consisting of developers, administrators and load testers would normally
perform this test. If UI changes were made, then UI designers and end–users need
to be involved. The regression tests must also be kept up to date with planned
changes in the application. As the application evolves, so must the tests.

External and Connectivity Testing

External testing deals with checking the effect of external factors on the
application. Example of external factors would be the web server, the database
server, the browser, network connectivity issues, etc., Examples of external testing
are:

• Database unavailability test (e.g., is login or further access to the application

permitted should the database go into a scheduled maintenance window).
• Database error detection and recovery test (e.g., simulate loss of database
connectivity, the application should detect this, and report an error
accordingly). The application should be able to recover without human
intervention when the database returns.
• Database authentication test (check access privileges to the database).
• Connection pooling test (ensure that database connections are used sparingly,
and will not run out under load).
• Web page authentication test.
• Browser compatibility tests – for example, does the application behave the
same way on multiple browsers, does the JavaScript work the same way, etc.,

Connectivity testing involves determining if the servers and clients behave

appropriately under varying circumstances. This testing is difficult to accomplish
from a server perspective since it is expected that the servers will be operating
with standby power supplies as well as being in a highly available configuration.
Thus the server tests need not be run using a power–off scenario; simply
removing the network connection to the PC may be sufficient. There are two
aspects of connectivity testing; the voluntary, where a user actively interacts with
the system in an unexpected way, and involuntary, where the system acts in an
unpredictable manner.

Page 14 of 17
divine QA Testing

Examples of involuntary connectivity checking includes some of the following;

• Forcing the browser to prematurely terminate during a page load using a task
manager to kill the browser, or hitting the ESC key and reloading or revisiting
the same page via a bookmark. The testing should cover both a small delay (<
10secs) in reinstating the browser as well as a long delay (> 10mins). In the
latter case the user should not be able to connect back to the application
without being redirected to the login page.
• Simulation of Hub Failure between PC and the Web Server. Removing the
network cable from the PC, attempt to visit a page; abort the visit, and then
reconnect the cable can simulate this. The test should use two time delays; the
first should be under 15 seconds, and the second delay around 15 minutes
before reconnecting. After reconnecting, attempt to reload the previous page.
The user should be able to continue with the session unless a specified timeout
has occurred in which case the user should be redirected to a login page.
• Web server on/off test. Shutdown the web server, then restart the server. The
user should be able to connect back to the application without being redirected
to the login page; this will prove the statelessness of individual pages. Note
the shutdown is only for the web server. Do not attempt this with an
application server, as that is a separate test.
• Database server on/off test. Shutdown the database server and restart it. The
user should be able to connect back to the application without being redirected
to the login page. It may be that a single transaction needs to be redone, and
the application should detect this and react accordingly.
• Application server on/off test. Shutdown the application server and restart it.
There are two possible outcomes for this depending on how session
management is implemented. The first outcome is that the application
redirects to an error page indicating loss of connectivity, and the user is
requested to login and retry. The second outcome is the application continues
normally since no session information was lost because it was held in a
persistent state that transcends application server restarts.

Examples of voluntary connectivity testing include;

• Quit from session without the user saving state.

• Quit from session with the user saving state.
• Server – forced quit from session due to inactivity.
• Server – forced quit from session due to server problem.
• Client forced quit from session due to visiting another site in the middle of a
session for a brief period of time.
• Client – forced quit from session due to visiting another site/application for an
extended period of time.
• Client – forced quit due to browser crashing.

Page 15 of 17
divine QA Testing

Browser compatibility also falls under the external or connectivity – testing

domain. The number of potential customers who may experience viewing
problems because of an incompatibility in browsers could be quite high. In
general, large audience web applications should work consistently with the
following browsers unless specifically excluded by the application requirements:

• Netscape Navigator 4.X

• Internet Explorer 4.X / 5.X
• AOL

The items that should be checked carefully are layout differences, JavaScript
behavior, image behavior, etc.,

Extended Session Testing

Extended Session testing should involve checking some or all of the following
items:
• Remaining in a session for an extended period of time and click items to
navigate the screen. The session must not be terminated by the server except
in the case of a deliberate logout initiated by the user.
• Remaining on a single page for an extended length of time. The session
should be automatically terminated and the next click by the user should take
the user to a page indicating why the session was terminated and the option to
log back into the system should be present. The page may have timed redirect
associated with it, and if so, a page indicating a timed out session should be
displayed. The following must be tested
• The user's session should have been saved and may optionally be restored on
re login
• The user's state must reflect the last complete action the user performed
• Leaving the application pages to visit another site or application and then
returning to original application via a bookmark or the back button should
result in a restoration of state, and the application should continue as if the
person had not left.

Power Hit/Reboot/Other Cycle Testing

Power Hit/Cycle testing involves determining if the servers and clients act
appropriately during the recovery process. The testing will be difficult to perform
from a server perspective since it is expected that the servers will be operating
with standby power supplies as well as being in a highly available configuration.
(i.e., hot spares, RAID, application fail over, etc.) However, the clients can be
tested this way by removing power from the PC's. Below are the tests performed.

• Client power off/on test.

Page 16 of 17
divine QA Testing

• Client hub power off/on test.

• Client network connection removal/reinsertion test.
• Server power off/on test.
• Server Hub power off/on test.
• Server network connection removal/reinsertion test.

Business Logic Testing

All the tests listed previously may pass with flying colors, but if the business
logic is not encapsulated by the application, then the application has not met its
requirements. Business requirements vary in nature and are thus difficult to
generalize.

The best way to quantify the business is to start with an existing business process
model, and extrapolate. The model can derive specific requirement for the web
application, and may or may not involve significant back end mainframe or mini
computer technologies. The key is that the web application and middle ware logic
meets the overall business requirements. Representative from the business team
must participate in defining these requirements.

Part of business logic testing is validation that certain sequences to in fact

generate the correct outputs back to the application user as well any backend users
be they human or machine. This is the black box approach, and works very well
for models that follow strict input/output process models. For those models in
which multiple outputs occur at various points in the application, a more
sophisticated approach must be used. Part of that approach is to use tools to
validate data at various intermediate points during the application run.

Electronic equipment such as televisions, cell phones, etc, use test points to which
probes may be attached and measurements taken to determine if the equipment is
behaving within specific tolerance levels. The same approach may be taken for
software, and will be a valuable aid in determining application success. This is
where the majority of the test teams time will be taken, automated tools may be
purchased to augment load testing etc, but the tools to test specific points in the
application must be written, or the data examined by hand.

The test team is responsible for constructing valid set of tests, scripts, and tools
that may be generated to facilitate testing. These tools will become the core tools
for performing regression testing as the application's capability increase over
time.

Page 17 of 17