Testing Strategy

Version 2.0
Author: PSC Consulting

PeopleSoft, the PeopleSoft logo, PeopleTools, PS/nVision, PeopleCode, PeopleBooks, and Red Pepper are registered trademarks, and The
Vantive Corporation, PeopleTalk, and "Applications for eBusiness" are trademarks of PeopleSoft, Inc. All other company and product names
may be trademarks of their respective owners. The information contained herein is subject to change without notice. Copyright 2002
PeopleSoft, Inc. All rights reserved.

Document Control
Change Record
Date

Author

Version

Change Reference

11.18.2002

PSC Consulting

1.0

Original Document

11.24.2002

PSC Consulting

2.0

Incorporated Changes

Sign-Off
Date

Reviewer Name

Position

Copy No.

Name

Mark Cianca

Reviewers
Sign-Off

Distribution

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

-1-

Table of Contents
1

INTRODUCTION ................................................................................................................ 2
1.1

1.2

Objectives ..............................................................................................................................2
1.1.1

AIS Project Management .........................................................................................2

1.1.2

Development & Testing Team..................................................................................2

Approach................................................................................................................................2
1.2.1

Document Scope......................................................................................................2

GOALS AND OBJECTIVES OF TESTING ........................................................................ 2

TESTING PHASES ............................................................................................................ 2

3.1

Business Testing....................................................................................................................2

3.2

Performance Testing..............................................................................................................2

TESTING PROCESS FLOW .............................................................................................. 2

4.1

Definitions of Testing Phases ................................................................................................2

4.1.1

Testing Subtypes .....................................................................................................2

4.1.2

Black Box Testing ....................................................................................................2

4.1.3

Acceptance Testing..................................................................................................2

TESTING TEAM STRUCTURE.......................................................................................... 2

5.1

Introduction ............................................................................................................................2

5.2

AIS Technical Project Manager .............................................................................................2

5.3

Testing Lead ..........................................................................................................................2

5.4

Module Testing Leads............................................................................................................2

5.5

Data Conversion Testing Lead ..............................................................................................2

TESTING SCRIPTS ........................................................................................................... 2

TESTING SCHEDULE ....................................................................................................... 2

TESTING INFRASTRUCTURE DEPLOYMENT ................................................................ 2

8.1

Testing Environments ............................................................................................................2

8.2

AIS Testing Instances ............................................................................................................2

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

-2-

ISSUE RESOLUTION PROCESS...................................................................................... 2

9.1

Issue Management Process ..................................................................................................2

9.2

Issue Tracking System...........................................................................................................2

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

-3-

1 Introduction
1.1 Objectives
This document deals with the goals and objectives of testingthat is, the purpose of
testing, testing phases and processes, resource allocations, testing environments and
an issue resolution. This document also tries to identify testing activities.
The AIS Testing Strategy Document forms the high level testing methodology for the
AIS project implementation and should be employed as a guide for the entire testing
effort. Although the general intent of this document is applicable to implementation
testing, the standards set forth should be applied to the greatest extent possible to
ensure the integrity of the system. The document should be used by each of the
following audience groups in the manner described below.
1.1.1

AIS Project Management

Project Management may use this testing strategy as a communication tool between
the project team and other interested parties, and to establish expectations for what
will be executed and monitored during the testing process. The Management team
may also employ this strategy to understand the different phases of testing, resource
requirements, and the estimated timeline necessary to ensure a comprehensive and
rigorous test of the applications.
1.1.2

Development & Testing Team

This document is primarily intended for project management to guide the testing
process but it may also be a useful tool for development and testing teams. The
development and testing teams may review this strategy to understand the different
phases of testing, resource requirements, the test management process, sign-offs, and
the estimated timeline. The strategy explains the flow and the details of the testing
process and the team members roles within this process. Successfully executing a
comprehensive and rigorous set of tests will minimize the risk of deploying the
applications.

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

-4-

1.2 Approach
This document focuses on the following areas:

Goals and Objectives of Testing: This section identifies the aims and
specific purpose of testing.

Testing Phases: This section describes the phases of the testing progression
strategy.

Testing Process: The testing process describes the testing cycle.

Testing Team Structure: This section details the different roles and
associated responsibilities during the testing phase.

Testing Scripts: This purpose of this section is to explain the reasons and
methods for creating testing scripts.

Testing Schedule: This portion identifies important concepts for

determining the testing schedule.

Testing Infrastructure Deployment: The section provides explanation of

testing instances.

Issue Resolution Process: This portion explains the steps for issue
resolution.

1.2.1

Document Scope

This document is not intended to cover the following areas:

Details of bug-reporting and bug-fixing activities
Details of bug/enhancement tracking system
The strategy for correcting problems identified after the initial release of the
AIS PeopleSoft Implementation. The strategy assumes that the procedures
existing at go-live will be followed in tracking and prioritizing identified
problems, resolving these problems, and releasing new software versions into
production.
The testing deliverable does not include any testing conducted outside the scope of
the AIS project implementation.
The testing process will interface with the development cycle and will precede the
release to production.

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

-5-

2 Goals and Objectives of Testing

The purpose of testing is to validate that the proposed system solution will satisfy the
business needs as specified in the functional specifications. Testing can do this by
identifying and tracking issues and facilitating resolution of those issues. Issues that may
arise during testing can be related to erroneous code, faulty code integration, data
conversion errors, instance setup, application setup, code migrations, and inaccurate
specifications. As these problems are found during testing, they will be recorded in
problem tracking system, prioritized, resolved, and re-tested.

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

-6-

3 Testing Phases
Testing phases give structure to the testing effort and bring to light any dependencies
among the different testing events. During the course of the testing process various
types of tests will be conducted to certify that the systems are suitable for release to
the production environment. The tests can be broadly categorized into the following
two areas:

3.1 Business Testing

Business system testing aims to test the quality of all elements of an application. It
does not address performance and data conversion issues.
Included in Business Testing:
Unit Testing
Link Testing
Functional testing
Systems Integration Testing
Regression Testing
Acceptance Testing

3.2 Performance Testing

The objective of Performance Testing is to define, construct, and execute a
performance test on an entire system or on a particular component of a system. By
implementing the process, you can establish the performance of the system or
component under test, and use the results to make decisions on whether the
performance is acceptable for the business.

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

-7-

4 Testing Process Flow

This section outlines the testing process flow to be used and the phase dependencies.
The testing phases follow a process in which to ensure rigorous testing: unit testing is
a pre-requisite for link testing, which is a pre-requisite for functional testing, which is
a pre-requisite for systems integration testing. After systems integration testing has
been performed, data conversion testing, failure recovery testing, and acceptance
testing may be performed in any order or simultaneously. Systems integration testing
is not a pre-requisite for data conversion testing, failure recovery testing, or
acceptance testing. Ideally, systems integration testing will employ a fully converted
data set.
Testing phases flow into each other as follows:
Data Conversion
Testing

Unit Testing

Link Testing

Functional Testing

Systems Integration
Testing

Failure Recovery
Testing

Acceptance
Testing

Figure 1

Testing Process Flow

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

-8-

4.1 Definitions of Testing Phases

The different phases of testing are defined as follows:

Unit Testing: To perform unit testing is to test and ensure the functioning of
each portion of a product or segment of code separately.

Link Testing: When a given unit is deemed to be operating successfully, it

is integrated with another successfully operating related unit (eventually
making up one business process), and the combination is tested, with
particular attention paid to ensuring that the upstream feed is appropriately
accepted, and the downstream feed is appropriately generated.

Functional Testing: The functional test is performed after signoff on the

unit and link tests. The entire system is tested, with emphasis on ensuring
successful interaction between business processes.

Systems Integration Testing (SIT): After signoff on the functional test, the
interaction of the system with neighboring systems should be tested. System
integration testing tests the following: the proper functioning of all
interfaces, proper processing of upstream feeds, and the proper generation of
downstream feeds.

Data Conversion Testing: After data conversion is run, the results are
tested to ensure that all the necessary data was migrated, and all conversion
is correct. Data conversion may take place before systems integration
testing, so that proper data is in place for such testing; or it may be delayed
until after the system has been proven robust. If data conversion takes place
before systems integration testing, data conversion testing should also take
place before system integration testing.

Failure Recovery Testing: Failure recovery testing ensures that the area
being tested has the ability to recover from hardware, software, process,
procedure, or security failure.

Regression Testing: After each defect is corrected, tests should be run to

verify that the defect has actually been corrected, and that no new defect has
been introduced.

Acceptance Testing: This test type confirms that the system as a whole,
meets end user business requirements. Actual end-users perform actual
functions in a testing environment using migrated data to ensure the system
meets all business requirements.

4.1.1

Testing Subtypes

Each type of testing has subtypes associated with it. Specifically, it is recommended
that thorough branch testing (that is, testing all logical data paths through the unit,
linked units, system, or systems) be employed wherever possible. As it is rarely
practical, especially when dealing with anything larger than a single unit, to perform

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

-9-

complete branch testing, a combination of branch and condition testing is an

acceptable compromise. Condition testing, instead of walking through every logical
path, tests the individual components that can be combined in different ways. That
is, should it not be practical to test every possible combination of circumstances, it is
acceptable to test the most likely ones, and then to test that each circumstance
functions properly on its own. Either error guessing or functional importance should
be used to decide which aspects are given a thorough branch testing and which are
condition tested.
4.1.2

Black Box Testing

Underlying each step in the unit test -> link test -> functional test -> systems
integration test flow is the concept of the black box test. A black box test is a
straightforward test of function, in which the object being tested (a segment of code,
a function of the program, the entire deliverable) is treated as a function box. Based
on the functional specifications, test cases are written that stimulate the black box to
provide certain specified outputs. A list of inputs and their expected outputs is kept,
and each test is marked simply pass or fail. At each phase (unit, link, system, and
systems integration) both positive and negative black box tests should be runthat
is, both tests that expect success and tests that expect failure.
Types of black box tests include:

Boundary Analysis A boundary analysis inputs variables on the very edge

of what the black box being tested is supposed to take, both the smallest and
the largest of what is a possible input, to ensure correct functioning at both
extremes.

Stress Testing To stress test a black box is to deliberately feed it inputs that
it is not designed to take, to ensure that no unacceptably negative
consequence results. Types of stress tests include any type of negative test,
as well as volume testing, in which the tester inputs a larger volume of data
into the system.

Error Guessing This test is designed by an experienced designer,

programmer, tester, or even end user, who draws upon his or her past
experience to suggest the areas most likely to fail, and to design black box
tests to catch those specific bugs.

4.1.3

Acceptance Testing

Acceptance testing is a separate situation governed by different rules. If the end

users were involved in the design process, black box tests can be derived from the
functional specifications, and translated into test scripts. In other circumstances, the
simplest form of acceptance testing might be to let the end users use the system as
they would in their day-to-day work, and generate from that experience comments
about adherence to business requirements, performance, and ease of use.

10

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

- 10 -

5 Testing Team Structure

5.1 Introduction
The testing process can require the creation of new roles for members involved in the
AIS implementation. During the start of the testing phase, team members will be part
of the testing team.

5.2 AIS Technical Project Manager

The AIS Technical Project Manager ensures that all necessary levels of testing are
conducted on the system. He prepares the testing schedule and testing deliverables.

5.3 Testing Lead

The Testing Lead is a separate role in the AIS implementation project team as
identified in the Roles and Responsibilities deliverable. The Testing Lead has the
day-to-day responsibility for all testing activities. He or she will aid the Technical
Project Manager in preparing the testing schedule and testing deliverables. The
Testing Lead ensures that testing activities are on schedule and that adequate
resources are available to the team to carry out the testing activities.

5.4 Module Testing Leads

The Team Leads from all the modules assume the Test Lead responsibilities for the
specific module once the testing phase begins. He or she will coordinate and manage
all activities related to functional testing. Functional testing consists of tests such as
unit testing and system integration testing. The functional testing team is made up of
Process Managers, Testers, and Programmer/Analysts. The Module Testing Lead
coordinates the development of the testing scripts and ensures that testing activities
are carried out on schedule.

5.5 Data Conversion Testing Lead

The Process Managers assume the role of Data Conversion Test Leads during the
data conversion phase. The Data Conversion Testing Lead coordinates and manages

11

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

- 11 -

all activities related to migrating data from the legacy systems to the project
database. Data conversion testing consists of tests such as conversion unit testing,
conversion functional testing, and conversion validation testing etc. The Data
Conversion Testing team is made up of Database Administrators, Process Managers,
Testers, and Programmer/Analysts.

12

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

- 12 -

6 Testing Scripts
Test scripts are the building blocks of any testing process. Test scripts cover the core
functions of the software, i.e., functions that users will use most often. Unit test
scripts covering a wide range of the software's functionality should be developed
early in the development cycle. When test scripts are run frequently during the
development process, they help identify and isolate defects, at a place in time where
defects are easier to find and less costly to correct. Test scripts are written for both
positive (valid) testing and negative (invalid) testing. Positive testing proves that the
software does what it should do; negative testing proves that the software doesnt do
what it shouldnt do.
The following rules apply to test scripts:
Test script functionality should be kept in sync with the application code.
Test scripts should be version controlled, just like the application code.
The version/revision number of the test script must be the same as the
application.
Test scripts are written for each level of testing - unit testing, functional testing,
systems integration testing, and acceptance testing. Each level of testing focuses on
various areas for script writing, for example, logic, file access, program
documentation, input, and performance. The testing lead will decide the
areas/features for the test scripts written for each of the levels of testing (unit testing,
functional testing, integration testing and acceptance testing)
The code-based test script should be approached in the same way as the code that it is
testing; it should be designed, developed and commented.

13

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

- 13 -

7 Testing Schedule
The testing schedule serves to match resources to the job. The Testing Lead creates a
detailed project plan including all the testing tasks based on the critical path for
success. Testing resources are identified and allocated to tasks depending on the
following conditions:

Availability of Resources: In case testers are not available or dont have the
requisite skills, adequate time is provided in the testing schedule to recruit
the right resources or to train the existing testing personnel.

Criticality of the Milestone: If the project has a critical milestone, more

than the required resources are put on the testing tasks to ensure that the
milestone is achieved.

Required Skill Level : Experienced testers are allotted to any testing

activities that require a high skill level, while easier testing tasks are
performed by less experienced testers. Experienced testers usually formulate
the test plans, test scripts and test scenarios while new testers execute the test
scripts and test scenarios. Allocating resources by considering the skill level
of testers builds up a buffer in the schedule and provides for contingencies by
keeping the more experienced people free to concentrate on problemresolution.

Availability of Testing Platforms: If a testing task requires the availability

of a special testing tool or a testing platform that is not easily available in the
market, the time needed for procurement has to be added to the testing
schedule.

14

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

- 14 -

8 Testing Infrastructure Deployment

8.1 Testing Environments
Examples of typical testing environments include:

Demo Instance: All new application patches and updates are first applied to
the demo instance. Process Managers must test all new functionality and
verify that it does not break business requirements or conflict with any
customizations. In the AIS project the demo instance is SADMO.

Development Instance: The development instance is used to continually test

the customized code, bug fixes and enhancements created by the developers.
Effective source code control techniques are necessary to keep the
development instance up-to-date so that the new/updated code is always
tested against the latest releases. The test data must be managed to prevent
testers/developers from overwriting each others data as multiple team
members work in the same environment. One way to accomplish this is to
assign value ranges to each unit tester. The AIS project will have two
development environments: SADEV and SADMD.

Functional Testing Instance: The testing instance is ideally a replica of the

production instance. The PeopleSoft Administrator will migrate AIS
Releases from the development environment to the testing environment. In
the AIS project the testing environment is SATST.

Data Conversion Testing Instance: In a data conversion testing instance,

the application is tested with real time input (data) for data conversion
accuracy. In the AIS project the data conversion testing instance is SACVT.

Quality Assurance Instance: The quality assurance (QA) instance is the last
testing area before changes are migrated into production. Tested data
migration runs from the data conversion testing instance and tested AIS
Releases from the functional testing instance are combined together in the
QA instance. They are tested one last time for functionality and for
performance. The QA instance in the AIS Project is SAQA.

Parallel /Multi-platform testing: The need for a prototype and an instance

setup that allows parallel testing has to be discussed with respect to the
project. For example, many Y2K projects required parallel testing. Parallel
testing requires two separate instances (in this example one instance was
Y2K compliant and the other instance was the existing instance in use) that
must run the same data side by side, in order for programmers to locate
errors in the repaired systems code. Also, if the software has to support
multiple platforms, one may need multiple sets of machines for each project

15

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

- 15 -

to do parallel testing. Alternatively, one might decide to constantly swap

software.

8.2 AIS Testing Instances

Listed below are the current instance names and the associated testing:
AIS Testing Instances

Associated Testing

SADMD

Unit Testing

SADEV

Unit Testing

SATST

Link and Functional Testing

SACVT

Data Conversion Testing

SAQA

System Integration Testing

Table 1
AIS Testing Environments

16

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

- 16 -

9 Issue Resolution Process

9.1 Issue Management Process
During the Development phase of the project, the AIS Technical Project Manager
and the Testing Lead along with the Project Managers and the Module Technical
Leads will design an issue resolution process which will ensure that proper
mechanism exists for resolving all issues which are reported during testing. A typical
issue resolution process is demonstrated in the process flow chart below.

17

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

- 17 -

Issue
Management
Role

Issue
Management
Process

A1. Identify
Issue
Issue Originator

A0. Raise Issue

A2. Submit Issue

Reporting Form

B1. Review
Issue

B0. Register Issue

No

B2. Is Issue
applicable to
project?

AIS Technical Project

Manager/Testing
Lead

Yes

B3. Update
Issue Tracking
System and
Assign Priority

C1. Review
Logged Issues

C3. Close issue

& update Issue
Tracking System

Yes

C2. Has issue

been resolved?

No

C0. Assign
Issue Actions

Issue
Change
Request

Yes

C4. Does issue

require a change?

Project
Review Group

No

Raise
Risk
Project

Yes

C5. Does issue

raise a risk?

No

C6. Assign Issue

Actions

D0. Implement
Issue Actions

D1. Complete
Issue Actions .

Project Team

9.2 Issue Tracking System

This process ensures that each issue identified within the project environment is
documented, prioritized and resolved within an appropriate timescale.

18

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

- 18 -

A reporting system is required which gives a status of bugs reported and resolved.
Its output should feed into an overall project management system, which allows
schedules and plans to be reworked based on this testing status.

19

PROPRIETARY AND CONFIDENTIAL

AIS Testing Strategy

- 19 -