Engineering Procedure

SAEP-750 14 November 2016

Testing Procedures for Process Automation Systems (PAS)
Document Responsibility: Process Control Standards Committee

Contents
1 Scope ................................................................ 2
2 Applicable Documents ....................................... 2
3 Acronyms and Definitions .................................. 3
4 Instructions ........................................................ 4
5 Responsibilities ................................................ 17
Revision Summary................................................. 18

Appendix A - Recommended Testing Procedures

for TMS Systems for Use in Bulk Product
Distribution Plants ...................................... 19

Previous Issue: New Next Planned Update: 14 November 2019

Page 1 of 19
Contact: Kinsley, John A. (kinsleja) on phone +966-13-8801831

©Saudi Aramco 2016. All rights reserved.

Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

1 Scope

This procedure defines recommended procedures to conduct Factory Acceptance

Testing (FAT), Integrated Factory Acceptance Testing (IFAT), and Site Acceptance
Testing (SAT) for Process Automation Systems (PAS). Execution of FAT, IFAT and
SAT are mandatory requirements for all PAS projects as defined in SAEP-16, Project
Execution Guide for Process Automation Systems. The purpose of these tests is to
verify the PAS design, manufacturing, and configuration programming to ensure it
meets project requirements and also all mandatory Saudi Aramco requirements.

Inspection requirements are not included in the scope of this document. Inspection
procedures shall be developed for each project as part of the project detailed inspection
and testing plan.

2 Applicable Documents

All referenced Procedures, Standards, Specifications, Codes, Forms, Drawings, and

similar material or equipment supplied shall be considered part of this Procedure to the
extent specified herein and shall be of the latest issue (including all revisions, addenda,
and supplements) unless stated otherwise.

2.1 Saudi Aramco References

Saudi Aramco Engineering Procedures

SAEP-16 Project Execution Guide for Process Automation
Systems
SAEP-98 Removable Media Usage for Process Automation
Systems
SAEP-99 Process Automation Networks and Systems Security
SAEP-302 Instructions for Obtaining a Waiver of a Mandatory
Saudi Aramco Engineering Requirement

Saudi Aramco Engineering Standard

SAES-J-904 FOUNDATION™ fieldbus (FF) Systems

2.2 Industry Codes and Standards

American National Standards Institute

ANSI/ISA 62381-2011 Automation Systems in the Process Industry -
Factory Acceptance Test (FAT), Site Acceptance
Test (SAT) and Site Integration Test (SIT)

Page 2 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

3 Acronyms and Definitions

3.1 Acronyms
CCS - Compressor Control System
DAHS - Data Acquisition and Historization
DCS - Distributed Control System
ESD - Emergency Shutdown System
FAT - Factory Acceptance Testing
FSD - Functional Specification Document
IFAT - Integrated Factory Acceptance Testing
LTSK - Lumps Sum Turn Key
OO - Operating Organization
P&CSD - Process & Control Systems Department
PAS - Process Automation System
PLC - Programmable Logic Controller
RMPS - Rotating Machinery Protection System
SAT - Site Acceptance Testing
SAEP - Saudi Aramco Engineering Procedure
SAPMT - Saudi Aramco Project Management Team
SCADA - Supervisory Control and Data Acquisition Systems

3.2 Definitions

Company: The Saudi Aramco entity which is responsible for the procurement
of the PAS system under test. For capital projects, this is normally SAPMT.
For smaller sized projects, this may be an operating organization representative.

Contractor: means LSTK contractor or Engineering, Procurement and

Construction (EPC) contractor depending on the procurement method selected
for the project. There may be more than one CONTRACTOR involved in the
project. In this case, the term CONTRACTOR refers to all applicable
contractors.

Non-Material Requirements: The complete set of documentation required for

the design of a PAS project. There are three categories of NMRs:
601 NMRs Preliminary drawings for review and approval

Page 3 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

602 NMRs Certified drawings, literature, photographs, and parts

data/requirements
603 NMRs Operations, maintenance manuals, installation instructions,
test certificates, etc.

Operating Organization: The department responsible for operating the facility

where the PAS will be installed; sometimes called Proponent.

Process Automation System: A network of computer-based or

microprocessor-based modules whose primary purpose is process automation.
The functions of a PAS may include process control, safety, data acquisition,
advanced control and optimization, historical archiving, and decision support.
Examples of process automation systems are DCS, SCADA, ESD, RMPS, CCS,
and PLC-based systems.

Vendor: The original equipment manufacturer of the PAS system under test.
This party is also responsible for the engineering and design of the PAS per the
project specifications and applicable Saudi Aramco requirements.

4 Instructions

This section details the recommended testing procedures for PAS during FAT, I-FAT
and SAT. The procedure defines recommended testing for each test phase and is
optimized to minimize duplication of testing during subsequent testing phases. It is
important to consider these recommendations in their entirety. Recommended testing
criteria for SAT assumes that certain testing has been completed during FAT. If the
recommended FAT / I-FAT procedures have not been completed, then additional SAT
testing may be required to ensure the integrity of the system at site.

Separation of hardware testing from software verification is recommended to allow

for concurrent testing of software and hardware. Technologies, such as virtualization,
enable software testing to be conducted on virtual servers, not the actual system.
This methodology enables software verification to be completed much earlier in the
testing cycle reducing the overall testing schedule.

The recommended testing below assumes that a full Pre-FAT, Pre-IFAT have been
conducted by the vendor and the system has passed all required Pre-FAT procedures.

4.1 Factory Acceptance Testing (FAT) Procedures

The purpose of Factory Acceptance Testing is to verify that the system meets
the job specification and all mandatory Saudi Aramco standards requirements.
FAT shall focus on the design and functionality developed and implemented for
the project, and not on standard system features.

Page 4 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

Section 4.1.1 below defines the recommended FAT testing categories for all PAS
types. Detailed test criteria shall be developed for each category, along with the
required test equipment and expected outcome of each test. Section 4.1.2
provides additional recommended testing for specific PAS types.

4.1.1 Recommended FAT Test Categories

Hardware FAT
1) Physical Verification: All components supplied with the system
shall be visually checked and verified against the applicable project
documentation. The intention is to verify that all components have
been supplied as per the project bill of materials and to ensure that
the materials provided are ‘as new’ and free from visual defect and
installed as per the Vendor guidelines and Saudi Aramco
requirements.
2) Wiring, Tagging and Cabinet Integration: The design and
installation of wiring for all cabinets and consoles shall be
physically inspected and verified to ensure consistency with the
relevant project drawings and to ensure they meet the intent of the
relevant material and engineering standards. Tug testing shall be
performed, per Saudi Aramco standards to ensure integrity of
physical wire terminations.
3) Power and Grounding: Verify electrical design and wiring for all
power supply and distribution circuits. Verify grounding of all
equipment meets Saudi Aramco and Vendor requirements, including
AC safety and DC grounding.
4) Spare and Expansion: Verify that the system provided meets the
project and standards requirements for spare capacity and expansion
capabilities. Items such as number of spare IO of each type per
operating area, spare slots in the controller and IO chassis or
baseplates to meet expansion requirements, controller spare
capacity, spare ports for network switches and fiber optic patch
panels, and power supply capacities shall be verified for compliance
with applicable project and standard requirements.
5) Revision Level: All components supplied with the system shall be
checked to verify they are the latest approved software and
hardware revision applicable for the project.
Note: A representative physical sample can be checked; however, all
components as listed in the bill of materials shall be to the
appropriate revision level.

Page 5 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

6) Redundancy Testing: A representative sample, minimum one of

each type of hardware modules and software supplied in redundant
configuration, shall be tested to ensure failover and redundancy is
fully functional and that ‘automatic failover’ of redundant modules
is functioning as expected. This shall include as a minimum:
controllers, IO modules, power supplies, networking components
and applicable application software.
Commentary Note:

P&CSD performs extensive redundancy testing during the product

approval process. It is not necessary for each project to test
redundancy of all modules. FAT testing should verify a minimum
sample of each type to confirm the system is configured and working
as expected.

7) Hardwired Loop Tests: The purpose of hardwired loop testing is to

verify the integrity of the wiring from the marshalling rack to the IO
module and to verify that the IO module is correctly configured and
communicating with the system. Verification of one channel of
each IO module is recommended to verify the integrity of the wiring
and installation only. Complete IO loop testing, including range
checks, configuration, graphics display, alarming, historization, etc.,
should be performed using software simulation of the IO cards only
as described in software loop tests below.
Commentary Note:

100% Hardwired IO loop testing is not required or recommended

during FAT. Time should be spent verifying the application and
configuration (i.e., software) rather than the physical IO. This practice
is recommended since 100% Io testing is typically conducted during
Pre-FAT and again during pre-commissioning / commissioning.
Duplication of this testing during FAT does not add value to the testing
and takes an enormous amount of time / manhours.

8) Fail-safe Configuration: The purpose of Fail-safe configuration

testing is to ensure that critical control outputs are properly
configured to ‘Hold last Value4” on loss of communications either
between controllers (peer-to-peer control) or loss of communications
between the controller and the IO card. A representative sample of
outputs shall be tested to confirm proper operation of outputs during
fault conditions.

Software FAT
1) Control Database Configuration: The purpose of this section is to
verify the configuration and operation of all ‘control strategy
templates’ used to develop the complete control application.

Page 6 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

Where control logic has been built using templates, complete testing
of each control template shall be performed. Verify that the
configuration of the template is consistent with the provided
specification documents to ensure the correct template and
parameterization has been implemented for each control scheme.
Functionality, such as initialization, action on bad input detection
and output fail-safe action shall be confirmed.
2) Software Loop Tests: The purpose of Software Loop testing is to
verify that the required database of inputs, outputs, controllers, etc.,
is properly configured in the system. This can be done on a per loop
basis (i.e. input-controller-output) by soft simulating the input signal
and verifying the configuration through to the associated output
signal (if applicable). The following are recommended to be
verified during software loop testing:

Applicable for all tags:

 Block scaling and descriptions
 Alarm settings and priorities
 Verification of display on process graphics
 Verification of alarming on process graphics and alarm summary
displays

Applicable for PID controllers:

 Control action (Increase / Decrease or Increase / Increase)
 Risk Area segregation requirements
 Action on Bad Input and Initialization
Commentary Note:

If control strategies are built from control templates, Bad Input

response and initialization only need to be confirmed once in the
template.

 Cascade, Split-range, ratio or other multi-loop control operation.

Applicable for Outputs:

 Verify the fail-safe action has been properly configured.
3) Graphics Checkout: Verify the content, structure, layout and
operability of graphics provided for the system. Graphics checkout
shall be conducted using soft simulation of IO points. Testing of
individual display elements, tag descriptions, alarms, etc., is

Page 7 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

intended to be completed during software loop tests. Additional

testing of graphics shall include, display navigation, user security
and most importantly, operator usability. Usability shall focus on
ensuring the operator has easy access to all information and actions
necessary to complete critical tasks, such as startup of equipment,
change-over of products, etc.
4) Application Testing: Verify the basic functionality of all application
software / packages provided with the system. This shall include
the following as a minimum if provided as part of the system:
System diagnostics, historian, trending, reporting, alarm
management, and any advanced control software supplied to meet
the project requirements.
5) Performance Testing: Verify the system meets the minimum
performance requirements defined in the project specification and
relevant standard and/or material specification. Verification of CPU
loading, memory utilization and other performance parameters shall
be conducted on all servers while the application is running.
Spot checks on operator / engineering workstations shall also be
conducted.
6) System Security and Access Control: Verify the system /
applications meet the company requirements for system security and
access control as defined in SAEP-99, Process Automation
Networks and Systems Security. The following functionality should
be verified as a minimum:
 User role configuration: Verify that the appropriate user roles
have been configured and are enforced on the system.
 User accounts: Verify that individual user accounts are
configured for each user role and that the user’s inherits the
proper access privileges when logging into the system.
Note that only Operator accounts may use generic user accounts.
All other general user accounts, such as engineer, administrator,
etc., shall be disabled.
 Password complexity: Verify that password complexity rules
are configured and enforced.
 System hardening – workstations and servers: Verify that
unused services and protocols are disabled on all workstations
and servers as per the security baseline design documents.

Page 8 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

 System hardening – networking: Verify that unused ports on

network switches are disabled. Verify that network devices
have appropriate route / access control as per the security
baseline document.
 Anti-virus: Verify that anti-virus software is installed and
configured on all stations connected to the system. Verify that
distribution of AV dat files from a centralized server is working
as expected.
 Patches: Verify that the latest windows OS and application
patches are installed on all workstations and servers. Also, verify
that all vendor software / application patches have been installed
on the system. If the system is delivered with the capability of
automatically distributing Windows patches, verify that this
functionality is working on all workstations and servers.
 Audit logs: Verify that audit logging is enabled on all
workstation, servers, and process automation network devices.
 Backup and recovery: Verify that backup and recovery software
is installed on all nodes and configured to automatically backup
nodes periodically to a centralized backup repository. Verify the
recovery procedure from the automated backups for a random
sampling of equipment.
 USB and removable media devices: Verify that the system is
able to recognize authorized USB devices and will allow access
to authorized USB devices while denying access to unauthorized
devices. Verify that only USB ports identified in the
‘Removable Media Usage Scheme’ (if available) are enabled
and are able to recognize authorized devices. Verify that all
USB ports and/or removable media which is not identified in the
Removable Media Usage Scheme’ have been disabled.
 At completion of FAT, verify that any temporary user accounts
added to facilitate testing have been removed from the system.
7) Unstructured Testing: Following satisfactory completion of
structured testing, company representatives may conduct additional
tests to further validate the robust operation and response of the
system under a variety of potential scenarios.

4.1.2 System Specific FAT Testing

The following testing is recommended in addition to the above for the

Page 9 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

specific type of Process Automation System listed.

1) Distributed Control Systems (DCS)
 Advanced Regulatory Control: Verify the configuration and
operation of any advanced control strategies developed for the
project against the approved control narrative document.
 Sequential Logic or Batch Control: Verify the configuration and
operation of any sequence logic or batch control logic developed
for the project against the approved control narrative document.
 Alarm Management Performance Reports: Verify that the alarm
management system is configured to produce periodic alarm
management performance reports as per requirements.
 DCS Historization: Verify the configuration of tags in the DCS
historian; including tagname (source), scaling, scan rate, and
storage deadband (if applicable). Verify that the database is
configured to automatically archive or delete files after a pre-
defined storage period (i.e., 3 months data storage).
2) Emergency Shutdown Systems (ESD)
 Functional Logic Testing: All ESD logic functionality shall be
checked against logic drawings and dynamically tested and verified
for proper ESD sequence and functionality. (Note: For parallel
processes or equipment, logic shall be verified for each
equipment). The dynamic test will involve soft simulation of
inputs and outputs in their proper operational sequence, and
verifying that specified ESD application program logic is executed
properly. Testing to be done with the DCS during IFAT.

 ESD Bypass Testing: Verify that all inputs have associated

input bypass tags configured. Verify that bypasses function as
expected by enabling a bypass for selected inputs, setting the
input beyond the trip setpoint and confirm the normal operation
of the ESD logic.
 Time Delay to Trip settings: Verify the response of the system to
a bad input signal. Verify that an operator alarm is generated and
a ‘time-delay’ before the trip signal is initiated. These settings
shall be verified to ensure accuracy with the SIF specification
sheet.

Page 10 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

3) Compressor Control Systems (CCS)

 Anti-surge control: Verify the data used to configure the surge
limit line (SLL), surge control line (SCL), and any parameters
used for automatic recalculation / adjustments to SCL.
 Fallback strategies: Verify the response of the controller to
failures (i.e., bad input) to flow, pressure and temperature
measurements used for anti-surge control.
 Performance control / load sharing: Verify the operation of
load-sharing control (if applicable), when a compressor is put in-
service / out-of-service.
 Compressor operating map: During Integrated Factory
Acceptance Testing, verify the proper display of compressor
map on the DCS operator console.
4) Terminal Management Systems (for product distribution, aka bulk,
plants)
 Refer to Appendix A for list of recommended test procedures for
TMS systems.
5) FOUNDATION Fieldbus
 Refer to SAES-J-904, Section 12 for Factory Acceptance
Testing requirements for the FOUNDATION Fieldbus portion of
control systems.

4.2 Integrated Factory Acceptance Testing (I-FAT) Procedures

The purpose of the I-FAT is to verify the configuration of the interface between
the main control system (i.e., PCS) and systems supplied by other Vendors,
referred to as 3rd party sub-systems. It is also to confirm the reliability, integrity
and security of the integrated PCS. I-FAT should focus on the integration
design developed for the project, and not on standard system features.

A minimum of one 3rd party system of each type (i.e., CCS, VMS, PLC, etc.)
shall be physically connected and tested with the overall PCS during I-FAT.
The following are recommended integration testing, as a minimum. Detailed
test criteria shall be developed for each section.

4.2.1 Physical and Logical Architecture: The purpose of this section is to

verify that the physical connection of 3rd party systems meets project and
mandatory Saudi Aramco standard requirements for redundancy,
segregation and segmentation. It is also to review the architecture

Page 11 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

against the integration requirements stated in the Integration

Specification Document (ISD) to ensure the integration of applications
and external data sources have been successfully implemented.

4.2.2 Communications Integrity: The purpose of this test section is to validate

the communications, both hardware and communication protocols,
between the PCS and 3rd party systems connected to the PCS. One of
each 3rd party system type (i.e., CCS, VMS, PLC, etc.) should be
physically connected to the PCS and detailed testing conducted to verify
the integrity of the communications link. Testing should include
verification of data exchange at the appropriate scan rate, verification of
fault handling (i.e., failure and reconnection of the communications
link), verification of alarming within the PCS on fault detection and
verification of communications loading, if applicable. Testing should
include failure / reconnection and fail-over (redundant links) for the PCS
interface device, the sub-system communications device, and any
modems or protocol converters used to provide communications between
the two systems.

4.2.3 Data Exchange: The purpose of this test section is to verify proper
configuration of the data exchange between the PCS and 3rd party
sub-systems. This testing typically involves sending data from the
3rd party system to the PCS and verification on the PCS that the proper
values, ranges and alarms are presented to the operator. It is also used to
verify the commands sent from the PCS to the sub-system are properly
received and processed within the sub-system control logic. It is not
necessary to use the actual 3rd party system to verify all data exchange
configuration. Once an appropriate sub-set of both Input Data and PCS
commands has been verified using the actual sub-system, software
simulation (i.e., Modbus or OPC simulators) can be used for the
remainder of the testing. Simulator usage in place of the 3rd party system
must be approved by a 3rd party representative who should attend the
IFAT to fix his side of any problems identified during the test.

4.2.4 Security: The purpose of this section is to test the integration of 3rd party
systems into the overall PCS security design infrastructure. This testing
shall include integration of 3rd party username and passwords into the
PCS domain controllers, integration of Anti-virus, backup-recovery and
ePO services into the PCS design and other common plant-wide security
services. Integration testing shall also ensure that any workstation or
server delivered as part of the overall PCS meets the minimum security
requirements defined in SAEP-99, Process Automation Networks and
System Security.

Page 12 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

4.2.5 Time Synchronization: The purpose of this section is to verify that all
systems connected to the PCS are automatically synchronized to the
same time clock. The master time station is typically an NTP server with
a GPS antenna which is connected to the Plant Automation Network.

4.2.6 Sub-systems supplied by the same vendor: For sub-systems provided by

the same vendor (i.e., DCS / ESD, etc.). Testing of the integration of the
sub-systems should be included in the FAT for the specific sub-systems.

4.2.7 Untestable Interfaces: For sub-systems where it is not practical to

physically bring the equipment to the I-FAT location, the project team
shall apply reasonable controls to ensure that the proposed integration
design is ‘proven’ at other customer locations. The I-FAT procedure
should clearly define which interfaces are not tested. Recommended
procedures for verifying the interface during SAT shall be incorporated
into the I-FAT plan or a reference to the applicable SAT procedures
included.

4.3 Site Acceptance Testing Procedures

The purpose of the Site Acceptance Test (SAT) is to verify the integrity of the
systems after shipment to site, installation and power-up. It is not to retest items
which have already been tested during FAT. SAT shall consist of the following
as a minimum:

4.3.1 System Inspection: A representative sample of equipment (Hardware,

cabinets, modules, etc.) will be inspected to ensure the integrity of the
system after installation and that no damage has occurred during
shipment. System diagnostic programs shall be used to determine the
health of each individual module after power-up. Visual inspection of
each modules is not required. Note that the intention of this testing is not
complete system verification as was conducted during FAT.

4.3.2 Installation Checks: Physical verification of the following for ALL

equipment:
 Power wiring with redundancy testing
 Grounding
 System cable installation

4.3.3 Hardware I/O checks: Spot check of I/O to verify the integrity of system
cables from I/O cards or FTA’s to marshalling. Signals to be injected
from the marshalling panel and verified through to the operator interface.
Note that 100% loop testing from the field devices through to the system is
typically performed during pre-commissioning / commissioning activities.

Page 13 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

4.3.4 3rd Party sub-system interfaces: The intent is not to test all tag / data
transfer between the PCS and 3rd party sub-systems. The purpose is to
verify that communications to each system is working by verification of
a sample of tags from each system. Any ‘Untestable’ interfaces
identified during I-FAT shall be thoroughly tested during SAT.

4.3.5 Network / Communications Integrity: Verify the operation of redundant

communications links by failing one half of a redundant link and
confirming the system is still functioning properly. Testing should focus
on major communications paths; such as communications from PIBs to
CCR and communications between edge switches and root switches.
Redundant links between controllers to local switches can be spot
checked; 100% redundancy testing is not required.

Network Management System (NMS) configuration and functionality

should be verified. Confirm that all network devices connected to the
NMS are displaying accurate information and that device status alarms
are received via SNMP or other means in the NMS.

4.3.6 Security

The intent of SAT for System Access and Security is not to duplicate the
complete FAT testing. The intent is to verify that the security profile of
the system has not changed since the completion of FAT. Compliance to
SAEP-99 should have been verified during FAT and IFAT. For this
reason, spot checks of security requirements should be sufficient, unless
it is found that something has changed. This would necessitate complete
testing. Complete security compliance testing is only required for any /
all workstation, servers or network devices which were not tested during
FAT and for similar 3rd party equipment which was not available or
tested during IFAT.
 Verify that user access and account management is working through
the Domain Controller
 Verify that any test accounts used during FAT have been removed or
disabled.
 Verify system hardening has not changed since FAT. Spot checks of
various workstations / servers to ensure the hardened configuration is
still active.
 Verify the operation of the Anti-virus (AV) distribution server is
working by updating AV dat files and pushing these to all stations
connected to the PCS.

Page 14 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

 Verify switch and network device hardening by spot checking switch

configuration files to ensure that only used ports are active and that
the proper access lists are still in effect.
 Verify that audit logging is enabled on all workstations, servers and
process automation network devices. If complete testing was
conducted during FAT/IFAT, then spot checks of various equipment
to ensure the system is functioning as testing during FAT/IFAT is
sufficient.
 Verify that the system is able to meet the requirements for USB and
Removable media devices as defined in SAEP-98, Removable Media
Usage for Process Automation Systems. If complete testing of the
Removable Media Usage Scheme was conducted during FAT;
then, spot checks to verify that the system has not been altered are
acceptable. If not, then complete testing shall be conducted.
 Verify that any temporary or guest user accounts added to facilitate
testing have been removed from the system.

4.3.6 FOUNDATION Fieldbus

 Refer to SAES-J-904, Section 13 for requirements for SAT and
commissioning of the FOUNDATION Fieldbus portion of control
systems.

4.3.7 Punch List Items: Retesting of any punch list or exception items from
FAT / I-FAT which have not been closed shall be conducted.

4.4 Documentation

During each of the test phases (FAT, I-FAT, SAT), the following documentation
shall be developed and completed, as a minimum, in order to document the test
results. Results shall be documented sufficiently such that re-tests can be
verified and testing which has been deferred to a later testing phase is clear.
The test documentation shall include the following, as a minimum:

4.4.1 Test Plan

A detailed testing plan shall be developed by the contractor and

approved by Saudi Aramco. It is recommended that a single plan be
developed which encompasses all three test phases (FAT, I-FAT and
SAT). The test plan shall include the following as a minimum:
 Testing schedule
 Personnel requirements from vendor, contractor (if applicable) and

Page 15 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

company.
 Details of any special testing tools or simulation software which will
be used during the testing.
 A written description of each of the major testing areas with high
level procedures to be followed for each.
Note: Detailed testing procedures with customer sign-off sheets should
be included in the Test Report, not the testing plan.

 Procedures to be followed to record and resolve any exception items

or deficiencies found during the testing.

The testing plan shall be submitted to Saudi Aramco no less than sixty
(60) days prior to the start of testing.

4.4.2 Test Report

A separate test report shall be developed and completed for each testing
phase (FAT, I-FAT, SAT). The purpose of the test report is to document
the specific tests performed with the expected results and the actual
observed results. Specific items include:
 Detailed procedures for each test case. Note that each test case shall
be numbered in order to facilitate referencing of individual tests.
 Expected results for each of test case.
 A record of the observed results during the testing
 An area to record any deficiencies observed during the test case.
Note that if a deficiency is found, it should be recorded in the
Exception log below and the exception number listed in the
individual test report.
 A signature block for vendor, contractor (if applicable) and company
to confirm that the testing was completed and the results documented
properly.
 Any supporting documentation associated with the test, such as
screen captures, report, etc., shall be referenced in the test case and
attached to the report as an appendix.

4.4.3 Exception Items / Deficiencies Log

An exception item / deficiency log shall be maintained for each phase of

testing. The log shall contain a summary sheet which lists all

Page 16 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

deficiencies with item#, title and status of each item. For each item, the
following information shall be listed:
 A detailed description of the problem found.
Note: Screen captures or other relevant information should be included
when necessary in order to clearly describe the problem.

 A description of the action taken by vendor or contractor to resolve

the problem.
 A signature block for vendor, contractor (if applicable) and company
to verify completion or resolution of the exception item.

Note that it is recommended that a meeting be held at the end of each

day of testing to review the exception item / deficiency log and develop a
plan for resolution.

4.4.4 Test Completion Certificate

A test completion certificate shall be created for each phase of testing.

The certificate shall contain a signature block for vendor, contractor (if
applicable) and company. Testing shall not be considered completed until
all parties have signed the certificate. The certificate shall also contain a
section which lists all outstanding exception items or items not tested,
which will be deferred to later testing phases. Copies of outstanding
exception items which will be corrected and retested during a subsequent
testing phase shall be attached to the Test Completion Certificate.

5 Responsibilities

5.1 Saudi Aramco Project Management Team (SAPMT)

SAMPT has overall responsibility for coordinating and conducting FAT, I-FAT
and SAT. PMT signature is required on the test completion certificate.

5.2 Process & Control Systems Department (P&CSD)

P&CSD has responsibility for maintaining this procedure and for consultations
during any of the testing phases. For projects considered ‘high risk’ or which
are deploying new technologies, P&CSD shall be invited to attend FAT, I-FAT
or SAT testing by PMT.

5.3 Operating Organization / Proponent

Proponent has responsibility to participate in FAT, I-FAT and SAT.

Proponent signature is required on the test completion certificate.

Page 17 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

Revision Summary
14 November 2016 New Saudi Aramco Engineering Procedure that replaces existing test plan preparation
documents. It is intended to consolidate testing requirements for Process Automation
Systems (PAS). It will replace three existing procedures which cover FAT, IFAT, and SAT
individually (SAEP-1634, SAEP-1630, and SAEP-1638).

Page 18 of 19
Document Responsibility: Process Control Standards Committee SAEP-750
Issue Date: 14 November 2016
Next Planned Update: 14 November 2019 Testing Procedures for Process Automation Systems (PAS)

Appendix A - Recommended Testing Procedures

for TMS Systems for Use in Bulk Product Distribution Plants

1. Operation Forms covering the following functionalities:

 Security Levels
 Shipment with Truck & Card Information View
 Security In-gate – Truck /Trailer / Driver Registration
 Truck Validation & Driver Verification
 Shipment Generation
 Card Allocation and re-allocation
 Bay Allocation and re-allocation
 Bill of Lading (BOL) Generation
 Security Out-gate Registration
2. Driver and Truck Identification System
 Card Validation at Entry and Exit Gate
3. Truck Loading Sequence covering
 Bay Control Unit (BCU) Remote Loading Operation (with / without interrupts)
 BCU Local Loading Operation
4. Engineering Forms or Configuration Entry Forms for
 Authorized Login
 Gantry, Truck, Card, Product, Arm, Bay & BCU
 Owner & Terminal
 User Configuration
5. Reports for
 Loaded quantity Report – By Product / By Bay / By BCU
 Utilization Report – Bay wise / Gantry wise
 Truck Turn Around Time Audit Report
 BCU Auto/Manual/Maintenance Log Report
 BCU Local Filling Report (By BCU)
 BCU Totalizer Report
 Product Reconciliation Report – Daily, Cumulative-Monthly, Monthly Variance
6. System Redundancy covering
 BCU Communication Port Redundancy
 TMS Server redundancy
 Tank Application Server (TAS) application and server redundancy
 Network Redundancy

Page 19 of 19