Deploying Oracle Data Guard

GDBA Engineering
 Deploying Oracle Data Guard

DOCUMENTATION
Module Owner : HPIT GDBA
Prepared by : Bethany Lapaglia
Document Version : D.02.00.0
Preparation Date : July 24, 2009
Last Updated : October 5, 2017

REVIEW AND APPROVAL

Reviewed by Name Department
Author Bethany Lapaglia HP-IT GDBA Engineering
Technical Editor(s) Kevin Riibe HP-IT GDBA Engineering
M. Bala Murugan HP-IT GDBA DOC

VERSION HISTORY
Version Date Revised by Description of Changes
D.01.00.00 7/24/2009 BL Initial Draft
D.01.00.01 10/2/07 BL Added delay
D.01.00.02 11/30/07 BL Modified the db_unique_name to remove the cluster/server
name for simplicity and to remove a required db bounce in
setup
12/12/07 BL Added note on no broker support for active/passive clusters,
note on SAP support of Flashback, and removed the
maximum availability
12/17/07 BL Added note under EM section regarding Data Guard Broker
requirement. Also added the Backup Setup section.
D.01.00.02 2/6/08 BL Fixed query used to find all tablespaces of type temporary
5/12/08 BL Removed note that flashback was not supported by SAP. It
is now supported.
6/12/08 BL Modified the recommended sizes for flashback setup
8/18/08 BL Added steps for a manual failover
7/21/09 BL Modified to apply to both 10g and 11g, added static
registration for listener.ora and new DGMGRL alias to
tnsnames.ora, added create audit directory on the standby
and setting ADR_BASE for 11g listener. Added new section
for Oracle Active Data Guard.
D.01.00.02 10/23/2009 KR/MM Revised Format
Added section for Fast Start Failover and Observer.
D.01.00.03 4/23/2010 WC Added section for Apply Lag.
D.01.00.04 09/30/2010 CL Added OEM Configurations
D.01.01.00 10/08/2010 CL Reorganized
10/11/2011 BL Removed sample tnsnames.ora and listener.ora files in
appendix and referenced the standards wiki page
12/13/2011 BL Added better examples and reordered some steps

TABLE OF CONTENTS
Documentation...................................................................................................................................... 2
Review and Approval ........................................................................................................................... 2

HP IT Confidential Page 2
 Deploying Oracle Data Guard
Version History ..................................................................................................................................... 2
Introduction 4
Article I. Installation Instructions .................................................................................................. 4
Section 1.01 System and Database Requirements ........................................................................... 4
Section 1.02 High Level Steps ........................................................................................................... 5
Article II. Create Standby ................................................................................................................ 5
Section 2.01 Configure the Primary Database for Data Guard .......................................................... 5
Section 2.02 Setup the SQL*Net Connectivity ................................................................................... 7
Section 2.03 Creating the Standby..................................................................................................... 9
Section 2.04 Modify the log_archive_dest parameters on the primary and standby ....................... 13
Section 2.05 Verify the Status of the Standby Database ................................................................. 13
Section 2.06 Enable Flashback Database ....................................................................................... 14
Section 2.07 Turn on Maximum Availability in Database ................................................................. 15
Section 2.08 Required Changes for EM ........................................................................................... 15
Section 2.09 Backup Setup .............................................................................................................. 16
Article III. Configuring the Data Guard Broker ............................................................................ 16
Section 3.01 Configure/Reconfigure the Database Service Name in CRS...................................... 16
Section 3.02 Start the Broker Process ............................................................................................. 16
Section 3.03 Configuration of Data Guard Broker............................................................................ 17
Section 3.04 Enable the Data Guard Broker .................................................................................... 18
Section 3.05 Modify the log_archive_dest parameters on the primary and standby ....................... 19
Article IV. Active Data Guard ......................................................................................................... 20
Section 4.01 Converting Data Guard to Active Data Guard ............................................................. 20
Article V. Enable Data Guard Fast Start Failover ....................................................................... 20
Section 5.01 Create the Standby online Redo log files for FSFO .................................................... 20
Section 5.02 Enable Fast Start Failover ........................................................................................... 21
Section 5.03 Starting the Observer .................................................................................................. 22
Section 5.04 Apply Lag .................................................................................................................... 24
Article VI. Switchover and Failover ............................................................................................... 25
Article VII. Client Configurations .................................................................................................... 25
Section 7.01 Setup Cluster Services For Client Access .................................................................. 25
(a) Add cluster wide read+write service: ...................................................................................... 26
(b) Add cluster wide read only service.......................................................................................... 26
(c) Adding a single Node read+write service: ............................................................................... 26
(d) Adding a single Node read only service .................................................................................. 26

HP IT Confidential Page 3
 Deploying Oracle Data Guard
(e) Services relocation and role change after installation:............................................................ 27
(f) EXAMPLE Client TNS Entry ....................................................................................................... 27
Article VIII. Appendix ........................................................................................................................ 28
Section 8.01 Standard Service Levels ............................................................................................. 28
Section 8.02 References .................................................................................................................. 28
Article IX. Troubleshooting Guide ................................................................................................. 28

INTRODUCTION
The purpose of the migration guide is to provide step-by-step instructions for setting up Oracle Data
Guard standby server. This includes installation and configuration of Oracle Data Guard Broker, Active
Data Guard and Data Guard Fast Start Failover (FSFO) configuration guides.

There are now TWO types of Data Guard standby databases GDBA will deploy:

CONTENT RECOVERY standby database is our current standard deployment. This has the a general
delay in the application of the archive redo log files to allow for recovery from logical corruption and
physical data file corruption. When databases transition to Z2Z, the CONTENTRECOVERY standby will
be moved to an alternate zone and remain in delay.

FAST START FAILOVER standby database for Business Continuity (Site-to-Site). Deployment of Oracle
on Linux requires we use Oracle Data Guard with Fast Start Failover (FSFO), a real time database
failover option. This configuration allows application teams to connect to the Active Data Guard solution
for read-only purposes such as reporting. This is certified for use in 11.2 and higher and will be the S2S
solution for Linux and HPUX on 11.2.

Please note any topics and words in RED. These need to be reviewed and updated dependent on the
specific use case.

Article I. INSTALLATION INSTRUCTIONS

Section 1.01 SYSTEM AND DATABASE REQUIREMENTS
Important system and database requirement notes:

1. Standard Platform build that is certified by GDBA to support Oracle database. This includes RAC
Cluster, Service Guard, and standalone server configurations.
2. The ORACLE_HOME for the standby database is installed and patched with the same one-offs
and/or bundles as the primary.
3. The storage is ready and configured on the standby server and equals the primary database server.
4. Data Guard Broker works only with Oracle RDBMS version 10.2.0.3 or higher.
5. Data Guard Broker does NOT support Service Guard active/passive cluster.
6. The number and the size of the online redo logs should be identical on the Primary and Standby
databases. This is critically important for using Fast Start Failover.
7. Changes as of FY10
o Introduction of Linux and Automatic Storage Management will change the file management
naming and definitions from traditional conventions.

HP IT Confidential Page 4
 Deploying Oracle Data Guard

Section 1.02 HIGH LEVEL STEPS

Setup and configuration of Data Guard environment:

1. Configure the Primary Database for Data Guard

2. Setup the SQL*Net Connectivity
3. Create the Standby and Make Sure it is Working
4. Configure Flashback Database
5. Configure and Enable the Broker
6. Configure the Observer
7. Perform the Switchover

For physical standby database servers which are used for CONTENT RECOVERY, standby databases
will be configured with a delay to try to protect against Oracle logical corruption. This does not delay the
transfer of the archive logs to the standby server but delays the application of the log files to the standby
database. This is configured by adding the DELAY option to the log_archive_dest_2 parameter.

The standard value for this delay is 24 hours.

IMPORTANT NOTE: If the primary database generates more logs than can be recovered on the standby
database to meet the Time to Fix requirement documented in the Appendix, the delay minutes must be
reduced and properly communicated to the application team. The decision for this delay value must be
derived by calculating the number of archive logs that can be applied to the standby database in 60% of
the TTF for that database. For example, if the database can recover at most 4 hours of logs on the
standby database and still meet the TTF for that database, then the value for DELAY would be 240.

For physical standby databases used for FAST START FAILOVER (FSFO) for maximum availbilty, the
databases will be configured with no delay to try to protect against systemic failover on the primary
environment. Like normal standby database to protect from logical corruption, this does not delay the
transfer of the archive logs to the standby server but log files will be applied immediately to the standby
database.

Article II. CREATE STANDBY

Section 2.01 CONFIGURE THE PRIMARY DATABASE FOR DATA GUARD
The work in this section should be done on the existing primary database to prepare it for Data Guard.

1. Create a password file for each instance of the primary database if one does not exist following
the steps below.

orapwd file=orapw<DB_UNIQUE_NAME>.ora password=<syspassword> entries=500

Example:
[UNIX]$ cd $ORACLE_ADMIN/DGTEST/pfile
[UNIX]$ orapwd file=orapwDGTEST password=syspassword
[UNIX]$ ln s $ORACLE_ADMIN/DGTEST/pfile/orapwDGTEST $ORACLE_HOME/dbs/orapwDGTEST

where DGTEST is the name of the database and syspassword is the sys password

2. Enable archiving and set the remote_login_passwordfile parameter if not already set. This is
done with the steps below.

HP IT Confidential Page 5
 Deploying Oracle Data Guard
SQL> shutdown immediate;
SQL> startup mount;
SQL> alter database archivelog;
SQL> alter system set remote_login_passwordfile=exclusive scope=spfile;
SQL> alter database open;

3. Enable force logging with this command:

SQL> alter database force logging;

4. Configure the following primary database initialization parameters.

a. The db_unique_name is required and follows the naming standard. This means that if
the database name is DGTEST, the db_unique_name for the primary database is
DGTEST. The standby database will be called DGTESTS and the db_unique_name will
be DGTESTS.
b. The db_unique_name for the primary and standby are used in the log_archive_config
parameter.
c. The service= in the log_archive_dest_2 must be a valid net service name for the standby
database and the db_unique_name used in this paramter is the db_unique_name for the
standby database.
d. Include the parameters that will be used when this server becomes the standby database
after a failover/switchover.
e. The fal_server entries contains the service_name for the current standby server.
f. The fal_client is the service_name of this current primary server

HP-UX Version

######## Used when functioning in the Primary Role ########

*.db_unique_name=DGTEST
*.service_names=DGTEST
*.log_archive_config=dg_config=("DGTEST", DGTESTS")
*.log_archive_dest_2=service=DGTESTS delay=1440 valid_for=(online_logfiles,
primary_role) db_unique_name=DGTESTS
*.log_archive_dest_state_2=defer
*.archive_lag_target=900 *** DO NOT SET FOR VERSIONS 11.2 or higher
######## Used when functioning in the Standby Role ########
*.db_file_name_convert=/u05/oradata/DGTESTS,/u05/oradata/DGTEST/
*.log_file_name_convert=/u02/origlog/DGTESTS, /u02/origlog/DGTEST,
/u03/mirrlog/DGTESTS, /u03/mirrlog/DGTEST
*.standby_file_management=AUTO
*.fal_server=DGTESTS
*.fal_client=DGTEST

Linux (ASM) Version

######## Used when functioning in the Primary Role ########

*.db_unique_name='LNXHADR'
*.service_names=LNXHADR
*.log_archive_config='dg_config=("LNXHADR","LNXHADRS")'
*.log_archive_dest_2='service="lnxhadr" delay=1440 valid_for=(online_logfiles,
primary_role) db_unique_name=LNXHADRS
*.log_archive_dest_state_2=defer
*.archive_lag_target=900 *** DO NOT SET FOR VERSIONS 11.2 or higher
######## Used when functioning in the Standby Role ########
*.db_file_name_convert='+GCT20210_DATA_DG,+GCT20207_DATA_DG','+GCT20210_DATA_DG/LNX
HADRS','+GCT20207_DATA_DG/LNXHADR
*.log_file_name_convert='+GCT20210_REDO01_DG','+GCT20207_REDO01_DG','+GCT20210_REDO02
_DG','+GCT20207_REDO02_DG','+GCT20210_REDO01_DG/LNXHADRS','+GCT20207_REDO01_DG/LNXHAD
R','+GCT20210_REDO02_DG/LNXHADRS','+GCT20207_REDO02_DG/LNXHADR'
*.standby_file_management=AUTO
*.fal_server='LNXHADRS'
*.fal_client='LNXHADR'

5. Create the standby redo log files on the primary server. These must exist so that they can be
used in the event of a failover/switchover. When creating the standby log files, you need to
create the exact number of redo logs that are on all instances of the primary database plus one.

HP IT Confidential Page 6
 Deploying Oracle Data Guard
The formula for the number of standby logs is as follows: (# of redo log groups on the primary +
1) * # of threads in the primary database. For example, if the primary database has 3 log groups
and 2 threads, you will create 8 standby logs ((3 + 1) * 2). The standby logs must be created the
same size as the primary database and on both the primary and standby servers. Below is an
example of the statements to create the standby logs. Start with the next available group in the
database.

Standard file system version

SQL> alter database add standby logfile thread 1

group 7 ('/u02/origlog/DGTEST/DGTESTlog_t1_g7_m1.dbf',
'/u03/mirrlog/DGTEST/DGTESTlog_t1_g7_m2.dbf') size 500M,
group 8 ('/u02/origlog/DGTEST/DGTESTlog_t1_g8_m1.dbf',
'/u03/mirrlog/DGTEST/DGTESTlog_t1_g8_m2.dbf') size 500M,
group 9 ('/u02/origlog/DGTEST/DGTESTlog_t1_g9_m1.dbf',
'/u03/mirrlog/DGTEST/DGTESTlog_t1_g9_m2.dbf') size 500M,
group 10 ('/u02/origlog/DGTEST/DGTESTlog_t1_g10_m1.dbf',
'/u03/mirrlog/DGTEST/DGTESTlog_t1_g10_m2.dbf') size 500M;
SQL> alter database add standby logfile thread 2
group 11 ('/u02/origlog/DGTEST/DGTESTlog_t2_g11_m1.dbf',
'/u03/mirrlog/DGTEST/DGTESTlog_t2_g11_m2.dbf') size 500M,
group 12 ('/u02/origlog/DGTEST/DGTESTlog_t2_g12_m1.dbf',
'/u03/mirrlog/DGTEST/DGTESTlog_t2_g12_m2.dbf') size 500M,
group 13 ('/u02/origlog/DGTEST/DGTESTlog_t2_g13_m1.dbf',
'/u03/mirrlog/DGTEST/DGTESTlog_t2_g13_m2.dbf') size 500M,
group 14 ('/u02/origlog/DGTEST/DGTESTlog_t2_g14_m1.dbf',
'/u03/mirrlog/DGTEST/DGTESTlog_t2_g14_m2.dbf') size 500M;

Oracle ASM Version

SQL> alter database add standby logfile thread 1

group 7 ('+GCT20207_REDO01_DG', '+GCT20207_REDO02_DG') SIZE 128M REUSE,
group 8 ('+GCT20207_REDO01_DG', '+GCT20207_REDO02_DG) SIZE 128M REUSE,
group 9 ('+GCT20207_REDO01_DG, '+GCT20207_REDO02_DG') SIZE 128M REUSE,
group 10 ('+GCT20207_REDO01_DG', '+GCT20207_REDO02_DG') SIZE 128M REUSE;

SQL> alter database add standby logfile thread 2

group 11 ('+GCT20207_REDO01_DG', '+GCT20207_REDO02_DG) SIZE 128M REUSE,
group 12 ('+GCT20207_REDO01_DG, '+GCT20207_REDO02_DG) SIZE 128M REUSE,
group 13 ('+GCT20207_REDO01_DG', '+GCT20207_REDO02_DG') SIZE 128M REUSE,
group 14 ('+GCT20207_REDO01_DG', '+GCT20207_REDO02_DG') SIZE 128M REUSE;

Section 2.02 SETUP THE SQL*NET CONNECTIVITY

All of the required service names must be configured in the TNSNAMES.ORA file on both the primary and
the standby RAC cluster before the new init.ora parameters can go into effect. Each service name must
exist in the TNSNAMES.ORA file which can cause some confusion with all the different entries. Below is
a sample list of the different type of names in a Data Guard environment.

TNSNAMES Entry Type RAC Database Server Standby Database Server

Database Name DGTEST DGTESTS
Db_unique_name DGTEST DGTESTS
Instance Name DGTEST1, DGTEST2 DGTESTS1,DGTESTS2
LOCAL_LISTENERS LOCAL_LISTENER1, LOCAL_LISTENER1,
LOCAL_LISTENER2 LOCAL_LISTENER2
REMOTE_LISTENERS SCAN_LISTENER SCAN_LISTENER
Dataguard Broker Name DGTEST_DGMGRL DGTEST_DGMGRL

The local_listener init.ora parameter must be set for database dynamic registration. The database will not
be able to register with the listener if it is not running on the default port. Therefore, the local_listener
init.ora parameter along with a TNSNAMES.ORA entry will tell the pmon process where to find the
listener when the database starts up.

HP IT Confidential Page 7
 Deploying Oracle Data Guard
The remote_listener init.ora parameter must be set in a RAC environment to allow the listeners to cross
register with each other. This parameter only allows one entry even though there may be multiple
listeners in the cluster. The way to get around this is to set the parameter to point to a TNSNAMES entry
that lists all nodes in the cluster that run a listener. For 11.2 clusters, rather than listing all nodes in the
cluster, you must list the three IP scan addresses for that cluster.

To enable DGMGRL to restart instances during the course of broker operations, a service with a specific
name must be statically registered with the local listener of each instance. A static service registration is
also required to enable the observer to restart instances as part of automatic reinstatement of the old
primary database after Fast-Start Failover has occurred.

Edit the listener.ora on both the primary and standby and add the static information for the database into
the SID_LIST_LISTENER section. The value for the GLOBAL_DBNAME attribute must be set to a
concatenation of db_unique_name_DGMGRL.db_domain.

For 11g databases, edit the listener.ora on both the primary and the standby and make sure the following
line exists: (where listenername is the name of the listener) ADR_BASE_listenername =
/u01/app/oracle/product/11.1.0/db_1/network/log

Below is an example of a listener.ora file with both the static registration and the ADR_BASE parameter
for 11g:

LISTENER = (DESCRIPTION =
(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=host_name)
(PORT=port_num))))
SID_LIST_LISTENER_D3U0105C =
(SID_LIST =
(SID_DESC =
(SID_NAME = DGTEST)
(ORACLE_HOME = /u01/app/oracle/product/11.1.0/db_1)
(GLOBAL_DBNAME=DGTEST_DGMGRL)
)
)
ADR_BASE_LISTENER_D3U0105C = $ORACLE_HOME/network/log

Where the port_num is the standard Oracle SQL*Net port and host_name is the host server name

MAKE SURE THE LISTENER IS RUNNING ON BOTH THE PRIMARY AND STANDBY SERVERS.

Refer to the following page on the internal wiki for further details on the standards for the network
configuration files and paramters.

FOR 11g DATABASES ONLY:

1. On the standby server (first node), create a duplicate listener by adding the following entry in the
listener.ora file under the $GRID_HOME

LISTENER_DUP =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)
(HOST = g9t0320c.houston.hp.com)
(PORT = 1521)(IP = FIRST))
)
)
SID_LIST_LISTENER_DUP =
(SID_LIST =
(SID_DESC =
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1)
(SID_NAME = STRMBZ)

HP IT Confidential Page 8
 Deploying Oracle Data Guard
)
)

2. Start the listener using the command lsnrctl start listener_dup

3. On the primary server (first node), add a tns entry for connecting to the standby database and make
sure to include the UR=A below:

STRMBZ =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = gct90320.houston.hp.com)(PORT = 1521))
(CONNECT_DATA =
(SERVICE_NAME = STRMBZ)(UR=A)
)
)
4. On the Standby database, add a tns entry for connecting to the primary database

Section 2.03 CREATING THE STANDBY

The standby database must be created on the standby server. This involves several important steps to
be completed before starting the recovery for the standby database.

1. For ASM standby databases, make sure the diskgroups exist and are mounted. This can be
verified using the ASM command line utility (asmcmd).
2. Create the standby initialization parameter file for each instance of the standby database.
3. On the primary database, create an initialization file from the spfile and transfer to the standby
server(s).

Create pfile from $ORACLE_ADMIN/<DB_UNIQUE_NAME>/spfile<DB_UNIQUE_NAME>.ora;

4. Transfer this init.ora file to $ORACLE_HOME/dbs directory on the first node of the standby
cluster.
5. Edit the initialization parameter file to include the required instance specific parameters and then
create an spfile from this pfile for the standby database

######## Used when functioning in the Primary Role ########

*.db_unique_name=DGTESTS
*.service_names=DGTESTS
*.log_archive_config=dg_config=("DGTEST", DGTESTS")
*.log_archive_dest_2=service=DGTEST db_unique_name="DGTEST" delay=1440
valid_for=(online_logfiles, primary_role)
*.Log_archive_dest_state_2=defer
######## Used when functioning in the Standby Role ########
*.db_file_name_convert=/u05/oradata/DGTEST, /u05/oradata/DGTESTS
*.log_file_name_convert=/u02/origlog/DGTEST, /u02/origlog/DGTESTS,
/u03/mirrlog/DGTEST, /u03/mirrlog/DGTESTS
*.standby_file_management=AUTO
*.fal_server=DGTEST
*.fal_client=DGTESTS

(Using Oracle ASM the parameters db_file_name_convert and log_file_name_convert

would look like)
*.db_file_name_convert='+GCT20207_DATA_DG,+GCT20210_DATA_DG','+GCT20207_DATA_DG/LNXHADR
','+GCT20210_DATA_DG/LNXHADRS
*.log_file_name_convert='+GCT20207_REDO01_DG','+GCT20210_REDO01_DG','+GCT20207_REDO02_DG'
,'+GCT20210_REDO02_DG','+GCT20207_REDO01_DG/LNXHADR','+GCT20210_REDO01_DG/LNXHADRS','+GCT
20207_REDO02_DG/LNXHADR','+GCT20210_REDO02_DG/LNXHADRS'

6. Create the password file on each standby host. The sys password must match the sys password
on the primary database. Issue the orapwd command to create the password file or just copy the
file from the primary and rename. Create a soft link to this orapw file from the
$ORACLE_HOME/dbs directory.

HP IT Confidential Page 9
 Deploying Oracle Data Guard
7. Create the same directory as is used on the primary to hold the audit files.
(/u01/app/oracle/admin/<DB_NAME>/adump)
8. Startup nomount the database using this new parameter file.
9. Using RMAN, issue a backup (either to disk or tape depending on disk space availability)
a. If backing up to disk:
i. Connect to the primary database: rman target /
ii. Issue the following command to backup the database and archive logs:

RMAN> backup device type disk format '/u01/app/oracle/reorg/backup/%U'

database plus archivelog;

[DBA Advisory] Adding a script to perform this function + compress (Pending)

NOTE: make sure you specify the correct location for the backup files followed
by the %U

iii. Issue the following command to backup the controlfile:

RMAN> backup device type disk format '/u01/app/oracle/reorg/backup/%U'

current controlfile for standby;

NOTE: make sure you specify the correct location for the backup files followed
by the %U

b. If backing up to tape: (NOTE: This is a sample script only and needs to change
depending on your environment).

run {
allocate channel dev_0 type sbt
'ENV=(OB2BARTYPE=Oracle08,OB2APPNAME=<DBNAME>,OB2BARLIST='Medium_Fullbackup<DBNAME
>)';
allocate channel dev_1 type sbt
'ENV=(OB2BARTYPE=Oracle08,OB2APPNAME=<DBNAME>,OB2BARLIST='Medium_Fullbackup<DBNAME
>)';
allocate channel dev_2 type sbt
'ENV=(OB2BARTYPE=Oracle08,OB2APPNAME=<DBNAME>,OB2BARLIST='Medium_Fullbackup<DBNAME
>)';
allocate channel dev_3 type sbt
'ENV=(OB2BARTYPE=Oracle08,OB2APPNAME=<DBNAME>,OB2BARLIST='Medium_Fullbackup<DBNAME
>)';
execute global script global_mediumfull_level0;
backup
format T1_DB_full_small_<application_%s:%t:%p.dbf>'
current controlfile;
}
c. If creating a standby database for 11g, you can use RMAN to create the standby. A
sample of the command is below. (NOTE: This is a sample script only and needs to
change depending on your environment).

run {
allocate channel prmy1 type disk;
allocate channel prmy2 type disk;
allocate channel prmy3 type disk;
allocate channel prmy4 type disk;
allocate channel prmy5 type disk;
allocate channel prmy6 type disk;
allocate auxiliary channel stby1 type disk;
allocate auxiliary channel stby2 type disk;
duplicate target database for standby from active database nofilenamecheck
spfile
parameter_value_convert 'strmb','strmbz'
set db_unique_name 'STRMBZ'
set db_file_name_convert
'+GCT90318_DATA_DG','+GCT90320_DATA_DG','+GCT90318_RED01_DG','+GCT90320_RED01_DG',
'+GCT90318_RED02_DG','+GCT90320_RED02_DG','+GCT90318_DATA_DG/STRMB','+GCT90320_DAT
A_DG/STRMBZ','+GCT90318_RED01_DG/STRMB','+GCT90320_RED01_DG/STRMBZ','+GCT90318_RED
02_DG/STRMB','+GCT90320_RED02_DG/STRMBZ'

HP IT Confidential Page 10
 Deploying Oracle Data Guard
set log_file_name_convert
'+GCT90318_DATA_DG','+GCT90320_DATA_DG','+GCT90318_RED01_DG','+GCT90320_RED01_DG',
'+GCT90318_RED02_DG','+GCT90320_RED02_DG','+GCT90318_DATA_DG/STRMB','+GCT90320_DAT
A_DG/STRMBZ','+GCT90318_RED01_DG/STRMB','+GCT90320_RED01_DG/STRMBZ','+GCT90318_RED
02_DG/STRMB','+GCT90320_RED02_DG/STRMBZ'
set control_files '+GCT90320_DATA_DG/STRMBZ/strmbz_1.ctl'
set log_archive_max_processes '5'
set fal_client 'STRMBZ'
set fal_server 'STRMB'
set audit_file_dest '/u01/app/oracle/admin/STRMBZ/adump'
set remote_listener 'SCAN_LISTENER'
set standby_file_management 'AUTO'
set cluster_database 'false'
set log_archive_config='dg_config=(STRMB,STRMBZ)'
set log_archive_dest_2='service=STRMB LGWR SYNC OPTIONAL
max_failure=10 max_connections=1 reopen=15 register net_timeout=30
valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=STRMB';
}

10. If the backup was done to storage on the primary server, copy over the backup to the same
location on the standby database server and restore the backup to the standby by issuing the
following commands when connected to the standby database (see the appropriate commands
based on a disk or tape backup):
a. If backup was to disk:

rman target sys/manager1@DGTEST auxiliary / catalog rman/dc_rman@HOUR2D

(where DGTEST is primary db and HOUR2D is the RMAN catalog)

RMAN> duplicate target database for standby;

NOTE: If restoring to the exact filesystem layout on the standby server, RMAN gets
confused since the db_file_name_convert parameter is not used and you can get a
RMAN-05501 error. To get past this error, add nofilenamecheck to the command as in
this example:

duplicate target database for standby nofilenamecheck;

b. If backup was to tape: (NOTE: This is a sample script only!!)

run {
allocate auxiliary channel aux_C0 device type disk;
allocate auxiliary channel aux_C1 device type disk;
allocate auxiliary channel aux_C2 device type disk;
allocate auxiliary channel aux_C3 device type disk;
allocate auxiliary channel aux_C4 device type disk;
allocate auxiliary channel aux_C5 device type disk;
allocate auxiliary channel aux_C6 device type disk;
allocate auxiliary channel aux_C7 device type disk;
allocate auxiliary channel aux_C8 device type disk;
allocate auxiliary channel aux_C9 device type disk;
allocate auxiliary channel aux_C10 device type disk;
allocate auxiliary channel aux_C11 device type disk;
allocate auxiliary channel aux_C12 device type disk;
allocate auxiliary channel aux_C13 device type disk;
allocate auxiliary channel aux_C14 device type disk;
allocate channel dev0 device type sbt 'DP parameters';
allocate channel dev1 device type sbt 'DP parameters';
allocate channel dev2 device type sbt 'DP parameters';
allocate channel dev3 device type sbt 'DP parameters';
allocate channel dev4 device type sbt 'DP parameters';
allocate channel dev5 device type sbt 'DP parameters';
allocate channel dev6 device type sbt 'DP parameters';
duplicate target database for standby;
}
c. If standby was setup using RMAN on 11g, then the standby database should have been
restored and started in a mounted state. The spfile was copied from the primary and will
have parameters matching the primary database so it must be changed to match the
standby. On the standby, create a pfile from the spfile and change all mention of the

HP IT Confidential Page 11
 Deploying Oracle Data Guard
primary database to the standby database name and make sure cluster_database is set
to TRUE. Use this pfile to create a new spfile and restart (mount) the standby database.

11. If the database is not running ASM, make sure that all datafiles for the temporary tablespace
have been created for this standby database. These are not needed when running in standby
mode but will be required when this database becomes the primary.
12. Create the standby log files as was done in step 5 of section Configure the Primary Database for
Data Guard above. They need to match the number and size of the primary files. See example
below:

SQL> alter database add standby logfile thread 1

group 7 ('/u02/origlog/DGTESTS/DGTESTSlog_t1_g7_m1.dbf',
'/u03/mirrlog/DGTESTS/DGTESTSlog_t1_g7_m2.dbf') size 500M,
group 8 ('/u02/origlog/DGTESTS/DGTESTSlog_t1_g8_m1.dbf',
'/u03/mirrlog/DGTESTS/DGTESTSlog_t1_g8_m2.dbf') size 500M,
group 9 ('/u02/origlog/DGTESTS/DGTESTSlog_t1_g9_m1.dbf',
'/u03/mirrlog/DGTESTS/DGTESTSlog_t1_g9_m2.dbf') size 500M,
group 10 ('/u02/origlog/DGTESTS/DGTESTlSog_t1_g10_m1.dbf',
'/u03/mirrlog/DGTESTS/DGTESTSlog_t1_g10_m2.dbf') size 500M;

SQL> alter database add standby logfile thread 2

group 11 ('/u02/origlog/DGTESTS/DGTESTSlog_t2_g11_m1.dbf',
'/u03/mirrlog/DGTESTS/DGTESTSlog_t2_g11_m2.dbf') size 500M,
group 12 ('/u02/origlog/DGTESTS/DGTESTSlog_t2_g12_m1.dbf',
'/u03/mirrlog/DGTESTS/DGTESTSlog_t2_g12_m2.dbf') size 500M,
group 13 ('/u02/origlog/DGTESTS/DGTESTSlog_t2_g13_m1.dbf',
'/u03/mirrlog/DGTESTS/DGTESTSlog_t2_g13_m2.dbf') size 500M,
group 14 ('/u02/origlog/DGTESTS/DGTESTSlog_t2_g14_m1.dbf',
'/u03/mirrlog/DGTESTS/DGTESTSlog_t2_g14_m2.dbf') size 500M;

(Using Oracle ASM)

SQL> alter database add standby logfile thread 1
group 7 ('+GCT20207_REDO01_DG', '+GCT20207_REDO02_DG/LNXHADR) SIZE 128M REUSE,
group 8 ('+GCT20207_REDO01_DG, '+GCT20207_REDO02_DG') SIZE 128M REUSE,
group 9 ('+GCT20207_REDO01_DG, '+GCT20207_REDO02_DG') SIZE 128M REUSE,
group 10 ('+GCT20207_REDO01_DG, '+GCT20207_REDO02_DG') SIZE 128M REUSE;
SQL> alter database add standby logfile thread 2
group 11 ('+GCT20207_REDO01_DG', '+GCT20207_REDO02_DG') SIZE 128M REUSE,
group 12 ('+GCT20207_REDO01_DG', '+GCT20207_REDO02_DG') SIZE 128M REUSE,
group 13 ('+GCT20207_REDO01_DG', '+GCT20207_REDO02_DG) SIZE 128M REUSE,
group 14 ('+GCT20207_REDO01_DG', '+GCT20207_REDO02_DG) SIZE 128M REUSE;

13. Verify that the LOCAL_LISTENER parameters and entries in the tnsnames.ora file are correct for
both the primary and standby databases.
14. For the 11g standby databases, make sure the duplicate listener has been stopped using the
command: lsnrctl stop listener_dup. On the standby reload the default listener lsnrctl reload.
15. Modify the primary nsnames.ora entry for the standby database and remove the UR=A entry
and make sure the port is set for the correct port of the listener on the standby server (1525 or
1526)
16. FOR RAC ONLY: Define the RAC standby database using srvctl by issuing the following
commands:

[UNIX]$ Srvctl add database d DGTESTS o $ORACLE_HOME r PHYSICAL_STANDBY s mount;

[UNIX]$ Srvctl add instance d DGTESTS i DGTESTS1 n d3u0105c;
[UNIX]$ Srvctl add instance d DGTESTS i DGTESTS2 n d3u0106c;

17. Make sure the spfile and orapw files are setup correctly on each node of the standby with the link
under the $ORACLE_HOME/dbs directory pointing to the correct standard location under the
admin/pfile location.
18. Shutdown all instances of the standby database and restart using the srvctl start database
command.

HP IT Confidential Page 12
 Deploying Oracle Data Guard
19. On the primary database, enable the deferred remote archive destination:

SQL> Alter system set log_archive_dest_state_2=enable scope=both;

20. Place the standby in managed recovery:

SQL> Recover managed standby database disconnect;

Section 2.04 MODIFY THE LOG_ARCHIVE_DEST PARAMETERS ON THE

PRIMARY AND STANDBY
NOTE: the delay=360 should be set to delay=0 if the standby will be used as an active data guard
standby.

Examples:

PRIMARY (original)

SQL> alter system set log_archive_dest_2='service=DGTESTS ARCH SYNC NOAFFIRM delay=0

OPTIONAL max_failure=10 max_connections=1 reopen=15 delay=360 db_unique_name="DGTESTS"
register net_timeout=30 valid_for=(online_logfile,primary_role)';

PRIMARY (new)

SQL> alter system set log_archive_dest_2='service=DGTESTS LGWR SYNC NOAFFIRM OPTIONAL

max_failure=10 max_connections=1 reopen=15 delay=1440 db_unique_name="DGTESTS" register
net_timeout=30 valid_for=(online_LOGFILES,primary_ROLE)' scope=both;

STANDBY (original)

SQL> alter system set log_archive_dest_2='service=DGTEST delay=1440

valid_for=(online_logfiles, primary_role) db_unique_name=DGTEST';

STANDBY (new)

SQL> alter system set log_archive_dest_2='service=DGTEST LGWR SYNC NOAFFIRM OPTIONAL

max_failure=10 max_connections=1 reopen=15 delay=1440 db_unique_name="DGTEST" register
net_timeout=30 valid_for= (online_LOGFILES,primary_ROLE)' scope=both;

In Oracle 10g only: In the appropriate pfiles, set the log_archive_dest_1 and the
standby_archive_dest the same directory on both the primary and standby. Example from the
primary:

(Standard UNIX file system)

log_archive_dest_1 location="/u01/app/oracle/arch/DGTEST/",
valid_for=(ALL_LOGFILES,ALL_ROLES)
standby_archive_dest="/u01/app/oracle/arch/DGTEST/"

Section 2.05 VERIFY THE STATUS OF THE STANDBY DATABASE

1. On the primary database, verify that the log files are being created on the standby database
server. To do this, issue the statement below to force a log switch

SQL> ALTER SYSTEM SWITCH LOGFILE;

2. Next, execute this query on the primary database. If an error message is returned such as
heartbeat failed, check the tns entry for the standby database on the primary cluster.

set lines 200 pages 200

select name,open_mode,protection_mode, protection_level, database_role,
switchover_status
from v$database;

HP IT Confidential Page 13
 Deploying Oracle Data Guard
select to_char(timestamp,'YYYY-MON-DD HH24:MI:SS')||' '||message
from v$dataguard_status;
select thread#, max(sequence#) "Last Primary Seq Generated"
from v$archived_log val, v$database vdb
where val.resetlogs_change# = vdb.resetlogs_change#
group by thread# order by 1;
3. On the standby database, run the following.

set lines 200 pages 200

select name, open_mode,protection_mode, protection_level, database_role,
switchover_status
from v$database;

select process, status, thread#, sequence#, block#, blocks from

v$managed_standby;

select thread#, max(sequence#) "Last Standby Seq Received"

from v$archived_log val, v$database vdb
where val.resetlogs_change# = vdb.resetlogs_change#
group by thread# order by 1;
select thread#, max(sequence#) "Last Standby Seq Applied"
from v$archived_log val, v$database vdb
where val.resetlogs_change# = vdb.resetlogs_change#
and val.applied='YES'
group by thread# order by 1;

4. If there are any errors in applying the log files, execute the following query on the primary
database to get the error message:

set lines 200 pages 200

col archive_dest for a40
SELECT DEST_ID "DEST_ID", -
STATUS "DB_status", -
DESTINATION "Archive_dest", -
ERROR "Error" -
FROM V$ARCHIVE_DEST WHERE DEST_ID <=10;

Section 2.06 ENABLE FLASHBACK DATABASE

If it is not already enabled on the primary and standby databases, Flashback Database must be enabled.
Flashback should be configured in the following locations based on whether or not the database has a
standby.

Does the Database have a Enable on the Primary Enable on the Standby
Standby?
Yes Yes if using FSFO
No X

Use the details in the table below for sizing recommendation

Tier DB_RECOVERY_FILE_DEST_SIZE DB_FLASHBACK_RETENTION_TARGET

(in minutes)
ALL 100% of archive logs size for 1 days 1440

Issue the following statements on each database.

SQL> ALTER SYSTEM SET UNDO_RETENTION=86400 SCOPE=SPFILE;

SQL> ALTER SYSTEM SET UNDO_MANAGEMENT='AUTO' SCOPE=SPFILE;
SQL> SHUTDOWN IMMEDIATE;
SQL> STARTUP MOUNT;
SQL> SHOW PARAMETER UNDO;
SQL> ALTER SYSTEM SET DB_FLASHBACK_RETENTION_TARGET=1440 SCOPE=BOTH;

HP IT Confidential Page 14
 Deploying Oracle Data Guard
SQL> ALTER SYSTEM SET DB_RECOVERY_FILE_DEST_SIZE=4G SCOPE=BOTH;
SQL> ALTER DATABASE ARCHIVELOG;
SQL> ALTER DATABASE FLASHBACK ON;
SQL> ALTER DATABASE OPEN;

To verify that flashback database has been turned on, issue this command on both the primary and
standby servers:

SQL> select flashback_on from v$database;

FLASHBACK_ON
------------------
YES

Section 2.07 TURN ON MAXIMUM AVAILABILITY IN DATABASE

Next, you have to tell the database to function in maximum availability mode. To to this, follow these
commands:

SQL> shutdown immediate;

SQL> startup mount;
SQL> alter database set standby database to maximize availability;
SQL> alter database open;

Note: If it is a RAC database, shutdown all the instances and open one of the instance in MOUNT state
and issue the above "alter database" command.

After the first instance is OPEN, then start the other instances.

To verify the availability of the database, issue the following command on both the primary and standby
servers:

SQL> select protection_mode, protection_level from v$database;

Section 2.08 REQUIRED CHANGES FOR EM

EM requires a connection with SYSDBA privileges in the database. Issue the following statement in the
primary database:

SQL> grant sysdba to dbsnmp;

SQL> Copy the primary orapwd file to the standby;

Verify the connection by logging in as DBSNMP on the standby database.

When adding the standby database in EM, make sure the database is configured to use DBSNMP with
SYSDBA privileges versus the normal privileges.

Once the primary and standby databases have been properly discovered in EM, verify the configuration.

To do this, select the primary database in EM.

On this screen, click on the Maintenance tab.
Click on the Setup and Manage option under the High Availability/Data Guard section.
You will be prompted to connect. Supply the DBSNMP id and password and connect as
SYSDBA.
The following steps are valid for those using the Data Guard Broker only.
o On the Data Guard screen, select the option at the bottom under Additional
Administration called Verify Configuration.
o This will go through verification checks for the Data Guard configuration.
o Correct any issues found and rerun this check if required.

HP IT Confidential Page 15
 Deploying Oracle Data Guard
Contact your OEM lead to push the latest database monitoring template to the primary and
standby databases. Once complete, confirm that the following metrics exist: transport lag, apply
lag.

Guide to install the EM Agent is located @

http://prime.sharepoint.hp.com/teams/GDBAProcessDocumentation/Internal/Wiki%20Pages/How
%20do%20I%20install,%20upgrade%20or%20patch%20the%20EM%20Agent.aspx.

Guide to setting up a new database target can also be found on the Wiki @
http://prime.sharepoint.hp.com/teams/GDBAProcessDocumentation/Internal/Wiki%20Pages/How
%20do%20I%20configure%20and%20name%20my%20EM%20targets.aspx.

Guide to preparing a target for MTP can be found on the Wiki @

http://prime.sharepoint.hp.com/teams/GDBAProcessDocumentation/Internal/Wiki%20Pages/How
%20do%20I%20enable%20montoring%20on%20a%20new%20target%20prior%20to%20MTP.as
px.

Section 2.09 BACKUP SETUP

Archive log backups are required on the standby server. For details in setting up this backup,
refer to Appendix: References section for Backup Archivelog Thresholds document. These are not
Data Protector scheduled backups so you do not have to work with the ITIO backup team for these
backups.

Article III. CONFIGURING THE DATA GUARD BROKER

Section 3.01 CONFIGURE/RECONFIGURE THE DATABASE SERVICE NAME IN
CRS
For RAC, the database needs to be registered in CRS using the db_unique_name and must have the
start option set to MOUNT. To do this, use the appropriate srvctl statement as follows:

If the database is registered but is not using the db_unique_name, you must remove the entry and re-
add.

[UNIX]$ srvctl remove database d <DB_NAME>

Use srvctl to add the database using the db_unique_name and each instance as follows:

[UNIX]$ srvctl add database -d <db_unique_name> -o <$oracle_home> -s mount

[UNIX]$ srvctl add instance d <db_unique_name> -I <instance_name> -n <node_name>

If the database is current registered using the db_unique_name, then modify the start options with the
statement below:

[UNIX]$ srvctl modify database -d <db_unique_name> -o <$oracle_home> -s mount

Section 3.02 START THE BROKER PROCESS

Set the instance specific paramters in spfile for all database instances (primary and standby). Make sure
you create any required subdirectories.

Primary:

DGTEST1.log_archive_format=DGTESTarch %t_%s_%r.dbf

HP IT Confidential Page 16
 Deploying Oracle Data Guard
DGTEST2.log_archive_format=DGTESTarch %t_%s_%r.dbf
DGTEST1.log_archive_trace=0
DGTEST2.log_archive_trace=0
DG_BROKER_CONFIG_FILE1 = /u01/app/oracle/admin/DGTEST/broker/DR1.DAT
DG_BROKER_CONFIG_FILE2 = /u01/app/oracle/admin/DGTEST/broker/DR2.DAT

Standby:

DGTESTS1.log_archive_format=DGTESTSarch%t_%s_%r.dbf
DGTESTS2.log_archive_format=DGTESTSarch%t_%s_%r.dbf
DGTESTS1.log_archive_trace=0
DGTESTS2.log_archive_trace=0
DG_BROKER_CONFIG_FILE1 = /u01/app/oracle/admin/DGTESTS/broker/DR1.DAT
DG_BROKER_CONFIG_FILE2 = /u01/app/oracle/admin/DGTESTS/broker/DR2.DAT

(Using Oracle ASM)

Primary:

LNXHADR1.log_archive_format=+LNXHADR_RECO_DG/LNXHADR/LNXHADRarch %t_%s_%r.dbf
LNXHADR2.log_archive_format=+LNXHADR_RECO_DG/LNXHADR/LNXHADRarch %t_%s_%r.dbf
LNXHADR1.log_archive_trace=0
LNXHADR2.log_archive_trace=0
DG_BROKER_CONFIG_FILE1 = +GCT20207_DATA_DG/LNXHADR/broker/DR1.DAT
DG_BROKER_CONFIG_FILE2 = +GCT20207_DATA_DG/LNXHADR/broker/DR2.DAT

Standby:

LNXHADRS1.log_archive_format=+LNXHADRS_RECO_DG/LNXHADRS/LNXHADRSarch%t_%s_%r.dbf
LNXHADRS2.log_archive_format=+LNXHADRS_RECO_DG?LNXHADRS/LNXHADRSarch%t_%s_%r.dbf
LNXHADRS1.log_archive_trace=0
LNXHADRS2.log_archive_trace=0
DG_BROKER_CONFIG_FILE1 = +GCT20210_DATA_DG/LNXHADRS/broker/DR1.DAT
DG_BROKER_CONFIG_FILE2 = +GCT20210_DATA_DG/LNXHADRS/broker/DR2.DAT

Need the following details to configuration and enable the Data Guard Broker:

Primary Standby

db_unique_name DGTEST DGTESTS

connect alias DGTEST DGTESTS

Turn on the broker on both the primary and standby database servers.

SQL> Alter system set dg_broker_start=true scope=both;

This will create a new process called dmon.

Section 3.03 CONFIGURATION OF DATA GUARD BROKER

Create the broker configuration and add the primary database (must be done on primary database). The
example below is screen response.

[UNIX]$ dgmgrl
DGMGRL for HPUX: Version 10.2.0.2.0 - 64bit Production
Copyright (c) 2000, 2005, Oracle. All rights reserved.

Welcome to DGMGRL, type "help" for information.

HP IT Confidential Page 17
 Deploying Oracle Data Guard
DGMGRL> connect sys
Password:
Connected.
DGMGRL> create configuration 'DGTEST' as primary database is 'DGTEST' connect identifier is
DGTEST;
Configuration "DGTEST" created with primary database "DGTEST"

Comments:

Database name used (example here is "DGTEST") must be the db_unique_name

Service Name (example here is "DGTEST") must be a valid service_name/alias in the
tnsnames.ora file for this database and must be in tnsnames on all servers in config.

DGMGRL> show configuration

Configuration
Name: DGTEST
Enabled: NO
Protection Mode: MaxAvailability
Fast-Start Failover: DISABLED
Databases:
DGTEST - Primary database
Current status for "DGTEST":
DISABLED

Add the standby database to the configuration (must be done on primary database)

DGMGRL> add database 'DGTESTS' as connect identifier is DGTESTS maintained as physical;

Comments:
Database name used (example here is "DGTESTS") must be the db_unique_name
Service Name (example here is "DGTESTS") must be a valid service_name/alias in the
tnsnames.ora file for this database and must be in tnsnames on all servers in configuration.

Database "DGTESTS" added

DGMGRL> show configuration

Configuration
Name: DGTEST
Enabled: NO
Protection Mode: MaxPerformance
Fast-Start Failover: DISABLED
Databases:
DGTEST - Primary database
DGTESTS - Physical standby database
Current status for "DGTEST":
DISABLED

Section 3.04 ENABLE THE DATA GUARD BROKER

Issue the command below to enable the broker:

DGMGRL> enable configuration;

Enabled.

This will create the configuration files for each database under the structure /u01/app/oracle/admin/. It
will also create a DataGuard Broker log file located in /u01/app/oracle/admin/DGTEST/bdump directory.
The name of the log file is drcDGTEST.log

You will now see the configuration enabled.

DGMGRL> show configuration;

Configuration
Name: DGTEST
Enabled: YES
Protection Mode: MaxPerformance
Fast-Start Failover: DISABLED

HP IT Confidential Page 18
 Deploying Oracle Data Guard
Databases:
DGTEST - Primary database
DGTESTS - Physical standby database

Current status for "DGTEST":

SUCCESS

DGMGRL> show database verbose 'DGTEST';

DGMGRL> show database verbose DGTESTS;

Make sure the broker has LogXptMode = 'SYNC'. If not, issue the command below to switch the log
transport mode to SYNC for both the primary and standby databases:

DGMGRL> edit database 'DGTEST' set property LogXptMode='SYNC';

Property "logxptmode" updated
DGMGRL> edit database ' DGTESTS' set property LogXptMode='SYNC';
Property "logxptmode" updated

Section 3.05 MODIFY THE LOG_ARCHIVE_DEST PARAMETERS ON THE

PRIMARY AND STANDBY
See the example below (Changes in RED):

PRIMARY [Original]

SQL> alter system set log_archive_dest_2='service=DB01TDG ARCH SYNC NOAFFIRM delay=0 OPTIONAL
max_failure=0 max_connections=1 reopen=300 db_unique_name="DB01T_S" register net_timeout=180
valid_for= (online_logfile,primary_role)';

PRIMARY [New]

Dgmgrl> edit database 'db_name set property 'ApplyInstanceTimeout'='0';

Dgmgrl> edit database 'db_name set property 'ApplyParallel'='AUTO';
Dgmgrl> edit database db_name set property AlternateLocation=USE_DB_RECOVERY_FILE_DEST;
Dgmgrl> edit database db_name set property ArchiveLagTarget=900;
Dgmgrl> edit database db_name set property DelayMins=0 (on ADG standby db)
Dgmgrl> edit database db_name set property DelayMins=1440 (on DR standby db)
Dgmgrl> edit database db_name set property LogXptMode=SYNC
Dgmgrl> edit database db_name set property NetTimeout=30;
Dgmgrl> edit database db_name set property Binding=OPTIONAL;
Dgmgrl> edit database db_name set property MaxFailure=10;
Dgmgrl> edit database db_name set property ReopenSecs=15;
Dgmgrl> edit database 'db_name set property 'LogArchiveMaxProcesses'='4';
Dgmgrl> edit database 'db_name set property 'LogArchiveMinSucceedDest'='1';
Dgmgrl> edit database 'db_name set property 'RedoCompression'='ENABLE';
Dgmgrl> edit database 'db_name set property 'StandbyFileManagement'='AUTO';

STANDBY [Original]

SQL> alter system set log_archive_dest_2='service=DB01T valid_for=(online_logfiles,

primary_role) db_unique_name=DB01T_P';

STANDBY [New]

Dgmgrl> edit database db_name set property 'ApplyInstanceTimeout'='0';

Dgmgrl> edit database db_name set property 'ApplyParallel'='AUTO';
Dgmgrl> edit database db_name set property AlternateLocation=USE_DB_RECOVERY_FILE_DEST;
Dgmgrl> edit database db_name set property ArchiveLagTarget=900;
Dgmgrl> edit database db_name set property DelayMins=0 (on ADG standby db)
Dgmgrl> edit database db_name set property DelayMins=1440 (on DR standby db)
Dgmgrl> edit database db_name set property LogXptMode=SYNC
Dgmgrl> edit database db_name set property NetTimeout=30;
Dgmgrl> edit database db_name set property Binding=OPTIONAL;
Dgmgrl> edit database db_name set property MaxFailure=10;
Dgmgrl> edit database db_name set property ReopenSecs=15;
Dgmgrl> edit database db_name set property 'LogArchiveMaxProcesses'='4';
Dgmgrl> edit database db_name set property 'LogArchiveMinSucceedDest'='1';
Dgmgrl> edit database db_name set property 'RedoCompression'='ENABLE';
Dgmgrl> edit database db_name set property 'StandbyFileManagement'='AUTO';

HP IT Confidential Page 19
 Deploying Oracle Data Guard
LOG_ARCHIVE_DEST_2 is currently ignored in the standby because this destination is valid only for the
primary role. If a switchover occurs and this instance becomes the primary database, then it will transmit
the redo data to the remote DB01T destination. If you put ALL_ROLES, it will try to transmit the redo data
to the primary when it is in the standby role and ORA-16009: invalid redo transport destination will be
raised in the standbys alert log.

Set the log_archive_dest_1 and the standby_archive_dest same directory on both the primary and
standby. Example:

SQL> log_archive_dest_1 location="/u01/app/oracle/arch/DB01T", MANDATORY

valid_for=(ALL_LOGFILES,ALL_ROLES) standby_archive_dest = "/u01/app/oracle/arch/DB01T";

Article IV. ACTIVE DATA GUARD

Section 4.01 CONVERTING DATA GUARD TO ACTIVE DATA GUARD

HP does not have licensing at this time to deploy active dataguard. Please do not configure any standby
databases to be read-only without management approval.

1. Cancel redo apply on the standby

ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;

2. Open the database in READ ONLY mode (the DB will be in MOUNT state at this time)

ALTER DATABASE OPEN READ ONLY;

3. Continue REDO apply

ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE

DISCONNECT;

Note: If it is RAC, issue alter database open read only command on each instance

Article V. ENABLE DATA GUARD FAST START FAILOVER

Section 5.01 CREATE THE STANDBY ONLINE REDO LOG FILES FOR FSFO
When creating the standby log files for FSFO, you need to create the exact number of redo logs that are
on all instances of the primary database plus one. The formula for the number of standby logs is as
follows: (# of redo log groups on the primary + 1) * # of threads in the primary database. For example, if
the primary database has 5 log groups and 2 threads, you will create 12 standby logs ((5 + 1) * 2).

As a reminder, the standby online redo logs must be created the same size as the primary database and
on both the primary and standby servers. NOTE: this should have been done when the standby was
created.

HP IT Confidential Page 20
 Deploying Oracle Data Guard

Section 5.02 ENABLE FAST START FAILOVER

You can enable fast-start failover while connected to any database system in the broker configuration,
including on the observer computer. For example:

DGMGRL> ENABLE FAST_START FAILOVER;

Enabled.

Sample output in the DataGuard Broker log file on the primary server:

DG 2007-08-28-14:58:08 0 2 631808424 DMON: Fast-Start Failover (FSFO) has been enabled

between:
DG 2007-08-28-14:58:08 0 2 631808424 Primary = "DB01T_P"
DG 2007-08-28-14:58:08 0 2 631808424 Standby = "DB01T_S"
DG 2007-08-28-14:58:08 0 2 631808424 DMON: EDIT_DRC: success

Sample output from the DataGuard Broker log file on the standby servers:

DG 2007-08-28-14:58:08 0 2 0 drcx: FSFO storing state flags=0x101, version=1, obid=0x0

(0), threshold=1800
DG 2007-08-28-14:58:08 0 2 0 drcx: FSFO storing state flags=0x101, version=1,
obid=0x743 (1859), threshold=1800
DG 2007-08-28-14:58:08 0 2 0 drcx: FSFO storing state flags=0x103, version=2,
obid=0x743 (1859), threshold=1800
DG 2007-08-28-14:58:08 0 2 0 DRCX: Start receiving metadata file:
/u01/app/oracle/product/10.2.0/db_1/dbs/dr2DB01T_S.dat
DG 2007-08-28-14:58:08 0 2 0 DRCX: Receiving block #1, 54 blocks.
DG 2007-08-28-14:58:08 0 2 0 DRCX: End receiving metadata file: opcode EDIT_DRC
DG 2007-08-28-14:58:08 0 2 631808424 DMON: chief lock convert for resync
DG 2007-08-28-14:58:08 0 2 631808424 DMON: status from posting standby instances for
RESYNCH = ORA-00000
DG 2007-08-28-14:58:08 0 2 0 INSV: Received message for inter-instance publication
DG 2007-08-28-14:58:08 0 2 0 req_id 1.1.631808424, opcode EDIT_DRC, phase
RESYNCH, flags 8005
DG 2007-08-28-14:58:08 0 2 631808424 DMON: entered rfm_metadata_avail
DG 2007-08-28-14:58:08 0 2 0 Executing SQL [ALTER SYSTEM REGISTER]
DG 2007-08-28-14:58:08 0 2 0 SQL [ALTER SYSTEM REGISTER] Executed successfully
DG 2007-08-28-14:58:08 0 2 0 DMON: Checking critical status of this database.
DG 2007-08-28-14:58:08 0 2 0 INSV: All instances have replied for message
DG 2007-08-28-14:58:08 0 2 0 req_id 1.1.631808424, opcode EDIT_DRC, phase
RESYNCH
DG 2007-08-28-14:58:08 0 2 631808424 DMON: Entered rfm_release_chief_lock for EDIT_DRC
DG 2007-08-28-14:58:12 0 2 0 RSM0: HEALTH CHECK WARNING: ORA-16819: Fast-Start
Failover observer not started

Edit the following Fast-Start Failover properties. To do this, issue this command:

DGMGRL> edit configuration set property ByStandersFollowRoleChange=ALL;

DGMGRL> edit configuration set property CommunicationTimeout=180;
DGMGRL> edit configuration set property FastStartFailoverAutoReinstate=TRUE;
DGMGRL> edit configuration set property FastStartFailoverPmyShutdown=TRUE;
DGMGRL> edit configuration set property FastStartFailoverLagLimit=10;
DGMGRL> edit configuration set property FastStartFailoverThreshold=1800;
DGMGRL> edit configuration set property OperationTimeout=60;

A show configuration command will now show that the Fast-Start Failover has been enabled. It will also
report an Oracle error ORA-16608 working message since the observer has not been started yet.

DGMGRL> show configuration verbose;

Configuration
Name: DB01T_P
Enabled: YES
Protection Mode: MaxAvailability
Fast-Start Failover: ENABLED
Databases:
DB01T_P - Primary database
DB01T_S - Physical standby database
- Fast-Start Failover target
Fast-Start Failover
Threshold: 1800 seconds
Observer: (none)

HP IT Confidential Page 21
 Deploying Oracle Data Guard

Current status for "DB01T_P":

Warning: ORA-16608: one or more databases have warnings

Section 5.03 STARTING THE OBSERVER

To start the observer, only the primary database must be running. The observer should be started on a
server outside of the the data centers that host the primary and standby servers. The DataGuard Broker
command line utility must be installed. This is done either via the SQL*Net client Administration install or
the full Oracle binary installation. Entries for Primary and Standby must be made in the tnsnames.ora file
that match the connect identifiers provided to the Data Guard Broker.

In the DataGuard Broker command line utility on the observer server, connect to the configuration on the
primary server. Review these instructions on using EM to setup a Primary and Alternate observer.

If running multiple observers on the same host, you must specify a different logfile and configuration file
for each observer.

nohup /u01/app/oracle/product/10.2.0/db_1/bin/dgmgrl -logfile

/u01/app/oracle/product/10.2.0/db_1/log/observer_DGTEST username/password@database_name
"start observer FILE='/u01/app/oracle/product/10.2.0/db_1/log/DGTEST.dat'" &

The Observer is a continuous foreground process so the command line will not return. If you need to stop
the observer, you will need to log into this server and issue the stop observer command in the
DataGuard command line utility from another session.

The observer creates a fsfo.dat file in the home directory from where the Observer is invoked. When
using the logfile option, it will also create a log file.

[DBA Advisory] Starting the Observer

Oracle recommends that connection credentials be supplied as a command parameter to the DGMGRL
CONNECT command, as shown in the example, rather than as a command line parameter to the
DGMGRL command. However, Oracle does not display the accout/password outright. For now, use this
method.

We will be looking to implement something like Oracle Wallet (Oracle Database Advanced Security
Administrator's Guide) at a later date.

The status of the observer can be checked from within both the primary and standby databases using this
command (you should see the same output from both databases:

SQL> select fs_failover_observer_present,fs_failover_observer_host,fs_failover_threshold from

v$database;

FS_FAIL FS_FAILOVER_OBSERVER_HOST FS_FAILOVER_THRESHOLD

------- ------------------------------------------------- --------------------------
YES d3u0014c 1800

Now, the show configuration should show that the observer name and the show database verbose
commands should return SUCCESS for both primary and standby.

DGMGRL> show configuration verbose;

Configuration
Name: DB01T_P
Enabled: YES
Protection Mode: MaxAvailability
Fast-Start Failover: ENABLED
Databases:
DB01T_P - Primary database

HP IT Confidential Page 22
 Deploying Oracle Data Guard
DB01T_S - Physical standby database
- Fast-Start Failover target
Fast-Start Failover
Threshold: 1800 seconds
Observer: d3u0014c

Current status for "DB01T_P":

SUCCESS

DGMGRL> show database verbose 'DB01T_P';

Database
Name: DB01T_P
Role: PRIMARY
Enabled: YES
Intended State: ONLINE
Instance(s):
DB01T
Properties:
InitialConnectIdentifier = 'db01t'
LogXptMode = 'SYNC'
Dependency = ''
DelayMins = '0'
Binding = 'OPTIONAL'
MaxFailure = '10'
MaxConnections = '1'
ReopenSecs = '15'
NetTimeout = '30'
LogShipping = 'ON'
PreferredApplyInstance = ''
ApplyInstanceTimeout = '0'
ApplyParallel = 'AUTO'
StandbyFileManagement = 'AUTO'
ArchiveLagTarget = '0'
LogArchiveMaxProcesses = '2'
LogArchiveMinSucceedDest = '1'
DbFileNameConvert = ''
LogFileNameConvert = ''
FastStartFailoverTarget = 'DB01T_S'
StatusReport = '(monitor)'
InconsistentProperties = '(monitor)'
InconsistentLogXptProps = '(monitor)'
SendQEntries = '(monitor)'
LogXptStatus = '(monitor)'
RecvQEntries = '(monitor)'
HostName = 'd3u0016'
SidName = 'DB01T'
LocalListenerAddress =
'(ADDRESS=(PROTOCOL=TCP)(Host=d3u0016.houston.hp.com)(Port=1525))'
StandbyArchiveLocation = '/u01/app/oracle/product/10.2.0/db_1/archh/'
AlternateLocation = ''
LogArchiveTrace = '0'
LogArchiveFormat = 'primary_%t_%s_%r.dbf'
LatestLog = '(monitor)'
TopWaitEvents = '(monitor)'

Current status for "DB01T_P":

SUCCESS

DGMGRL> show database verbose 'DB01T_S';

Database
Name: DB01T_S
Role: PHYSICAL STANDBY
Enabled: YES
Intended State: ONLINE
Instance(s):
DB01T
Properties:
InitialConnectIdentifier = 'db01tdg'
LogXptMode = 'SYNC'
Dependency = ''
DelayMins = '0'
Binding = 'OPTIONAL'
MaxFailure = '10'
MaxConnections = '1'
ReopenSecs = '15'
NetTimeout = '30'

HP IT Confidential Page 23
 Deploying Oracle Data Guard
LogShipping = 'ON'
PreferredApplyInstance = ''
ApplyInstanceTimeout = '0'
ApplyParallel = 'AUTO'
StandbyFileManagement = 'AUTO'
ArchiveLagTarget = '0'
LogArchiveMaxProcesses = '2'
LogArchiveMinSucceedDest = '1'
DbFileNameConvert = ''
LogFileNameConvert = ''
FastStartFailoverTarget = 'DB01T_P'
StatusReport = '(monitor)'
InconsistentProperties = '(monitor)'
InconsistentLogXptProps = '(monitor)'
SendQEntries = '(monitor)'
LogXptStatus = '(monitor)'
RecvQEntries = '(monitor)'
HostName = 'd3u0017'
SidName = 'DB01T'
LocalListenerAddress =
'(ADDRESS=(PROTOCOL=TCP)(Host=d3u0017.houston.hp.com)(Port=1525))'
StandbyArchiveLocation = '/u01/app/oracle/product/10.2.0/db_1/archh/'
AlternateLocation = ''
LogArchiveTrace = '0'
LogArchiveFormat = 'standby_%t_%s_%r.dbf'
LatestLog = '(monitor)'
TopWaitEvents = '(monitor)'
Current status for "DB01T_S":
SUCCESS

Section 5.04 APPLY LAG

Apply lag is a measure of the degree to which the data in a standby database lags behind the data in the
primary database, due to delays in propagating and applying redo to the standby database. The apply lag
helps to identify any problems that may exist with both the redo transport services and the log apply
services.

If you are using real-time query to offload queries from a primary database to the physical standby
database, you may want to monitor the apply lag to ensure that it is within acceptable limits.

The current apply lag is the difference, in elapsed time computed in nearest second, between when the
last applied change became visible on the standby and when that same change was first visible on the
primary.

1. To display the apply lag for each managed standby database via dgmgrl

DGMGRL> show database 'TWCTS';

Database - TWCTS
Role: PHYSICAL STANDBY
Intended State: APPLY-ON
Transport Lag: 0 seconds
Apply Lag: 0 seconds
Real Time Query: ON
Instance(s):
TWCTS
Database Status:
SUCCESS

2. To obtain the apply lag via the V$DATAGUARD_STATS view.

SQL> SELECT name, value, datum_time, time_computed FROM V$DATAGUARD_STATS WHERE name like
'apply lag';
NAME VALUE DATUM_TIME TIME_COMPUTED

HP IT Confidential Page 24
 Deploying Oracle Data Guard
------------ ------------------------- ------------------------------ -------------------
--
apply lag +00 00:00:00 04/14/2010 13:22:56 04/14/2010 13:22:57

where DATUM_TIME column contains a timestamp of when the data was last received by the
standby database and TIME_COMPUTED column contains a timestamp taken when the apply
lag metric was calculated. The difference between the values in these columns should be less
than 30 seconds.

3. To obtain a histogram that shows the history of apply lag values since the standby instance was
last started, query the V$STANDBY_EVENT_HISTOGRAM

SQL> SELECT * FROM V$STANDBY_EVENT_HISTOGRAM WHERE NAME = 'apply lag' AND COUNT > 0;
NAME TIME UNIT COUNT LAST_TIME_UPDATED
------------ ---------- ---------------- ---------- --------------------
apply lag 0 seconds 77667 04/14/2010 13:27:24
apply lag 1 seconds 35 04/14/2010 13:00:25
apply lag 2 seconds 4 04/14/2010 00:33:26
apply lag 3 seconds 2 04/14/2010 00:33:27
apply lag 4 seconds 3 04/14/2010 00:33:28
apply lag 5 seconds 1 04/14/2010 00:33:29
apply lag 7 minutes 1 04/13/2010 15:42:50

4. Apply Lag tolerance in a real-time query environment can be set via the
STANDBY_MAX_DATA_DELAY session parameter. This capability allows queries to be safely
offloaded from the primary database to the physical standby database, because it is possible to
detect if the physical standby database has become unacceptably stale.

If STANDBY_MAX_DATA_DELAY is set to the default value of NONE, queries issued to the

physical standby database will be executed regardless of the apply lag on that database.

If STANDBY_MAX_DATA_DELAY is set to a non-zero value, a query issued to the active

standby will be executed only if the apply lag is less than or equal to
STANDBY_MAX_DATA_DELAY.

If STANDBY_MAX_DATA_DELAY is set to 0, a query issued to the physical standby database is

guaranteed to return the exact same result as if the query were issued on the primary database.
Use the ALTER SESSION SQL statement to set STANDBY_MAX_DATA_DELAY.

SQL> ALTER SESSION SET STANDBY_MAX_DATA_DELAY=2\

Article VI. SWITCHOVER AND FAILOVER

For Data Guard switchover and failover details see
http://prime.sharepoint.hp.com/teams/GDBAProcessDocumentation/Internal/Wiki%20Pages/Data%20Gu
ard%20Playbook.aspx

Article VII. CLIENT CONFIGURATIONS

Section 7.01 SETUP CLUSTER SERVICES FOR CLIENT ACCESS
Application connect strings should be set up to use a Service, this service can fail back and forth with the
primary database without manual intervention.

HP IT Confidential Page 25
 Deploying Oracle Data Guard

(a) ADD CLUSTER WIDE READ +WRITE SERVICE :

On PRIMARY cluster:

srvctl add service -d <db_name primary> -s <service_name> -r <list of all db instance> -l

PRIMARY -q TRUE -e SELECT -m BASIC -w 10 -z 150

On STANDBY:

srvctl add service -d <db_name physican standby> -s <service_name> -r <list of all db

instance> -l PRIMARY -q TRUE -e SELECT -m BASIC -w 10 -z 150

(b) ADD CLUSTER WIDE READ ONLY SERVICE

On PRIMARY cluster:

srvctl add service -d <db_name primary> -s <service_name> -r <list of all db instance> -l

PHYSICAL_STANDBY -q TRUE -e SELECT -m BASIC -w 10 -z 150

On STANDBY:

srvctl add service -d <db_name physican standby> -s <service_name> -r <list of all db

instance> -l PHYSICAL_STANDBY -q TRUE -e SELECT -m BASIC -w 10 -z 150

In addition to the above execute following on PRIMARY:

SQLPLUS> EXECUTE DBMS_SERVICE.CREATE_SERVICE('<service_name>', '<service_name>', NULL, NULL,

TRUE, 'BASIC', 'SELECT', 150, 10, NULL);

(c) ADDING A SINGLE NODE READ+WRITE SERVICE :

On PRIMARY cluster:

srvctl add service -d <db_name primary> -s <service_name> -r <instance_name> -a <list of all

remaining instance> -l PRIMARY -q TRUE -e SELECT -m BASIC -w 10 -z 150

On STANDBY

srvctl add service -d <db_name physican standby> -s <service_name> -r <instance_name> -a

<list of all remaining instance> -l PRIMARY -q TRUE -e SELECT -m BASIC -w 10 -z 150

(d) ADDING A SINGLE NODE READ ONLY SERVICE

On PRIMARY cluster:

srvctl add service -d <db_name primary> -s <service_name> -r <instance_name> -a <list of all

remaining instance> -l PHYSICAL_STANDBY -q TRUE -e SELECT -m BASIC -w 10 -z 150

On STANDBY

srvctl add service -d <db_name physican standby> -s <service_name> -r <instance_name> -a

<list of all remaining instance> -l PHYSICAL_STANDBY -q TRUE -e SELECT -m BASIC -w 10 -z 150

In addition to the above execute following on PRIMARY:

HP IT Confidential Page 26
 Deploying Oracle Data Guard
EXECUTE DBMS_SERVICE.CREATE_SERVICE('<service_name>', '<service_name>', NULL, NULL, TRUE,
'BASIC', 'SELECT', 150, 10, NULL);

(e) SERVICES RELOCATION AND ROLE CHANGE AFTER INSTALLATION :

AUTOP is the primary database, and AUTOZ is currently physical standby:

On primary cluster:

[oracle@d3t0012c HP_AWM_service]$ srvctl status service -d AUTOP

Service AUTOC is running on instance AUTOP1,AUTOP2
Service AUTOCR is not running.

On physical standby cluster:

[oracle@d3t0014c HP_AWM_service]$ srvctl status service -d AUTOZ

Service AUTOCR is running on instance AUTOZ1,AUTOZ2
Service AUTOC is not running.

After failover AUTOZ is now the primary database, and AUTOP is now physical standby:

On primary cluster:

[oracle@d3t0012c HP_AWM_service]$ srvctl status service -d AUTOZ

Service AUTOC is running on instance AUTOZ1,AUTOZ2
Service AUTOCR is not running.

On physical standby cluster:

[oracle@d3t0014c HP_AWM_service]$ srvctl status service -d AUTOP

Service AUTOCR is running on instance AUTOP1,AUTOP2
Service AUTOC is not running.

(f) EXAMPLE CLIENT TNS ENTRY

Note that the addresses below are the SCAN addresses these should always be used when connecting.
VIP IP addresses should no longer be used.

AUTOC =
(DESCRIPTION =
(SDU = 32768)
(enable = broken)
(LOAD_BALANCE = yes)
(ADDRESS = (PROTOCOL = TCP)(HOST = gct0001.houston.hp.com)(PORT = 1526))
(ADDRESS = (PROTOCOL = TCP)(HOST = gct0002.houston.hp.com)(PORT = 1526))
(CONNECT_DATA =
(SERVICE_NAME = AUTOC)
)
)
AUTOCR =
(DESCRIPTION =
(SDU = 32768)
(enable = broken)
(LOAD_BALANCE = yes)
(ADDRESS = (PROTOCOL = TCP)(HOST = gct0001.houston.hp.com)(PORT = 1526))
(ADDRESS = (PROTOCOL = TCP)(HOST = gct0002.houston.hp.com)(PORT = 1526))
(CONNECT_DATA =
(SERVICE_NAME = AUTOCR)
)
)

HP IT Confidential Page 27
 Deploying Oracle Data Guard

Article VIII. APPENDIX

Section 8.01 STANDARD SERVICE LEVELS

HPIT Standard Service Levels

- go-forward model (From 8/ 2007 CIO Council)
Date: June 26, 2007
SLA Key Data HPIT-Normal
HPIT-Mission Critical HPIT-Entity Essential
Sheet (and non-Production) *)
Opening hours: 24 * 5 Opening hours: 9 * 5
Opening hours: 24 * 7
(Mon - Fri) (Mon-Fri)
Service call and
incident management Time To Own Time To Fix Time To Own Time To Fix Time To Own Time To Fix Thresholds
Priority

Top 15min 4hr n/a n/a n/a n/a <70% >80%

High 1 hr 8 hr 1 hr 8 hr n/a n/a <70% >80%

24 hr 24 hr 18 hr
Medium 4 hr = 1 calendar 4 hr = 1 business 4 hr = 2 business <70% >80%
day day +) days +)
48 hr 48 hr 36 hr
Low 8 hr = 2 calendar 8 hr = 2 business 8 hr = 4 business <70% >80%
days days +) days +)

Comments
TTO = Time to own TTO - Ticket submitted, Ownership assigned, and caller is notified of Agent working the ticket ( automation enabled )

TTF = Time to fix TTF - Issue resolved and user notified. Ticket status is "resolved" and will be auto-closed after 7 calendar days if caller is not objecting.

*) including CIs with higher criticality, but in "non-production" status (ie. Staging, Dev, Integration, pre-production)
10
/ 2/
+) where business days = Mon-Fri
20
26
07 1-26
2 October 2007

Section 8.02 REFERENCES

Backup Archive Thresholds http://enhanced1.sharepoint.hp.com/teams/GDBA/eng
ineering/Shared%20Documents/Oracle/Backup%20an
d%20Recovery/Archivelog%20threshold.doc

Article IX. TROUBLESHOOTING GUIDE

Description of the Error Problem & Corrective Action

HP IT Confidential Page 28
 Deploying Oracle Data Guard

HP IT Confidential Page 29