D74942GC20 Ag PDF

These eKit materials are to be used ONLY by you for the express purpose SELF STUDY.
SHARING THE FILE IS STRICTLY PROHIBITED.
D80415
April 2013
Edition 2.0
D74942GC20
Activity Guide
Administration
Oracle Solaris Cluster 4.x
Oracle University and (Oracle Corporation) use only.

Authors Copyright © 2013, Oracle and/or it affiliates. All rights reserved.
Raghavendra JS Disclaimer
Zeeshan Nofil This document contains proprietary information and is protected by copyright and
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.
Venu Poddar other intellectual property laws. You may copy and print this document solely for your
own use in an Oracle training course. The document may not be modified or altered
Technical Contributors in any way. Except where your use constitutes "fair use" under copyright law, you
may not use, share, download, upload, copy, print, display, perform, reproduce,
and Reviewers publish, license, post, transmit, or distribute this document in whole or in part without
the express authorization of Oracle.
Thorsten Fruauf
Harish Mallya The information contained in this document is subject to change without notice. If you
Hemachandran find any problems in the document, please report them in writing to: Oracle University,
Namachivayam 500 Oracle Parkway, Redwood Shores, California 94065 USA. This document is not
warranted to be error-free.
Pramod Rao
Tim Read Restricted Rights Notice
Sambit Nayak

Venugopal Navilugon If this documentation is delivered to the United States Government or anyone using
the documentation on behalf of the United States Government, the following notice is
Shreedhar applicable:
Lisa Shepherd
Mahesh Subramanya U.S. GOVERNMENT RIGHTS
Venkateswarlu Tella The U.S. Government’s rights to use, modify, reproduce, release, perform, display, or
disclose these training materials are restricted by the terms of the applicable Oracle
license agreement and/or the applicable U.S. Government contract.
Graphic Designer
Trademark Notice
Rajiv Chandrabhanu
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names
Editors may be trademarks of their respective owners.
Raj Kumar
Malavika Jinka
Daniel Milne
Publishers
Jayanthy Keshavamurthy
Veena Narasimhan
Table of Contents
Practices for Lesson 1: Introduction ..............................................................................................................1-1
Practice for Lesson 1: Overview.....................................................................................................................1-2

Practices for Lesson 2: Planning the Oracle Solaris Cluster Environment ................................................2-1
Practice 2-1: Guided Tour of the Virtual Training Lab ....................................................................................2-3
Practices for Lesson 3: Establishing Cluster Node Console Connectivity .................................................3-1
Practice 3-1: Connecting to the Cluster Node Console ..................................................................................3-3
Practices for Lesson 4: Preparing for the Oracle Solaris Cluster Installation ............................................4-1
Practices for Lesson 4: Overview ...................................................................................................................4-2
Practice 4-1: Preparing for Installation ...........................................................................................................4-3

Practices for Lesson 5: Installing and Configuring the Oracle Solaris Cluster Software ..........................5-1
Practice 5-1: Installing and Configuring the Oracle Solaris Cluster Software .................................................5-3
Practices for Lesson 6: Performing Cluster Administration ........................................................................6-1
Practice 6-1: Performing Basic Cluster Administration ...................................................................................6-3
Practices for Lesson 7: Using ZFS with Oracle Solaris Cluster Software ..................................................7-1
Practice 7-1: Configuring Volume Management by Using ZFS ......................................................................7-3
Practices for Lesson 8: Using Solaris Volume Manager with Oracle Solaris Cluster Software ................8-1
Practice 8-1: Configuring Volume Management by Using Solaris Volume Manager ......................................8-3
Practices for Lesson 9: Managing the Public Network with IPMP ...............................................................9-1
Practice 9-1: Configuring and Testing IPMP ..................................................................................................9-3
Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS .................................10-1
Practices for Lesson 10: Overview .................................................................................................................10-2
Practice 10-1: Installing and Configuring HA for NFS ....................................................................................10-3
Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
...........................................................................................................................................................................11-1
Practice 11-1: Installing and Configuring Apache as a Scalable Service on Oracle Solaris Cluster ..............11-3
Practices for Lesson 12: Using Oracle Solaris Zones in Oracle Solaris Cluster ........................................12-1
Practice 12-1: Running Failover Zones with HA for Zones and solaris10 Branded Zones .............................12-3
Practice 12-2: Building a Zone Cluster ...........................................................................................................12-10
Practice 12-3: Configuring a Scalable Application in a Zone Cluster .............................................................12-14
Practice 12-4: Integrating Oracle 11g into Solaris Cluster Software as a Failover Application (Optional) ......12-18
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Oracle Solaris Cluster 4.x Administration Table of Contents

iii

Practices for Lesson 1: Introduction

Chapter 1 - Page 1
Chapter 1
Introduction

Practices for Lesson 1:

Practice for Lesson 1: Overview
Practice Overview
There is no practice for Lesson 1.
Practices for Lesson 1: Introduction

Chapter 1 - Page 2

Planning the Oracle Solaris
Cluster Environment
Chapter 2
Practices for Lesson 2: Planning the Oracle Solaris Cluster Environment

Chapter 2 - Page 1
Practice Overview
In this practice, you identify the various hardware and software components that are made
available inside the virtual training environment for you to install, configure, and administer the
Oracle Solaris Cluster software.

Chapter 2 - Page 2
Practice 2-1: Guided Tour of the Virtual Training Lab
Overview
In this practice, you participate in a guided tour of the virtual training lab. This tour helps you
review the essentials of the Oracle Solaris Cluster hardware components.
Tasks
You participate in a guided tour of the virtual training lab with your instructor. While participating
in the guided tour, you identify the various Oracle Solaris Cluster hardware components
including the cluster nodes, the Administrative Workstation, and the storage device.
The virtual training lab is created using VirtualBox software and consists of the following Oracle
Solaris Cluster hardware components in the form of virtual machines:
• Sun_ZFS_Storage_7000: The ZFS Storage 7000 storage appliance simulator

• Cluster_Node_1: The first cluster node
• Cluster_Node_2: The second cluster node
• Administrative Workstation: The Administrative Workstation to manage your cluster
The following table lists the login credentials for the cluster nodes and the administration
workstation:
Virtual Machines Regular User/Password Superuser/Password
Cluster Node 1 oracle1/oracle1 root/2secure
Cluster Node 2 oracle1/oracle1 root/2secure
Administration Workstation oracle1/oracle1 root/2secure

Chapter 2 - Page 3
Chapter 2 - Page 4

Establishing Cluster Node
Console Connectivity
Chapter 3
Practices for Lesson 3: Establishing Cluster Node Console Connectivity

Chapter 3 - Page 1
Practice Overview
In this practice, you install, configure, and use the Parallel Console Access (pconsole) utility
on the Administrative Workstation to manage a cluster environment.

Chapter 3 - Page 2
Practice 3-1: Connecting to the Cluster Node Console
Overview
In this practice, you install and configure the Parallel Console Access (pconsole) utility on an
Administrative Workstation to manage a cluster environment.
Preparation
Before proceeding with the tasks in this practice, start the Sun_ZFS_Storage_7000,
Cluster_Node_1, Cluster_Node_2, and Administration Workstation virtual
machines in that order. Wait for the virtual machines to boot properly until you see the login
prompt.
This practice assumes that the Oracle Solaris 11.1 operating system is already installed on the
two cluster nodes and the Administrative Workstation.

For the purpose of this practice, use the Administrative Workstation (adminws) system
to install, configure, and use the pconsole utility.
Task 1: Updating Host Name Resolution

Perform the following steps:
1. Log in to your Administrative Workstation as the oracle1 user.
2. Verify that the Oracle Solaris 11.1 operating system is installed on the Administrative
Workstation.
$ cat /etc/release
Oracle Solaris 11.1 X86
Copyright (c) 1983, 2012, Oracle and/or its affiliates. All rights reserved.
Assembled 19 September 2012
3. Ensure that the /etc/inet/hosts file has entries for the IP addresses and host names of
the clnode1 and clnode2 cluster nodes.
$ cat /etc/inet/hosts
…
192.168.1.101 cluster-1 clnode1
…
4. Verify that the config/host property of the system/name-service/switch service is

set to “files nis” to ensure that the name resolution first happens by using the
/etc/inet/hosts file before trying any other name service:
$ svcprop system/name-service/switch | grep config/host
config/host astring\ files\ nis

Chapter 3 - Page 3
If the config/host property is not set, use the svccfg command as the root role to set
the config/host property of the system/name-service/switch service to “files
nis”:
# svccfg -s system/name-service/switch
svc:/system/name-service/switch> setprop config/host = astring: "files nis"
svc:/system/name-service/switch> quit
# svcadm refresh name-service/switch
# svcprop system/name-service/switch | grep config/host
config/host astring\ files\ nis
Note: Ensure that you exit the root role before moving on to the next task.
Task 2: Installing the pconsole Utility


1. Ensure that you are logged in as the oracle1 user.
2. Verify that the solaris and ha-cluster publishers correspond to the Oracle Solaris 11.1
operating system repository and Oracle Solaris Cluster 4.1 repository, respectively:
$ pkg publisher
PUBLISHER TYPE STATUS P LOCATION
solaris origin online F http://adminws:10001/
ha-cluster origin online F http://adminws:10002/
Note: By default, the pkg publisher command should list the solaris publisher having
https://pkg.oracle.com as the URI. If not set, type the following command to set the origin
for the solaris publisher to get the Oracle Solaris 11.1 operating system software
packages and the ha-cluster publisher to get the Oracle Solaris Cluster 4.1 software
packages from a local repository.
$ pfexec pkg set-publisher –G ‘*’ –g http://adminws:10001/ solaris
$ pfexec pkg set-publisher –g http://adminws:10002/ ha-cluster
$ pkg publisher
3. Install the terminal/pconsole package.

$ pfexec pkg install terminal/pconsole
$ pkg info terminal/pconsole
4. Install the Oracle Solaris Cluster man page packages:
$ pfexec pkg install ha-cluster/system/manual
$ pkg info ha-cluster/system/manual
$ pfexec pkg install ha-cluster/system/manual/data-services
$ pkg info ha-cluster/system/manual/data-services
Installing the Oracle Solaris Cluster man page packages on the Administrative Workstation
enables you to view the various man pages of the Oracle Solaris Cluster software on the
Administrative Workstation before you actually install the software on the cluster nodes.
Task 3: Configuring the pconsole Utility

1. Ensure that you are logged in as the oracle1 user.

Chapter 3 - Page 4
2. Assign the /usr/cluster/man directory path to the PATH environment variable in the
.profile file of the oracle user:
$ vi ~/.profile
...
export PATH=/usr/cluster/man:/usr/bin:/usr/sbin
...
3. Set the environment variables by running the .profile login script.

$ . .profile
$ echo $PATH
4. Assume the root role.

5. Grant the Parallel Console Access rights profile to the oracle1 user.
# usermod –P +“Parallel Console Access” oracle1

Note: Throughout the practices of this course, you use the pconsole utility as the oracle
user having the Parallel Console Access rights profile.
6. Exit the root role.
7. Verify that the Parallel Console Access rights profile is granted to the oracle1 user.
$ profiles | grep –i parallel
Parallel Console Access
Task 4: Using the pconsole Utility

1. Ensure that the Cluster_Node_1 and Cluster_Node_2 virtual machines acting as the
two cluster nodes are powered on or booted.
2. Ensure that you are logged in to the Administrative Workstation as the oracle1 user.
3. Start the pconsole utility to manage the cluster nodes.
$ pconsole clnode1 clnode2 &
The previous command opens a host terminal window for each cluster node. It also opens a
central or master console window that propagates what you input there to each of the
cluster node connections that you open.
The pconsole utility uses the Secure Shell (SSH) protocol to establish a connection with
the cluster nodes. Therefore, on the host terminal windows of the cluster nodes, you should
see a message asking to confirm the connection to the cluster nodes.
4. Type yes and then specify oracle1 as the password in the master console window to
accept an SSH connection to the cluster nodes.
5. Place the cursor in the master console window, and press the Enter key a couple of times.
You should see a response in the host terminal window of each cluster node. The
pconsole utility is now ready to use for various cluster operations.

Chapter 3 - Page 5
6. Adjust the two host terminal windows for each cluster node and the master console window
on the desktop of the Administrative Workstation, as shown in the following screenshot:

Note: Adjusting the two host terminal windows and the master console window is just for
your convenience and is not a mandatory step.

Chapter 3 - Page 6

Preparing for the Oracle
Solaris Cluster Installation
Chapter 4
Practices for Lesson 4: Preparing for the Oracle Solaris Cluster Installation
Chapter 4 - Page 1
Practices for Lesson 4: Overview
Practice Overview
In this practice, you verify the Oracle Solaris 11.1 OS installed on your lab systems, identify a
cluster topology, select quorum devices, verify the cluster interconnect configuration, and select
public network interfaces.
Chapter 4 - Page 2
Practice 4-1: Preparing for Installation
Overview
In this practice, you prepare your assigned cluster nodes for installing the Oracle Solaris Cluster
4.1 software.
Preparation
To begin this practice, you must be connected to the cluster nodes through the pconsole utility
from the Administrative Workstation and you should be logged in to them as the oracle1 user.
Task 1: Verifying the Oracle Solaris 11.1 Environment

1. Verify that the Oracle Solaris 11.1 operating system is installed on the cluster nodes.

$ cat /etc/release
Oracle Solaris 11.1 X86
Copyright (c) 1983, 2012, Oracle and/or its affiliates. All rights reserved.
Assembled 19 September 2012
2. Use the /usr/sbin/prtconf command to record the size of (physical) memory on the
designated cluster nodes.
$ /usr/sbin/prtconf | grep –i memory
Node 1 memory: _______________
Node 2 memory: _______________
3. Use the df -h command and the zpool status command to determine whether ZFS is
the root file system on the cluster nodes.
$ df -kh
$ zpool status
$ zpool list
$ zfs list
4. Use the swap -l command to verify that at least 750 MB of swap space is given to each
cluster node. The swap space should be on zvol.
$ swap -l
Task 2: Identifying a Cluster Topology

1. Record the desired topology configuration of your cluster in the following table:
Number of nodes
Number of storage arrays
Types of storage arrays
2. Verify that the storage arrays in your cluster are properly connected for your target
topology. Recable the storage arrays if necessary.
In this course, you will be creating a two-node cluster by using a preconfigured
Sun_ZFS_Storage_7000 appliance simulator as the storage device.
Chapter 4 - Page 3
Task 3: Selecting Quorum Devices
Perform the following tasks:
1. Record the number of quorum devices that you must configure after the cluster host
software installation.
Number of quorum devices: __________
Note: Consult with your instructor if you are not sure about your quorum device
configuration.
2. Decide whether the scinstall utility will automatically be able to choose a quorum device
(it can for a single quorum device for a two-node cluster).
Automatic quorum configuration: (yes/no)?
3. If there can be no automatic quorum configuration (for a three-node cluster, for example),
enter the format command and record the logical path to the disks that you want to use as

quorum disk drives in your storage arrays.
Quorum disks: ________________________________
4. Press Ctrl + D to exit the format utility.
You will be configuring only a two-node cluster as you perform the practices of this course
inside the VirtualBox environment, and will ignore steps 3 and 4. Therefore, you should
specify yes for automatic quorum device selection and configuration by the scinstall
utility in step 2.
Task 4: Verifying the Cluster-Private Interconnect Configuration

This task describes how to verify the cluster-private interconnect configuration. Skip this section
if your cluster-private interconnect is not point-to-point.
Perform the following steps to configure an Ethernet-based, point-to-point interconnect:
1. Determine the names of your cluster-private interconnect adapters by using the dladm
show-phys command followed by the ipadm show-addr command.
$ dladm show-phys
$ ipadm show-addr
Refrain from assigning IP addresses to the existing public network interfaces.
2. Determine the cluster-private interconnect interfaces for as many interfaces as are
applicable in your lab environment.
Chapter 4 - Page 4
The following figure is a schematic representation of a cluster using an Ethernet-based,
point-to-point cluster-private interconnect configuration:
Perform the following steps to configure a switch-based Ethernet interconnect:

1. Record the logical names of the cluster interconnect interfaces.

Note: You can use the strategy of determining the cluster transport interfaces by using the
dladm show-phys command followed by the ipadm show-addr commands. Do not use
IP addresses in the existing public network space.
$ dladm show-phys
$ ipadm show-addr
2. Determine the interconnect interfaces for as many interfaces as are applicable in your lab
environment.
The following figure is a schematic representation of a cluster that uses an Ethernet-based
cluster-private interconnects with switches.
3. Verify that each Ethernet interconnect interface is connected to the correct switch. If you
are remote, and do not want to take your instructor’s word for it, you can verify that all
nodes can ping each other across the private switches.
Note: If you have any doubt about the private interconnect cabling, consult with your
instructor now. Do not continue this practice until you are confident that your cluster-private
interconnects are cabled correctly, and that you know the names of the cluster-private
transport adapters.
Note: In your given cluster setup environment inside VirtualBox, net1 and net3 are the
designated switch-based cluster-private interconnects.
Chapter 4 - Page 5
Task 5: Selecting Public Network Interfaces
Ask your instructor for help from in identifying public network interfaces on each cluster node
that can be used in IPMP groups.
Record the logical names of the potential IPMP Ethernet interfaces on each cluster node, in the
following table:
System Primary IPMP interface Backup IPMP interface
Node 1
Node 2
Note: It is important that you are sure about the logical name of each public network
interface.
In your given cluster setup environment inside VirtualBox, net0 is the primary IPMP

interface while net2 is the backup IPMP interface.
Chapter 4 - Page 6

Installing and Configuring the
Oracle Solaris Cluster
Software
Chapter 5
Practices for Lesson 5: Installing and Configuring the Oracle Solaris Cluster Software
Chapter 5 - Page 1
Practice Overview
In this practice, you install and configure the Oracle Solaris Cluster software.
Chapter 5 - Page 2
Practice 5-1: Installing and Configuring the Oracle Solaris Cluster
Software
Overview
In this practice, you install and configure the Oracle Solaris Cluster software. To do this, you
verify the lab environment, update local name resolution, install the Oracle Solaris Cluster
packages, configure a new cluster, verify an automatically selected quorum device, configure a
quorum device, and verify the cluster configuration and status.
Preparation
Cluster_Node_1, Cluster_Node_2, and Administrative Workstation virtual
machines, in that order. Wait for the virtual machines to boot properly until you see the login

prompt.
Before you begin this practice, ensure that you have completed the tasks covered in the
previous practices.
For the purpose of this practice, use the Administrative Workstation (adminws) system
to install and configure the Oracle Solaris Cluster software by using the pconsole utility.
Task 1: Establishing Connection to the Cluster Nodes

1. Ensure that the Cluster_Node_1 and Cluster_Node_2 virtual machines acting as the
two cluster nodes are powered on.
2. Log in to the Administrative Workstation as the oracle1 user and open a terminal window.
3. Start the pconsole utility to open the host terminal window for each cluster node and a
master console window.
$ pconsole clnode1 clnode2 &
4. Use the master console window to log in to the two cluster nodes with oracle1 as the
username and password.
You perform the remaining tasks of this practice in the master console window, unless
otherwise instructed.
Task 2: Preparing the Cluster Node Environment

Perform the following steps in the master console window on the Administrative Workstation:
1. Verify that the Oracle Solaris 11.1 operating system is installed on the cluster nodes.
$ cat /etc/release
2. Ensure that the CONSOLE=/dev/console line in the /etc/default/login file is

commented out on the cluster nodes.
3. Choose the primary group package of the Oracle Solaris Cluster 4.1 software to install.
Primary group package: _________________________________________________
For the purpose of this practice, you select ha-cluster-full primary group package of
the Oracle Solaris Cluster 4.1 software to install on the cluster nodes.
Chapter 5 - Page 3
4. If necessary, edit the /etc/inet/hosts file of the cluster nodes and add the IP
addresses of your Administrative Workstation and of all your cluster nodes.
…
192.168.1.100 adminws
…
5. Verify that the Network Auto-Magic feature is disabled on the cluster nodes.
$ netadm list
TYPE PROFILE STATE
ncp Automatic disabled
ncp DefaultFixed online
loc Automatic offline

loc NoNet offline
loc DefaultFixed online
6. Grant the Service Management and Service Operator rights profiles to the oracle1
user on the cluster nodes.
a. While logged in as the oracle1 user on the cluster nodes, assume the root role.
b. Grant the Service Management and Service Operator rights profiles to the
oracle1 user.
# usermod –P +”Service Management”,”Service Operator” oracle1
c. Exit the root role to return to the prompt of the oracle1 user on the cluster nodes.
7. Verify that the local_only property of the network/rpc/bind service is set to false
to enable external access to remote procedure call (RPC) communication.
$ svcprop network/rpc/bind:default | grep local_only
config/local_only boolean false
If the local_only property is set to true, use the svccfg command as follows to set the
local_only property of network/rpc/bind service to false.
$ svccfg
svc:> select network/rpc/bind
svc:/network/rpc/bind> setprop config/local_only=false
svc:/network/rpc/bind> quit
$ svcadm refresh network/rpc/bind:default
$ svcprop network/rpc/bind:default | grep local_only
config/local_only boolean false
Task 3: Installing the Oracle Solaris Cluster Packages

1. Set the solaris and ha-cluster publishers for the Oracle Solaris 11.1 operating system
and Oracle Solaris Cluster 4.1 software packages.
$ pkg publisher
Chapter 5 - Page 4
Note: By default, the pkg publisher command should list the solaris publisher having
https://pkg.oracle.com as the URI. For the purpose of this practice, you will reset the origin
for the solaris publisher to get the Oracle Solaris 11.1 operating system software
packages from a local repository.

$ pfexec pkg set-publisher –G ‘*’ –g http://adminws:10001/ solaris
$ pfexec pkg set-publisher –g http://adminws:10002/ ha-cluster
$ pkg publisher
2. Install the primary group package of the Oracle Solaris Cluster 4.1 software that you
selected in step 3 of Task 2.

$ pfexec pkg install ha-cluster-full
3. Verify the Oracle Solaris Cluster software package installation.
$ pkg info ha-cluster-full
4. Assign the /usr/cluster/bin and /usr/cluster/man directory paths to the PATH

environment variable in the .profile login script of the oracle1 user.
$ vi .profile
...
export PATH=/usr/cluster/bin:/usr/cluster/man:/usr/bin:/usr/sbin
...

$ . .profile
$ echo $PATH
Task 4: Configuring a New Cluster: The All-Nodes-at-Once Method

If you prefer to use the one-node-at-a-time method, skip this task and complete Task 5 instead.
Perform the following steps to configure the entire cluster by using the all-nodes-at-once
method.
Complete the following steps in the host terminal window of clnode1 cluster node:
1. While logged in as the oracle1 user on the clnode1 cluster node, assume the root role.
environment variable in the .profile login script of the root role.
# vi .profile
...
...

# . .profile
# echo $PATH
4. Start the scinstall utility.

# /usr/cluster/bin/scinstall
Chapter 5 - Page 5
5. As the installation proceeds, make the following choices:
a. From the Main menu, select option 1, Create a new cluster or add a
cluster node.
b. From the New Cluster and Cluster Node menu, select option 1, Create a new
cluster.
c. Specify Yes when asked whether you want to continue.
The scinstall utility now checks whether the value of the local_only property of
network/rpc/bind service is correctly set to false.
d. Press the Enter key to continue.
e. From the Typical or Custom Mode menu, select option 1, Typical.
f. Specify a cluster name, such as cluster1.
g. Specify the name of the other cluster node, such as clnode2.

h. Press Ctrl + D to finish adding the cluster nodes to display the complete list of cluster
nodes added.
i. Specify Yes to confirm the listing of the cluster nodes.
j. Select the cluster transport adapters that will be used as the cluster-private
interconnect.
For this practice, select net1 and net3, one after the other, as the cluster-private
interconnect.
k. Specify No when asked whether you want to disable the automatic quorum device
selection.
l. Specify Yes when asked whether it is okay to create the new cluster.
m. Specify No when asked to interrupt cluster creation for cluster check errors.
The cluster configuration begins. Somewhere toward the end of the cluster configuration,
the clnode2 cluster node reboots due to which you lose the pconsole connection to the
node.
After the clnode2 cluster node successfully reboots and joins the cluster, the scinstall
utility then proceeds to configure the clnode1 cluster node. After the configuration, the
clnode1 cluster node also reboots to join the cluster. At this point, you now lose the
pconsole connection to the clnode1 cluster node too and only the master console
window remains open. Press Ctrl + D to close the master console window.
After the clnode1 cluster node also successfully reboots and joins the cluster, reestablish
the pconsole connection to the cluster nodes to continue with the cluster configuration
and administration.
Note: Refer to the various messages displayed in the console window of the cluster nodes
when they reboot.
6. You observe the following error messages as the node reboots in the console window of
the clnode1 cluster node.
clnode1 did: WARNING: did_admin_open: FEXCL EBUSY
/usr/cluster/bin/scdidadm: Could not load /dev/did/admin –
Device busy.
These error messages are safe to ignore.
Chapter 5 - Page 6
Task 5: Configuring a New Cluster: The One-Node-at-a-Time Method
Perform the following steps to configure the first node in your cluster (the one that will be
assigned node ID 1). You must wait for this node to complete and reboot into the cluster before
configuring the other node.

Note: If you chose to install your cluster by using the all-nodes-at-once method, do not do
this task.
Complete the following steps in the host terminal window of the clnode1 cluster node:
# vi .profile
...

...

# . .profile
# echo $PATH

cluster node.
b. From the Install menu, select option 2, Create just the first node of a new
cluster on this machine.
c. Answer Yes to confirm the creation of just the first node of a new cluster.
d. Press Enter to continue after the local_only property value is checked to be false.
e. From the Type of Installation menu, select option 1, Typical.
f. Furnish your assigned cluster name.
g. Do not allow the cluster check to run.
h. Furnish the name of the other nodes to be added later.
i. Verify the list of node names.
j. Select the transport adapters. If you are asked (with a tagged VLAN-capable adapter)
whether it is a dedicated cluster transport adapter, answer Yes.
k. For a two-node cluster, do not disable the automatic quorum selection (answer No).
l. Reply Yes to the automatic reboot question.
m. Examine the scinstall command options for correctness. Accept them if they seem
appropriate.
You must wait for this node to complete rebooting to proceed to the second node.
Note: After cluster configuration completes, the node reboots, causing the host terminal
window to close. You have to reestablish the connection to the clnode1 cluster node by
using the pconsole utility. Refer to messages displayed in the console window of the
cluster nodes.
Chapter 5 - Page 7
6. You observe the following error messages as the node reboots in the console window of
the clnode1 cluster node. They are safe to ignore.
clnode1 did: WARNING: did_admin_open: FEXCL EBUSY
/usr/cluster/bin/scdidadm: Could not load /dev/did/admin –

Device busy.
Complete the following steps in the host terminal window of the clnode2 cluster node:
# vi .profile
...

...

# . .profile
# echo $PATH

cluster node.
b. From the Install menu, select option 3, Add this machine as a node in an
existing cluster.
c. Answer Yes to confirm the addition of a node to an existing cluster.
d. Press Enter to continue after the local_only property value is checked to be false.
e. From the Type of Installation menu, select option 1, Typical.
f. Provide the name of a sponsoring node, clnode1.
g. Provide the name of the cluster that you want to join.
Note: Use cluster show -t global on the first cluster node (the sponsoring
node) if you have forgotten the name of the cluster.
h. Press Enter to continue.
i. Do not allow the cluster check to run.
j. Use auto-discovery for the transport adapters.
k. Reply Yes to the automatic reboot question.
l. Examine and approve the scinstall command-line options.
You must wait for this node to complete rebooting to proceed to the next step.
6. Exit the master console window on the Administrative Workstation.
Note: After the cluster configuration completes, the node reboots, causing the host terminal
window to close. You have to reestablish the connection to the clnode2 cluster node by using
the pconsole utility. Refer to the messages displayed in the console window of the cluster
nodes.
Chapter 5 - Page 8
Task 6: Verifying the Cluster Configuration and Status
Perform the following steps on any one of the cluster nodes as the oracle1 user to verify the
cluster configuration:
1. Run the clquorum status command.

$ clquorum status
You should see the correct node and disk quorum votes.
2. Run the clinterconnect status command.
$ clinterconnect status
You should see active redundant interconnect paths between all pairs of nodes.
3. Run the cldevice list -v command.
$ cldevice list -v

Each shared (dual-ported) DID device should show a logical path from each cluster node.
4. Run the cluster show and cluster status commands.
$ cluster show
$ cluster status
The cluster status, node names, transport configuration, and quorum device information
should be complete.
Chapter 5 - Page 9
Chapter 5 - Page 10

Performing Cluster
Administration
Chapter 6
Practices for Lesson 6: Performing Cluster Administration

Chapter 6 - Page 1
Practice Overview
In this practice, you perform basic cluster administration tasks such as verifying basic cluster
configuration and status, reassigning a quorum device, adding a quorum server quorum device,
preventing cluster amnesia, and changing the cluster private IP address range.

Chapter 6 - Page 2
Practice 6-1: Performing Basic Cluster Administration
Overview
In this practice, you perform basic cluster administration tasks such as verifying basic cluster
configuration and status, reassigning a quorum device, adding a quorum server quorum device,
preventing cluster amnesia, and changing the cluster private IP address range.
Preparation
prompt.

previous practices.
For the purpose of this practice, run the pconsole utility on the Administrative
Workstation (adminws) system to open a master console window and the host terminal
window for each cluster node.
Task 1: Verifying Basic Cluster Configuration and Status

Perform the following steps as the oracle1 user in the host terminal window of clnode1
cluster node:
1. Verify the cluster global properties and node properties:
$ cluster show -t global
$ clnode show
2. Use the clquorum (clq) command to verify the current cluster membership and the
quorum status.
$ clq status
3. Record the quorum configuration from the previous step.
Quorum votes needed: _____
Quorum votes present: _____
Quorum votes possible: _____
4. Verify all cluster disk paths.
$ cldev status
5. Verify the interconnect status.
$ clintr status
6. Verify the revision of the currently installed Oracle Solaris Cluster software on each node.
$ clnode show-rev –v

Chapter 6 - Page 3
Task 2: Reassigning a Quorum Device
Assume that your quorum device (or one of your quorum devices) has failed. Perform the
following tasks from any one node in the cluster to reassign the quorum device:
1. Verify the status of your current quorum device(s):
$ clq status
$ cldev status d#
2. Show all disk paths:
$ cldev list -v
3. Choose a different shared disk to be the new quorum device, and add your new quorum
device:
$ clq add new-d-#

$ clq status
$ clq status new-d-#
4. Remove your old broken quorum device:
$ clq remove old-d-#
5. Verify the quorum status again:
$ clq status
Task 3: Configuring Oracle Solaris Cluster Quorum Server Software

Perform the following steps to configure your Administrative Workstation as the quorum
server:
1. Ensure that the quorum server machine is connected to a public network that is accessible
to the cluster nodes.
2. Ensure that you are logged in to the selected quorum server machine as an oracle1 user.
3. Ensure that the solaris and ha-cluster publishers are valid.
adminws$ pkg publisher
4. Install the Quorum Server group package.
adminws$ pfexec pkg install ha-cluster-quorum-server-full
5. Add the Oracle Solaris Cluster Quorum Server binary location to your PATH environment
variable by adding the following entry to the .profile file.
adminws$ . .profile

7. Specify quorum configuration information in the /etc/scqsd/scqsd.conf file as follows:
/usr/cluster/lib/sc/scqsd -d /var/scqsd -i quorumserver -p 9000
8. Save and close the /etc/scqsd/scqsd.conf file.

9. Start the newly configured quorum server.
adminws# /usr/cluster/bin/clquorumserver start quorumserver


Chapter 6 - Page 4
Task 4: Adding a Quorum Server Quorum Device
To get the IP address of the quorum server that you configured in step 3, either use the ipadm
command or refer to the /etc/inet/hosts file. In your cluster lab environment, the IP
address of the Administrative Workstation, now acting as a quorum server, is 192.168.1.100.

You can add the quorum device just by using the IP address, or you can use the host name, if it
is resolvable on your cluster nodes.
Perform the following steps on any one-cluster node as the oracle1 user:
1. Add the quorum server quorum device.
$ clq add -t quorum_server -p qshost=qserver_name_or_IP -p port=9000 myqserver
$ clq status
The name that you give to the device on the cluster side as the last argument to the
command is not important. You can use any name.

2. Delete your disk quorum device(s).
$ clq remove old-d-# [old-d#]
$ clq status
3. Add back the disk quorum devices and remove the quorum server quorum device.
$ clq add d#
$ clq status
$ clq remove myqserver
$ clq status
If you have a cluster with nonstorage nodes, having a quorum server quorum device is not
necessarily exactly what you want, because it enables the nonstorage node to stay alive
even if both the storage nodes have failed.
Task 5: Preventing Cluster Amnesia

Perform the following steps to demonstrate how the cluster prevents cluster amnesia, by using
persistent reservations on the quorum device.
1. Verify that all nodes are active cluster members.
2. As the root user role, shut down the clnode1 and clnode2 cluster nodes with init 0.
# init 0
3. Boot the clnode1 cluster node.

The clnode1 cluster node should hang waiting for operational quorum. This is because the
reservation key for the clnode1 cluster node has been removed from the quorum disk(s).
4. Now boot the clnode2 cluster node.
Both the cluster nodes should complete the cluster software startup sequence and join the
clustered operations.
Task 6: Changing the Cluster Private IP Address Range

1. Shut down the entire cluster by entering the following command on any one node as the
root user:
# cluster shutdown –g0 -y
Note: If for some reason a cluster node does not shut down by using the preceding
command, use the VirtualBox menu options to either send a shutdown signal or power off
the system.

Chapter 6 - Page 5
2. Boot all nodes to noncluster mode.
a. In the GRUB menu, use the arrow keys to select the appropriate Oracle Solaris 11.1
entry and type e to edit its commands.
b. In the boot parameters screen, use the arrow keys to select the $multiboot entry.
c. Append -x at the end of the $multiboot entry to specify system boot in noncluster
mode.
$multiboot /ROOT/solaris/@/$kern $kern –B $zfs_bootfs -x
d. Press Ctrl + x to boot the node into noncluster mode.
3. Choose (any) single node on which to perform the following commands. On the chosen
node only, run the following procedure to change the cluster-private IP address range:
a. As the root user, run the clsetup command.
b. Select option 1, Change Network Addressing and Ranges for the Cluster

Transport, to change the configuration.
c. Answer yes when asked whether you want to change the settings.
d. Answer no when asked whether you want to accept the default base address.
e. Enter a new available private address base address, such as 192.16.0.0.
Consult your instructor if you are not sure. Do not accidentally conflict with the public
network address space.
f. Answer no when asked whether you want to accept the default netmask.
g. Enter 4 for the maximum number of nodes.
h. Enter 4 for the maximum number of private networks.
i. Accept the suggested netmask value of 255.255.255.0.
j. Confirm that you want to proceed with the update.
4. When the command succeeds, reboot all nodes back into the cluster.

Chapter 6 - Page 6
5. Verify that the private network addresses have changed.
a. Use the ipadm show-addr command and verify the information for the physical
private network adapters and for the clprivnet0 adapter.
$ ipadm show-addr
b. Verify the output of the cluster show -t global command.

$ cluster show –t global

Chapter 6 - Page 7

Chapter 6 - Page 8

Practices for Lesson 7: Using
ZFS with Oracle Solaris
Cluster Software
Chapter 7
Practices for Lesson 7: Using ZFS with Oracle Solaris Cluster Software
Chapter 7 - Page 1
Practice Overview
In this practice, you configure volume management by using ZFS.
Chapter 7 - Page 2
Practice 7-1: Configuring Volume Management by Using ZFS
Overview
In this practice, you create a ZFS pool and file system for cluster data, create a snapshot and
then modify your data, and manually migrate your ZFS pool to another node.
Preparation
prompt.
previous practices.

Task 1: Creating a ZFS Pool and File System for Cluster Data
Perform the following steps as the oracle1 user in the host terminal window of the clnode1
cluster node:
1. Select disk drives from the shared storage to use in a new ZFS pool for data.
$ zpool status
$ cldev list –v
$ clq status
Make sure that you do not use any disks already in use in the existing device group. Note
the logical device name of the identified disk, referred to as c#t#d# in the next step.
Consult with your instructor if you are unsure.
3. Create a ZFS pool that provides mirroring for the selected disks.
# zpool create nfspool mirror c#t#d# c#t#d#
# zpool status
4. Create a ZFS file system within the pool.

# zfs create nfspool/nfs
# zfs list -r nfspool
# df -kh
5. Change the mount point.
# zfs set mountpoint=/testnfs nfspool/nfs
# df –kh
6. Populate your file system with some simple files.

# cd /testnfs
# touch file1 file2 file3
# vi my_letter
ZFS is much better than that legacy stuff
# cd
Chapter 7 - Page 3
Task 2: Creating a Snapshot and Then Modifying Your Data
Perform the following steps as the root user in the host terminal window of the clnode1
cluster node:
1. Snapshot your ZFS file system.

# zfs snapshot nfspool/nfs@SAVEMYWORK
2. Modify the content of your file system.

# cd /testnfs
# touch file4
# rm file2
# vi my_letter
ZFS is much better than that legacy stuff

Here is more information about my class
# cd
Task 3: Manually Migrating Your ZFS Pool to Another Node

Perform the following steps as the root role.
On the host terminal window of the clnode1 cluster node:
1. Export your nfspool pool.
# zpool export nfspool
# zpool status
# zfs list
On the host terminal window of the clnode2 cluster node:

3. Import the nfspool pool.
# zpool import
# zpool import nfspool
# zpool status
# df -kh
4. Recover the snapshot that had migrated along with the pool.
# cd /testnfs
# ls
# zfs rollback nfspool/nfs@SAVEMYWORK
# ls
Chapter 7 - Page 4

Practices for Lesson 8: Using
Solaris Volume Manager with
Software
Chapter 8
Practices for Lesson 8: Using Solaris Volume Manager with Oracle Solaris Cluster Software
Chapter 8 - Page 1
Practice Overview
In this practice, you configure Solaris Volume Manager (SVM) to use with Oracle Solaris
Cluster.
Chapter 8 - Page 2
Practice 8-1: Configuring Volume Management by Using Solaris
Volume Manager
Overview
In this practice, you configure Solaris Volume Manager (SVM).
Preparation
Cluster_Node_1, Cluster_ Node_2, and Administrative Workstation virtual
prompt.
previous practices.

At least one local disk on each cluster node must have a small unused slice that you can use for
the local metadb replicas. The examples in this practice use slice 6 of the boot disk, which
should be reserved ahead of time. During this practice, you create two data service disk sets,
each of which contains a single mirrored volume.
Task 1: Initializing the SVM Local metadb Replicas

Before you can use SVM to create disk sets and volumes, you must initialize the state database
and create one or more replicas.
1. While logged in as the oracle1 user on the cluster nodes, assume the root role.
2. Verify that the boot disk has a small slice, s6, reserved for creating state database replica.
# zpool status
# prtvtoc /dev/rdsk/c#t#d#s0
Note: Your boot disk path may be different on different nodes.
3. Use the metadb command to create three database replicas on the unused boot disk slice.
# metadb -a -f -c 3 c#t#d#s6
4. Verify that the replicas are configured and operational.

# metadb
Task 2: Selecting the SVM Demo Volume Disk Drives

Perform the following steps in the host terminal window of the clnode1 cluster node to select
the SVM demo volume disk drives:
1. As the oracle1 user or root role, type the cldev list -v command to list all of the
available DID drives.
$ cldev list –v
$ clq status
Make note of the DID numbers of shared disks that are already in use. Make sure that you
do not use those shared disks in this practice.
Chapter 8 - Page 3
2. Record the logical path and DID path numbers of two disks that you will use to create the
demonstration disk set, webds, and volumes in the following table:
Disk set Volumes Primary Disk Mirror Disk
Webds d100
Note: You need to record only the last portion of the DID path. The first part is the same for
all DID devices: /dev/did/rdsk.
Note: Make sure that the disks you select are not local devices. They must be dual-hosted
and available to more than one cluster node.
For the purpose of this practice, select the d9 and d10 shared disks for the webds disk set.
Task 3: Configuring SVM Disk Sets and Volumes for a Web Server
Complete the following steps to create demonstration disk sets and volumes for use in later

practices:
2. Create the webds disk set, and configure the nodes that are physically connected to it.
# metaset -s webds -a -h clnode1 clnode2
2. Add the same nodes as disk set mediators to each disk set.
# metaset -s webds -a -m clnode1 clnode2
3. Add the two disks chosen in Task 2 to the webds disk set.
# metaset -s webds -a /dev/did/rdsk/d9 /dev/did/rdsk/d10
4. Verify the status of the new disk set.
# metaset -s webds
# medstat -s webds
# cldg status
5. Create a submirror on each of your disks in the webds disk set.

# metainit -s webds d0 1 1 /dev/did/rdsk/d9s0
# metainit -s webds d1 1 1 /dev/did/rdsk/d10s0
6. Create a mirror volume, d99, by using the d0 submirror.

# metainit -s webds d99 -m d0
7. Attach the second submirror, d1, to the mirror volume d99.

# metattach -s webds d99 d1
8. Create a 500 MB soft partition, d100, on top of your mirror. This is the volume that you
actually use for your file system data.
# metainit -s webds d100 -p d99 500m
9. Verify the status of the new volume.
# metastat -s webds
Task 4: Creating a Cluster File System

Complete the following steps to create a cluster file system in the webds disk set:
1. Ensure that you are logged in as the oracle1 user on the clnode1 cluster node and have
assumed the root role.
Chapter 8 - Page 4
2. Create a file system on d100 in the webds disk set by performing the following command,
and type y to construct a new file system.
# newfs /dev/md/webds/rdsk/d100
3. On all nodes, create a global mount point for the new file system.
# mkdir /global/web
4. On all nodes, add a mount entry in the /etc/vfstab file for the new file system with the
global mount option.
/dev/md/webds/dsk/d100 /dev/md/webds/rdsk/d100 \
/global/web ufs 2 yes global
Note: Do not use the line continuation character (\) in the vfstab file.
5. On the clnode1 cluster node, mount the /global/web file system.

# mount /global/web
6. Verify that the file system is mounted and available on all nodes.
# mount | grep global/web
# ls /global/web
lost+found
# df –kh | grep global/web
Task 5: Testing Cluster File Systems

Complete the following steps to test the accessibility of the webds cluster file system from
different cluster nodes:
1. Ensure that you are logged in as the oracle1 user on the clnode1 cluster node and have
assumed the root role.
2. On the clnode2 cluster node, move into the /global/web file system.
# cd /global/web
3. On the clnode1 cluster node, try to unmount the /global/web file system.
# umount /global/web
You should get an error that the file system is busy.

4. On the clnode2 cluster node, move out of the /global/web file system (cd /) and try to
unmount it again on the clnode1 cluster node.
5. On the clnode2 cluster node, mount the /global/web file system.
6. Try unmounting and mounting /global/web from all nodes.
Ensure that the /global/web cluster file system remains mounted before you proceed to
the next task.
Task 6: Managing Disk Sets

Perform the following steps to migrate a disk set between cluster nodes:
1. Ensure that you are logged in as the oracle1 user on the clnode1 cluster.
2. Make sure that the webds disk set is online.
$ cldg status
3. Verify the current demonstration disk set configuration.

$ cldg show
Chapter 8 - Page 5
4. Assume the root role and shut down the clnode1 cluster node.
# shutdown –g0 -y
5. On the clnode2 cluster node, use the cldg status command to verify that the webds
disk set has automatically migrated to the clnode2 cluster node.

$ cldg status
6. Boot the clnode1 cluster node and assume root role.

The webds disk set should remain mastered by the clnode2 cluster node.
7. Use the cldg switch command from either node to migrate the webds disk set to the
clnode1 cluster node.
# cldg switch -n clnode1 webds
# cldg status
Chapter 8 - Page 6

Managing the Public Network
with IPMP
Chapter 9
Practices for Lesson 9: Managing the Public Network with IPMP

Chapter 9 - Page 1
Practice Overview
In this practice, you will configure and test IPMP.

Chapter 9 - Page 2
Practice 9-1: Configuring and Testing IPMP
Overview
In this practice, you verify the local-mac-address? variable, verify the adapters for the IPMP
group, and enter test addresses in the /etc/hosts file. Next, you verify that IPMP is
configured and test IPMP failover and failback.
Preparation
prompt.

previous practices.
Task 1: Verifying the local-mac-address? Variable

2. Configure the cluster nodes to use unique MAC addresses.
# eeprom local-mac-address?=true
3. Reboot the cluster nodes to enable unique MAC address assignment.

# init 6
Task 2: Verifying the Adapters for the IPMP Group

Perform the following steps from the host terminal window of each cluster node, one at a time:
1. Ensure that you are logged in as the oracle1 user on the cluster nodes.
2. Identify the redundant network adapters on the public network.
You might have already made a note of this during an earlier practice.
3. Your primary public network adapter should be the only one currently configured on the
public network.
$ ipadm show-if
$ ipadm show-addr
4. You can verify your secondary public network adapter by:
• Making sure that it is not configured as a private transport
• Making sure that it can snoop public network broadcast traffic

Chapter 9 - Page 3
Assume the root role and use the following commands:
# ipadm create-ip net2
# snoop -d net2
In another terminal window of the same cluster node or other cluster node, use the
following command:
$ ping -s pubnet_broadcast_addr
For example, ping the broadcast address, 192.168.1.255:
$ ping -s 192.168.1.255
Task 3: Verifying or Entering Test Addresses in the /etc/inet/hosts File

It is a good idea, although not required, to have test IP addresses listed in the hosts file.
Although one cluster node does not need to know anything about another cluster node’s test

addresses, it is advisable to have all test addresses in the /etc/inet/hosts file for all nodes
to indicate that these addresses are reserved and what they are reserved for.
Perform the following steps from the host terminal window of each cluster node:
2. Verify the IP addresses to be used for each network adapter on each cluster node.
3. Type the IP addresses in /etc/inet/hosts on each node, if they are not already there.
The following is a sample entry of test addresses in the /etc/inet/hosts file of a cluster
node. It really does not matter whether you use the same names as another group working
on another cluster as long as the IP addresses are different:
# IPMP Test Addresses
192.168.1.150 clnode1-local-net2
192.168.1.151 clnode1-net0-ipmp-test
192.168.1.153 clnode2-local-net2
Task 4: Configuring IPMP with Public Network Adapters and Test Addresses
Perform the following steps in the host terminal window of each cluster node, one at a time:
2. View the existing IPMP group information.
# ipmpstat -g
Note: During cluster configuration, the scinstall utility creates an IPMP group called
sc_ipmp0.
3. Add the second public network adapter to the IPMP group:
# ipadm add-ipmp -i net2 sc_ipmp0
4. Add a data address to the second public network adapter, net2.

# ipadm create-addr -T static -a 192.168.1.150/24 sc_ipmp0/v4add2

Chapter 9 - Page 4
5. Add test addresses to the first and second public network adapters:
# ipadm create-addr -T static -a 192.168.1.151/24 net0/test
Similarly, perform steps 3 through 5 on the other cluster node. Ensure that you use the
correct second public network adapter, the data address, and the test addresses.
For example, run the following commands on the other cluster node:
# ipadm add-ipmp -i net2 sc_ipmp0
# ipadm create-addr -T static -a 192.168.1.153/24 sc_ipmp0/v4add2
Task 5: Verifying That IPMP Is Configured

Perform the following steps in the host terminal window of each cluster, one at a time:

1. Ensure that you are logged in as the oracle1 user to the cluster node.
2. Verify the IPMP configuration.
$ ipmpstat -a
$ ipmpstat -t
$ ipmpstat –i
3. Verify IPMP cluster-wide status with clnode status -m.

$ clnode status -m
Task 6: Verifying IPMP Failover and Failback

Perform the following steps from the host terminal window of a cluster node:
1. As the oracle1 user in the Administrative Workstation system, launch ping -s
nodename, and keep it running.
adminws$ ping –s nodename
2. Simulate the unplugging of Ethernet cable from the network adapter that currently has the
physical interface of the cluster node on it.
3. Observe the cluster node messages in the console or in the /var/adm/messages file.
4. Observe the output of the clnode status -m command.
$ clnode status -m
5. Observe the behavior of your command from step 1 (keep it running).
6. Simulate the reattachment of the broken Ethernet cable.
7. Observe the messages and the behavior of your command from step 1.
8. Observe the output of the clnode status -m command.
$ clnode status -m

Chapter 9 - Page 5

Chapter 9 - Page 6

Managing Data Services,
Resource Groups, and HA-
NFS
Chapter 10
Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS
Chapter 10 - Page 1
Practice Overview
In this practice, you install and configure HA for NFS.
Chapter 10 - Page 2
Practice 10-1: Installing and Configuring HA for NFS
Overview
In this practice, you install and configure HA for NFS.
Preparation
prompt.
previous practices.

Workstation (adminws) system to open the host terminal window for each cluster node and a
Task 1: Installing and Configuring the HA for NFS Agent and Server
In earlier practices, you created the cluster file system for NFS. Confirm that this file system is
available and ready to configure for Oracle Solaris Cluster HA for NFS.
Perform the following steps on the specified cluster nodes as the oracle1 user, unless
otherwise specified:
In the master console window:
1. Verify that the HA for NFS software package is already installed on both cluster nodes.
$ pkg info ha-cluster/data-service/nfs
2. Lower the value of the grace_period property of the NFS protocol.

$ sharectl set -p grace_period=10 nfs
Lowering the grace period will speed up switchovers and failovers of HA for NFS resource.
Oracle Solaris 11.1 operating system uses NFS Version 4 by default. This is a stateful
protocol, which will intentionally delay resumption of NFS activity so that clients have a
chance to reclaim their state any time that the server is recovering, which will include any
cluster switchover or failover. The grace_period property controls the length of the delay.
3. Ensure that the /etc/inet/hosts file on the two cluster nodes and the Administrative
Workstation already contains an entry for the logical host name resource, cluster1-nfs.
$ cat /etc/inet/hosts | grep cluster1-nfs
192.168.1.120 cluster1-nfs
4. Identify the node that is currently importing the ZFS storage pool, nfspool, which you
created in the lesson titled “Using ZFS with Oracle Solaris Cluster Software.”
$ zpool status
5. On the node identified in step 4, assume the root role and mount the nfspool/nfs ZFS
file system under /global/nfs.
# zfs set mountpoint=/global/nfs nfspool/nfs
# df –kh | grep nfspool/nfs

In the host terminal window of the cluster node identified in step 4
Chapter 10 - Page 3
8. Create a directory within the /global/nfs file system for the HA for NFS resource to
maintain administrative information.
# cd /global/nfs
# mkdir SUNW.nfs
9. Create the dfstab.nfs-res file in the /global/nfs/SUNW.nfs directory and add the
entry to share the /global/nfs/data directory.
# cd SUNW.nfs
# vi dfstab.nfs-res
share -F nfs -o rw /global/nfs/data
10. Create the directory specified in the dfstab.nfs-res file.

# cd /global/nfs

# mkdir data
# touch data/sample.file
# chmod 777 data
# cd

Note: You change the mode of the data directory only for the purpose of demonstration. In
practice, you would be more specific about the share options in the dfstab.nfs-res file.
Task 2: Registering and Configuring the Solaris Cluster HA for NFS Data Service
In the host terminal window of any one cluster node:
1. Register the SUNW.nfs and SUNW.HAStoragePlus resource types.
$ clrt register SUNW.nfs
$ clrt register SUNW.HAStoragePlus
$ clrt list -v
2. Create the NFS failover resource group.

$ clrg create –n clnode1,clnode2 -p Pathprefix=/global/nfs nfs-rg
3. Create the logical host name resource for the NFS failover resource group.
$ clrslh create -g nfs-rg cluster1-nfs
4. Create the SUNW.HAStoragePlus resource and set the value of Zpools to nfspool.
$ clrs create -t SUNW.HAStoragePlus -g nfs-rg -p Zpools=nfspool nfs-stor
5. Bring the resource group to a managed state and then bring it online.
$ clrg online -M nfs-rg
Chapter 10 - Page 4
6. Create the SUNW.nfs resource and set the value of Resource_dependencies to nfs-
stor resource.
$ clrs create -t SUNW.nfs -g nfs-rg -p Resource_dependencies=nfs-stor nfs-res

7. Verify that the data service is online.
$ clrs list -v
$ clrs status
$ clrg status
$ df -kh
Task 3: Verifying Access by NFS Clients

In this task, you verify that NFS clients can access the HA for the NFS cluster file system.

On the Administrative Workstation:
1. Verify that you can access the cluster file system.
oracle1@adminws:~$ ls /net/cluster1-nfs/global/nfs/data
sample.file
2. Copy the test.nfs file from the /opt/ora/lab directory into your home directory.
oracle1@adminws:~$ cp /opt/ora/lab/test.nfs $HOME
3. View the $HOME/test.nfs script content.

oracle1@adminws:~$ cat test.nfs
#!/bin/ksh
cd /net/cluster1-nfs/global/nfs/data
while :
do
date > date.out
cat date.out
rm date.out
sleep 1
done
oracle1@adminws:~$
When you run the test.nfs script in the next task, it will create and write a file containing
a timestamp, to the NFS-mounted cluster file system. The script will also display the file to
standard output (stdout). This script helps you determine the duration for which the NFS
data service is interrupted during switchovers and takeovers.
Task 4: Observing Oracle Solaris Cluster HA for NFS Failover Behavior

1. Determine the name of the node that is currently primary for the nfs-rg resource group.
$ clrg status nfs-rg
Chapter 10 - Page 5
2. Run the test.nfs script.
$ chmod u+x test.nfs
$ ./test.nfs
3. Transfer control of the NFS resource to the other node.
$ clrg switch -n dest-node nfs-rg
Substitute the name of your offline node for dest-node.

4. Observe the messages displayed by the test.nfs script.
5. How long was the data service interrupted during the switchover from one physical host to
another?

__________________________________________________
6. Stop the test.nfs script.
In the master console window:
7. Use the share command to verify which file systems the nodes are now mounting and
exporting.
__________________________________________________
__________________________________________________
__________________________________________________
8. Use the ipadm show-addr command to observe the additional IP address associated
with the Logical Hostname resource configured as a virtual interface on one of the adapters
in your IPMP group.
$ ipadm show-addr
9. On one node of the cluster, use the clrg switch command to transfer control of the NFS
service back to its preferred host.
$ clrg switch -n dest-node nfs-rg
Task 5: Generating Cluster Failures and Observing the Behavior of the NFS
Failover
Generate failures in your cluster to observe the recovery features.
Try to generate the following failures:
• Power down a cluster node.
• Simulate the unplugging of network cable of one public network interface.
On any one cluster node, try your tests while the resource group is in its normal, nonsuspended
state. Repeat some tests after suspending the resource group.
$ clrg suspend nfs-rg
Note: By suspending a resource group, you will temporarily be suspending the automatic
recovery actions of the resource group. You might need to suspend the automatic recovery
of a resource group to investigate and fix a problem in the cluster.
When you are satisfied with your results, remove the suspension.
$ clrg resume nfs-rg
Chapter 10 - Page 6
Task 6: Making a Customized Application Fail Over with a Generic Data Service
Resource
In this task, you can see how easy it is to get any daemon to fail over in the cluster, by using the
Generic Data Service.
Perform the following steps on the specified cluster nodes:
1. On each cluster node, create a daemon that represents your customized application.
$ vi /var/tmp/myappdaemon
#!/bin/ksh
while :
do
sleep 10
done

2. Make sure that the file is executable on all nodes.
3. From any one cluster node, create a new failover resource group for your application.
$ clrg create –n clnode1,clnode2 myapp-rg
4. From one cluster node, register the Generic Data Service resource type.
$ clrt register SUNW.gds
5. From one cluster node, create the new resource and enable the resource group and
resources.
$ clrs create -g myapp-rg -t SUNW.gds -p Start_Command=/var/tmp/myappdaemon \
-p Probe_Command=/bin/true -p Network_aware=false myapp-res
$ clrg online -M myapp-rg
$ clrg status myapp-rg
$ clrs status myapp-res
$ ps –ef | grep myappdaemon
6. Verify the behavior of your customized application.
a. Verify that you can manually switch the group from the other cluster node.
$ clrg switch –n dest_node myapp-rg
$ clrg status myapp-rg
b. On the destination cluster node, kill the myappdaemon process.

$ pfexec kill -9 pid
$ clrs status
Wait a little while and note that the myappdaemon process restarts on the same node.
Also, wait until clrs status shows that the resource is fully online again.
c. Repeat step 6b a few times.
Eventually, the resource group switches over to the other node.
Chapter 10 - Page 7
Chapter 10 - Page 8

Configuring Scalable
Services and Advanced
Resource Group
Relationships
Chapter 11
Chapter 11 - Page 1
Practice Overview
In this practice, you will install and configure Apache web server as a scalable data service on
Oracle Solaris Cluster software.
Chapter 11 - Page 2
Practice 11-1: Installing and Configuring Apache as a Scalable
Service on Oracle Solaris Cluster
Overview
In this practice, you install and configure Apache as a scalable service on Oracle Solaris Cluster
software.
Preparation
prompt.

previous practices.
Task 1: Preparing for Apache Data Service Configuration

Perform the following steps on each node of the cluster:
1. Ensure that you are logged in as the oracle1 user on the cluster node.
2. Verify that the Solaris Cluster Apache data service software package is installed.
$ pkg info apache
3. Ensure that the /etc/inet/hosts file on the two cluster nodes and the Administrative
Workstation already contains an entry for the logical host name resource, cluster1-web.
$ cat /etc/inet/hosts | grep cluster1-web
192.168.1.121 cluster1-web
Task 2: Configuring the Apache Environment

Perform the following steps on any one-cluster node:
1. While logged in as the oracle1 user on the cluster node, assume the root role.
2. Make a resource-specific copy of the /usr/apache2/bin/apachectl script and edit it.
# mkdir /global/web/bin
# cp /usr/apache2/2.2/bin/apachectl /global/web/bin/
# chmod u+w /global/web/bin/apachectl
# vi /global/web/bin/apachectl
a. Add a line to make an Apache runtime directory. This directory is deleted every time
you reboot. Letting this script create it resolves the problem. Just add the line in bold as
the second line of the file.
#!/bin/sh
mkdir -p /var/run/apache2
b. Locate the line:

HTTPD='/usr/apache2/2.2/bin/httpd'
And change it to:
HTTPD='/usr/apache2/2.2/bin/httpd -f /global/web/conf/httpd.conf'
Chapter 11 - Page 3
3. Copy the sample /etc/apache2/httpd.conf-example file to the
/global/web/conf/httpd.conf file.
# mkdir /global/web/conf
# cp /etc/apache2/2.2/httpd.conf /global/web/conf/httpd.conf
4. Edit the /global/web/conf/httpd.conf file, and change the following entries, as

shown in the following table:
Old Entry New Entry
Listen 80 Listen cluster1-web:80
ServerName 127.0.0.1 ServerName cluster1-web
DocumentRoot "/var/apache2/2.2/htdocs" DocumentRoot "/global/web/htdocs"
<Directory "/var/apache2/2.2/htdocs"> <Directory "/global/web/htdocs">

ScriptAlias /cgi-bin/ ScriptAlias /cgi-bin/ "/global/web/cgi-
"/var/apache2/2.2/cgi-bin/" [one line] bin/" [one line]
<Directory "/var/apache2/2.2/cgi-bin"> <Directory "/global/web/cgi-bin">
Note: The changes mentioned in the table are shown in the order of their appearance in the
httpd.conf file, so you can search for the first place to change, change it, then search for
the next, and so on.
5. Create directories for the HTML and CGI files and populate with the sample files.
# cp -rp /var/apache2/2.2/htdocs /global/web
# cp -rp /var/apache2/2.2/cgi-bin /global/web
6. Copy the test-apache.cgi file from the classroom server to /global/web/cgi-bin.

# cp /opt/ora/lab/test-apache.cgi /global/web/cgi-bin/
You use the test-apache.cgi file to test the scalable service.

7. Make sure that test-apache.cgi can be executed by all users.
# chmod 755 /global/web/cgi-bin/test-apache.cgi
Task 3: Testing the Server on Each Node Before Configuring the Data Service
Resources
Perform the following steps on both the cluster nodes, but one node at a time:
1. While logged in as the oracle1 user on a cluster node, assume the root role.
2. Temporarily configure the logical shared address (on one node).
# ipadm create-addr -T static -a 192.168.1.121/24 net0/web
3. Start the server (on that node).

# /global/web/bin/apachectl start
Chapter 11 - Page 4
4. Verify that the server is running.
clnode1:/# ps –ef | grep apache2
webservd 4604 4601 0 10:20:05 ? 0:00 /usr/apache2/bin/httpd -f
/global/web/conf/httpd.conf -k start
root 4601 1 0 10:20:04 ? 0:01 /usr/apache2/bin/httpd -f
5. Connect to the Apache web server from the web browser on your Administrative

Workstation by using the following URL:
http://cluster1-web/
6. Stop the Apache web server.
# /global/web/bin/apachectl stop
7. Verify that the server has stopped.
# ps -ef | grep apache2
8. Delete the logical IP address.

# ipadm delete-addr net0/web
Task 4: Registering and Configuring the Apache Data Service

2. Register the resource type required for the Apache data service.
$ clrt register SUNW.apache
3. Create a failover resource group for the shared address resource. If you have more than
two nodes, you can include more than two nodes after the -n.
$ clrg create -n clnode1,clnode2 sa-rg
4. Add the SharedAddress logical host name resource to the resource group.
$ clrssa create -g sa-rg cluster1-web
5. Bring the failover resource group online.

$ clrg online -M sa-rg
6. Create a scalable resource group to run on all nodes of the cluster. (The example assumes
two nodes.)
$ clrg create -S -n clnode1,clnode2 web-rg
7. Add a storage resource to the scalable group.

$ clrs create -g web-rg -t SUNW.HAStoragePlus \
-p FilesystemMountPoints=/global/web -p AffinityOn=true web-stor
Chapter 11 - Page 5
8. Add an application resource to the scalable resource group.
$ clrs create -g web-rg -t SUNW.apache -p Bin_dir=/global/web/bin \
-p Scalable=true -p Resource_dependencies=cluster1-web,web-stor \
apache-res
9. Bring the scalable resource group online.

$ clrg online -M web-rg
Note: You may see an error that PIDFILE is not defined. It is defined, but as PIDFILE and
not as PidFile. This does not hamper normal operation.
10. Verify that the data service is online.
$ clrg status
$ clrs status

Task 5: Verifying Apache Web Server Access
Perform the following steps on the Administrative Workstation:
1. Open a web browser, such as Firefox.
2. Connect to the web server from a browser anywhere other than from a cluster node by
using http://cluster1-web/cgi-bin/test-apache.cgi.
3. Click the Refresh or Reload button repeatedly on the browser.
The test-apache.cgi script shows the name of the cluster node that is currently
servicing the request. The load balancing is not performed on a round-robin basis, so you
might see several consecutive requests serviced by the same node. Over time, however,
you should see the load balancing to be about 50-50.
Task 6: Observing Cluster Failures

Cause as many of the following failures (one at a time, and fix each one before going on to the
next one) as you can, given your physical access to the cluster.
Induce the following failure scenarios to observe the behavior of the scalable service:
• Fail a single public network interface on the node where the SharedAddress resource
is online.
• Fail all public network interfaces on that node.
• Reboot one of the nodes running the scalable service.
• Shut down one of the cluster nodes running the scalable service.
Task 7: Configuring Advanced Resource Group Relationships

In this task, you observe the effects of configuring variations of the RG_affinities property.
1. Ensure that you are logged as the oracle1 user on the cluster node.
2. If you have not yet done so, complete Task 6 titled “Making a Customized Application Fail
Over with a Generic Data Service Resource” of Practice 10-1.
3. Put your myapp-rg resource group offline.
$ clrg offline myapp-rg
Chapter 11 - Page 6
4. Create a strong negative affinity with the Apache resource group as the source and
myapp-rg as the target. This means that Apache categorically refuses to run on any node
where myapp-rg is running.
$ clrg set -p RG_affinities=--myapp-rg web-rg
5. Switch myapp-rg to any one-cluster node where Apache is running. Observe what
happens to the Apache data service. Also, observe the console messages.
$ clrg status web-rg
$ clrg switch -n apacheclusternode myapp-rg
$ clrg status
6. Switch myapp-rg to another node where Apache is running. Observe what happens to the
Apache data service. Does it come back online on the first node?
$ clrg switch -n apacheclusternode myapp-rg

$ clrg status
7. Switch myapp-rg offline. Can Apache come back online on both the cluster nodes? Now
remove the relationship.
$ clrg status
$ clrg set -p RG_affinities="" web-rg
8. Set a weak positive affinity so that myapp-rg (source) always prefers to run on the same
node as nfs-rg (target).
$ clrg set -p RG_affinities=+nfs-rg myapp-rg
9. Print out the value of the Nodelist for myapp-rg.

$ clrg show -p Nodelist myapp-rg
10. Switch your nfs-rg so that it is not on the preferred node of myapp-rg.
$ clrg switch -n nonpreferrednode nfs-rg
11. Bring myapp-rg online without specifying the node. Where does it end up? Why does it not
end up on the nonpreferred node of its own node list?
$ clrg online myapp-rg
$ clrg status
12. Switch myapp-rg so it is no longer on the same node as nfs-rg. Check whether this can
be done. Notice that a weak affinity is a preference and not a requirement.
$ clrg switch -n othernode myapp-rg
$ clrg status
13. Switch myapp-rg offline and now change the affinity to a strong positive affinity with
delegation. What is the difference between ++ and +++ (use +++)? The answer is a few
steps further on.
$ clrg set -p RG_affinities=+++nfs-rg myapp-rg
14. Bring myapp-rg online without specifying the node. Identify the node on which it becomes
online.
$ clrg online myapp-rg
$ clrg status
Chapter 11 - Page 7
15. Switch myapp-rg so that it is no longer on the same node as nfs-rg. Note what happens.
$ clrg switch -n othernode myapp-rg
$ clrg status
16. Switch the target group nfs-rg. Note what happens.

$ clrg switch -n othernode nfs-rg
$ clrg status
17. What happens if Resource Group Manager (RGM) wants to fail over the source myapp-rg
because its fault monitor indicates application failure?
Kill myappdaemon on whichever node it is running a few times (be patient with restarts) and
observe the console messages.
Chapter 11 - Page 8

Using Oracle Solaris Zones in
Chapter 12
Practices for Lesson 12: Using Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 1
Practice Overview
In this practice, you:

• Use the HA for Zones agent to create a failover zone using the solaris10 brand
• Create a zone cluster and re-create your scalable web application within the zone
cluster
• Configure the Apache scalable application, running in the global cluster, as a scalable
application in the zone cluster by reusing the Apache binaries of the global cluster.
Chapter 12 - Page 2
Practice 12-1: Running Failover Zones with HA for Zones and
solaris10 Branded Zones
Overview
In this practice, you use the HA for Zones agent to create a failover zone using the brand,
solaris10. Such a zone could even run on hardware where Oracle Solaris 10 is not natively
supported. The HA for Zones feature on Oracle Solaris 11 supports branded zones of type
solaris and solaris10.
Preparation
Before proceeding with the tasks in this practice, start Sun_ZFS_Storage_7000,
machines, one after the other. Wait for the virtual machines to boot properly until you see the

login prompt.
Before you begin this practice, please ensure that you have completed the tasks covered in the
previous practices.
Task 1: Verifying the Oracle solaris10 Branded Zone Support

Perform the following steps from within the master console window on the administrative
workstation:
2. Verify that the brand-solaris10 package is already installed on the cluster nodes.
$ pkg info brand-solaris10
Name: system/zones/brand/brand-solaris10
Summary: Solaris 10 Zones (solaris10 branded zones)
Description: Support for the Solaris 10 operating environment within a non-
global zone via the solaris10(5) brand.
Category: System/Virtualization
State: Installed
Publisher: solaris
Version: 0.5.11
Build Release: 5.11
Branch: 0.175.1.0.0.24.2
Packaging Date: September 19, 2012 06:53:02 PM
Size: 804.26 kB
FMRI: pkg://solaris/system/zones/brand/brand-solaris10@0.5.11,5.11-
0.175.1.0.0.24.2:20120919T185302Z
oracle1@clnode1:~$
If not, use the pkg installs command to install the brand-solaris10 package.
$ pfexec pkg install brand-solaris10
$ pkg info brand-solaris10
Chapter 12 - Page 3
Task 2: Verifying the HA for Zones Agent Support
workstation:
2. Verify that the ha-zones package is already installed on the cluster nodes.
# pkg info ha-zones
Name: ha-cluster/data-service/ha-zones
Summary: Oracle Solaris Cluster HA for Solaris Containers
Description: Oracle Solaris Cluster HA for Solaris Containers
Category: System/HA Cluster
State: Installed
Publisher: ha-cluster
Version: 4.1

Build Release: 5.11
Branch: 0.18
Packaging Date: October 1, 2012 08:38:48 PM
Size: 129.26 kB
FMRI: pkg://ha-cluster/ha-cluster/data-service/ha-zones@4.1,5.11-
0.18:20121001T203848Z
oracle1@clnode1:~$
If not, use the pkg install command to install the ha-zones package.
$ pfexec pkg install ha-zones
$ pkg info ha-zones
Task 3: Finding a Free Disk and Creating a Failover ZFS Pool and ZFS File
System for the Zonepath
Perform the following steps as the oracle1 user, unless otherwise specified:
In the master console window
1. Select an unused disk from shared storage for a new ZFS pool.
$ metaset
$ zpool status
In the host terminal window of any one cluster node
$ clq status
$ cldev list -v
Make sure that you do not use any disks already in use in existing device groups.
3. Create a failover ZFS pool and file system on the identified shared storage. Refer to the
listings in step 1.
# zpool create s10zonepool c#t#d#
# zfs create s10zonepool/s10zone
# zfs set mountpoint=/s10zone s10zonepool/s10zone
# chmod 700 /s10zone
Chapter 12 - Page 4
Task 4: Creating an Oracle solaris10 Branded Zone with Zonepath in the Shared
Storage
Perform the following steps only on the cluster node that currently mounts the s10zonepool
shared storage pool:

2. Identify the cluster node that currently mounts the s10zonepool shared storage pool.
$ df –kh
$ zpool list
$ zpool status

4. Create an Oracle Solaris 10 branded zone:
# zonecfg -z s10zone

Use 'create' to begin configuring a new zone.
zonecfg:s10zone> create -t SYSsolaris10
zonecfg:s10zone> set zonepath=/s10zone
zonecfg:s10zone> set autoboot=false
zonecfg:s10zone> set ip-type=shared
zonecfg:s10zone> add attr
zonecfg:s10zone:attr> set name=osc-ha-zone
zonecfg:s10zone:attr> set type=boolean
zonecfg:s10zone:attr> set value=true
zonecfg:s10zone:attr> end
zonecfg:s10zone> add net
zonecfg:s10zone:attr> set physical=sc_ipmp0
zonecfg:s10zone:attr> set address=192.168.1.122/24
zonecfg:s10zone:attr> end
zonecfg:s10zone> remove anet
zonecfg:s10zone> verify
zonecfg:s10zone> commit
zonecfg:s10zone> exit
5. Verify the zone configuration.
# zoneadm list -cv
ID NAME STATUS PATH BRAND IP
0 global running / solaris shared
- s10zone configured /s10zone solaris10 shared
6. Install the zone using a minimal Oracle Solaris 10 flash archive created on a physical
machine and provided for the course.
# zoneadm -z s10zone install –a /opt/ora/lab/s10flar/s10u10.flar –u
Progress being logged to /var/log/zones/zoneadm.xxx.s10zone.install
Installing: This may take several minutes...
Postprocessing: This may take a while...
Postprocess: Updating the image to run within a zone
Result: Installation completed successfully.

Log saved in non-global zone as
/s10zone/root/var/log/zones/zoneadm.xxx.s10zone.install
Chapter 12 - Page 5
7. Boot the zone.
# zoneadm -z s10zone boot
# zoneadm list -cv

3 s10zone running /s10zone solaris10 shared
8. Connect to the zone console and configure the zone.
# zlogin -C s10zone
[Connected to zone 'z10zone' console]
Navigate through the configuration screens. Select the default values for most of the
configuration options. Ensure that you get your terminal type correct. X Terminal
Emulator (xterms) seems to work better in VirtualBox environment.
•

Host name: s10zone
• Password: root_password
When you are finished with the system configuration of the zone, it will display the console
login prompt.
9. Disconnect from the zone console using ~. or ~~. key combinations.
Task 5: Creating a Logical Host Name Entry for the Zone

Perform the following steps on all the cluster nodes, one at a time:
1. While you are logged in as the oracle1 user on the cluster nodes, assume the root role.
2. Create a logical host name entry for the s10zone zone in the /etc/inet/hosts file.
# vi /etc/inet/hosts
192.168.1.123 s10zone-lh
Using the logical host name for a failover zone is optional, but will allow the zone to failover
if there is public network failure on a cluster node.
Task 6: Creating a Skeleton Resource Group for the Failover Zone

You need to configure a resource group for the failover zone storage and optional logical host
name before you can configure the failover zone resource.
Perform the following steps only on the cluster node that currently mounts the s10zonepool
shared storage pool:
2. Identify the cluster node that currently mounts the s10zonepool shared storage pool.
$ df –kh
$ zpool status
Chapter 12 - Page 6
3. On the identified cluster node, create the skeleton resource group and its resources and
ensure that they are online.
$ clrg create -n clnode1,clnode2 s10zone-rg
$ clrslh create -g s10zone-rg s10zone-lh

$ clrs create -g s10zone-rg -t SUNW.HAStoragePlus \
-p Zpools=s10zonepool s10zone-stor
$ clrg online -M -n node_you_are_typing_on s10zone-rg
$ clrg status s10zone-rg
Task 7: Halting the Zone and Replicating the Zone Configuration on the Other
Cluster Node
Perform the following steps only on the cluster node that is hosting the s10zone zone and the
s10zone-rg resource group:

2. Identify the cluster node that is hosting the s10zone zone and s10zone-rg resource
group.

4. Halt the running zone on the identified node.
# zoneadm -z s10zone halt
# zoneadm list -cv
- s10zone installed /s10zone solaris10 shared
5. Copy the file /etc/zones/s10zone.xml to the other storage node that will host the
failover zone.
6. Copy the single-line entry for the s10zone zone in the /etc/zones/index file to the
same file on other cluster node that will host the failover zone.
Following is a sample entry of the s10zone in the /etc/zones/index file:
s10zone:installed:/s10zone:97606842-c887-465d-a9cc-ee8f6d2a9be1
Task 8: Switching the Skeleton Resource Group to the Other Cluster Node
Perform the following steps on the cluster node that is hosting the s10zone zone and the
s10zone-rg resource group:
2. Switch the resource group to the other cluster node for the failover zone:
$ clrg switch -n other_node s10zone-rg
3 On the new node now hosting the s10zone-rg resource group, assume the root role.
Chapter 12 - Page 7
4. Start the s10zone zone and verify its installation and configuration.
# zoneadm -z s10zone boot
# zlogin s10zone
# uname –a
# cat /etc/release
# exit
5. Halt the s10zone zone:

# zoneadm list -cv

Task 9: Creating the Resource to Control the Failover Zone
Perform the following steps on the cluster node that is currently hosting the s10zone zone and
the s10zone-rg resource group:
2. Configure the zone resource using the template. The added/edited values are highlighted
and can be reproduced literally:
# cd /opt/SUNWsczone/sczbt/util
# vi sczbt_config
.
.
RS=s10zone-rs
RG=s10zone-rg
PARAMETERDIR=/s10zone/params
SC_NETWORK=true
SC_LH=s10zone-lh
FAILOVER=true
HAS_RS=s10zone-stor
.
.
Zonename="s10zone"
Zonebrand="solaris10"
Zonebootopt=""
Milestone="svc:/milestone/multi-user-server"
LXrunlevel="3"
SLrunlevel="3"
Mounts=""
3. Create a directory in the shared storage to store the parameter file mentioned in the
configuration file above (this is in the failover storage):
# mkdir /s10zone/params
Chapter 12 - Page 8
4. Create and enable the zone resource:
# ./sczbt_register
# cd
# clrs enable s10zone-rs

# clrs status s10zone-rs
# zlogin s10zone
# ifconfig -a
# exit
You should now see the s10zone-lh IP address automatically moved into the zone.
5. Halt the s10zone zone:
# zoneadm list –cv

Task 10: Investigating Node Failures and Zone Failures with the Failover Zone
Cause as many of the following failures as you can, given your physical access to the cluster.
Ensure that you induce the failure only one at a time, and fix each one before going on to induce
the next failure.
Induce the following failure scenarios to observe the behavior of the failover branded zone:
• Crash or reboot the physical node hosting the failover zone.
• Manually halt the failover zone when it is under control of the cluster.
• Cause a public network outage on the node hosting the zone, if possible.
Note: In the VirtualBox delivery environment, some amount of patience is required
when you cause a public network outage on a cluster node by simulating disconnection
of network cable, as it takes a while for the cluster node to recognize network failure
and to take corrective action.
Chapter 12 - Page 9
Practice 12-2: Building a Zone Cluster
Overview
In this practice, you create a zone cluster and re-create your scalable Web application within the
zone cluster.
Preparation
login prompt.
previous practices.

Task 1: Creating Static Addresses for Zone Cluster Nodes

On all cluster nodes, edit the /etc/inet/hosts file as the root role and provide static IP
addresses on your public subnet for the two nodes (zones) of the zone cluster.
192.168.1.140 zcnode1
192.168.1.141 zcnode2
Task 2: Configuring a Zone Cluster

Perform the following steps on the specified cluster node to configure a zone cluster:
1. Assume the root role on a cluster node, such as clnode1.
Chapter 12 - Page 10
2. Create a zone cluster, myzc.
# clzc configure myzc
myzc: No such zone cluster is configured.
Use 'create' to begin configuring a new cluster zone

clzc:myzc> create
clzc:myzc> set zonepath=/zones/myzc
clzc:myzc> add node
clzc:myzc:node> set physical-host=clnode1
clzc:myzc:node> set hostname=zcnode1
clzc:myzc:node> add net
clzc:myzc:node:net> set address=zcnode1
clzc:myzc:node:net> set physical=sc_ipmp0
clzc:myzc:node:net> end

clzc:myzc:node> end
clzc:myzc> add node
clzc:myzc:node> set physical-host=clnode2
clzc:myzc:node> set hostname=zcnode2
clzc:myzc:node> add net
clzc:myzc:node:net> set address=zcnode2
clzc:myzc:node:net> set physical=sc_ipmp0
clzc:myzc:node:net> end
clzc:myzc:node> end
clzc:myzc> commit
clzc:myzc> exit
3. Verify the zone cluster configuration.
# clzc verify myzc
Task 3: Installing the Zone Cluster

Perform the following steps on the specified cluster node to install a zone cluster:
1. Ensure that you have assumed the root role on both the global cluster nodes.
2. Create a zone cluster, myzc using the clzc install command.
The -v option shows the output of the underlying zoneadm command on the console of
each cluster node:
# clzc status myzc
# clzc install -v myzc
Waiting for zone install commands to complete on all the nodes of the zone
cluster "myzc"...
// zoneadm output for installation appears on the physical node consoles only
# clzc status myzc
Note: The zone cluster installation could take anywhere from 30-45 minutes.
Task 4: Booting the Zone Cluster
From any global cluster node, as a root role, boot all zones in the zone cluster.
Note: The zone name (from the point of view of the zone commands in the global zone) will
be the same as the name of the zone cluster (myzc), on all physical nodes.
# clzc boot myzc
Waiting for zone boot commands to complete on all the nodes of the zone
cluster "myzc"...
# clzc status myzc
Task 5: Accessing the Zone Consoles

In this task, you will access the zone console and customize your zone.
Perform the following steps on each global cluster node, one at a time.

1. Ensure that you have assumed the root role on each global cluster node.
2. Connect to the zone console on each global cluster node.
# zlogin -C myzc
You still see the zone doing its first boot. Navigate through the System Configuration Tool
screens and specify the following values, when asked for:
• Computer name: zcnode1/zcnode2
• Do not configure DNS: Select
• Name Service: None
• Time Zone: timezone
• Root password: 2secure
• Username: oracle1
• User password: oracle1
When you are finished with the system configuration of each zone cluster node, it will
display the console login prompt.
zcnode console login:
3. Log in to each zone cluster node as the oracle1 user and assume the root role.
4. Set your zone cluster environment:
a. Create a better login environment.
# vi ~/.profile
...
export PATH=/usr/cluster/bin:/usr/bin:/usr/sbin
...
# . .profile
# cd
b. Make any other desired customizations (for example, change your shell if you want, but
do it carefully).
5. While still logged in to each zone cluster node, explore the zone cluster environment from
the point of view of the zone:
# uname -a
# cat /etc/inet/hosts
# clnode status
# clq status // see an error: You cannot run this command from zone cluster
# clrg status
# clrs status
# clrt list –v
6. Exit from the zone console using ~. or ~~. key combinations.
Task 6: Exploring the Zone Cluster from the Global Zone Point of View
Perform the following steps on the global cluster node:

1. Ensure that you are logged as the oracle1 user on the global cluster node, not the zone
cluster node.
2. Explore various cluster resource and resource group status options:
$ clnode status
$ clnode status -Z myzc
$ clnode status -Z all
$ clrg status
$ clrg status -Z myzc
$ clrg status -Z all
$ clrs status
$ clrs status -Z myzc
$ clrs status -Z all
$ clrs list -v
$ clrs list -v -Z myzc
$ clrs list -v -Z all
$ clrt list -v
$ clrt list -v -Z myzc
$ clrt list -v -Z all
Task 7: Experimenting with Zone Cluster Failures

Cause as many of the following failures as you can, given your physical access to the cluster.
Ensure that you induce the failure only one at a time, and fix each one before going on to induce
the next failure.
Induce the following failure scenarios to observe the behavior of the zone cluster:
• Halt a zone in a zone cluster.
• Reboot a zone in a zone cluster.
• Halt or crash a physical node hosting a zone in a zone cluster.
• Reboot a cluster node hosting a zone in a zone cluster.
Practice 12-3: Configuring a Scalable Application in a Zone Cluster
Overview
In this practice you configure the Apache scalable application, running in the global cluster, as a
scalable application in the zone cluster by reusing the Apache binaries of the global cluster.
Preparation
login prompt.
previous practices.

Task 1: Deleting the Scalable Apache Data Service from the Global Cluster
Perform the following steps on a global cluster node:
1. Ensure that you are logged in as the oracle1 user on the global cluster node.
2. From any node of the global cluster (logged into the physical node, not the zone), delete the
Apache resource groups and all their contained resources:
$ clrg delete -F web-rg
$ clrg delete -F sa-rg
Task 2: Adding the Cluster File System to a Zone Cluster

Perform the following steps to add a cluster file system for Apache web server data service in
the zone cluster:
1. Ensure that you are logged in as the oracle1 user on any one global cluster node and
assume the root role.
2. Add the /zonecfs/web-lofs highly available local file system for use by the myzc zone
cluster. Ensure that the Oracle Solaris Cluster does not manage the loopback file system
during the initial zone cluster configuration by setting the value of the cluster-control
property to false.
clzc:myzc> add net
clzc:myzc:net> set address=web-lofs-lh
clzc:myzc:net> end
clzc:myzc> add fs
clzc:myzc:fs> set dir=/zonecfs/web-lofs
clzc:myzc:fs> set special=/global/web
clzc:myzc:fs> set type=lofs
clzc:myzc:fs> set cluster-control=false

clzc:myzc:fs> end
clzc:myzc> verify
clzc:myzc> commit
clzc:myzc> exit
$ clzc show –v myzc
3. Assume the root role and log in to the myzc zone.

# zlogin myzc
4. Create the /zonecfs/web-lofs directory on both the zone cluster nodes.

# mkdir –p /zonecfs/web-lofs
5. Ensure that the /etc/inet/hosts file on both the zone cluster nodes include entries for
the two zone clusters and for web-lofs-lh.
# cat /etc/inet/hosts
...
192.168.1.140 zcnode1
192.168.1.141 zcnode2
...
192.168.1.124 web-lofs-lh
...
6. Exit the root role and then reboot the myzc zone cluster from any one global cluster node.
# clzc reboot myzc
Task 3: Configuring the Apache Web Server Environment in the Zone Cluster
Perform the following steps on any one zone cluster node:
1. Ensure that you are logged in as the root role on a zone cluster node.
# zlogin myzc
2. Edit the HTTPD entry in the /zonecfs/web-lofs/bin/apachectl script:

HTTPD='/usr/apache2/2.2/bin/httpd -f /global/web/conf/httpd.conf'
And change it to:
HTTPD='/usr/apache2/2.2/bin/httpd -f /zonecfs/web-lofs/conf/httpd.conf'
3. Edit the /zonecfs/web-lofs/conf/httpd.conf file, and change the following entries,
as shown in the following table:
Old Entry New Entry
Listen cluster1-web:80 Listen web-lofs-lh:80

ServerName cluster1-web ServerName web-lofs-lh
DocumentRoot "/global/web/htdocs" DocumentRoot "/zonecfs/web-lofs/htdocs"
<Directory "/global/web/htdocs"> <Directory "/zonecfs/web-lofs/htdocs">
ScriptAlias /cgi-bin/ /global/web/cgi- ScriptAlias /cgi-bin/ "/zonecfs/web-
bin/" [single line] lofs/cgi-bin/" [single line]
<Directory "/global/web/cgi-bin"> <Directory "/zonecfs/web-lofs/cgi-bin">
Note: The changes mentioned in the table above are shown in the order of their

appearance in the httpd.conf file, so you can search for the first place to change,
change it, then search for the next, and so on.
4. Exit the zone cluster node to return to the global cluster node.
5. Change the myzc zone cluster configuration to ensure that Oracle Solaris Cluster now
manages the loopback file system by setting the value of the cluster-control property
to true.
clzc:myzc> select fs dir=/zonecfs/web-lofs
clzc:myzc:fs> set cluster-control=true
clzc:myzc:fs> end
clzc:myzc> verify
clzc:myzc> commit
clzc:myzc> exit
# clzc show –v myzc
6. Reboot the myzc zone cluster.

# clzc reboot myzc
Task 4: Creating a Scalable Apache Data Service in the Zone Cluster

Perform the following steps on any one zone cluster node:
1. Log in as the root role on a zone cluster node.
# zlogin myzc
2. Register resource types in the zone cluster:
# clrt list -v
# clrt register SUNW.apache
# clrt register SUNW.HAStoragePlus
# clrt list -v
3. Create a scalable resource group containing the shared address:
# clrg create -n zcnode1,zcnode2 sa-rg
# clrssa create -g sa-rg web-lofs-lh
# clrg online -M sa-rg
# clrg status
# clrs status
4. Create a scalable group containing the Apache resource:
# clrg create -S -n zcnode1,zcnode2 web-rg
# clrs create -g web-rg -t SUNW.HAStoragePlus \
-p FilesystemMountPoints=/zonecfs/web-lofs -p AffinityOn=false web-stor

# clrg online -M web-rg
# clrg status
# clrs status
# clrs create -g web-rg -t SUNW.apache -p Bin_dir=/zonecfs/web-lofs/bin \
-p Scalable=true -p Resource_dependencies=web-lofs-lh,web-stor apache-res
# clrg status
# clrs status
Task 5: Verifying That the Web Server Works

Perform the following steps on the Administrative Workstation:
1. Open a Web browser, such as Firefox.
2. Navigate to http://web-lofs-lh/cgi-bin/test-apache.cgi
3. Click the reload or refresh button several times to verify the behavior of the scalable
application.
Practice 12-4: Integrating Oracle 11g into Solaris Cluster Software as
a Failover Application (Optional)
Overview
In this practice, you integrate Oracle 11g into Solaris Cluster software as a failover application.
In this practice, you complete the following tasks:
• Task 1: Creating a Logical IP Entry in the /etc/hosts File
• Task 2: Creating oracle user and required group accounts
• Task 3: Setting system parameters (On both nodes as root role)
• Task 4: Creating a Shared Storage File System (SVM or ZFS pool)
• Task 5: Preparing the oracle User Environment

• Task 6: Download and Install packages
• Task 7: Installation of required packages
• Task 8: Disabling Access Control of X Server on the Admin Workstation
• Task 9: Running the runInstaller, netca, and dbca Installation Script
• Task 10: Preparing an Oracle Instance for Cluster Integration
• Task 11: Registering the resource types
• Task 12: Creating Resources and Resource Groups for Oracle
• Task 13: Verifying That Oracle Runs Properly in the Cluster
Preparation
The Oracle database installation is very time consuming and could take anywhere between 120
minutes and 150 minutes. Therefore, to save time, you are given access to a new set of virtual
machines with pre-installed Oracle database using SVM only (Refer Task 4 of this practice).
Before proceeding with the tasks in this practice, perform the following steps:
• Power-off the following virtual machines:
• Sun_ZFS_Storage_7000
• Cluster_Node_1
• Cluster_Node_2
• Boot the following virtual machines in the given order, one after the other:
• Sun_ZFS_Storage_7000_DB
• Cluster_Node_1_DB
• Cluster_Node_2_DB
Task 1 to Task 9 of this practice is already performed for you to save the time and to speed up
the lab. However, the said tasks are documented here for your reference purpose and you just
need to verify them before proceeding to the next step.
You will start integrating the Oracle database instance with cluster from Task 10 onwards.
If you wish to install the Oracle database using ZFS, follow all the instructions from Task 1 to
Task 13 on the following VM’s:
• Sun_ZFS_Storage_7000
• Cluster_Node_1
• Cluster_Node_2
Note: For Task 4, only perform Step 2 to install the Oracle database using ZFS
Task 1: Creating a Logical IP Entry in the /etc/hosts File

Assume the root role and create an entry for a logical IP address for Oracle in the

/etc/hosts file on all the cluster nodes:
...
192.168.1.126 ora-lh
Note: This task is already performed for you.
Task 2: Creating oracle user and required group accounts

To create the oracle user, dba and oinstall group accounts, perform the following steps
from within the master console window on the administrative workstation:
1. Create dba and oinstall accounts on both the cluster nodes.
# groupadd dba
# groupadd oinstall
2. Create the oracle user on both the cluster nodes.
# useradd -d /export/home/oracle -m -s /bin/bash -g dba \
-G oinstall oracle
80 blocks
3. Create the oracle user password on both the cluster nodes.
# passwd oracle
New Password: oracle1
Re-enter new Password: oracle1
passwd: password successfully changed for oracle
#
4. Provide administrative access to the oracle user by assuming root role on both the
cluster nodes.
# usermod -R root oracle
# usermod -P +"System Administrator" oracle
5. Verify the administrative access provided for the oracle user on both the cluster nodes.
# su - oracle
Oracle Corporation SunOS 5.11 11.1 September 2012
oracle@clnode1:~$ roles
root
oracle@clnode1:~$ exit
Task 3: Setting system parameters (On both nodes as root role)

workstation as root role:
1. Configure system kernel parameters for the oracle user.

# projmod -U oracle -K "project.max-shm-memory=(priv,6g,deny)" default
# projmod -sK "project.max-sem-nsems=(priv,512,deny)" default
# projmod -sK "project.max-sem-ids=(priv,128,deny)" default
# projmod -sK "project.max-shm-ids=(priv,128,deny)" default
# projects –l default
2. Set the system swap space.
Oracle database installation requires swap space of up to 5GB.
Increase the swap space to meet the Oracle database installation requirement:
# swap -lh
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 274,2 4K 1.0G 1.0G
# swap -d /dev/zvol/dsk/rpool/swap
# zfs volsize=5G rpool/swap
# swap -a /dev/zvol/dsk/rpool/swap
# swap -lh
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 274,2 4K 5.0G 5.0G
#
Task 4: Creating a Shared Storage File System (SVM or ZFS Pool)

You can use either SVM or ZFS pool to create a shared storage file system for the Oracle
database software.
Note: This task is already performed for you using SVM.
Perform any one of the following two steps of this task:
1. Creating a shared storage file system for Oracle database using SVM
Perform the following steps on any node physically connected to the storage. This lab
assumes that local metadb’s have been created already.
Just to speed up the lab, we will create unmirrored volumes. You can always add another
disk and mirror the parent volume later at your leisure, or pretend that you were going to.
a. Verify that the metadb replicas are configured and operational on both the cluster
nodes.
root@clnode1:~# metadb
flags first blk block count

a u 16 8192 /dev/dsk/c7t0d0s6
a u 8208 8192 /dev/dsk/c7t0d0s6
a u 16400 8192 /dev/dsk/c7t0d0s6
root@clnode1:~#
b. Select a disk from shared storage. Make sure that you do not use any disks already in
use in existing device groups.
root@clnode1:~# cldev list -v
. . .

d14 clnode2:/dev/rdsk/ c0t600144F0AE8319BA0000508E37DC000Ed0
. . .
root@clnode1:~#
Note: For the purpose of this practice, select d14 shared disks for orads disk set.
c. Create an orads diskset using the shared disk identified in the previous step.
While logged in as the oracle1 user on the clnode1 cluster node, assume the root
role.
root@clnode1:~# metaset -s orads -a -h clnode1 clnode2
root@clnode1:~# metaset -s orads -a /dev/did/rdsk/d14
d. Create a soft partition (d200) of a mirror volume for Oracle. (SVM still calls it a mirror,
although we only have one sub-mirror. This gives us flexibility to add the other half of
the mirror at our leisure later.)
root@clnode1:~# metainit -s orads d21 1 1 /dev/did/rdsk/d14s0
orads/d21: Concat/Stripe is setup
root@clnode1:~# metainit -s orads d20 -m d21
orads/d20: Mirror is setup
root@clnode1:~# metainit -s orads d200 -p d20 8g
d200: Soft Partition is setup
root@clnode1:~#
e. Create a cluster file system
root@clnode1:~# newfs /dev/md/orads/rdsk/d200
f. Create a mount point and an entry in /etc/vfstab on all the nodes:
root@clnode1:~# mkdir /oracledb
root@clnode1:~# vi /etc/vfstab
. . .
/dev/md/orads/dsk/d200 /dev/md/orads/rdsk/d200 /oracledb ufs \
2 yes global
root@clnode1:~#
g. On the node on which you created the storage, mount the file system and modify its
ownership:
root@clnode1:~# mount /oracledb
root@clnode1:~# chown oracle:dba /oracledb

h. Verify that the file system is mounted and available on both the the cluster nodes
# mount | grep /oracledb
# ls /oracledb
lost+found
# df -kh | grep /oracledb
2. Create a shared storage file system for an Oracle database using ZFS
Perform the following steps on any one node physically connected to the storage.
To simplify, we will create a ZFS storage pool with one disk. You can always add another

disk and mirror the existing disk later.
a. Select a disk from shared storage. Make sure that you do not use any disks already in
use in existing device groups.
# zpool status
# metaset
# cldev list -v
. . .
. . .
#
Note: For the purpose of this practice, select d14 shared disks for to create oraclepool.
b. Create a ZFS storage pool and a file system for Oracle. Verify that your file system is
mounted and change its permissions:
root@clnode1:~# zpool create oraclepool c0t600144F0AE8319BA0000508E382B0010d0
root@clnode1:~# zfs create -o mountpoint=/oracledb oraclepool/orafs
root@clnode1:~# df -h
root@clnode1:~# chown oracle:dba /oracledb
Task 5: Preparing the oracle User Environment

To configure environment variables required for the Oracle software, run the following
commands from both the cluster nodes:
1. Switch to the oracle user:
root@clnode1:~# su - oracle
Oracle Corporation SunOS 5.11 11.1 September 2012
oracle@clnode1:~$ pwd
/export/home/oracle
oracle@clnode1:~$
2. Edit the .profile file with the following entries.
oracle@clnode1:~$ vi .profile
...
...
ORACLE_BASE=/oracledb
ORACLE_HOME=$ORACLE_BASE/11.3/db_1
ORACLE_UNQNAME=orcl
ORACLE_SID=orcl
PATH=$PATH:/usr/openwin/bin:$ORACLE_HOME/bin
export ORACLE_BASE ORACLE_HOME ORACLE_UNQNAME ORACLE_SID PATH
...

oracle@clnode1:~$ . .profile
oracle@clnode1:~$ exit
Task 6: Download the packages for Oracle Database installation

1. While logged in as the oracle1 user on the clnode1, assume the root role.
2. Create a directory oracledb11g under and change the ownership as follows assuming
root role.
root@clnode1:~# cd
root@clnode1:~# mkdir /oracledb11g
3. Copy the oracle database files from the directory /opt/ora/oracledb11g/ to
/oracledb11g as follows:
root@clnode1:~# cp /opt/ora/lab/oracledb11g/p10404530_112030_Solaris86-
64_1of6.zip /oracledb11g/
root@clnode1:~# cp /opt/ora/lab/oracledb11g/p10404530_112030_Solaris86-
64_2of6.zip /oracledb11g/
4. Unzip the copied files.
root@clnode1:~# cd /oracledb11g
root@clnode1:/oracledb11g# unzip p10404530_112030_Solaris86-
64_1of6.zip
root@clnode1:/oracledb11g# unzip p10404530_112030_Solaris86-
64_2of6.zip
root@clnode1:/oracledb11g# chown -R oracle:dba database
root@clnode1:/oracledb11g# ls
database
p10404530_112030_Solaris86-64_1of6.zip
p10404530_112030_Solaris86-64_2of6.zip
root@clnode1:/oracledb11g# exit
Task 7: Installation of required packages
workstation as root role to verify and install the required packages.

1. Verify the packages.
# pkginfo -i SUNWxwplt SUNWmfrun SUNWarc SUNWhea SUNWlibm xclock
system SUNWarc Lint Libraries (usr)
system SUNWhea SunOS Header Files
system SUNWlibm Math & Microtasking Library Headers & Lint Files
(Usr)
ERROR: information for "SUNWxwplt" was not found
ERROR: information for "SUNWmfrun" was not found

ERROR: information for "xclock" was not found
#
2. Install the missed packages identified in the step 1.
# pkg install compatibility/packages/SUNWxwplt
# pkg install SUNWmfrun
# pkg install xclock
# pkginfo -i SUNWxwplt SUNWmfrun xclock
Task 8: Disabling Access Control of X Server on the Admin Workstation

1. To allow client GUIs to be displayed, run the following command on the admin workstation:
oracle1@adminws:~$ /usr/openwin/bin/xhost +
access control disabled, clients can connect from any host
2. From the admin workstation login as oracle user to verify client GUI as mentioned below
on both the nodes.
$ ssh –X oracle@clnode1
Passwd:
$ xclock
Note: Ensure that the clock appears on the screen.
Task 9: Running the runInstaller Installation Script

1. From the admin workstation login to the clnode1 as oracle user and ensure that the
xclock appears:
passwd:
$ xclock
2. Change directory to the location of the Oracle software:
oracle@clnode1:~$ cd /oracledb11g/database
3. Run the runInstaller script:
oracle@clnode1:/oracledb11g/database$ ./runInstaller &
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 180 MB. Actual 3753 MB
Passed
Checking swap space: must be greater than 150 MB. Actual 4183 MB
Passed
Checking monitor: must be configured to display at least 256 colors.
Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from
/tmp/OraInstall2012-11-21_08-23-32AM. Please wait ..
. . .
4. Respond to the dialog boxes.

Step Window/Page Choices or Values
Description
a. Configure Security Unselect I wish to receive security
Update update via My Oracle Support and click
Next.
Click Yes in the My Oracle Support
Username/Email Address Not Specified
dialog box
b. Download Software Select Skip software updates and click
Updates Next
c. Select installation Option Select Install database software only
and click Next
d. Grid Installation Option Select Single instance database
installation and click Next
e. Product Languages Select English and click Next
f. Database Edition Select Standard Edition One (4.6GB) and
click Next
g. Installation Location Ensure Oracle Base is set as /oracledb
Ensure Software Location is set as
/oracledb/11.3/db_1 and click Next
h. Create Inventory Change the Inventory Directory from
/oraInventory to
/oracledb/oraInventory by Browse and
click Next
Click Yes to ignore the message in the [INS-
32055] The central Inventory is
located in the Oracle base dialog box,
and then click Next.
i. Operating system groups Ensure dba in the Database Administrator
(OSDBA) Group and click Next
Step Window/Page Choices or Values
Description
j. Perform Prerequisite The tests will either all succeed or you will be
Checks warned about swap space requirements.
Click Next, and if you get a pop-up warning
window, click Yes to proceed.
k. Summary Save the db.rsp file and click Install
l. Install Product Installation starts and it will take around 30 to 45

minutes depending on the system configuration.
Installation time varies from system to system.
Open a terminal window as user root on the
installer node (clnode1) and run the script:

/oracledb/oraInventory/orainstRoot.sh
Now run
/oracledb/11.3/db_1/root.sh
Give /usr/bin as the default path name for the
local bin directory.
Click OK in the script prompt window.
m. Finish Click Close to complete the installation
5. Run netca on any one node (clnode1)

a. From admin work station login to the clnode1 as oracle user and run the netca:
passwd:
$ netca &
b. Respond to the netca dialog boxes
1. Select Listener configuration and click Next
2. Select Add and click Next
3. Type LISTENER in the Listener name text box and click Next
4. Ensure TCP is the selected protocol and click Next
5. Select the Use the standard port number of 1521 radio button and click
Next
6. Select No when asked to configure another listener and click Next
7. Click Next and Finish
6. Run dbca on any one node (clnode1)
Caution: While working inside VirtualBox environment, we observe that the cluster
nodes behave absurdly and many a time reports short of memory. We advise you to
reboot the two cluster nodes along with the storage appliance node before starting the
dbca script. In such a case, after reboot, you will observe that the oraclepool pool
goes into FAULTED state. Run the following commands as the root role to bring the
state of oraclepool pool to ONLINE and to mount the oraclepool/orafs file
system:
# zpool clear oraclepool
# zfs mount oraclepool/orafs
a. From admin work station login to the clnode1 as oracle user and run the dbca:

passwd:
$ dbca &
b. Respond to the dbca dialog boxes
1. Click Next on the welcome screen
2. Select Create a Database and click Next
3. Select General Purpose or Transaction Processing and click Next
4. Type orcl in Global Database Name and SID text boxes and click Next
5. Unselect Configure Enterprise manager option and click Next
6. Select Use the Same Administrative Password for All Accounts
and give the password as India1947
7. Ensure that the Use Database File Locations from Template option is
selected and click Next
8. Unselect Specify Fast Recovery Area option and click Finish
9. Accept default values (Sample Schemas) and click Next
10. Accept default values (Memory, Sizing, Character set, connection mode) and click
Next
11. Click Next on Database Storage screen
12. Select Create Database and click Finish
13. Verify Database summary on Confirmation screen and click OK
14. Database Configuration Assistant (on clnode1) will start installation. After the
installation click Exit.
Note: Installation may take 30 to 45 minutes depending on the system configuration.
Task 10: Preparing an Oracle Instance for Cluster Integration
On the same node from which the Oracle software was installed, perform the following steps as
oracle user to prepare the Oracle instance for Solaris Cluster software integration:
1. Configure an Oracle user for the fault monitor (as user oracle):
$ sqlplus /nolog
SQL> connect / as sysdba
SQL> startup
SQL> create user sc_fm identified by sc_fm;
SQL> grant create session, create table to sc_fm;
SQL> grant select on v_$sysstat to sc_fm;
SQL> alter user sc_fm default tablespace users quota 1m on users;
SQL> quit

2. Test the new Oracle user account used by the fault monitor (as user oracle):
$ sqlplus sc_fm/sc_fm
SQL> select * from sys.v_$sysstat;
SQL> quit
3. Create a sample table (as user oracle). You can be creative about your data values:
$ sqlplus /nolog
SQL> create table mytable (Name VARCHAR(20), age NUMBER(4));
SQL> insert into mytable values ('vincent', 14);
SQL> insert into mytable values ('theo', 24);
SQL> commit;
SQL> select * from mytable;
SQL> quit
Note: This sample table is created only for a quick verification that the database is running
properly.
4. Shut down the Oracle instance:
$ sqlplus /nolog
SQL> shutdown immediate
SQL> quit
5. Stop the Oracle listener:
$ lsnrctl stop
6. Configure the Oracle listener (as user oracle):
a. Edit the $ORACLE_HOME/network/admin/listener.ora file
$ vi $ORACLE_HOME/network/admin/listener.ora
b. Add the following lines at the top:
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(SID_NAME = orcl)
(ORACLE_HOME = /oracledb/11.3/db_1)
(GLOBAL_DBNAME = orcl)
)
)
c. Modify the HOST variable to match logical host name ora-lh.
LISTENER =
(DESCRIPTION_LIST =

(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = ora-lh)(PORT = 1521))
)
)
ADR_BASE_LISTENER = /oracledb
7. Configure the tnsnames.ora file (as user oracle):
Modify the HOST variable to match logical host name ora-lh of the
$ORACLE_HOME/network/admin/tnsnames.ora.
oracle@clnode1:~$ vi /oracledb/11.3/db_1/network/admin/tnsnames.ora
# tnsnames.ora Network Configuration File:
/oracledb/11.3/db_1/network/admin/tnsnames.ora
# Generated by Oracle configuration tools.
ORCL =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = ora-lh)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = orcl)
)
)
oracle@clnode1:~$
Task 11: Registering the resource types

1. Registering the SUNW.HAStoragePlus Type
If necessary (if not done in a previous practice), register the SUNW.HAStoragePlus type
from any one node in the cluster:
# clrt register SUNW.HAStoragePlus
2. Register the SUNW.oracle_server and SUNW.oracle_listener resource types from
one cluster node:
# clrt register SUNW.oracle_server
# clrt register SUNW.oracle_listener
Task 12: Creating Resources and Resource Groups for Oracle

Perform the following steps from the node that owns the Oracle storage as oracle1 user, to
create the resource group and resources necessary for the Oracle data service:
1. Create an empty resource group:
# clrg create -n clnode1,clnode2 ora-rg
2. Add a LogicalHostname resource:
# clrslh create -g ora-rg ora-lh

3. Add an HAStoragePlus resource for Oracle database software.
For SVM:
# clrs create -g ora-rg -t HAStoragePlus \
-p FilesystemMountPoints=/oracledb \
-p AffinityOn=true ora-stor
For ZFS:
# clrs create -g ora-rg -t HAStoragePlus -p Zpools=oraclepool ora-stor
4. Put the resource group online (required by the dependency specification in the next step):
# clrg online -M ora-rg
5. Create an oracle_server resource by creating and running a script. (The command is too
long to type interactively in some shells.):
a. Write a script containing the clrs command:
# vi /var/tmp/setup-ora-res.sh
#!/bin/sh
/usr/cluster/bin/clrs create -g ora-rg \
-t oracle_server \
-p Resource_dependencies=ora-stor \
-p ORACLE_SID=orcl -p ORACLE_HOME=/oracledb/11.3/db_1 \
-p Alert_log_file=/oracledb/diag/rdbms/orcl/orcl/trace/alert_orcl.log\
-p Connect_string=sc_fm/sc_fm \
-p START_TIMEOUT=3600 \
ora-server-res
#
b. Run the script:
# chmod 700 /var/tmp/setup-ora-res.sh
# sh /var/tmp/setup-ora-res.sh
6. Create an oracle_listener resource:
# clrs create -g ora-rg -t oracle_listener \
-p ORACLE_HOME=/oracledb/11.3/db_1 \
-p Listener_name=LISTENER \
-p Resource_dependencies=ora-stor \
ora-listener-res
7. Verify the application:
# clrg status
# clrs status
# ps -ef | grep oracle
Task 13: Verifying That Oracle Runs Properly in the Cluster

Perform the following steps as indicated to verify that the Oracle software properly runs in the
cluster:
1. Switch user to oracle user on the cluster node currently running the ora-rg resource group:
# su - oracle
2. Query the database from the cluster node currently running the ora-rg resource group:
$ sqlplus /nolog
SQL> quit
3. As user root (from any cluster node), switch the resource group to another node:
# clrg switch -n other-node ora-rg
4. Query the database from the cluster node currently running the ora-rg resource group (as
user oracle):
$ sqlplus /nolog
SQL> insert into mytable values (‘daniel’, 34);
SQL> commit;
SQL> quit
5. Verify that the database fails over properly under various failure conditions:
a. Complete failure of the node on which the resource group is running

D74942GC20 Ag PDF

Uploaded by

Copyright:

Available Formats

D74942GC20 Ag PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

D74942GC20 Ag PDF

Uploaded by

Copyright:

Available Formats

What is the document about?

What is the document about?

What steps are required to create Oracle resources and resource groups in the cluster?

What steps are required to create Oracle resources and resource groups in the cluster?

These eKit materials are to be used ONLY by you for the express purpose SELF STUDY.

SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle University and (Oracle Corporation) use only.

Oracle University and (Oracle Corporation) use only.

Practice for Lesson 1: Overview.....................................................................................................................1-2

Oracle University and (Oracle Corporation) use only.

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Oracle Solaris Cluster 4.x Administration Table of Contents

Oracle University and (Oracle Corporation) use only.

Practices for Lesson 1: Introduction

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Oracle University and (Oracle Corporation) use only.

There is no practice for Lesson 1.

Oracle University and (Oracle Corporation) use only.

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction

Oracle University and (Oracle Corporation) use only.

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 2: Planning the Oracle Solaris Cluster Environment

Oracle University and (Oracle Corporation) use only.

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 2: Planning the Oracle Solaris Cluster Environment

Oracle University and (Oracle Corporation) use only.

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 2: Planning the Oracle Solaris Cluster Environment

Oracle University and (Oracle Corporation) use only.

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity

Oracle University and (Oracle Corporation) use only.

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity

Oracle University and (Oracle Corporation) use only.

Task 1: Updating Host Name Resolution

4. Verify that the config/host property of the system/name-service/switch service is

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity

Task 2: Installing the pconsole Utility

Oracle University and (Oracle Corporation) use only.

3. Install the terminal/pconsole package.

Task 3: Configuring the pconsole Utility

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity

3. Set the environment variables by running the .profile login script.

4. Assume the root role.

Oracle University and (Oracle Corporation) use only.

Task 4: Using the pconsole Utility

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity

Oracle University and (Oracle Corporation) use only.

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity

Oracle University and (Oracle Corporation) use only.

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Oracle University and (Oracle Corporation) use only.

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.