HP Serviceguard - Configuration Guide

- serviceguard cluster commands : cmviewcl, cmhaltpkg, cmrunpkg

HP Serviceguard
HP Serviceguard was the very first solution for high availability clustering software. Serviceguard uses resource virtualization to ensure that critical applications, databases, or services are continually up and running. Users typically dont care which server they run their application onthey just want to make sure that application is running. Serviceguard hides the details of which server an application is actually running on at any given time, which means it can move applications between servers on the fly.

What is a Serviceguard cluster?

Any host system running in a Serviceguard cluster is called an active node. Under normal conditions, a fully operating Serviceguard cluster monitors the health of the clusters components on all its active nodes. Most Serviceguard packages are failover packages. When you configure a failover package, you specify which active node will be the primary node where the package will start, and one or more other nodes, called adoptive nodes, that can also run the package.

Creating a Serviceguard cluster

Before you configure the cluster, you will need to create a volume group on the shared disks, so that Service Guard can create a Cluster Lock Disk.

On testnode1 # pvcreate /dev/rdsk/c0t4d0 # pvcreate /dev/rdsk/c1t4d0 # mkdir /dev/vg01 # mknod /dev/vg01/group c 64 0010000 # vgcreate vg01 /dev/dsk/c0t4d0 /dev/dsk/c1t4d0 # mkdir /etc/cmcluster/maps # vgexport p s m vg01 /etc/cmcluster/maps/vg01 On testnode2 # mkdir /dev/vg01 # mknod /dev/vg01/group c 64 0010000 # mkdir /etc/cmcluster/maps # rcp testnode1:/etc/cmcluster/maps/vg01 /etc/cmcluster/maps/vg01 # vgimport s m /etc/cmcluster/maps/vg01 vg01 # vgchange a y vg01 # vgchange a n vg01 You can now go ahead and create the cluster. # cd /etc/cmcluster # cmquerycl v C cmclconfig.ascii n testnode1 -n testnode2

After creating this file, you will need to edit it to suit your cluster. You will need to alter the following lines: # vi cmclconfig.ascii CLUSTER_NAME simeon1 NODE_NAME testnode1 NETWORK_INTERFACE lan0 HEARTBEAT_IP 192.168.100.101 NETWORK_INTERFACE lan1 HEARTBEAT_IP 164.39.8.92 NETWORK_INTERFACE lan2 NODE_NAME testnode2 NETWORK_INTERFACE lan0 HEARTBEAT_IP 192.168.100.102 NETWORK_INTERFACE lan1 HEARTBEAT_IP 164.39.8.93 NETWORK_INTERFACE lan2 HEARTBEAT_INTERVAL 2000000 NODE_TIMEOUT 5000000 MAX_CONFIGURED_PACKAGES 10 To check that there are no mistakes in the file, use the check command. # cmcheckconf C /etc/cmcluster/cmclconfig.ascii To apply the configuration file, usr the apply command. # cmapplyconf C /etc/cmcluster/cmclconfig.ascii You are now ready to startup the cluster. On testnode1 # vgchange a n vg01

# cmruncl v # cmviewcl v # vgchange c y vg01 On Both nodes # netstat -in # vi /etc/rc.config.d/cmcluster AUTOSTART_CMCLD=1 On testnode1 # cmhaltcl v # vgchange a n vg01

Creating a basic package

Serviceguard packages group together applications and the services and resources they depend on. The typical Serviceguard package is a failover package that starts on one node but can be moved (failed over) to another if necessary. Creating or modifying a package requires the following broad steps: 1. Generate the package ASCII file 2. Edit the package ASCII file 3. Generate the package control script 4. Edit the package control script 5. Distribute the control script to the cluster nodes 6. Apply the package ASCII file Once the cluster is up and running, we can create a package to run in it. This package will run an xclock on a client PC, and mount a file system, using the IP Address 164.39.8.101 The first step is to make a logical volume. On node1 # vgchange a e vg01 # lvcreate L 20 vg01

# lvextend m 1 /dev/vg01/lvol1 # newfs F vxfs /dev/vg01/rlvol1 # mkdir /pkg_1 # vi /etc/fstab #/dev/vg01/lvol1 /pkg_1 vxfs delaylog 0 2 # MC/SG # vgchange a n vg01 # vgexport p s m vg01 /etc/cmcluster/maps/vg01 On Node 2 # vgexport vg01 # mkdir /dev/vg01 # mknod /dev/vg01/group c 64 0010000 # vgimport s m /etc/cmlcuster/maps/vg01 vg01 # mkdir /pkg_1 # vi /etc/fstab #/dev/vg01/lvol1 /pkg_1 vxfs delaylog 0 2 # MC/SG Now we can make the package. On Node1 # cd /etc/cmcluster # mkdir pkg_1 # cd pkg_1 # cmmakepkg p pkg_1.conf The file pkg_1.conf will need to be edited to suit. # vi pkg_1.conf PACKAGE_NAME pkg_1 NODE_NAME testnode1 NODE_NAME testnode2

RUN_SCRIPT /etc/cmcluster/pkg_1/pkg_1.cntl RUN_SCRIPT_TIMEOUT NO_TIMEOUT HALT_SCRIPT /etc/cmcluster/pkg_1/pkg_1.cntl HALT_SCRIPT_TIMEOUT NO_TIMEOUT SERVICE_NAME pkg_1_service SUBNET 164.39.8.0 Add the IP address that will be used for the package in the hosts file. # vi /etc/hosts 164.39.8.101 pkg_1 xclock # cmmakepkg s pkg_1.cntl The file pkg_1.cntl will need to be edited to suit # vi pkg_1.cntl VG[0]=vg01 LV[0]=/dev/vg01/lvol1; FS[0]=/pkg_1; FS_MOUNT_OPT[0]=" IP[0]=164.39.8.101 SUBNET[0]=164.39.8.0 SERVICE_NAME[0]=pkg_1_service SERVICE_CMD[0/etc/cmcluster/pkg_1/service_pkg_1" #SERVICE_RESTART[0]=" function customer_defined_run_cmds { # ADD customer defined run commands. if [ $(hostname) = "testnode1" ] then COLOUR=blue else COLOUR=red fi

/usr/bin/X11/xclock -bg $COLOUR -update 1 display \ 164.39.11.232:0.0 & test_return 51 } function customer_defined_halt_cmds { # ADD customer defined halt commands. if ps -ef |grep -v grep |grep -q xclock then kill -9 $(ps -ef |grep -v grep |grep xclock|cut -c10-14) fi test_return 52 } # vi pkg_1_service #!/usr/bin/sh #This is a script to monitor that xclock is running while true do if ps -ef |grep -v grep |grep -q xclock then sleep 5 else echo Package xclock failed at $(date) from node \ $(hostname) > /etc/cmcluster/pkg_1/pkg_1.cntl.log

exit 99 fi done # chmod u+x pkg_1_servce # cd /etc/cmcluster # rcp r testnode1:/etc/cmcluster/pkg_1 On node 1 # cmcheckconf -P pkg_1.conf # cmapplyconf P pkg_1.conf # cmmodpkg e n testnode1 -n testnode2 pkg_1 # cmviewcl v p pkg_1 # cmhaltpkg pkg_1 # cmviewcl v|more # tail /etc/cmcluster/pkg_1/pkg_1.log # tail /var/adm/syslog/syslog.log

Starting and stopping a Serviceguard Package

Normally, when a cluster starts up, the packages configured as part of the cluster will start up on their configured nodes. You may need to start a package manually after it has been halted manually. You can do this either in Serviceguard Manager or on the Serviceguard command line. If you want to stop a package, use the following commands # cmviewcl v p <package name> # cmahalt <package name> # cmviewcl v p <package name> You halt a Serviceguard package when you wish to bring the package out of use but wish the node to continue in operation. You can halt a package using Serviceguard Manager or on the

Serviceguard command line. Halting a package has a different effect than halting the node. When you halt the node, its failover packages may switch to adoptive nodes. If you want to start a package, then use the following commands # cmviewcl v p <package name> # cmmodpkg e <package name> OR # cmrunpkg n <host name> <package name> # cmmodpkg e <package name>

Deleting a Serviceguard Package

To delete a package, use the following commands # cmdeleteconf -p <package name>

Stopping and starting the Serviceguard Cluster

The cmhaltcl command can be used to halt the entire cluster. This command causes all nodes in a configured cluster to halt their Serviceguard daemons. You can use the -f option to force the cluster to halt even when packages are running. To stop the cluster, use the following commands
The cmhaltcl command can be used to halt the entire cluster. This command causes all nodes in a configured cluster to halt their Serviceguard daemons. You can use the -f option to force the cluster to halt even when packages are running.

To stop the cluster, use the following commands # cmiewcl v # cmhaltcl OR # cmhaltcl f To start the cluster: -

# cmruncl v # cmviewcl v

Automatically Restarting the Serviceguard Cluster

You can configure your cluster to automatically restart after an event, such as a long-term power failure, which brought down all nodes in the cluster. This is done by setting AUTOSTART_CMCLD to 1 in the

/etc/rc.config.d/cmcluster file.

Stopping and starting a node in the Serviceguard Cluster

Starting a node will not cause any active packages to be moved to the new node. However, if a package is DOWN, has its switching enabled, and is able to run on the new node, that package will automatically run there. To stop a node in the cluster # cmhaltnode n <node name> # cmviewcl v # vi /etc/rc.config.d/cmcluster AUTOSTART_CMCLD=0 To start a node in the cluster. # cmrunnode n <name name> # cmviewcl v # vi /etc/rc.config.d/cmcluster AUTOSTART_CMCLD=1

Deleting a cluster
When there are no packages left in the cluster, and you want to delete the cluster, use the following commands.

# vgchange c n vg01 # cmhaltcl f # cmdeleteconf c <cluster_name> # vgchange a y vg01 # lvremove /dev/vg01/lvol1 # vgremove vg01 # rm r /dev/vg01