Beginners Lesson VERITAS Volume

Manager for Solaris

+ In this post we will be Discussing about : [hide]

1. Physical Disks

Check the disks recognized by Solaris

2. Solaris Native Disk Partitioning
Steps to add new disk to Solaris:
3. Initialize Physical Disks under VxVM control

o
o
o

Device Naming Schemes

c#t#d# Naming Scheme
Enclosure-Based Naming Scheme
Steps to Recognize new disks under VxVM control
4. Virtual Objects (DiskGroups / Volumes / Plexs ) in VxVM

o
o
o

Disk Groups
Volumes
Plexes:
Summary Transformation of Physical disks into Veritas Volumes
Solaris Volume Manager Vs Veritas Volume Manager

o
o

Other Popular Posts

Share & Discuss

Veritas Volume Manager is a storage management application by Symantec , which allows you to
manage physical disks as logical devices called volumes.
VxVM uses two types of objects to perform the storage management
1. Physical objects are direct mappings to physical disks
2 . Virtual objects are volumes, plexes, subdisks and diskgroups.
a. Disk groups are composed of Volumes
b. Volumes are composed of Plexes and Subdisks
c. Plexes are composed of SubDisks
d. Subdisks are actual disk space segments of VxVM disk ( directly mapped from the physical disks)

1. Physical Disks

Physical disk is a basic storage where ultimate data will be stored. In Solaris physical disk names uses
the convention like c#t#d# where c# refers to controller/adapter connection, t# refers to the SCSI
target Id , and d# refers to disk device Id. Below figure illustrates how the disk name changes
depending on the connection.
Physical disks could be coming from different sources within the servers e.g. Internal disks to the
server , Disks from the Disk Array and Disks from the SAN.

Check the disks recognized by Solaris

#echo|format
Searching for disksdone
AVAILABLE DISK SELECTIONS:
0. c0t0d0
/sbus@1f,0/SUNW,fas@e,8800000/sd@0,0
1. c0t1d0
/sbus@1f,0/SUNW,fas@e,8800000/sd@1,0

2. Solaris Native Disk Partitioning

In solaris, physical disks will partitioned into slices numbered as S0,S1,S3,S4,S5,S6,S7 and the slice
numberS2 normally called as overlap slice and points to the entire disk. In Solaris we use the format
utility used to partition the physical disks into slices.
Once we added new disks to the Server, first we should recognize the disks from the solaris level
before proceeding for any other storage management utility.

Steps to add new disk to Solaris:

If the disks that are recently added to the server not visible, you can use below procedure
Option 1: Reconfiguration Reboot ( for the server hardware models that doesnt support
hot swapping/dynamic addition of disks )
# touch /reconfigure; init 6
or
#reboot -r ( only if no applications running on the machine)
Option 2: Recognize the disks added to external SCSI, without reboot
# devfsadm
# echo | format
Option 3: Recognize disks that are added to internal scsi, hot swappable, disk
connections.
Just run the command cfgadm -al and check for any newly added devices in
unconfigured state, and configure them.
# cfgadm -al
Ap_Id
c0
c0::dsk/c0t0d0
c0::dsk/c0t0d0
c0::rmt/0

Type
Receptacle Occupant Condition
scsi-bus connected configured unknown
disk
connected configured unknown
disk
connected configured unknown
tape
connected configured unknown

c1
c1::dsk/c1t0d0
c1::dsk/c1t1d0
configured

scsi-bus
connected configured unknown
unavailable connected unconfigured unknown
unavailable connected unconfigured unknown < == disk not

# cfgadm -c configure c1::dsk/c1t0d0

# cfgadm -c configure c1::dsk/c1t0d0
# cfgadm -al
Ap_Id
c0
c0::dsk/c0t0d0
c0::rmt/0
c1
c1::dsk/c1t0d0
c1::dsk/c1t1d0

Type
Receptacle Occupant Condition
scsi-bus connected configured unknown
disk
connected configured unknown
tape
connected configured unknown
scsi-bus
connected configured unknown
disk
connected configured unknown
disk
connected configured unknown

# devfsadm
#echo|format
Additional Procedure : Click here to check the procedure to add Internal FC-AL disks

3. Initialize Physical Disks under VxVM control

A formatted physical disk is considered uninitialized until it is initialized for use by VxVM. When a disk
is initialized, partitions for the public and private regions are created, VM disk header information
is written to the private region and actual data is written to Public region . During the notmal
initialization process any data or partitions that may have existed on the disk are removed.
Note: Encapsulation is another method of placing a disk under VxVM control in which existing data on
the disk is preserved
An initialized disk is placed into the VxVM free disk pool. The VxVM free disk pool contains disks that
have been initialized but that have not yet been assigned to a disk group. These disks are under
Volume Manager control but cannot be used by Volume Manager until they are added to a disk group

Device Naming Schemes

In VxVM, device names can be represented in two ways:

Using the traditional operating system-dependent format c#t#d#
Using an operating system-independent format that is based on enclosure names

c#t#d# Naming Scheme

Traditionally, device names in VxVM have been represented in the way that the operating system
represents them. For example, Solaris and HP-UX both use the format c#t#d# in device naming, which
is derived from the controller, target, and disk number. In VxVM version 3.1.1 and earlier, all disks are
named using the c#t#d# format. VxVM parses disk names in this format to retrieve connectivity
information for disks.

Enclosure-Based Naming Scheme

With VxVM version 3.2 and later, VxVM provides a new device naming scheme, called enclosure-based
naming. With enclosure-based naming, the name of a disk is based on the logical name of the
enclosure, or disk array, in which the disk resides.

Steps to Recognize new disks under VxVM control

1. Run the below command to see the available disks under VxVM control
# vxdisk list
in the output you will see below status

error indicates that the disk has neither been initialized nor encapsulated
by VxVM. The disk is uninitialized.
online indicates that the drive has been initialized or encapsulated.
online invalid indicated that disk is visible to VxVM but not controlled by
VxVM
If disks are visible with format command but not visible with vxdisk list command,
run below command to scan the new disks for VxVM
# vxdctl enable

Now you should see new disks with the status of Online Invalid
2. Initialize each disk with vxdisksetup command
#/etc/vx/bin/vxdisksetup -i
after running this command vxdisk list should see the status as online for all the
newly initialized disks

4. Virtual Objects (DiskGroups / Volumes / Plexs ) in

VxVM
Disk Groups
A disk group is a collection of VxVM disks ( going forward we will call them as VM Disks ) that share a
common configuration. Disk groups allow you to group disks into logical group of Subdisks called
plexes which in turn forms the volumes.

Volumes
A volume is a virtual disk device that appears to applications, databases, and file systems like a
physical disk device, but does not have the physical limitations of a physical disk device. A volume
consists of one or more plexes, each holding a copy of the selected data in the volume.

Plexes:

VxVM uses subdisks to create virtual objects called plexes. A plex consists of one or more subdisks
located on one or more physical disks.
From the below diagram you can observer below points
The Diskgroup named Diskgroup_o1 is created using 4 different VM disks named as

vxdisk_0x, vxdisk_oy, vxdisk_oz and vxdisk_oa

The diskfgroup
Diskgroup_01 was configured

1. Concat_vol 2. striped_vol 3. mirror_vol and 4. raid5_vol

Concat_vol is a concatenation volume with single

and con_plex01 was build up using 4 subdisks of different size

Striped_vol is
a
striped volume
with single
plex

and stripe_plex01 was build up using 4 subdisks of same size

mirror_vol is a mirrored volume with two plexes named mplex01 and mplex02 inside,

each plex is copy of other. Both the plexes formed with different subdisks of either same size or
different size.
raid5_vol is a raid5 ( striped with parity) volume which build up using 3 different plexes
formed with the subdisks of 3 different VM disks.

with

different

plex

volumes

i.e.

i.e. con_plex01 inside,

i.e. stripe_plex01 inside,

Summary Transformation of Physical disks into

Veritas Volumes
Below diagram shows you the complete transformation of a physical disk into a veritas volume. And
below is the summary of complete process
1. Recognize disks under solaris using devfsadm, cfgadm or reconfiguration reboot , and verify
usingformat command
2.
Recognize
the
disks
under
VxVM
using
vxdctl
enable
3.
Initialize
the
disks
under
VxVM
using vxdisksetup
4.
Add
the
disks
to
Veritas
Disk
Group
using vxdg commands
5.
Create
Volumes
under
Disk
Group
using vxmake or vxassist commands
6. Create filesystem on top of volumes using mkfs or newfs, and you can create
either VXFS filesystem or UFS filesystem

Solaris Volume Manager Vs Veritas Volume

Manager
Below diagram explains how Solaris volume manager differs from the Veritas volume manager during
the process of creating a new file system on top of physical storage.

VxVM Beginners Commands and

Examples Part 2

+ In this post we will be Discussing about : [hide]

VxVM Disk Operations

VxVM Subdisk Operations
VxVM Volume Operations
VxVM Advanced Administration
Examples for VxVM Admistration using Commands

o
o

Other Popular Posts

Share & Discuss

This is a continuation post for our previous post Beginners Lesson VERITAS Volume Manager for
Solaris
Learning VxVM administration will be much easier by understanding below three things>
1. How Veritas Volume Manager treats Operating System Native Disks into Veritas Volumes
2. What are the operations we can do in VxVM and what commands are used to perform those
operations
3. What are the background services/daemons/Configuration files required to make VxVM work in
normal way

Below Diagram will gives you complete picture how VxVM converts Solaris native disks into Diskgroups,
Volumes, Plexes and Subdisks.

Transformation of Solaris Disks into VERITAS Volumes

Below given Command Reference for Veritas Volume Manager helps you to perform VxVM administration
operations using commands

VxVM Disk Operations

VxVM Disk related administration can done using the commands

vxdisksetup / vxdiskupstup

vxdisk
vxedit
vxunreloc

VxVM Disk Operations

VxVM DiskGroup Operations

VxVM diskgroup related operations can be done by using the command vxdg

VxVM DiskGroup Operations

VxVM Subdisk Operations

VxVM disk/Subdisk Operations can be done using below commands

vxmake
vxedit
vxsd

VxVM Subdisk Operations

VxVM Plex Operations

VxVM plex related Operations can be done using the command vxplex

VxVM Plex Operations

VxVM Volume Operations

VxVM Volume related operations cab be done using the below commands
vxassist
vxresize
vxvol / vxrecover

VxVM Volume Operations

VxVM Advanced Administration

To administrate Dynamic Multipathing and Device Discovery layer we use the commands
vxdmpadm
vxddladm

Advanced VxVM Commands

Examples for VxVM Admistration using Commands

Initialize disks c1t0d0, c1t1d0, c1t2d0, c2t0d0, c2t1d0, and c2t2d0:
# vxdisksetup -i c1t0d0
# vxdisksetup -i c1t1d0
# vxdisksetup -i c1t2d0

# vxdisksetup -i c2t0d0
# vxdisksetup -i c2t1d0
# vxdisksetup -i c2t2d0
Create a disk group named database1 and add the six disks:
# vxdg init database1 disk01=c1t0d0 disk02=c1t1d0 disk03=c1t2d0
# vxdg -g database1 adddisk disk04=c2t0d0 disk05=c2t1d0 disk06=c2t2d0
Using the top down (modern) technique, create a RAID-5 volume named db01 of size 2 GB on the six
disks (5 + log). Also, create and mount a UFS file system on the volume:
#
#
#
#

vxassist -g database1 make db01 2g layout=raid5 disk01 disk02 disk03 disk04 disk05 disk06
newfs /dev/vx/rdsk/database1/db01
mkdir /db01
mount /dev/vx/dsk/database1/db01 /db01

Remove the volume database1:

# umount /db01
# vxvol stop database1
# vxedit -g database1 -r rm db01
Using the bottom up technique, create a two-way mirrored volume named db02 of size 1 GB using
disks disk01, disk04:

1 GB = 2097152 sectors
Subdisks should be cylinder aligned.
If disk uses 1520 sectors/cylinder, subdisk size = 2097600 sectors.
#
#
#
#
#
#

vxmake -g database1
vxmake -g database1
vxmake -g database1
vxmake -g database1
vxmake -g database1
vxvol start db02

sd sd01 disk01,0,2097600
sd sd02 disk04,0,2097600
plex plex01 sd=sd01:0/0
plex plex02 sd=sd02:0/0
-U fsgen vol db02 plex=plex01,plex02

Change the permissions of the volume so that dba is the owner and dbgroup is the group:
# vxedit set user=dba group=dbgroup mode=0744 db02
Destroy volume and remove disks from disk group database1. Also, remove disks from
volume manager control:
#
#
#
#

vxedit -g database1 -rf rm db02

vxdg -g database1 rmdisk disk01 disk02 disk03 disk04 disk05
vxdg deport database1
vxdiskunsetup c1t1d0

# vxdiskunsetup c1t2d0
# vxdiskunsetup c1t3d0...
Advanced vxmake Operation: Create a three-way striped volume:
#
#
#
#
#
#

vxmake -g spice sd sd01 spice01,0,1520000

vxmake -g spice sd sd02 spice02,0,1520000
vxmake -g spice sd sd03 spice03,0,1520000
vxmake -g spice plex plex1 layout=stripe ncolumn=3 stwidth=64k sd=sd01:0/0,sd02:1/0,sd03:2/0
vxmake -g spice -U fsgen vol db5 plex=plex1
vxvol -g spice start db5

Advanced vxmake Operation: Create a RAID 0+1 volume with a DRL Log:
#
#
#
#
#
#
#
#
#
#

vxmake -g spice sd sd01 spice01,0,194560

vxmake -g spice sd sd02 spice02,0,194560
vxmake -g spice sd sd03 spice03,0,194560
vxmake -g spice sd sd04 spice04,0,194560
vxmake -g spice sd logsd spice01,194560,2
vxmake -g spice plex plex1 layout=stripe ncolumn=2 stwidth=64k sd=sd01:0/0,sd02:1/0
vxmake -g spice plex plex2 layout=stripe ncolumn=2 stwidth=64k sd=sd03:0/0,sd04:1/0
vxmake -g spice plex logplex log_sd=logsd
vxmake -g spice -U fsgen vol db6 plex=plex1,plex2,logplex
vxvol -g spice start db6

Disk Operations Task CommandInitialize

disk vxdisksetup -i device vxdiskadd devicevxdiskadm option 1, Add or initialize one or more
disksUninitialize disk vxdiskunsetup device
List disks vxdisk listList disk header vxdisk list diskname|deviceEvacuate a disk vxevac -g diskgroup
from_disk to_disk
Rename a disk vxedit -g diskgroup rename oldname newnameSet a disk as a spare vxedit -g diskgroup
set spare=on|off diskname
Unrelocate a disk vxunreloc -g diskgroup original_diskname
Disk Group OperationsTask CommandCreate disk group vxdg init diskgroup diskname=device
Add disk to disk group vxdg -g diskgroup adddisk diskname=device Deport disk group vxdg deport
diskgroup

Import disk group vxdg import diskgroup Destroy disk group vxdg destroy diskgroup List disk groups
vxdg list
List diskgroup details vxdg list diskgroup Remove disk from diskgroup vxdg g diskgroup rmdisk
diskname
Upgrade disk group version vxdg [-T version] upgrade diskgroup Move an object between DGs vxdg
move sourcedg targetdg object...
Split objects between DGs vxdg split sourcedg targetdg object... Join disk groups vxdg join sourcedg
targetdg
List objects affected by a disk group move operation vxdg listmove sourcedg targetdg object...Subdisk
Operations
Create a subdisk vxmake -g diskgroup sd subdiskname diskname offset length Remove a subdisk
vxedit -g diskgroup rm subdisk_name Display subdisk info vxprint st vxprint -l subdisk_name
Associate a subdisk to a plex vxsd assoc plex_name subdisk_name
Dissociate a subdisk vxsd dis subdisk_name
Plex Opertions Create a plex vxmake -g diskgroup plex plex_namesd=subdisk_name,
Associate a plex (to a volume) vxplex g diskgroup att vol_name plex_name
Dissociate a plex vxplex dis plex_name Remove a plex vxedit g diskgroup rm plex_name List all plexes
vxprint -lp
Detach a plex vxplex g diskgroup det plex_name Attach a plex vxplex g diskgroup att vol_name
plex_name
Volume Operations create a volume vxassist -g diskgroup make vol_name size layout=format
diskname
Or vxmake -g diskgroup vol vol_name len=size plex plex_name
Remove a volume vxedit -g diskgroup -rf rm vol_name or vxassist -g diskgroup remove volume
vol_name
Display a volume vxprint -g diskgroup -vt vol_name vxprint -g diskgroup -l vol_name Change

volume attributes vxedit -g diskgroup set field=value vol_name

vxvol field=value vol_name Resize a volume vxassist -g diskgroup growto
vol_name new_length vxassist -g diskgroup growby vol_name length_change
vxassist -g diskgroup shrinkto vol_name new_length vxassist -g diskgroup shrinkby vol_name
length_change
vxresize -g diskgroup vol_name [+|-]length Start/Stop volumes
start a volume vxvol start vol_name Start all volumes vxvol startall Start all volumes in a dg vxvol -g
diskgroup startall
Stop a volume vxvol stop vol_name Stop all volumes vxvol stopall Recover a volume vxrecover -sn
vol_name
List unstartable volumes vxinfo [vol_name] Mirror an existing plex vxassist -g diskgroup mirror
vol_name or
vxmake -g diskgroup plex plex_namesd=subdisk_name
vxplex -g diskgroup att vol_name plex_name vOLUME SNAPSHOTS Create a snapshot volume vxassist
g diskgroup
-b snapstart vol_name vxassist g diskgroup snapshot vol_name new_volume
Abort a snapshot vxassist -g diskgroup snapabort orig_vol_name Reassociate a snapshot vxassist -g
diskgroup snapback snapshot_vol
Dissociate a snapshot vxassist -g diskgroup snapclear snapshot_vol Print snapshot information
vxassist -g diskgroup snapprint vol_name
Relayout a volume vxassist -g diskgroup relayout vol_name layout=new_layout [attributes...]
To or from a layered vxassist -g diskgroup convert vol_name layout layout=new_layout
[attributes...] Add a log to a volume vxassist g diskgroup addlog vol_name Create and mount a VxFS
file system on a volume
mkfs -F vxfs /dev/vx/rdsk/diskgroup/vol_name mkdir /mount_point mount

-F vxfs /dev/vx/rdsk/diskgroup/vol_name /mount_point DMP, DDL, and Task Management

Task Command Manage tasks vxtask list vxtask monitor
Manage device discovery layer (DDL) List supported disk arrays vxddladm listsupport
Exclude support for an array vxddladm excludearray libname=library
vxddladm excludearray vid=vid pid=pid Reinclude support vxddladm includearray libname=library
vxddladm includearray vid=vid pid=pid
List excluded arrays vxddladm listexclude Manage dynamic multipathing (DMP) List controllers
on system vxdmpadm listctlr all Display subpaths vxdmpadm getsubpaths ctlr=ctlr Display DMP
nodes
vxdmpadm getdmpnode nodename=nodename Enable/disable I/O to controller
vxdmpadm enable ctlr=ctlr vxdmpadm disable ctlr=ctlr
Display enclosure attributes vxdmpadm listenclosure all Rename an enclosure vxdmpadm setattr
enclosure orig_name name=new_name