Delivering A Highly Available Oracle Database As A Service: Emc Vplex, Vmware Vfabric Data Director

White Paper
DELIVERING A HIGHLY AVAILABLE ORACLE

DATABASE AS A SERVICE
EMC VPLEX, VMware vFabric Data Director, VMware vSphere,
EMC Symmetrix VMAX
• Self-service deployment of Oracle Database 11g
• Virtual storage
• Application mobility
• Active/active data centers
EMC Solutions Group
Abstract
This white paper describes the deployment of a database as a service
distributed over two sites for improved availability. The solution is enabled by
EMC® VPLEX® Metro, EMC Symmetrix® VMAX®, VMware vSphere High
Availability, VMware vFabric Data Director, and Oracle Database 11g.
September 2013
Copyright © 2013 EMC Corporation. All Rights Reserved.
EMC believes the information in this publication is accurate as of its

publication date. The information is subject to change without notice.
The information in this publication is provided as is. EMC Corporation makes no

representations or warranties of any kind with respect to the information in this
publication, and specifically disclaims implied warranties of merchantability or
fitness for a particular purpose.
Use, copying, and distribution of any EMC software described in this

publication requires an applicable software license.
For the most up-to-date listing of EMC product names, see EMC Corporation
Trademarks on EMC.com.
All trademarks used herein are the property of their respective owners.
Part Number H12235
Delivering a Highly Available Oracle Database as a Service 2

EMC VPLEX, VMware vFabric Data Director, VMware vSphere, EMC Symmetrix VMAX
Table of contents
Executive summary ............................................................................................................................. 6
Business case .................................................................................................................................. 6
Solution overview ............................................................................................................................ 6
Key benefits ..................................................................................................................................... 7
Introduction ....................................................................................................................................... 9
Purpose ........................................................................................................................................... 9
Scope .............................................................................................................................................. 9
Audience.......................................................................................................................................... 9
Solution overview ............................................................................................................................. 10

Introduction ................................................................................................................................... 10
Solution architecture...................................................................................................................... 10
Hardware resources ....................................................................................................................... 12
Software resources ........................................................................................................................ 13
EMC storage infrastructure ............................................................................................................... 14

Introduction ................................................................................................................................... 14
Overview ................................................................................................................................... 14
EMC Symmetrix VMAX 10K ............................................................................................................. 14
EMC Enginuity ................................................................................................................................ 14
EMC Symmetrix VMAX 20K ............................................................................................................. 15
EMC Unisphere for VMAX ............................................................................................................... 15
EMC VPLEX Metro infrastructure ....................................................................................................... 16

Introduction ................................................................................................................................... 16
Overview ................................................................................................................................... 16
EMC VPLEX ................................................................................................................................ 16
EMC VPLEX Metro ...................................................................................................................... 16
EMC VPLEX Witness ................................................................................................................... 16
EMC Unisphere for VPLEX .......................................................................................................... 17
EMC VPLEX high availability....................................................................................................... 17
VPLEX logical storage structures ................................................................................................ 18
VPLEX consistency groups ......................................................................................................... 18
Detach rules .............................................................................................................................. 19
VPLEX Metro solution configuration................................................................................................ 19
Consistency group ..................................................................................................................... 19
Configuration process ............................................................................................................... 19

VPLEX Witness configuration .......................................................................................................... 21
VPLEX data mobility ....................................................................................................................... 22
Overview ................................................................................................................................... 22
Data mobility configuration ....................................................................................................... 24
Data mobility in this solution ..................................................................................................... 25
Back-end array volume expansion.................................................................................................. 25
VPLEX performance monitoring ...................................................................................................... 28
VMware virtualized infrastructure ..................................................................................................... 30

Introduction ................................................................................................................................... 30
Overview ................................................................................................................................... 30
VMware vSphere 5..................................................................................................................... 30
VMware vCenter Server .............................................................................................................. 30
VMware vSphere Update Manager ............................................................................................. 30
VMware vSphere vMotion .......................................................................................................... 30
VMware vSphere High Availability ............................................................................................. 30
VMware vSphere Distributed Resource Scheduler ..................................................................... 31
EMC PowerPath/VE .................................................................................................................... 31
EMC Virtual Storage Integrator for VMware vSphere ................................................................... 31
VMware deployments on VPLEX Metro ........................................................................................... 31
VMware stretched cluster configuration ......................................................................................... 33
VMware vSphere HA configuration ................................................................................................. 35
Enabling VMware vSphere HA and VMware vSphere DRS ........................................................... 35
VM Monitoring ........................................................................................................................... 35
Virtual machine restart options ................................................................................................. 35
Datastore heartbeating .............................................................................................................. 36
VMware vSphere DRS configuration ............................................................................................... 36
VMware DRS host groups and virtual machine groups ............................................................... 36
VMware DRS affinity rules .......................................................................................................... 37
VMware vSphere Fault Tolerance .................................................................................................... 37
EMC Virtual Storage Integrator and VPLEX ...................................................................................... 39
VMware vFabric Data Director ........................................................................................................... 41

Introduction ................................................................................................................................... 41
Architecture ................................................................................................................................... 42
Organizational structures and user management ........................................................................... 42
Resource management................................................................................................................... 43
Installation and configuration ........................................................................................................ 45
Managing databases...................................................................................................................... 47
Database lifecycle ..................................................................................................................... 47
Base database templates .......................................................................................................... 48

Creating databases ................................................................................................................... 49
Cloning databases .................................................................................................................... 51
Ingesting databases .................................................................................................................. 52
Backup and restore ................................................................................................................... 52
Monitoring the Data Director environment ...................................................................................... 54
Conclusion ....................................................................................................................................... 55
Summary ....................................................................................................................................... 55
Findings ......................................................................................................................................... 55
References ....................................................................................................................................... 57
Product documentation.................................................................................................................. 57
White papers ................................................................................................................................. 57
Other documentation ..................................................................................................................... 57

Executive summary
Business case Global enterprises are experiencing an increasing demand to support a growing and
fragmented database infrastructure while enforcing policy and compliance. In
addition to managing an increasing number of production databases, DBA teams
must provide multiple copies of database environments to meet business needs such
as developing and testing application features, reporting, and application
troubleshooting.
In physical environments, these demands can involve the deployment of new servers
and infrastructure resources, resulting in long lead times and poor response to
business needs.
Database virtualization and automation tools have improved the process for the DBA
team, shortening the lead time for database services for both business and
developers. Database as a service (DBaaS) has built on this foundation, adding a
self-service lifecycle management framework to provide a secure, controlled, and
repeatable process for the database service user.
The EMC solution described in this white paper offers a highly available, distributed
strategy for DBaaS utilizing EMC® VPLEX® Metro and VMware vFabric Data Director.
This solution offers the following business benefits:

• CapEx savings—Reduced hardware and licensing costs through virtualization
and consolidation of physical servers
• OpEx savings—Automated database lifecycle management, self-service
database provisioning with backup, restore, and management of the virtual
database infrastructure through a single interface
• Quality of service—Increased application quality of service, with built-in
resource management, high availability, and the ability to scale quickly
• Secure multitenancy—Complete isolation between systems, even on the
same host, and an organizational framework to ensure isolation of databases
and resources to the correct application owner
Solution overview This white paper describes the deployment of a DBaaS distributed over two sites for
improved availability. The solution is enabled by EMC VPLEX Metro, EMC Symmetrix®
VMAX®, VMware vSphere High Availability (HA), VMware vFabric Data Director, and
Oracle Database 11g. It demonstrates how the following technologies create this
innovative business-continuity solution:
• EMC VPLEX Metro provides the virtual storage layer that enables an
active/active Metro data center.
• VPLEX data mobility enables nondisruptive movement of data at the storage
layer.
• VPLEX storage-volume expansion enables nondisruptive expansion of a
virtual volume to match the capacity of the underlying LUNs.

• VPLEX Witness supports continuous application availability, even in the event
of disruption at one of the data centers.
• EMC Symmetrix VMAX 10K and 20K series with Enginuity™ storage arrays,
proven five 9s availability, support for Fully Automated Storage Tiering (FAST),
and a choice of replication technologies provide the enterprise-class storage
platforms for the solution.
• VMware vFabric Data Director improves agility and drastically reduces
database total cost of ownership (TCO) through database-aware virtualization
and self-service lifecycle management, empowering administrators to
securely automate routine tasks including database provisioning, high
availability, backup, and cloning.
• VMware vSphere virtualizes application components and eliminates them as
single points of failure. VMware HA protects the virtual machines in the event
of physical server and operating system failures.
• Brocade Ethernet fabrics and MLXe core routers provide seamless networking
and Layer2 extension between sites.
• Brocade DCX 8510 Backbones provide redundant SAN infrastructure,
including fabric extension.
Key benefits This solution highlights the benefits of deploying and utilizing VMware vFabric Data
Director on a stretched VMware vSphere HA cluster using a virtualized storage layer
distributed between data centers. Data Director improves operational agility through
database-aware virtualization and offers secure self-service lifecycle management
that automates routine tasks.
To increase operational efficiency and empower both DBAs and developers to control
database provisioning, backup, and cloning, Data Director provides the following:
• Simplified database virtualization using built-in workflows
• Standardized environment using database templates
• Migration from physical to virtual using database ingestion
• Management and monitoring of the virtual database infrastructure through a
single pane of glass
Combining VMware Distributed Resource Scheduler (DRS) groups with VMware

vFabric Data Director resource bundles, restricts database virtual machines (DBVMs)
to a select group of hosts and resources across both sites. This improves service
availability and can be used to ensure software license compliance by restricting
where virtual machines are placed and where software is installed and/or running.
Underpinning and enabling the stretched VMware vSphere HA cluster is EMC VPLEX.
The VPLEX Metro configuration used in this solution provides highly available storage
virtualization across multiple sites. It encapsulates devices from heterogeneous
storage arrays and provides active/active, block-level access to data on two sites
within synchronous distances.

VPLEX as a storage virtualization technology enables the following:
• Nondisruptive movement of data between storage arrays without application

impact
• Growth of VMFS datastores through the nondisruptive expansion of a virtual
volume
VPLEX Metro as an active/active data protection product, combined with VMware

vSphere clustering, provides a stretched vSphere cluster with HA and DRS running
over distance between two data centers. This offers the following benefits:
• Increased utilization of hardware and software assets

• Automatic load balancing between data centers
• Zero downtime maintenance using vMotion
• Quick restart of virtual machines across data centers with vSphere HA
• Instant transition to a new host for the fault-tolerant virtual machine, offering
a highly available NFS service

Introduction
Purpose This white paper describes the deployment and operation of Data Director, a DBaaS
solution that empowers both DBAs and developers to securely self-provision, back
up, and clone Oracle 11g databases in virtualized environments.
In this solution, the highly available, virtualized storage layer is provided by EMC
VPLEX with active/active access across data centers in geographically separate
locations.
Scope This white paper includes the following information about this DBaaS solution:
• Overview of the key enabling technologies
• Solution architecture and design
• Configuration of key components
• Results of the tests performed to demonstrate the following:
 Empowerment of DBAs and developers
 Environmental resilience
 Agile and timely delivery of services
 Security and control
• Business benefits of the solution
Audience This white paper is intended for Oracle DBAs, developers, VMware administrators,
storage administrators, IT architects, and technical managers responsible for
designing, creating, and managing virtualized data centers and cloud services to
provide DBaaS.

Solution overview
Introduction Traditionally, the system and database deployment lifecycle in physical environments
has been lengthened by the need to purchase and configure additional IT
infrastructure.
Virtualization has simplified and improved the process but still requires intervention
by an administrator:
• VMware administrators create template virtual machines and customize their
deployment for use.
• DBAs spend time replicating database environments.
DBaaS adds a self-service lifecycle management framework to offer a secure,

controlled, and repeatable process for the database service user.
Data Director is a software-based solution easily deployed as a vApp—a collection of
virtual machines that are operated and monitored as a unit—on any VMware vSphere
cluster with HA and DRS enabled. VPLEX Metro then extends the high availability with
a clustering architecture that breaks the boundaries of the data center and allows
servers at multiple data centers to have read/write access to shared block storage
devices. This data center transformation takes traditional high availability to a new
level.
Solution Combining VMware vSphere clustering and VPLEX Metro results in a vSphere HA
architecture cluster stretched between two data centers. Figure 1 shows the physical architecture
of the solution, including the network components.

Figure 1. Physical layout of the stretched vSphere cluster with VPLEX Metro high
availability
Figure 2 shows the logical architecture of a vFabric Data Director vApp deployment on
a vSphere cluster and the required supporting infrastructure: Network Time Protocol
(NTP) server and optional Dynamic Host Configuration Protocol (DHCP) server.
vFabric Data Director architecture is discussed in detail under the section VMware
vFabric Data Director on page 41.

Figure 2. Logical architecture of vFabric Data Director
Table 1Table 1 lists the hardware components of the solution, and Table 2 lists the
software components.
Hardware Table 1. Solution hardware environment

resources
Purpose Quantity Configuration
Storage (Site A) 1 Symmetrix VMAX 20K series with Enginuity:
• 2 engines
• 171 x 450 GB FC drives
• 52 x 2 TB SATA drives
Storage (Site B) 1 Symmetrix VMAX 10K series with Enginuity:

• 1 engine
• 30 x 2 TB NL-SAS drives
• 79 x 600 GB SAS drives
Distributed virtual storage 2 VPLEX Metro cluster with 2 VS2 engines
VMware ESXi servers for 2 4 x eight-core CPUs, 128 GB RAM

templates and virtual (1 server per site)
machines hosting Oracle
Database

Purpose Quantity Configuration
VMware ESXi servers for 2 2 four-core CPUs, 128 GB RAM
vFabric Data Director vApp
VMware ESXi server for VPLEX 2 2 two-core CPUs, 48 GB RAM

Witness
Network switching and 2 Brocade DCX 8510 Backbone with:

routing platform • Fx8-24 FC extension card
• 2 x 48-port FC Blades with 16 Gb FC line
speed support
Brocade MLXe Router
4 Brocade VDX 6720 in VCS mode
Software resources Table 2. Solution software environment
Software Version Purpose

EMC Enginuity 5876.229.145 Symmetrix VMAX operating
environment
EMC VPLEX GeoSynchrony® 5.2 VPLEX operating environment
EMC VPLEX Witness 5.2 Monitor and arbitrator component for

handling VPLEX cluster failure and
inter-cluster communication loss
EMC UnisphereTM for VPLEX 5.2 VPLEX management software
EMC Unisphere T1.6.0.8 VMAX management software
EMC Virtual Storage 5.4.1.8 VMware storage integration

Integrator (VSI)
EMC PowerPath®/VE 5.8 (build 342) Multipathing software
Red Hat Enterprise Linux 5.5 Operating system for all DBVM
VMware vSphere 5.1 Update 1 Hypervisor hosting all virtual

machines
vFabric Data Director 2.5 and 2.7 Database virtualization software—

installed version 2.5 and upgraded to
2.7
Oracle Database 11g Enterprise Edition Oracle database and cluster software
11.2.0.3.6

EMC storage infrastructure
Introduction Overview
This section describes the storage platforms that are virtualized by VPLEX for this
solution:
• Symmetrix VMAX 10K array: Storage platform at Site B
• Symmetrix VMAX 20K array: Storage platform at Site A
The two storage arrays are deployed with a matching LUN configuration.
EMC Symmetrix Symmetrix VMAX 10K is a new enterprise storage platform built to provide leading
VMAX 10K high-end virtual storage capabilities to a growing number of IT organizations and
service providers with demanding storage requirements and limited resources.
Designed for easy installation, setup, and use, VMAX 10K is an ideal entry into a
Symmetrix storage infrastructure for customers who need increased failure-mode
performance. Leveraging the Virtual Matrix Architecture, VMAX 10K provides
enterprise-level reliability, availability, and serviceability.
VMAX 10K includes preconfiguration for easy setup in less than 4 hours.
VMAX 10K is designed for a 100-percent virtually provisioned environment.

Symmetrix Virtual Provisioning™ gives a host, application, or file system the view
that it has more storage than is physically provided. Physical storage is allocated only
when the data is written, rather than when the application is initially configured. This
eliminates manual calculations and can also reduce power and cooling costs by
decreasing the amount of idle storage capacity in the array.
Adding FAST VP for fully automated tiered storage makes the installation and daily
operation easier for IT organizations with limited resources and staff.
Building on EMC's industry-leading VMware integration, VMAX 10K now offers even
more efficient enterprise storage because of new integration with the VMware
vSphere 5 cloud infrastructure platform. EMC Virtual Storage Integrator (VSI) for
VMware simplifies the process of integrating EMC storage into a virtualized
environment.
EMC Enginuity The Enginuity operating environment provides the intelligence that controls all
components in an EMC Symmetrix storage array. Enginuity is an intelligent,
multitasking, preemptive storage-operating environment (SOE) that controls storage
data flow. It is wholly devoted to storage operations and optimized for service levels
required in high-end environments.

EMC Symmetrix VMAX 20K offers an excellent balance among ease of use, performance, high
VMAX 20K availability, and price for smaller enterprises or upper midtier customers who require
a multicontroller storage design. VMAX 20K provides higher levels of scale, capacity,
and performance for customers with more demanding requirements. All VMAX
systems are built on the industry-leading Virtual Matrix Architecture, run the same
Enginuity code, and share a single interface for centralized management—EMC
Unisphere for VMAX.
EMC Unisphere for Unisphere for VMAX is an advanced GUI that provides a common user experience
VMAX across storage platforms. Unisphere for VMAX enables customers to easily provision,
manage, and monitor VMAX environments, as shown in Figure 3.
Figure 3. Unisphere for VMAX

EMC VPLEX Metro infrastructure
EMC VPLEX Metro is the primary enabling technology for the solution. VPLEX Metro is
a SAN-based virtual storage solution that delivers both local and distributed storage
federation. Its breakthrough technology, AccessAnywhereTM, enables the same data
to exist in two separate geographical locations, and to be accessed and updated at
both locations simultaneously. With VPLEX Witness added to the solution,
applications continue to be available, with no interruption or downtime, even in the
event of disruption at one of the data centers.
This section describes the VPLEX Metro infrastructure for the solution, which
comprises the following components:
• EMC VPLEX Metro cluster at each data center (Site A and Site B)
• EMC VPLEX Witness in a separate failure domain (Site C)
EMC VPLEX
EMC VPLEX is a virtual storage solution for both EMC and non-EMC storage arrays.
EMC offers VPLEX in three configurations to address customer needs for high
availability and data mobility, as shown in Figure 4.
Figure 4. VPLEX topologies
EMC VPLEX Metro

This solution uses VPLEX Metro, which includes a unique clustering architecture to
help customers break the boundaries of the data center and allow servers at multiple
data centers to have read/write access to shared block storage devices. VPLEX Metro
delivers active/active, block-level access to data on two sites within synchronous
distances with a round-trip time of up to 5 ms.
EMC VPLEX Witness

VPLEX Witness is an optional external server that is installed as a virtual machine in a
separate failure domain from the VPLEX clusters. VPLEX Witness connects to both
VPLEX clusters using a virtual private network (VPN) over the management IP network,
requiring a round-trip time that does not exceed 1 second.

By reconciling its own observations with information reported periodically by the
clusters, VPLEX Witness enables the cluster(s) to distinguish between inter-cluster
network partition failures and cluster failures and to automatically resume I/O at the
appropriate site.
VPLEX Witness failure-handling semantics apply only to distributed volumes within a

consistency group and only when the detach rules identify a static preferred cluster
for the consistency group (see VPLEX consistency groups on page 18 for more
details).
EMC Unisphere for VPLEX

You can manage and administer a VPLEX environment with the Unisphere for VPLEX
web-based GUI, as shown in Figure 5, or you can connect directly to a management
server and start a VPlexcli (VPLEX command line interface) session.
Figure 5. EMC Unisphere for VPLEX
EMC VPLEX high availability

VPLEX Metro enables application and data mobility and, when configured with VPLEX
Witness, it provides a high-availability infrastructure for clustered applications such
as Oracle RAC. VPLEX Metro enables you to build an extended or stretch cluster as if it
was a local cluster and removes the data center as a single point of failure.
Furthermore, because the data and applications are active at both sites, the solution
provides a simple business continuity strategy.
You can achieve an even higher degree of availability by using a VPLEX Cross-Cluster
Connect configuration. In this case, each host is connected to the VPLEX clusters at
both sites. Such configuration ensures that the host has an alternate path to the
remaining VPLEX cluster in the unlikely event of a full VPLEX cluster failure.

VPLEX logical storage structures
VPLEX encapsulates traditional physical storage array devices and applies layers of
logical abstraction to these exported LUNs, as shown in Figure 6.
Figure 6. VPLEX logical storage structures
A storage volume is a LUN exported from an array and encapsulated by VPLEX. An

extent is the mechanism used by VPLEX to divide storage volumes, using all or part of
the capacity of the underlying storage volume. A device encapsulates an extent or
combines multiple extents or other devices into one large device with a specific RAID
type. A distributed device is a device that encapsulates other devices from two
separate VPLEX clusters.
At the top layer of the VPLEX storage structures are virtual volumes. These volumes
are created from a top-level device (a device or distributed device) and always use
the full capacity of the top-level device. Virtual volumes are the elements that VPLEX
exposes to hosts using its front-end ports. VPLEX presents a virtual volume to a host
through a storage view.
VPLEX can encapsulate devices across heterogeneous storage arrays, including

virtually provisioned thin devices and traditional LUNs.
VPLEX consistency groups

Consistency groups aggregate virtual volumes so that the same detach rules and
other properties can be applied to all volumes in the group. There are two types of
consistency groups:
• Synchronous consistency groups—Used in VPLEX Local and VPLEX Metro to
apply the same detach rules and other properties to a group of volumes in a
configuration. This simplifies configuration and administration on large
systems.

Synchronous consistency groups use write-through caching (known as
synchronous cache mode) and, with VPLEX Metro, are supported on clusters
separated by up to 5 ms of latency. VPLEX Metro sends writes to the back-end
storage volumes, and acknowledges a write to the application only when the
back-end storage volumes in both clusters acknowledge the write.
• Asynchronous consistency groups—Used for distributed volumes in VPLEX
Geo, where clusters can be separated by up to 50 ms of latency.
Detach rules
Detach rules are predefined rules that determine I/O processing semantics for a
consistency group when connectivity with a remote cluster is lost—for example, in the
case of a network-partitioning or remote cluster failure.
Synchronous consistency groups support the following detach rules to determine
cluster behavior during a failure:
• Static preference rule identifies a preferred cluster.
• No-automatic-winner rule suspends I/O on both clusters.
In the event of a cluster partition, the configured detach rule is automatically invoked.
However, VPLEX Witness can be deployed to override the static preference rule and
ensure that the nonpreferred cluster remains active if the preferred cluster fails.
VPLEX Metro Consistency group

solution Consistency groups are particularly important for databases and their applications.
configuration For example:
• Write-order fidelity—To maintain data integrity, all Oracle database LUNs (for
example, data, control and log files) should be placed together in a single
consistency group.
• Transactional dependency—Multiple databases often have transaction
dependencies, such as when an application issues transactions to multiple
databases and expects the databases to be consistent with each other. All
LUNs that require I/O dependency to be preserved should reside in a single
consistency group.
For the solution, a single synchronous consistency group—vORA_DS_DIST—contains all

the virtual volumes that hold the VMFS datastores. The detach rule for the
consistency group has cluster-1 as the preferred cluster.
Configuration process
For the solution, we configured the VPLEX Metro logical storage structures as follows:
• Storage volume—A storage volume is a LUN exported from an array and
encapsulated by VPLEX. Figure 7 shows several storage volumes created at
Site A, as displayed in the VPLEX Management Console.

Figure 7. EMC VPLEX storage volumes (Site A)
• Extent—In the solution, there is a one-to-one mapping between extents and

storage volumes, as shown in Figure 7 and Figure 8.
Figure 8. EMC VPLEX extent (Site A)
• Device—In the solution, there is a one-to-one mapping between devices and

extents. Figure 9 shows the option used to configure this one-to-one
mapping.
Figure 9. EMC VPLEX Device Creation wizard
• Distributed device—In the solution, the distributed devices were created by

mirroring a device remotely in a distributed RAID 1 (DR1) configuration, as
shown in Figure 10.

Figure 10. EMC VPLEX Distributed Devices detail
• Virtual volume—All top-level devices are distributed devices. These devices

are encapsulated by virtual volumes, which VPLEX presents to the hosts
through storage views. The storage views define which hosts access which
virtual volumes on which VPLEX ports.
• Consistency group—Figure 11 shows the consistency group created for the
solution—vORA_DS_DIST.
Figure 11. EMC VPLEX Consistency Group Properties window
VPLEX Witness The solution uses VPLEX Witness to monitor connectivity between the two VPLEX
configuration clusters and ensure continued availability in the event of an inter-cluster network
partition failure or a cluster failure. This is considered a VPLEX Metro HA configuration
as storage availability is ensured at the surviving site.
VPLEX Witness is deployed at a third, separate failure domain (Site C) and connected
to the VPLEX clusters at Site A and Site B. Site C is located at a distance of less than
1-second latency from Sites A and B.

When VPLEX Witness has been installed and configured, the VPLEX Management
Console displays the status of cluster witness components, as shown in Figure 12.
Figure 12. EMC VPLEX Witness components and status
VPLEX data Overview

mobility The VPLEX data mobility feature enables storage elements—extent or devices—to be
migrated nondisruptively within or between clusters. Data mobility offers the
following benefits:
• Consolidation of data centers
• Fast migration of data between data centers
• Movement and relocation of applications and data over distance
• Migration of data between dissimilar arrays to do the following:
 Relocate a "hot" device from slower to faster storage
 Relocate a "cold" device from faster to slower storage
 Retire a storage volume or array

Extent migrations
You use extent migrations to move data between extents in the same cluster. Storage
volumes may be located on the same or different arrays. Figure 13 illustrates extent
migration.
Figure 13. Extent migration on a VPLEX cluster
Device migrations
Nondistributed devices of type RAID 0, RAID 1, and RAID-C can use device migration
to move data between devices on the same cluster or different clusters. Devices can
be built on single or multiple extents or devices. Figure 14 illustrates device
migration.
Figure 14. Migration of a device within or between VPLEX clusters
This solution uses distributed devices. For this type of device, extent migrations are
used to migrate the underlying extent on each leg of the device. Device migration
between distributed devices is not supported.

Data mobility configuration
The prerequisites for the target device or extent are as follows:
• The target must be the same size as or larger than the source device or extent.
If the target is larger than the source, the extra space cannot be used.
• The target must not have any existing volumes on it.
Extent and device mobility jobs can be created, run, and monitored from the VPLEX
CLI or Unisphere for VPLEX GUI.
• VPLEX CLI—You can run migrations as one-time jobs or as batch jobs with
reusable migration plan files. Refer to the EMC VPLEX CLI Guide for more
information.
• Unisphere for VPLEX GUI—The GUI does not support batch mobility jobs.
However, you can migrate multiple extents or devices by using a wizard.
On a VPLEX cluster, up to 25 local and 25 distributed migrations can be in progress at

the same time. Any migrations beyond those limits are queued until an existing
migration is completed.
Figure 15 shows the creation of multiple extent mobility jobs with the Extent Mobility
wizard in Unisphere for VPLEX. It also shows the jobs being run and the job status.
Figure 15. Creating and monitoring extent mobility jobs

Data mobility in this solution
This solution uses an existing VPLEX Metro infrastructure. As part of a technology
refresh, the back-end storage arrays were swapped out and replaced with VMAX
arrays. VPLEX data mobility jobs were used for nondisruptive data migration from the
original arrays to the new arrays.
The systems, in this case the ESXi servers and virtual machines, were powered on and
available throughout the migration, as shown in Figure 16.
Figure 16. ESXi servers, virtual machines, and datastores available during the migration
Back-end array Two methods of volume expansion are available with VPLEX 5.2:
volume expansion
• RAID-C expansion—Online expansion of virtual volumes by concatenating
additional devices and storage volumes. This type of expansion does not
support DR1 expansion.
• Storage volume expansion—Online, nondisruptive expansion of the virtual
volume by taking the full "current capacity" of the storage volume and making
that "configured capacity" of the virtual volume. This type of expansion
supports DR1 expansion.
For this solution, we are interested in storage volume expansion to enable

nondisruptive expansion of a VMFS datastore that resides on a VPLEX virtual volume
built on a distributed device.

Figure 17 shows the size and state of the VMFS datastore and the VPLEX and VMAX
storage elements prior to expansion. Table 3 shows the storage elements.
Figure 17. VMFS datastore and VPLEX and VMAX storage elements before expansion
Table 3. VMFS datastore, VPLEX, and VMAX storage elements
Figure label # Description

1 vCenter 32 GB VMFS datastore, v_OR_GROW_DS_01
2 VPLEX 32 GB virtual volume, v_OR_GROW_DS_01
3 VPLEX distributed device, device_Grow_Symm5936_0A51_1
4 VPLEX extent, Extent_GROW_SYMM5936_0A51_1
5 VPLEX storage volume, GROW_SYMM5936_0A51 on Cluster-1
6 Concatenated metavolume, 0A51 on the VMAX
With GeoSynchrony 5.2 and later, the port enabled flag SCSI_Support (OS2007)
should be set on the initiator group of each VMAX connected to a VPLEX cluster. This
enables VPLEX to automatically detect, in the presence of host I/O, configuration
changes, such as an expanded LUN, on the array storage view.

After increasing the size of metavolume 0A51 to 72 GB, the VPLEX detects the change
and the storage volume shows the new size. You can see the new size in Unisphere
for VPLEX, as shown in Figure 18. LUNs on both arrays need to be grown.
Figure 18. Storage volume expansion after LUN size is increased on array
In Unisphere for VPLEX, choose to expand an expandable virtual volume and enter
the Expand Virtual Volumes dialog, as shown in Figure 19.
Figure 19. Expand Virtual Volumes dialog

When the expansion of the virtual volume has completed, increase the datastore
size, up to a maximum of 2 TB, in vCenter. Figure 20 shows the Increase Datastore
Capacity wizard review screen and the final datastore details.
Figure 20. vCenter Increase Datastore Capacity wizard
VPLEX VPLEX 5.2 delivers enhancements to performance monitoring through the

performance performance monitoring dashboard. This dashboard provides a customizable view
monitoring into the performance of the VPLEX system and enables you to compare different
aspects of system performance, down to the director level.
The dashboard provides various performance metrics, including the following:

• Front-end latency
• Front-end bandwidth
• Front-end throughput
• CPU utilization
• Rebuild status
• WAN link performance
• Back-end latency

Figure 21 shows the front-end and back-end throughput on cluster-2 (the Site B
VPLEX).
Figure 21. VPLEX performance monitoring dashboard

VMware virtualized infrastructure
This solution is fully virtualized using VMware vSphere 5.1. This section describes the
virtualization infrastructure, which uses these components and options:
• VMware vSphere 5.1 Update 1
• VMware vCenter Server
• VMware vSphere Update Manager
• VMware vSphere vMotion
• VMware vSphere High Availability (HA)
• VMware vSphere Distributed Resource Scheduler
• EMC PowerPath/VE for VMware vSphere, Version 5.8
• EMC Virtual Storage Integrator for VMware vSphere, Version 5.4
VMware vSphere 5
VMware vSphere 5 is the industry’s most complete and powerful virtualization
platform. Its infrastructure services transform IT hardware into a high-performance
shared computing platform, and its application services help IT organizations deliver
the highest levels of availability, security, and scalability.
VMware vCenter Server

VMware vCenter is the centralized management platform for vSphere environments,
enabling control and visibility at every level of the virtual infrastructure.
VMware vSphere Update Manager

VMware vSphere Update Manager simplifies vSphere management by managing,
tracking and automating patches and updates of vSphere hosts, virtual machines,
and vApps.
VMware vSphere vMotion

VMware vSphere vMotion supports live migration of virtual machines across servers
with no disruption to users or loss of service.
VMware Storage vMotion enables live migration of virtual machine storage without
any interruption in the availability of the virtual machine.
VMware vSphere High Availability

VMware vSphere High Availability (HA) is a vSphere component that provides high
availability for any application running in a virtual machine, regardless of its
operating system or underlying hardware configuration.

VMware vSphere Distributed Resource Scheduler
VMware vSphere Distributed Resource Scheduler (DRS) dynamically and
automatically balances load distribution and virtual machine placement across
multiple ESXi servers.
EMC PowerPath/VE
EMC PowerPath/VE for VMware vSphere delivers PowerPath multipathing features to
optimize VMware vSphere virtual environments. PowerPath/VE is installed as a kernel
module on the ESXi host and works as a multipathing plug-in (MPP) that provides
enhanced path management capabilities to ESXi hosts.
EMC Virtual Storage Integrator for VMware vSphere

EMC Virtual Storage Integrator (VSI) for VMware vSphere is a plug-in to the VMware
vSphere client that provides a single management interface for managing EMC
storage within the vSphere environment. VSI provides a unified and flexible user
experience that allows each feature to be updated independently and new features to
be introduced rapidly in response to changing customer requirements.
When PowerPath/VE is installed on an ESXi host, VSI presents important multipathing

details for devices, such as the load-balancing policy, the number of active paths,
and the number of dead paths.
VMware EMC VPLEX Metro delivers concurrent access to the same set of devices at two
deployments on physically separate locations and thus provides the active/active infrastructure that
VPLEX Metro enables geographically stretched clusters based on VMware vSphere. The use of
Brocade Virtual Link Aggregation Group (vLAG) technology enables extension of
VLANs, and hence subnets, across different physical data centers.
By deploying VMware vSphere features and components with VPLEX Metro, the
following functionality can be achieved:
• The ability to live-migrate virtual machines between sites, in anticipation of
planned events such as hardware maintenance, through the use of vMotion.
• The ability to migrate a virtual machine’s storage without any interruption in
the availability of the virtual machine, through the use of Storage vMotion.
This allows the relocation of live virtual machines to new datastores.
• Automatic load distribution and virtual machine placement across sites
through the use of DRS groups and affinity rules.
• Automatic application restart for any site-level disaster through the use of
VMware HA.
A VPLEX Metro environment configured with VPLEX Witness is considered a

VPLEX Metro HA configuration, as it ensures storage availability at the
surviving site in the event of a site-level failure. Combining VPLEX Metro HA
with a host failover clustering technology such as VMware HA provides
automatic application restart for any site-level disaster. Figure 22 illustrates
this HA architecture.

Figure 22. VMware HA with VPLEX Witness–logical view
• Increased protection of the VMware HA cluster through the addition of a

cross-cluster connect between the local VMware ESXi servers and the VPLEX
cluster on the remote site.
Local data unavailability events, which VMware vSphere 5.0 does not
recognize, can occur when there is not a full site outage. Cross-connecting
vSphere environments to VPLEX clusters protects against this and ensures
that failed virtual machines automatically move to the surviving site.
VPLEX Cross-Cluster Connect is available for up to 1 ms of distance-induced
latency.
This solution uses VPLEX Metro HA with Cross-Cluster Connect to maximize
the availability of the VMware virtual machines, as shown in Figure 23. 1
1
For detailed information, see the EMC TechBook: EMC VPLEX Metro Witness Technology and
High Availability.

Figure 23. VMware HA with VPLEX Witness and Cross-Cluster Connect–logical view
VMware stretched VMware and EMC support a stretched cluster configuration that includes ESXi hosts
cluster from multiple sites 2. For the solution, a single vSphere cluster—SiteAandSiteB—is
configuration stretched between Site A and Site B by using a distributed VPLEX virtual volume with
VMware HA and VMware DRS. There are four hosts in the cluster, two at each site.
VPLEX Metro HA Cross-Cluster Connect provides increased resilience to the
configuration.
In vCenter, it is easy to view the configuration of this cluster and the features enabled
for it, as shown in Figure 24. This view also shows the memory, CPU, and storage
resources available to the cluster.
Figure 24. vSphere cluster with HA and DRS enabled
2
For detailed requirements and scenarios, see the VMware Knowledge Base article 1026692:
Using VPLEX Metro with VMware HA

Each ESXi server is configured with two 10 GbE physical adapters to provide network
failover and high performance. A vSphere distributed switch (dvSwitch) 3 provides a
single, common switch across all hosts. The 10 GbE physical adapters (also referred
to as uplink adapters) are assigned to the dvSwitch.
Two distributed port groups are assigned to the dvSwitch:

• dvVMNetwork86—For virtual machine network traffic
• dvCenter Network—For VMkernel traffic and, in particular, vMotion traffic
Figure 25 shows the dvSwitch configuration. As both vSphere 5.1 distributed

switches and Brocade VCS switches support Link Layer Discovery Protocol (LLDP), the
properties of the associated physical switches can also be easily identified from
vCenter.
Figure 25. dvSwitch configuration and LLDP detail
Datastores v_ORA_DS_vFDD_vol1 and v_ORA_DS_vFDD_vol2 were created on 1.5 TB

VPLEX distributed volumes and presented to the ESXi hosts in the stretch cluster. Six
secondary datastores sized at 256 GB were also created. All virtual machines under
vFabric Data Director control would reside on these datastores.
3
A dvSwitch provides a network configuration that spans all member hosts and allows virtual
machines to maintain consistent network configuration as they migrate between hosts. For
further information, see the VMware vSphere Networking ESXi 5.1 document.

VMware vSphere Enabling VMware vSphere HA and VMware vSphere DRS
HA configuration vSphere HA leverages multiple ESXi hosts, configured as a cluster, to provide rapid
recovery from outages and cost-effective high availability for applications running in
virtual machines. 4 vSphere HA protects application availability in the following ways:
• It protects against a server failure by restarting the virtual machines on other
ESXi servers within the cluster.
• It protects against application failure by continuously monitoring a virtual
machine and resetting it in the event of guest OS failure.
For the solution, both vSphere HA and DRS were enabled, as shown in Figure 26.
Figure 26. vSphere HA wizard
VM Monitoring
VM Monitoring was configured to restart individual virtual machines if their heartbeat
is not received within 60 seconds.
Virtual machine restart options

The VM Restart Priority option for the infrastructure virtual machines was set to High.
This ensures that these virtual machines are powered on first in the event of an
outage. Figure 27 shows this setting and the Host Isolation Response setting
(default).
4
For further information on vSphere HA, see the VMware vSphere Availability ESXi 5.0
document.

Figure 27. VM Restart Priority and Host Isolation Response settings
Datastore heartbeating
When you create a vSphere HA cluster, a single host is automatically elected as the
master host. The master host monitors the state of all protected virtual machines and
of the slave hosts. When the master host cannot communicate with a slave host, it
uses datastore heartbeating to determine whether the slave host has failed, is in a
network partition, or is network isolated.
To meet vSphere HA requirements for datastore heartbeating, a minimum of two

datastores are required, as shown in Figure 28. In a production environment, vCenter
automatically selects two or more datastores for this purpose, based on host
visibility.
Figure 28. vSphere HA Cluster Status–heartbeat datastores
VMware vSphere VMware DRS host groups and virtual machine groups
DRS configuration DRS host groups and virtual machine groups simplify management of ESXi host
resources. These features were not required for this solution.

VMware DRS affinity rules
DRS uses affinity rules to control the placement of virtual machines on hosts within a
cluster. DRS provides two types of affinity rules:
• A VM-Host affinity rule specifies an affinity relationship between a group of
virtual machines and a group of hosts.
• A VM-VM affinity rule specifies whether particular virtual machines should run
on the same host or be kept on separate hosts.
For this solution DRS was utilized to control placement of the primary and secondary
fault-tolerant virtual machine.
Beginning with vFabric Data Director 2.7, virtual machine DRS groups can be assigned
resource bundles, enabling DBVMs to be restricted to a chosen group of hosts via
VM-Host affinity rules.
VMware vSphere A Red Hat Enterprise Linux 5 virtual machine, ftnfsvm, shown in Figure 29, was
Fault Tolerance created on an ESXi server on Site B, tce-orap-r910c. This virtual machine acts as an
NFS mount server, which allows for sharing of software and files in the vFabric Data
Director infrastructure.
Figure 29. Virtual machines on ESXi server tce-orap-r910c
vSphere Fault Tolerance with vSphere Distributed Resource Scheduler (DRS) was
enabled on the virtual machine with the secondary shadow virtual machine residing
on tce-orap-r910a, as shown in Figure 30.
Figure 30. Virtual machines on ESXi server tce-orap-r910a

The Enhanced vMotion Compatibility (EVC) feature was first enabled on the virtual
machine ftnfsvm, as shown in Figure 31.
Figure 31. Summary details of virtual machine ftnfsvm
This process allows fault tolerant virtual machines to benefit from better initial
placement and also to be included in the cluster's load balancing calculations.
In addition to the following information, see the VMware Fault Tolerance

Recommendations and Considerations white paper.
If either the host running the primary virtual machine or the host running the
secondary virtual machine fails, an immediate and transparent failover occurs. The
functioning ESXi host seamlessly becomes the primary virtual machine host without
losing network connections or in-progress transactions. With transparent failover,
there is no data loss and network connections are maintained.
After a transparent failover occurs, a new secondary virtual machine is re-spawned

and redundancy is re-established. The entire process is transparent and fully
automated, and occurs even if vCenter Server is unavailable.
With vSphere DRS enabled, the placement of the standby secondary virtual machine
after failover is enforced using rules. To create a DRS affinity rule, DRS cluster virtual
machine groups and cluster host groups are first created. In the DRS host group only
the required hosts are added. These nodes are selected, as they are on different
sites, improving availability in the event of a site failure. Figure 32 shows the DRS
groups and rules for virtual machine ftnfsvm.

Figure 32. DRS groups and rules for virtual machine ftnfsvm
The detail and application of the rule is shown in Figure 33. The rule forces the
placement of virtual machines to run only on the hosts in the host group in the event
of failover.
Figure 33. Detailed DRS rules for virtual machine ftnfsvm
EMC Virtual EMC Virtual Storage Integrator (VSI) provides enhanced visibility into VPLEX directly
Storage Integrator from the vCenter GUI. The Storage Viewer and Path Management features are
and VPLEX accessible through the EMC VSI tab, as shown in Figure 34.
In the solution, VPLEX distributed volumes host the Virtual Machine File System
(VMFS) datastores: v_ORA_DS_vFDD_vol1 and v_ORA_DS_vFDD_vol2. Storage Viewer
provides details of the datastore’s virtual volumes, storage volumes, and paths.

As shown in Figure 34, the LUNS that make up the VMFS datastore
v_ORA_DS_vFDD_vol2 are distributed across RAID 1 VPLEX Metro volumes accessible
via PowerPath.
Figure 34. VSI Storage Viewer–datastores

VMware vFabric Data Director
Introduction VMware vFabric Data Director is a unified DBaaS platform that enables DBAs and
cloud service providers to virtualize and automate lifecycle management of
heterogeneous databases. As of version 2.7, vFabric Data Director supports the
following databases:
• Oracle 10gR2 and 11gR2 on Linux
• Microsoft SQL Server 2008 and 2012
• VMware vFabric Postgres
• MySQL
vFabric Data Director delivers the following key sets of capabilities:

• Database-aware virtualization
• Automated lifecycle management
• Self-service
Figure 35 shows how vFabric Data Director enables self-service database

provisioning, cloning, and backup for simpler database management for DBAs and
developers.
Figure 35. vFabric Data Director service layers, support, and integration
vFabric Data Director also enables DBAs and system administrators to control access
and manage resource usage within the vFabric Data Director environment. Further, its
use of REST APIs enables integration with other provisioning and management tools.

Architecture The vFabric Data Director vApp is made up of two virtual machines:
• Management Server, which is accessed via the web console to control the
environment and deploy and manage databases
• DB Name Server
Data Director components communicate by using the following vSphere networking

resources:
• vCenter Network—Carries management traffic between vCenter Server and the

Data Director Management Server. This network carries commands that the
Management Server uses to interact with vCenter Server and all the ESXi
hosts managed by that vCenter Server.
• Web Console Network—Carries traffic between Web clients (console) and the
Data Director Management Server.
You specify the vSphere port groups to use for both vCenter Network and Web
Console Network during Data Director vApp deployment. With express
installation, these two networks are mapped to the same vSphere network.
• DB Access Network (one or more)—Carries SQL traffic between database
clients and the DBVM. Each DB Access Network must have DHCP enabled.
• Internal Network—Carries internal management traffic among databases, the
Data Director Management Server, and the DB Name Server. For security,
internal management traffic must have its own network, not sharing a network
with other types of traffic, and it must have DHCP enabled.
• DB Name Service Network—The DB Name Server performs database
connection services, translating database names to IP addresses, and must
be reachable from database clients.
Organizational The basic component of Data Director is the organization. Data Director system
structures and administrators create organizations, assign the initial organization administrator, and
user management allocate resources to the organization.
Unique organization names are assigned within Data Director. At the next level down,
databases are organized into database groups. Roles of administrator and template
user are assigned with applicable privileges to users within the organization.
Organization roles, policies, and templates apply only within that organization.
Resources allocated to an organization are reserved for that organization and cannot
be shared among multiple organizations. This restriction enhances security and
ensures resource isolation.
Figure 36 shows the Data Director organizational structure used in this solution along
with the assigned users and their level of access.

Figure 36. vFabric Data Director organizational structure example
Resource A resource bundle is a set of compatible IT resources for provisioning databases. It

management combines CPU, memory, storage, and network resources into a single entity that is
then assigned to an organization.
Resource bundles are associated with vSphere resource pools, but they also include
datastores and network IP addresses that will be used for databases that are
provisioned through vFabric Data Director, as shown in Figure 37.
Figure 37. Mapping vFabric Data Director resource bundles to vSphere

Figure 38 shows the Resource Bundles window with details of the default and
production organizations and the newly created, but unassigned, development
resource bundle.
Figure 38. vFabric Data Director Resource Bundles window
As of vFabric Data Director 2.7, resource bundles can also be associated with DRS
groups based on hardware requirements and site availability. Figure 39 shows the
assignment of the virtual machine DRS group and the mapping to the VM-Host affinity
rules configured in vCenter.

Figure 39. DRS group assignment to a resource bundle in vFabric Data Director 2.7
In this solution, the VM-Host affinity “must” rule ensures that the DBVMs are
restricted to two of the four physical servers, one on each site. It restricts the virtual
machines on which Oracle software is installed and/or running to these two physical
servers.
Installation and The vFabric Data Director vApp is deployed from an Open Virtualization Format (OVF)
configuration template using the vSphere Client. In this solution, vFabric Data Director 2.5 was
initially deployed, as shown in Figure 40.

Figure 40. Summary screen of OVF Deployment wizard
After the vApp is deployed, using a web browser, enter the static IP address assigned
during virtual machine deployment to log in to the Data Director application server
and complete configuration, as shown in Figure 40. Follow the Setup wizard, shown in
Figure 41, to quickly configure Data Director.
Figure 41. Summary screen of Data Director Setup wizard

The vFabric Data Director vApp was upgraded to the newly available version 2.7 using
vSphere Update Manager. The upgrade enabled use of new features such as storage
allocation by operating system, database log, and backup, as shown in Figure 42.
Figure 42. Storage allocation by type in vFabric Data Director 2.7
Managing Database lifecycle

databases In Data Director, the database lifecycle includes the following:
• Preparing base database templates from DBVMs
• Creating databases and allocating resources
• Managing the database schema and data
• Performing backup and recovery tasks
• Ingesting databases into Data Director
• Decommissioning databases
The vFabric Data Director interface provides a guided checklist to take you through
the steps required to prepare your environment for the database lifecycle, as shown
in Figure 43.

Figure 43. Getting Started Checklist in Data Director
Base database templates

You can download a prebuilt or blank Base DBVM template from www.vmware.com.
The prebuilt template comes fully configured with SUSE Linux Enterprise Server 11
and vFabric Data Director software installed. The blank DBVM template has the virtual
hardware configured but requires a custom OS installation.
Following the instructions in the VMware vFabric Data Director Administrator and User
Guide, we created a custom OS installation with Red Hat Enterprise Linux 5.5 and
initialized the vFabric Data Director software. This virtual machine was updated to
11.2.0.3.6 with the latest patch set update applied.
Before the base DBVM template can be deployed to a resource bundle, the following
two-step process is required:
• Conversion of the base DBVM
• Validation of the base DBVM template
The validation goes through the process of provisioning, backup, cloning, and so on
and requires resources to complete these steps successfully. EMC recommends that
the database group have at least 40 GB of available storage to successfully complete
the validation. Figure 44 shows an example of successful validation.

Figure 44. Validation of base DBVM template
After the validation is successful, the base DVBM template can be assigned and
enabled in a resource bundle within an organization, as shown in Figure 45.
Figure 45. Enabling base DBVM template in Data Director
Creating databases
As a DBA or application developer, you can create and provision databases within
your organization using several methods:
• Create a new empty database.
• Create a database from a catalog: When you require a database with known
characteristics and preloaded data, you can create and store read-only
catalog databases within Data Director. You can also clone a database to the
catalog.

• Create a database by cloning an existing database visible to your
organization.
• Ingest an external database: Either create a "golden clone," which cannot be
modified within Data Director, or refresh an existing database from an
external database. Golden clones are used as a source when you are cloning
databases.
Figure 46 shows the options presented by the Create Database wizard.
Figure 46. Creation Type options in the Create Database wizard
Figure 47 shows the creation of an empty Oracle database from a template in the
organization and database group Production.

Figure 47. Creating an empty database from a template
Cloning databases
You can clone databases and customize their deployment to suit your needs. You can
specify new resource settings, database parameter settings, and backup settings for
the clone, choose the clone point, set an immediate backup, and set an expiration
date for the clone. You can also choose to add to the catalog a clone of an existing
database as shown in Figure 48.

Figure 48. Adding a database to the catalog as a full clone
Ingesting databases
You can ingest an external database from an NFS share where the RMAN backup of
the source database is stored into a base database template of the same Oracle
version. The Create Database wizard guides and controls the process. In this solution,
the vSphere fault-tolerant virtual machine ftnfsvm provided the highly available NFS
share.
Backup and restore

Data Director contains a number of backup templates with settings that use a
combination of methods to safeguard data, provide consistent database backups,
and enforce limits on resource consumption. Figure 49 shows the backup template
screen from the Create Database wizard.

Figure 49. Custom backup template
Backups are highly configurable with Data Director and matched to your strategy and
needs.
For example, for a production database with a high transaction volume and business
rules that require the highest possible database resiliency, you might define the
following backups:
• Take full external backups twice a day.
• Take database snapshots every hour.
• Enable point-in-time recovery to keep a continuous log of all transactions as
they occur on the running database.
• Retain full backups for a month or more.
vFabric Data Director provides the following backup types:

• External backups, which are full copies of the database saved to a datastore
separate from the database.
• Snapshot backups, which capture the changes to the database after the
snapshot is taken. Snapshots initially use less storage than external backup
files and take just a few minutes regardless of database size.
• Point-in-time recovery, which uses a write-ahead log (WAL) that continuously
records every change made to the database while the database is running. In
the event of a failure, you can replay the WAL to restore the database to its
state at a point in time within the retention period of the database backups.

Monitoring the Data Director provides a customizable monitoring and reporting solution for system
Data Director and organization administrators. In the solution, each organization has a dashboard
environment that provides a single interface for monitoring the Data Director environment. As
shown in Figure 50, administrators can view tasks and alarms, and monitor
performance indicators such as CPU utilization and memory utilization.
Figure 50. Production organization dashboard after first database creation

Conclusion
Summary VMware vFabric Data Director improves agility and drastically reduces database TCO
through database-aware virtualization and self-service lifecycle management. It
empowers both administrators and developers by automating routine tasks such as
database provisioning, high availability, backup, and cloning in a secure, controlled
environment.
Customers can expect to achieve real CapEx savings through reduced license costs
resulting from the virtualization and consolidation of physical servers. By assigning
virtual machine DRS groups to vFabric Data Director resource bundles, customers can
use VM-Host affinity “must” rules for control of virtual machine placement and gain
improved software license compliance management. In this solution, the virtual
machines on which Oracle software is installed and/or running are restricted to two of
the four physical servers in the cluster.
VPLEX Metro underpins and enables the stretched VMware vSphere HA cluster in this
solution. This extends the vSphere HA and DRS functionality, required by Data
Director, across sites to improve service availability and increase utilization of
resources.
Fast, self-service database provisioning with built-in backup and restore and process
automation are managed through a single user interface. These features combined
with improved service availability offer meaningful OpEx savings and enable the DBA
and system administrator to concentrate on other duties.
Data Director’s organizational framework enables simple administration of a secure
multitenant, virtualized environment. This framework ensures isolation of databases
and gives control of access and resources to the correct application owner and users.
Findings The key findings of the testing performed for the solution include:
• The Data Director vApp can be deployed quickly and easily on any VMware
vSphere cluster with HA and DRS enabled using the vSphere Client.
• VPLEX as an active/active data protection product, combined with VMware
vSphere clustering, provides a stretched vSphere cluster with HA and DRS
running over distance between two data centers, offering the following
benefits:
 Increased utilization of hardware and software assets
 Automatic load balancing between data centers
 Zero downtime maintenance using vMotion
 Quick restart of virtual machines across data centers with vSphere HA
 Instant transition to a new host for the fault-tolerant virtual machine,
offering a highly available NFS service
• The Data Director application, accessed via a web browser, is simple to

configure using the Data Director Setup wizard.

• Data Director users and organizations provide secure, policy-based access
control to resources within an organization.
• You can deploy DBVM templates from prebuilt images or create a customized
operating-system and patched database install to meet your specific needs.
• You can create and deploy databases using a number of methods:
 Create a new empty database from a DB template assigned to your
organization.
 Create a new database with known characteristics and preloaded data
from a catalog-stored database.
 Create a database by cloning an existing database visible to your
organization.
 Ingest an external Oracle database from an RMAN backup presented from
an NFS share, which in this solution is supplied by a virtual machine using
VPLEX Metro with VMware HA and fault tolerance.
• VPLEX as a virtual storage technology enables the following:

 Nondisruptive movement of data between storage arrays without
application impact
 Growth of VMFS datastores through the nondisruptive expansion of a
virtual volume

References
Product For additional information, see the following product documents:
documentation
• EMC VPLEX Metro Witness Technology and High Availability TechBook
• EMC VPLEX with GeoSynchrony 5.2 Release Notes
• EMC VPLEX 5.2 Administration Guide
• VMware vFabric Data Director Installation Guide—vFabric Data Director 2.5
• VMware vFabric Data Director Administrator and User Guide—vFabric Data
Director 2.5
• VMware vFabric Data Director Administrator and User Guide—vFabric Data
Director 2.7
• VMware vFabric Data Director Installation Guide—vFabric Data Director 2.7
• VMware vSphere Networking—ESXi 5.0
• VMware vSphere Availability—ESXi 5.0
• Oracle Database Upgrade Guide—11g Release 2 (11.2)
White papers For additional information, see the following white papers:
• Using VMware vSphere with EMC VPLEX—Best Practices Planning
• Conditions for Stretched Hosts Cluster Support on EMC VPLEX Metro
• VMware Fault Tolerance Recommendations and Considerations
• Using VPLEX Metro with VMware High Availability and Fault Tolerance for
Ultimate Availability
• Understanding Oracle Certification, Support and Licensing for VMware
Environments
Other For additional information, see the following articles:
documentation
• VMware Knowledge Base article 1026692: Using VPLEX Metro with VMware
HA
• VMware Knowledge Base article 2007545: Implementing vSphere Metro
Storage Cluster (vMSC) using EMC VPLEX


Delivering A Highly Available Oracle Database As A Service: Emc Vplex, Vmware Vfabric Data Director

Uploaded by

Copyright:

Available Formats

Delivering A Highly Available Oracle Database As A Service: Emc Vplex, Vmware Vfabric Data Director

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Delivering A Highly Available Oracle Database As A Service: Emc Vplex, Vmware Vfabric Data Director

Uploaded by

Copyright:

Available Formats

White Paper

DELIVERING A HIGHLY AVAILABLE ORACLE

EMC Solutions Group

EMC believes the information in this publication is accurate as of its

The information in this publication is provided as is. EMC Corporation makes no

Use, copying, and distribution of any EMC software described in this

Part Number H12235

Delivering a Highly Available Oracle Database as a Service 2

Solution overview ............................................................................................................................. 10

EMC storage infrastructure ............................................................................................................... 14

EMC VPLEX Metro infrastructure ....................................................................................................... 16

Delivering a Highly Available Oracle Database as a Service 3

VMware virtualized infrastructure ..................................................................................................... 30

VMware vFabric Data Director ........................................................................................................... 41

Delivering a Highly Available Oracle Database as a Service 4

Delivering a Highly Available Oracle Database as a Service 5

This solution offers the following business benefits:

Delivering a Highly Available Oracle Database as a Service 6

Combining VMware Distributed Resource Scheduler (DRS) groups with VMware

Delivering a Highly Available Oracle Database as a Service 7

• Nondisruptive movement of data between storage arrays without application

VPLEX Metro as an active/active data protection product, combined with VMware

• Increased utilization of hardware and software assets

Delivering a Highly Available Oracle Database as a Service 8

Delivering a Highly Available Oracle Database as a Service 9

DBaaS adds a self-service lifecycle management framework to offer a secure,

Delivering a Highly Available Oracle Database as a Service 10

Delivering a Highly Available Oracle Database as a Service 11

Hardware Table 1. Solution hardware environment

Storage (Site B) 1 Symmetrix VMAX 10K series with Enginuity:

Distributed virtual storage 2 VPLEX Metro cluster with 2 VS2 engines

VMware ESXi servers for 2 4 x eight-core CPUs, 128 GB RAM

Delivering a Highly Available Oracle Database as a Service 12

VMware ESXi server for VPLEX 2 2 two-core CPUs, 48 GB RAM

Network switching and 2 Brocade DCX 8510 Backbone with:

Brocade MLXe Router

4 Brocade VDX 6720 in VCS mode

Software resources Table 2. Solution software environment

Software Version Purpose

EMC VPLEX GeoSynchrony® 5.2 VPLEX operating environment

EMC VPLEX Witness 5.2 Monitor and arbitrator component for

EMC UnisphereTM for VPLEX 5.2 VPLEX management software

EMC Unisphere T1.6.0.8 VMAX management software

EMC Virtual Storage 5.4.1.8 VMware storage integration

EMC PowerPath®/VE 5.8 (build 342) Multipathing software

VMware vSphere 5.1 Update 1 Hypervisor hosting all virtual

vFabric Data Director 2.5 and 2.7 Database virtualization software—

Delivering a Highly Available Oracle Database as a Service 13

VMAX 10K is designed for a 100-percent virtually provisioned environment.

Delivering a Highly Available Oracle Database as a Service 14

Figure 3. Unisphere for VMAX

Delivering a Highly Available Oracle Database as a Service 15

Figure 4. VPLEX topologies

EMC VPLEX Metro

EMC VPLEX Witness

Delivering a Highly Available Oracle Database as a Service 16

VPLEX Witness failure-handling semantics apply only to distributed volumes within a

EMC Unisphere for VPLEX

Figure 5. EMC Unisphere for VPLEX