Commvault Professional Course Guide
Commvault Professional Course Guide
Page 1 of 559
Legal Notices
Copyright
Information in this document, including URL and other website references, represents the current view of Commvault Systems, Inc.
as of the date of publication and is subject to change without notice to you.
Descriptions or references to third party products, services or websites are provided only as a convenience to you and should not
be considered an endorsement by Commvault. Commvault makes no representations or warranties, express or implied, as to any
third-party products, services or websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners. Unless
otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and
events depicted herein are fictitious.
Complying with all applicable copyright laws is the responsibility of the user. This document is intended for distribution to and use
only by Commvault customers. Use or distribution of this document by any other persons is prohibited without the express written
permission of Commvault. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or
introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or
otherwise), or for any purpose, without the express written permission of Commvault Systems, Inc.
Commvault may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject
matter in this document. Except as expressly provided in any written license agreement from Commvault, this document does not
give you any license to Commvault’s intellectual property.
COMMVAULT MAKES NO WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED, AS TO THE INFORMATION CONTAINED IN THIS
DOCUMENT.
©2018 Commvault Systems, Inc. All rights reserved. Commvault, Commvault and logo, the “C” hexagon logo, Commvault Systems,
Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick
Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell,
APSS, Commvault GO, Commvault HyperScale, IntelliSnap, ROMS, Commvault OnePass, CommVault Edge, Edge Drive, and
CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third-party brands, products, service
names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective
owners. All specifications are subject to change without notice.
Confidentiality
The descriptive materials and related information in the document contain information that is confidential and proprietary to
Commvault. This information is submitted with the express understanding that it will be held in strict confidence and will not be
disclosed, duplicated or used, in whole or in part, for any purpose other than evaluation purposes. All right, title and intellectual
property rights in and to the document is owned by Commvault. No rights are granted to you other than a license to use the
document for your personal use and information. You may not make a copy or derivative work of this document. You may not sell,
resell, sublicense, rent, loan or lease the document to another party, transfer or assign your rights to use the document or
otherwise exploit or use the Manual for any purpose other than for your personal use and reference. The document is provided "AS
IS" without a warranty of any kind and the information provided herein is subject to change without notice.
Page 2 of 559
Education Services Commvault® Professional Course May 2018
Table of Contents
Contents
Legal Notices ......................................................................................................................................................... 2
Table of Contents .................................................................................................................................................. 3
Introduction .......................................................................................................................................................... 8
Commvault® Professional Course Preliminaries ..................................................................................................... 9
Education Advantage .................................................................................................................................... 10
Class Resources ............................................................................................................................................ 11
CVLab On Demand Lab Environment .............................................................................................................. 12
Commvault® Education Career Path ............................................................................................................... 13
Commvault® On-Demand Learning ................................................................................................................ 14
Education Services V11 Certification ............................................................................................................... 15
Additional Resources ..................................................................................................................................... 17
Course Overview ........................................................................................................................................... 18
CommCell® Deployment and Configuration ............................................................................................................ 19
Next Generation Platform .................................................................................................................................. 20
Next Generation Platform Overview................................................................................................................ 21
Physical Architecture ..................................................................................................................................... 22
Logical Architecture ....................................................................................................................................... 25
Web Console ................................................................................................................................................ 27
Admin Console .............................................................................................................................................. 29
CommCell® Console....................................................................................................................................... 39
CommCell® Component Requirements ............................................................................................................... 46
CommServe® Server Requirements ................................................................................................................ 47
MediaAgent Requirements ............................................................................................................................. 49
Agent Requirements ...................................................................................................................................... 53
CommCell® Deployment .................................................................................................................................... 54
Download the Commvault® Software ............................................................................................................. 55
CommCell® Deployment Overview .................................................................................................................. 58
CommServe® Server – Gathering Information ................................................................................................. 59
CommServe® Server – Installation.................................................................................................................. 61
CommServe® Server – Post Installation Tasks ................................................................................................. 65
CommCell® License ....................................................................................................................................... 66
CommCell® Update Process ........................................................................................................................... 67
Page 3 of 559
Education Services Commvault® Professional Course May 2018
Page 4 of 559
Education Services Commvault® Professional Course May 2018
Servers ...........................................................................................................................................................234
Admin Console Concepts ..............................................................................................................................235
Server Navigation Structure ..........................................................................................................................238
Administrative Tasks ....................................................................................................................................242
Server Groups .................................................................................................................................................259
Server Groups (Client Computer Groups) .......................................................................................................260
File Server Solution .............................................................................................................................................269
File Server Solution Overview .......................................................................................................................270
Add a File Server Solution.............................................................................................................................271
Data Protection Plans ...................................................................................................................................275
Subclient .....................................................................................................................................................279
Subclient Configuration ................................................................................................................................282
Filtering .......................................................................................................................................................289
Job and Data Management ..................................................................................................................................296
Data Protection Overview .............................................................................................................................297
Synthetic Full and DASH Full Jobs .................................................................................................................300
Auxiliary Copy and DASH Copy Jobs ..............................................................................................................303
Job Initiation ...............................................................................................................................................313
Using Schedules and Schedule Policies ..........................................................................................................315
Automatic Schedules ....................................................................................................................................319
Managing Active Jobs ...................................................................................................................................324
Controlling Job Activity .................................................................................................................................336
Data Recovery ................................................................................................................................................351
Data Recovery Overview ..............................................................................................................................352
Using the Find Feature .................................................................................................................................354
Using Browse ...............................................................................................................................................356
Additional Recovery Methods ........................................................................................................................366
Virtualization Solution .........................................................................................................................................371
Virtual Protection Overview ..........................................................................................................................372
Transport Modes ..........................................................................................................................................374
Configuring the Virtualization Solution ..............................................................................................................378
Adding Virtualization Solution .......................................................................................................................379
VSA Filters ...................................................................................................................................................384
VSA AppAware Backup .................................................................................................................................391
Admin Console – VSA Tasks..........................................................................................................................395
Traditional File Recovery vs. Block-Level Browse ............................................................................................407
Page 5 of 559
Education Services Commvault® Professional Course May 2018
Page 6 of 559
Education Services Commvault® Professional Course May 2018
Page 7 of 559
Education Services Commvault® Professional Course May 2018
Introduction
Page 8 of 559
Education Services Commvault® Professional Course May 2018
Page 9 of 559
Education Services Commvault® Professional Course May 2018
Education Advantage
The Commvault® Education Advantage product training portal contains a set of powerful tools to enable Commvault
customers and partners to better educate themselves on the use of the Commvault software suite. The portal
includes:
Page 10 of 559
Education Services Commvault® Professional Course May 2018
Class Resources
Course manuals and activity guides are available for download for Instructor-Led Training (ILT) and Virtual Instructor-
Led Training (vILT) courses. It is recommended to download these documents the day prior to attending class to
ensure the latest document versions are being used.
Self-paced eLearning courses can be launched directly from the EA page. If an eLearning course is part of an ILT or
vILT course, it is a required prerequisite and should be viewed prior to attending class.
If an ILT or vILT class will be using the Commvault® Virtual Lab environment, a button will be used to launch the lab
on the first day of class.
Commvault® certification exams can be launched directly from the EA page. If you are automatically registered for an
exam as part of an ILT or vILT course, it will be available on the final day of class. There is no time limit on when the
exams need to be taken, but it is recommended to take them as soon as you feel you are ready.
Page 11 of 559
Education Services Commvault® Professional Course May 2018
The CVLab time can be purchased as standalone on-demand CVLab time, or to extend lab time for training courses
attended. Extending CVLab time must be purchased within 48-hours after class end time to maintain your lab
progress from the training course. Whether purchasing on-demand or extending; CVLab connect time may be
purchased in four-hour blocks in any quantity. Access will be available for 90 days from point of purchase and is
priced at just one Training Unit per four-hour block.
Page 12 of 559
Education Services Commvault® Professional Course May 2018
Page 13 of 559
Education Services Commvault® Professional Course May 2018
Commvault On-Demand Learning is a convenient, flexible, and cost-effective training solution that gives you the tools
to keep a step ahead of your company’s digital transformation initiatives. You and your company will benefit by:
Page 14 of 559
Education Services Commvault® Professional Course May 2018
Commvault's Certification Program offers Professional-level, Engineer-level, and Master-level certifications. This
Program provides certification based on a career path, and enables advancement based on an individual’s previous
experience and desired area of focus. It also distinguishes higher-level certifications such as Engineer and Master from
lower-level certification as a verified proof of expertise.
Key Points
• Certification is integrated with and managed through Commvault's online registration in the Education
Advantage Customer Portal.
• Cost of certification registration is included in the associated training course.
• Practice assessments are given in class.
• The Commvault Certified Professional Exam Prep course is also available.
• Students may take the online certification exam(s) any time after completing the course.
• Although it is recommended to attend training prior to attempting an exam, it is not required.
Page 15 of 559
Education Services Commvault® Professional Course May 2018
• CommCell Administration – user and group security, configuring administrative tasks, conducting data
protection and recovery operations, and CommCell monitoring.
• Storage Administration – deduplication configuration, disk library settings, tape library settings, media
management handling, and snapshot administration.
• CommCell Implementation – CommServe® server design, MediaAgent design and placement, indexing
settings, client and agent deployment, and CommCell maintenance.
Certification status as a Commvault Certified Professional requires passing one exam: Commvault® Professional Exam.
• Advanced Infrastructure Design – this exam validates expertise in deploying medium and enterprise level
CommCell® environments with a focus on storage design, virtual environment protection, and application data
protection strategies.
Certification status as a Commvault Certified Engineer requires certification as a Commvault Certified Professional and
passing the Advanced Infrastructure Design exam.
Certification status as a Commvault Certified Master requires certification as both a Commvault Certified Professional
and Certified Engineer, and successful completion of Master certification requirements. These Master certification
requirements include attending a Master class and passing the Master Certification exam.
Page 16 of 559
Education Services Commvault® Professional Course May 2018
Additional Resources
Maintenance Advantage:
http://ma.commvault.com/
Documentation:
http://documentation.commvault.com/commvault/v11
YouTube Channel:
https://www.youtube.com/user/commvault
https://www.youtube.com/channel/UC0UfIcY_5YFAyXW5INileRA
Page 17 of 559
Education Services Commvault® Professional Course May 2018
Course Overview
This course is intended for personnel responsible for day-to-day administration and management of Commvault ®
software. The course covers essential concepts, details, options, and best practices for user security and management,
system settings, policy configuration and use, media and library management, job activity and status monitoring, and
job management. Students also learn how to efficiently and effectively manage data movement (backup, auxiliary
copy, and restore) within a CommCell® environment. This course also provides knowledge for designing and
implementing a CommCell® environment. Focus is placed on the most common technical features including
deduplication, virtualization, Commvault ® agent configuration, encryption, and firewall configuration.
Page 18 of 559
Education Services Commvault® Professional Course May 2018
Page 19 of 559
Education Services Commvault® Professional Course May 2018
Page 20 of 559
Education Services Commvault® Professional Course May 2018
Deduplication processes will be used on the client to optionally compress data and then a signature will be generated
on the data block. The block can also optionally be encrypted over the network or on media. Index data for each job
will be managed in the MediaAgent’s index directory and will also be copied to protected storage when the job
completes.
Page 21 of 559
Education Services Commvault® Professional Course May 2018
Physical Architecture
Commvault® software is deployed in a cell-like structure called a CommCell® environment. One or more cells can be
deployed to manage small to enterprise global environments. The central component of a CommCell environment is
the CommServe® server which coordinates, manages and monitors all CommCell activity. Production data is protected
by installing agents which directly communicate with the operating system or application being protected. Any
production server with an agent installed is referred to as a client. Data is protected by transferring data through
MediaAgents to storage, which can be disk, cloud or tape.
Page 22 of 559
Education Services Commvault® Professional Course May 2018
CommServe® Server
The CommServe® Server is the central management system within a CommCell ® environment. All activity is
coordinated and managed by the CommServe server. The CommServe server runs on a Windows platform and
maintains a Microsoft SQL metadata database. This database contains all critical configuration information. It is
important to note that Commvault® software does not use a centralized catalog system like most other backup
products. This means the metadata database on the CommServe server will be considerably smaller than databases
that contain catalog data. Due to the small size of the database, an automated backup of the database is executed by
default every morning at 10:00 AM.
Page 23 of 559
Education Services Commvault® Professional Course May 2018
MediaAgents
A MediaAgent moves data from production systems to protected storage during data protection jobs and moves data
back to production systems during recovery operations. It is a software module that can be installed on most
operating systems. All tasks are coordinated by the CommServe® server. MediaAgents are also used during auxiliary
copy jobs when data is copied from a source library to a destination library such as off-site Disaster Recovery (DR)
copies.
There is a basic rule that all data must travel through a MediaAgent to reach its destination. One exception to this rule
is when conducting Network Data Management Protocol (NDMP) dumps directly to tape media. In this case the
MediaAgent is used to execute the NDMP dump and no data will travel through the MediaAgent. This rule is important
to note as it will affect MediaAgent placement.
Indexing
Commvault® software uses a distributed indexing structure where index data is kept on MediaAgents and is also
automatically copied to storage. Using a distributed indexing structure allows Commvault software to scale significantly
more than legacy backup products and keeps indexes local to where data is being protected.
When data is protected, indexes are automatically generated and written to the MediaAgent’s Index Directory location.
At the conclusion of the job, indexes or index logs are copied to the store location where the job resides. During
restore operations, the index database within the index directory is accessed. If the index database is not available, it
will automatically be restored from the storage media.
Commvault uses two primary indexing methods: the traditional V1 indexing and the new V2
indexing method.
Libraries
Disk library
A disk library is a logical container which is used to define one or more paths to storage called mount paths. These
paths are defined explicitly to the location of the storage and can be defined as a drive letter or a UNC path. Within
each mount path, writers are allocated, which defines the total number of concurrent streams for the mount path.
Tape Library
A tape or removable media library is a library where media can be added, removed and moved between multiple
libraries. The term removable media is used to specify various types of removable media supported by Commvault®
software including tape and USB disk drives, which can be moved between MediaAgents for data protection and
recovery operations.
Cloud Library
A cloud library is cost-effective storage that reduces the need to maintain hardware resources, such as tape or disk
storage devices. It also provides the ability to easily increase your storage capacity when required. Cloud Storage
provides centralized data access, better failover capabilities and reduces the day-to-day storage administration tasks.
Page 24 of 559
Education Services Commvault® Professional Course May 2018
Logical Architecture
Commvault® software logically manages data by containerizing production data, moving the data through logical
streams, and managing protected data using policies.
Page 25 of 559
Education Services Commvault® Professional Course May 2018
Storage Policies
The Commvault® software suite offers a wide range of features and options to provide great flexibility in configuring
and managing protected data. Protection capabilities such as standard backup, snapshots, archiving and replication
can all be incorporated in a single environment for a complete end-to-end data protection solution. No matter which
methods are used within a Commvault® environment, the concepts used to manage the data remain consistent.
Page 26 of 559
Education Services Commvault® Professional Course May 2018
Web Console
The Web Console, as its name implies, is a web-based application that allows end-users to interact and achieve tasks
in the CommCell® environment. Tasks are sorted into main categories.
• My Data – Allows an end-user to conduct backups and restores. If permissions permit, the user can also
configure backup schedules, contents to protect, encryption and alerts. Available restores are file-level for
computer file systems, SharePoint document-level, virtual machines and emails.
• Download Center – Provides option to browse and download prepackaged agents, such as laptop backup
agents. These packages must be preconfigured and complied by the backup administrator.
• Event Organizer - web-based application for creating and managing corporate events such as conferences,
seminars, or exhibitions. You can also create surveys and generate reports for tracking and monitoring events
and surveys.
• Virtual Machines – This section lists all the VMs owned by the user and allows to create new virtual
machines. The view provides information about the VMs such as the state, the IP address, and operating
system. The creation of virtual machines is based on templates provided by the administrator.
• Virtual Labs – This view displays information about virtual labs owned by a user. It also provides wizards to
create new virtual labs. This allows a user to manage a group of virtual machines.
• Forms – Forms and actions can be published to users either by the Commvault ® administrator, or by a
workflow that requires user inputs to continue. A good example is a workflow that would automatically install
an agent on an end-user laptop but would first require the department manager’s approbation to install it.
Page 27 of 559
Education Services Commvault® Professional Course May 2018
• Manage Databases – This section lists all Microsoft® SQL databases protected by Commvault® software.
This view allows database administrators to run queries to generate reports. It also provides wizards to easily
clone databases.
• Storage Replication – This section allows to configure the replication between NAS filer, potentially from
different geographical locations, using the Commvault® software DASH Copy feature. This mechanism
optimizes the replication in two ways; it uses minimum bandwidth and storage resources by removing
redundant data and it optimizes the data transfer through chaining and forking mechanisms.
• Admin Console – Provides access to the Admin Console to manage the CommCell® environment.
• Analytics – Offers dashboards providing a view of all the structured and unstructured data of the
organization. It can use several mechanisms that must be implemented and configured beforehand such as,
Commvault® software data analytics, Data Cube, NAS system discovery, Enterprise Explorer. A Web Analytics
tool can also be integrated to provide information and metrics about organization web sites.
Page 28 of 559
Education Services Commvault® Professional Course May 2018
Admin Console
The Admin Console is a powerful web-based user interface that provides a simplified, service-oriented approach to
managing on-premise and cloud infrastructures. With an easy-to-use and flexible interface, you can configure and
manage a broad range of solutions including file systems, databases, and hypervisors.
By using out of the box configurations and streamlined procedures, you can automate common tasks including:
Page 29 of 559
Education Services Commvault® Professional Course May 2018
Installation
The Admin Console is installed directly on the CommServe® server or on a dedicated web host in the DMZ – if access
to the console is provided over the Internet (such as in an MSP environment). Note that no matter which Admin
Console solutions you plan to use, you must first complete the Core Setup.
The Admin Console is included with the Web Console package, which is automatically installed
while installing the CommServe software.
1. Replace the following URL <webhost> by your Admin Console server’s fully qualified domain name (FQDN):
http://<webhost>/webconsole
2. Type your username and password, and then click Login.
Page 30 of 559
Education Services Commvault® Professional Course May 2018
Setup Process
Once the web-based console component is installed in the environment and you log into the Admin Console for the
first time, the Core Setup wizard is immediately launched. This interface guides you through the setup process, which
is required by the applications available in the Admin Console.
Page 31 of 559
Education Services Commvault® Professional Course May 2018
Page 32 of 559
Education Services Commvault® Professional Course May 2018
Page 33 of 559
Education Services Commvault® Professional Course May 2018
If you are looking for a specific section of the sidebar, use the Filter navigation box to type characters and sort the
sidebar content.
Page 34 of 559
Education Services Commvault® Professional Course May 2018
Jobs
The Job view provides monitoring functionality for all active jobs within the CommCell ® environment. The Jobs view
displays current running jobs by default. But it can also be used to see the job history of the last 24 hours or the last
three months. From this view, any job can be controlled using the Actions button to kill, suspend, resume and view
the job logs.
A progress bar clearly indicates the progress of the job, while its Status column indicates if the job is still running, and
if so in which phase it is, or if it failed, is suspended or was killed.
Page 35 of 559
Education Services Commvault® Professional Course May 2018
Jobs view
Page 36 of 559
Education Services Commvault® Professional Course May 2018
Events
All Commvault® software related events are reported in the Events view. By default, 500 events are displayed, but the
event log can maintain up to 10,000 events or 7 days of events. Events can be filtered by severity level and can also
provide Job ID and Event ID links that can be clicked to display detailed information about the event or the job that
triggered it.
Events view
Page 37 of 559
Education Services Commvault® Professional Course May 2018
Page 38 of 559
Education Services Commvault® Professional Course May 2018
CommCell® Console
The CommCell® console is the graphical user interface used to configure and manage the CommCell environment,
monitor and control active jobs and view events related to all activities. The console is accessed using local CommCell
accounts or by using Active Directory integrated account authentication.
• CommCell Toolbar – provides an easy to navigate ‘ribbon’ to manage and configure the CommCell
environment
• CommCell browser – is the main navigation window which contains a hierarchal structure of all categories
and components within the CommCell environment
• Content / Summary window – provides details based on what component is selected in the CommCell
browser
• Job Controller – provides viewing and control capabilities for all active jobs in the CommCell environment
• Event Viewer – provides information for all logged events within the CommCell environment
Page 39 of 559
Education Services Commvault® Professional Course May 2018
To log on to the CommCell console, launch the application locally or through a web browser. Required information to
log on is a user name, password and the CommServe host name. When using Active Directory accounts, the format
for username will be domain\user.
Page 40 of 559
Education Services Commvault® Professional Course May 2018
CommCell® Toolbar
The CommCell® Console uses a ribbon style toolbar to provide more efficient navigation and configuration.
Configuration options are organized within the toolbar to provide quick access to perform common tasks. By placing
the mouse on the toolbar, use the scroll wheel to quickly move through the different toolbars available. You can hide
the toolbar by clicking the arrow in the upper right corner.
Page 41 of 559
Education Services Commvault® Professional Course May 2018
Page 42 of 559
Education Services Commvault® Professional Course May 2018
Page 43 of 559
Education Services Commvault® Professional Course May 2018
Page 44 of 559
Education Services Commvault® Professional Course May 2018
Page 45 of 559
Education Services Commvault® Professional Course May 2018
Page 46 of 559
Education Services Commvault® Professional Course May 2018
The following summarizes key points for a new CommServe server deployment:
• Commvault software must be downloaded prior to installation. To avoid any deployment delays,
arrangements for the software download should be done in advance. Additionally, determine the ability to
routinely download updates and upload log files from the CommServe host. If the CommServe will not have
internet access, alternate methods should be discussed and documented.
• Determine the location for a local and remote Software Cache. The Software Cache is a directory where
Commvault updates and software packages are stored. These can be configured during the deployment and
Page 47 of 559
Education Services Commvault® Professional Course May 2018
typically help position the software to be routinely accessible throughout the organization – or prepare for a
disaster.
• Ensure the size of the environment has been assessed and there are adequate resources available for the
CommServe server:
o Based on the sizing assessment, determine if the CommServe server will be physical or virtual.
o Determine if the CommServe server needs to be deployed in a clustered configuration for high
availability.
o Ensure the operating system meets the Commvault specifications and patched with updates prior to
the installation.
o Determine if the method of deployment requires additional considerations for Disaster Recovery. For
example, configuring a ‘Floating Host Name’ for the CommServe server.
o Determine if additional components such as Metrics Reporting or the Workflow engine will be installed
on the CommServe server.
o Determine the methods for accessing the CommCell® console and/or Admin Console.
▪ The consoles are installed by default along with the CommServe components.
▪ IIS is required for the Web Server and Web Console and are automatically installed when IIS
is enabled on the CommServe server.
• Although not always required, reboots (powering off and on) may be required to complete an installation or
update. It is recommended to anticipate downtime and that the organization’s change request or
maintenance window process is accommodated in advance. In some cases, the organization may require the
changes be implemented after hours.
• Outline the firewall and network considerations prior to any installation. Unless performing a decoupled install,
all software components must communicate with the CommServe server during installation. Determine the
requirements for working with the organizations firewall configuration in advance.
• Identify any monitoring, security, or anti-virus software that will be installed on the same systems as
Commvault software. The installation and in many cases Commvault operations may be blocked or
performance severely degraded by such software. This can be avoided by applying the appropriate exceptions
or filters for the Commvault software in advance.
• Ensure any Service and Administrative accounts are preconfigured and known during the installation. The
account type and permissions required are determined by the components being deployed. A thorough review
of the deployment should help determine the needs.
o For the CommServe server, an account with local Administrator privileges is required for the software
installation.
o A password for the CommCell ‘admin’ account is configured during the installation. This password
should be a complex password and the primary administrator should always use this account when
managing the environment.
• A permanent license file must be applied after the CommServe software is installed. Ensure that any pending
purchase agreements are completed prior to the deployment of the Commvault software.
Page 48 of 559
Education Services Commvault® Professional Course May 2018
MediaAgent Requirements
MediaAgents are the multifunction workhorses of a Commvault ® software environment. They facilitate the transfer of
data from source to destination, hosts the deduplication database, metadata indexes, and run analytic engines.
For MediaAgent resource requirements and guidelines, refer to the Commvault Online
Documentation.
• Data Mover – moves data during data protection, data recovery, auxiliary copy, and content indexing jobs.
• Deduplication Database (DDB) – hosts one or more deduplication databases on high speed solid state or
PCI storage.
• Metadata indexes – hosts both V1 and V2 indexes on high speed dedicated disks.
• Analytics – runs various analytics engines including data analytics, log monitoring, web analytics, and the
Exchange index for the new Exchange Mailbox agent.
Page 49 of 559
Education Services Commvault® Professional Course May 2018
The MediaAgent is the high-performance data mover that transmits data from source to destination, such as from a
client to a library during data protection operations or vice-versa during data recovery. They are used during auxiliary
copy jobs when data is copied from a source library to a destination library. The MediaAgent software can be installed
on most operating systems in physical, virtual, and clustered environments. Note that all tasks are coordinated by the
CommServe® server.
Since all data moving to/from protected storage must move through a MediaAgent, resource provisioning for
MediaAgent hosts (e.g., CPU, memory, and bandwidth) must be adequate for both the volume and the concurrency of
data movement you expect it to handle.
In the scenario where the MediaAgent component is co-located on the same host as the client agent, the exchange of
data is contained within the host. This is called a SAN MediaAgent configuration, or sometimes referred to as LAN-
free backups, and has its advantages of keeping data off potentially slower TCP/IP networks by using local higher
performance transmission devices (e.g., Fibre Channel, SCSI, etc.). On the other hand, a MediaAgent component
located on a host by itself can provide dedicated resources and facilitate exchange of data over longer distances using
TCP/IP (e.g., LAN, WAN, etc.).
Page 50 of 559
Education Services Commvault® Professional Course May 2018
Deduplication Database
The Deduplication Database (DDB) maintains all signature records for a deduplication engine. During data protection
operations, signatures are generated on data blocks and sent to the DDB to determine if data blocks are duplicate or
unique. During data aging operations, the DDB is used to decrement signature counters for blocks from aged jobs and
subsequently prune signatures, and block records when the signature counter reaches zero. For these reasons, it is
critical that the DDB is located on high performance, locally attached solid state or PCI storage technology.
Metadata Indexes
Commvault® software uses a distributed indexing structure that provides for enterprise level scalability and automated
index management. This works by using the CommServe® database to only retain job-based metadata such as chunk
information, which keeps the database relatively small. Detailed index information, such as details of protected objects
is kept on the MediaAgent. The index location can maintain both V1 and V2 indexes. Ensure the index location is on
high speed dedicated disks.
Page 51 of 559
Education Services Commvault® Professional Course May 2018
Analytics
One or more analytics engines can be installed on a MediaAgent. The following provides a high-level overview of the
commonly used analytics engines:
• Data analytics – provides a view into unstructured data within an environment. Some capabilities include:
o identifying old files and emails
o identifying multiple copies of large files
o removing unauthorized file types
• Log monitoring – identifies and monitors any logs on client systems. The monitoring process is used to
identify specific log entries and set filters based on criteria defined within a monitoring policy.
• Exchange index engine – maintains V2 metadata indexing information for the new Exchange Mailbox
Agent. It is recommended when using the Exchange index server that no other analytic engines are installed
on the MediaAgent hosting the index.
You need to protect a smaller remote site and want to keep a local copy of data for quick restore. However,
you are concerned about hardware costs for a MediaAgent.
Solution: Virtualize the remote site MediaAgent and keep a shorter retention for the local copy, producing a
smaller footprint. Then replicate the data using DASH Copy to the main data center physical MediaAgent
where it can be kept for a longer retention.
Page 52 of 559
Education Services Commvault® Professional Course May 2018
Agent Requirements
It is important, when deploying agents, to validate requirements. Prerequisites differ from one agent type to another.
Even for components that you frequently deploy, always confirm as it may change when a new service pack is
released.
Page 53 of 559
Education Services Commvault® Professional Course May 2018
CommCell® Deployment
Page 54 of 559
Education Services Commvault® Professional Course May 2018
The Commvault Maintenance Advantage website provides the latest version of the Commvault installation media. By
clicking Downloads & Packages, you can access the most current software installations and service packs or select a
previous version.
• Resumable Download Manager – Based on the Bootstrapper download manager, this option is activated
by selecting the required files from the list and then selecting Launch Download Manager at the bottom of the
screen.
• Bootstrapper Direct Download – This option reduces deployment time by selecting only the required
Commvault software components and download of installation media.
Page 55 of 559
Education Services Commvault® Professional Course May 2018
Page 56 of 559
Education Services Commvault® Professional Course May 2018
The installer media can also be downloaded from the Download Center on the Commvault ® software Cloud Services
website.
To learn more about the Commvault's Cloud Services, go to the following link:
https://cloud.commvault.com/
Page 57 of 559
Education Services Commvault® Professional Course May 2018
Page 58 of 559
Education Services Commvault® Professional Course May 2018
Gathering Information
Proper documentation of the CommCell components being installed is essential for a smooth deployment. The
following chart provides a sample of the information that must be obtained for the CommServe deployment. Having
this information in advance will not only help the deployment go quicker – it can help bring any shortcomings to the
surface, such as a lacking resource. Furthermore, it can aid in verifying site readiness and serve as a template for
post deployment documentation.
Page 59 of 559
Education Services Commvault® Professional Course May 2018
Additional Packages File system agent, MediaAgent, Workflow engine, Web Server
DR Share \\DRCommserve\CSDR
Page 60 of 559
Education Services Commvault® Professional Course May 2018
Page 61 of 559
Education Services Commvault® Professional Course May 2018
The installation wizard also offers the opportunity to import an existing database. This function is useful when
performing a hardware refresh, or when rebuilding a CommServe server in a disaster scenario. When selecting the
‘Use an Existing Database’ option, browse to the location of the database export, and select the correct database. If
there is an SA password mismatch, you must supply the customer defined password.
When configuring the CommServe Database, the following occurs during installation:
Page 62 of 559
Education Services Commvault® Professional Course May 2018
Page 63 of 559
Education Services Commvault® Professional Course May 2018
%allusersprofile%\Commvault Systems\Galaxy\LogFiles\Install.log. If the error occurs after the summary page, check
the installation logs in the Software_Installation_Directory\Log Files directory.
Page 64 of 559
Education Services Commvault® Professional Course May 2018
Page 65 of 559
Education Services Commvault® Professional Course May 2018
CommCell® License
When deploying Commvault V11, specific information is required to acquire a full production license:
Page 66 of 559
Education Services Commvault® Professional Course May 2018
Additional hotfixes may follow the release of a service pack to address critical issues. The hotfixes should also be
applied as soon as they are available. Subsequently, if a new service pack or hotfix conflicts with an older installed
hotfix, the installation process removes them automatically.
By default, the system creates automatic schedules that download and install updates on Commvault ® servers, as well
as on clients. These schedules can be modified as desired.
Page 67 of 559
Education Services Commvault® Professional Course May 2018
• Primary cache location where all CommCell® resources pull software and updates, unless configured to use a
remote cache.
• Remote caches are used as secondary cache locations, which is beneficial for remote locations. Secondary
caches are set up on local clients at the location to pull software and updates locally avoiding additional WAN
traffic.
To configure the main software cache (CommCell® Console)
Page 68 of 559
Education Services Commvault® Professional Course May 2018
Page 69 of 559
Education Services Commvault® Professional Course May 2018
Page 70 of 559
Education Services Commvault® Professional Course May 2018
Page 71 of 559
Education Services Commvault® Professional Course May 2018
Page 72 of 559
Education Services Commvault® Professional Course May 2018
• System Created Download Software – Download the updates automatically in the software cache once a
week if new updates are available.
• System Created Install Software – Automatically install updates on Commvault® servers and clients once
a week if required.
For instance, many companies have change control procedures in place. Installing updates automatically on
servers might go against these procedures. In this case, the System Created Install Software schedule can be
modified or simply disabled.
Viewing or editing automatic update schedules
Page 73 of 559
Education Services Commvault® Professional Course May 2018
Page 74 of 559
Education Services Commvault® Professional Course May 2018
The next step is to ensure that the CommServe® server is up to date. This provides all the latest configuration options
available. Updates can be deployed from the software cache using the CommCell console.
Page 75 of 559
Education Services Commvault® Professional Course May 2018
CommServe® DR Backup
By default, every day at 10:00 AM, the CommServe DR backup process is executed. This process first dumps the
CommServe SQL database to a local folder path. An export process then copies the folder contents to a user defined
drive letter or UNC path. A backup phase subsequently backs up the DR Metadata and any user defined log files to a
location based on the storage policy associated with the backup phase of the DR process. All processes, schedules and
export/backup location are customizable in the DR Backup Settings applet in the Control Panel.
Additionally, a copy of the DR backup can be uploaded to Commvault® Cloud Services, which guarantees that an
offline copy exists and is accessible during recovery if a disaster was to occur.
Page 76 of 559
Education Services Commvault® Professional Course May 2018
Database Dump
During the dump phase, the system stores the dump files in the following location:
Export
The Export process copies the contents of the \CommServeDR folder to the user defined export location. A drive letter
or UNC path can be defined. The export location should NOT be on the local CommServe server. If a standby
CommServe server is available, define the export location to a share on the standby server.
By default, five metadata backups are retained in the export location. It is recommended to have enough disk space to
maintain one weeks’ worth of DR exports and adjust the number of exports to the DR backup schedule frequency.
Backup
The Backup process is used to back up the DR Metadata to protected storage. This is accomplished by associating the
backup phase with a storage policy. A default DR storage policy is automatically created when the first library is
configured in the CommCell environment. Although the backup phase can be associated with a regular storage policy,
it is recommended to use a dedicated DR storage policy to protect the DR Metadata.
DR Storage Policy
When the first library in a CommCell environment is configured, a CommServe Disaster Recovery storage policy is
automatically created. The Backup phase of the DR backup process is automatically associated with this storage
Page 77 of 559
Education Services Commvault® Professional Course May 2018
policy. If the first library configured is a disk library and a tape library is subsequently added, a storage policy
secondary copy is created and associated with the tape library.
There are several critical points regarding the DR storage policy and backup phase configurations:
• Although the Backup phase can be associated with any storage policy in the CommCell ® environment, it is
recommended to use a dedicated DR storage policy. Using a dedicated policy isolates DR Metadata on its own
set of media making it potentially easier to locate and catalog in a disaster situation.
• The most common reason the Backup phase is associated with regular data protection storage policies is to
reduce the number of tapes being sent off-site. If the backup phase is associated with a regular storage
policy, consider the following key points:
o Make sure the ‘Erase Data’ feature is disabled in the storage policy. If this is not done, the DR
Metadata will not be recoverable using the Media Explorer utility.
o When the storage policy secondary copy is created, ensure the DR Metadata is included in the
Associations tab of the policy copy.
o Make sure you are properly running and storing media reports. This is especially important when
sending large numbers of tapes off-site. If you don’t know which tape the metadata is on, you will
have to catalog every tape until you locate the correct media which is storing the DR Metadata.
http://cloud.commvault.com
Backup Frequency
By default, the DR backup runs once a day at 10:00 AM. The time the backup runs can be modified, and the DR
backup can be scheduled to run multiple times a day or saved as a script to be executed on demand.
Consider the following key points regarding the scheduling time and frequency of DR backups:
• If tapes are being sent off-site daily prior to 10:00 AM then the default DR backup time is not adequate. Alter
the default schedule so the backup can complete, and DR tapes can be exported from the library prior to
media being sent off-site.
• The DR Metadata is essential to recover protected data. If backups are conducted at night and auxiliary copies
are run during the day, consider setting up a second schedule after auxiliary copies complete.
• For mission critical jobs, consider saving a DR backup job as a script. The script can then be executed by using
an alert to execute the script upon successful completion of the job.
Page 78 of 559
Education Services Commvault® Professional Course May 2018
Page 79 of 559
Education Services Commvault® Professional Course May 2018
Locations
Multiple copies of the DR backup can be maintained in its raw (export) form using scripts. Multiple copies of the
backup phase are created within the DR storage policy by creating secondary copies, or by creating a data backup
storage policy and including the metadata in the secondary copy’s Association tab.
• On-site and off-site standby CommServe® servers should have an export copy of the metadata.
• Wherever protected data is located, a copy of the DR Metadata should also be included.
• Whenever protected data is sent off-site a copy of the DR Metadata should be included.
• Since DR Metadata does not consume a lot of space, longer retention is recommended.
Retention
By default, the export phase maintains five copies of the metadata. A general recommendation is to maintain a weeks’
worth of metadata exports if disk space is available. This means if the DR backup is scheduled to run two times per
day, then 14 metadata backups should be maintained.
For the metadata backup phase, the default storage policy retention is 60 days and 60 cycles. A general best practice
is that the metadata should be saved based on the longest data being retained. If data is being sent off-site on tape
for ten years, a copy of the DR database should be included with the data.
Page 80 of 559
Education Services Commvault® Professional Course May 2018
Metadata Security
Securing the location where the DR Metadata is copied to is critical since all security and encryption keys are
maintained in the CommServe database. If the metadata is copied to removable drives or network locations, best
practices recommend using disk-based encryption.
CommServe DR backup is an administrative task that protects the CommServe® server metadata database. The DR
backup protects the database in two phases. First, it creates an export of the database in a location provided in the
DR Backup applet from the Configuration menu. Then, it creates a copy of that export in a storage policy called
CommServeDR. This storage policy is automatically created by the system upon completion of the libraries
configuration.
At this point of the deployment, since no libraries are yet configured, only the export location can be defined. The
export location must be as far as possible from the CommServe®- server, preferably a remote location.
Page 81 of 559
Education Services Commvault® Professional Course May 2018
Page 82 of 559
Education Services Commvault® Professional Course May 2018
Page 83 of 559
Education Services Commvault® Professional Course May 2018
Page 84 of 559
Education Services Commvault® Professional Course May 2018
If your corporate mail server is secured, it is important to understand the level of security. Commvault ®
software uses a functionality called SMTP relay. This means that the email server relays emails generated and
sent by the CommServe® server. Therefore, SMTP relay must be allowed on the mail server for the
CommServe® server IP address. Refer to your software vendor documentation for more information about
SMTP relay and the mail server.
Page 85 of 559
Education Services Commvault® Professional Course May 2018
Page 86 of 559
Education Services Commvault® Professional Course May 2018
MediaAgent – Installation
Installing MediaAgent Software
The next component to install are the MediaAgents to which libraries will later be attached. MediaAgents move and
retrieve protected data from disk, tape, and cloud storage. Data can move at a speed of 4 plus terabytes per hour per
MediaAgent. The MediaAgent software is pushed to a server directly from the software cache using the CommCell®
Console.
When installing the MediaAgents, refer to the Commvault Online Documentation to ensure that all hardware
requirements are met.
Page 87 of 559
Education Services Commvault® Professional Course May 2018
Page 88 of 559
Education Services Commvault® Professional Course May 2018
Page 89 of 559
Education Services Commvault® Professional Course May 2018
Page 90 of 559
Education Services Commvault® Professional Course May 2018
Page 91 of 559
Education Services Commvault® Professional Course May 2018
Page 92 of 559
Education Services Commvault® Professional Course May 2018
The index directory location is modified by changing the ‘Index Directory’ in the Catalog tab of the MediaAgent
properties. When the path is changed, the system prompts the user to automatically copy the contents from the old
location to the new location.
Note that this is a copy operation, so the old index directory contents remain until deleted by the
user.
Page 93 of 559
Education Services Commvault® Professional Course May 2018
Page 94 of 559
Education Services Commvault® Professional Course May 2018
Page 95 of 559
Education Services Commvault® Professional Course May 2018
Storage Overview
Commvault® software logically addresses storage systems to allow virtually any library type to be used. The three
primary library types are disk, tape, and cloud.
• If using DAS or SAN, format mount paths using a 64KB block size.
• If using DAS or SAN, try to create multiple mount path. For instance, if there are 10 mount paths, and there is
a maintenance job, such as a defrag job running on one, the mount path can be set to read-only, leaving 90%
of the disk library available for backup jobs.
• Set mount path usage to Spill and Fill, even if using only one mount path. If additional mount paths are added
later, the streams will spill as expected.
• Share the disk library if required.
Page 96 of 559
Education Services Commvault® Professional Course May 2018
• Dedicated – disk libraries are created by first adding a disk library entity to the MediaAgent using either the
right-click All Tasks menu or the Control Panel’s Expert Storage Configuration tool. One or more mount paths
can be created/added to the library. Mount Paths are configured as Shared Disk Devices. The Shared Disk
Device in a dedicated disk library has only one Primary Sharing Folder.
• Shared – disk libraries are libraries with more than one Primary Sharing Folder configured on a Shared Disk
Device. This enables other MediaAgents access to the same shared volume resource. A shared disk library
can then be created and the ‘Shared Disk Devices’ added to the library. One path to the shared folder can be
direct while the others are Common Internet File System (CIFS) shared directory paths. CIFS protocol is used
to manage multiple MediaAgent access to the same directory. For UNIX hosted MediaAgents, Network File
Share (NFS) protocol can be used. NFS shared disks appear to the MediaAgent as local drives.
• Replicated – disk libraries are configured like a shared disk library with the exception that the Shared Disk
Device has a replicated data path defined to a volume accessible via another MediaAgent. Replicated folders
are read-only and replication can be configured for use with third party replication hardware.
Page 97 of 559
Education Services Commvault® Professional Course May 2018
There are three methods that disk library data paths can be configured:
Page 98 of 559
Education Services Commvault® Professional Course May 2018
When using SAN storage, each building block should use a dedicated MediaAgent, DDB and disk library. Although the
backend disk storage in the SAN can reside on the same disk array, it should be configured in the Commvault®
software as two separate libraries; where Logical unit numbers (LUNs) are presented as mount paths in dedicated
libraries for specific MediaAgents.
SAN storage provides fast and efficient movement of data but, if the building block MediaAgent fails, data cannot be
restored. When using SAN storage, either the MediaAgent can be rebuilt or the disk library can be re-zoned to a
different MediaAgent. If the disk library is rezoned, it must be reconfigured in the Commvault® software to the
MediaAgent that has access to the LUN.
Page 99 of 559
Education Services Commvault® Professional Course May 2018
Data transfers are achieved using secured channels (HTTPS) and are optionally encrypted to further secure the data
sent to the cloud.
• Properly plan and analyze if the cloud library scenario meets the needs (i.e. restoring an entire datacenter).
• If the link is shared with users, consider throttling Commvault ® bandwidth usage during business hours.
• If the MediaAgent does not have direct access to the internet, define the proxy settings in the Advanced tab of
the cloud library configuration page.
• If the cloud library is accessed through a high-speed internet link (1GB or higher), consider tuning the
connection. For more information, refer to the Commvault Online Documentation, ‘Cloud Connection
Performance Tuning’ section.
• If using deduplication, by default, jobs are not aged and pruned unless the DDB is sealed. If you want to age
and prune jobs as soon as retention is met, configure micro pruning. For more information, refer to the
Commvault Online Documentation, ‘Configuring Micro Pruning on Cloud Storage’ section.
The list of supported cloud providers for Commvault® software grew over the years — up to 20
providers as of Service Pack 7. For a complete list of supported providers, please refer to
Commvault Online Documentation.
A MediaAgent must be defined to act as a gateway and to send the data to the cloud. If the library is used for
secondary copies of data store in local library, it is recommended whenever possible to use the MediaAgent hosting
the primary copy to avoid unnecessary traffic. If the MediaAgent requires a proxy to reach the cloud, it can be defined
during the cloud library creation process by using the Advanced tab.
Deduplication
Deduplication Overview
In any modern data center, duplicate data exists on storage-based media, networks, and virtual servers. Some
examples include identical DLL files existing on different servers, or multiple users working on the same document—
each user modifies different blocks in the file while other blocks remain unchanged. Traditionally this redundant data is
stored on disk or tape, which requires a significant amount of space to protect. With Commvault ® deduplication
storage techniques, a single copy of redundant data (and any subsequent references to the same data) is stored only
once; reducing the amount of space needed to save data and protecting against data loss.
• Deduplication Storage Policy – performs deduplication on all data blocks written to each storage policy.
• Global Deduplication Storage Policy (optional) – writes blocks from multiple storage policies through a
single deduplicated policy. Using a global policy results in multiple policy data blocks being stored once on disk
storage.
The Global Deduplication Policy – defines the rules for the Deduplication Engine. These rules include:
Deduplication Database (DDB) – is the database that maintains records of all signatures for data blocks in the
Deduplication Store.
Deduplication Store – contains the protected storage using Commvault deduplication. The store is a disk library
which contains non-duplicate blocks, along with block indexing information, job metadata, and job indexes.
Client – is the production client where data is being protected. The client has a file system and/or an application
agent installed. The agent contains the functionality to conduct deduplication operations, such as creating data blocks
and generating signatures.
MediaAgent – coordinates signature lookups in the DDB and writes data to a protected storage. The signature
lookups operation is performed using the DDB on the MediaAgent.
The global deduplication policy wizard is used to create new global deduplication policies. Options during the wizard
include the name, library, MediaAgent, number of partitions and partition location, and the network interfaces used for
MediaAgent configurations. Additional settings including the use of transactional DDB and compression usage must be
set after completing the wizard.
A disk or cloud storage pool defines the relationship between a disk or cloud-based storage and a deduplication
database (DDB). The storage target can be created beforehand and simply selected when creating the storage pool.
This means that all datasets sent to this library will be deduplicated against each other, resulting in each unique block
being written only once in back-end or cloud storage.
Multiple storage pools can be created within a CommCell®, allowing to send protected data to different targets, and
replicating the backup data to a secondary location.
1. Signature is generated at the source - For primary data protection jobs using client-side deduplication, the
source location is the client. For auxiliary DASH copy jobs, the source MediaAgent generates signatures.
2. Based on the generated signature it is sent to its respective database. The database compares the signature
to determine if the block is duplicate or unique.
3. The defined storage policy data path is used to protect data – regardless of which database the signature is
compared in, the data path remains consistent throughout the job. If GridStor ® Round-Robin has been
enabled for the storage policy primary copy, jobs will load balance across MediaAgents.
Data Verification
With all the benefits of Commvault® deduplication, it is critical to consider the integrity of deduplicated data. A corrupt
block in the deduplication store can result in data from multiple jobs not being recoverable. Commvault® V11 provides
live data verification operations that are conducted while data protection jobs are running. To use data verification,
the MediaAgent options ‘Validation on Media’ and ‘Validation on Network’ must be enabled, which they are by default.
Incremental Verification
This method verifies data integrity for new jobs added since the last verification job. This option is available when
running ‘Verification of Deduplication Database’ or ‘Verification of Existing Jobs on Disk and the Deduplication
Database’ options. Commvault® introduced a DDB verification schedule that executes an incremental verification every
day, at 11 a.m. Since this method only verifies new jobs, full verification jobs should periodically be executed, such as
once a month or once a quarter.
The best way to protect against potential data corruption, whether using deduplication or not, is to
always have multiple copies of data.
• Configure the tape library cleaning method to use. Software cleaning (Commvault) or hardware cleaning
(library) can be used, but not both. A choice must be made.
• Share the tape library if required.
• Create a barcode pattern for cleaning tapes and assign it to the Cleaning Media group.
• If using multiple scratch media groups, create scratch groups and barcode patterns to use.
• Validate drive speed (from the CommCell Console) and document for future reference.
Tape libraries are divided into the following components:
• Library – is the logical representation of a library within a CommCell® environment. A library can be dedicated
to a MediaAgent or shared between multiple MediaAgents. Sharing of removable media libraries can be static or
dynamic depending on the library type and the network connection method between the MediaAgents and the
library.
• Master drive pool – is a physical representation of drives of the same technology within a library. An example
of master drive pools would be a tape library with different drive types like LTO4 and LTO5 drives within the
same library.
• Drive pool – is used to logically divide drives within a library. The drives can then be assigned to protect
different jobs.
• Scratch pool – is defined to manage scratch media, also referred to as spare media, which can then be
assigned to different data protection jobs.
o Custom scratch pools – can be defined and media can be assigned to each pool.
o Custom barcode patterns – can be defined to automatically assign specific media to different scratch
pools or media can manually be moved between scratch pools in the library.
Library Types
Commvault software supports the drives for tape libraries:
• Standalone
• Blind
• Dedicated
• Static Shared
• Dynamic Shared
• Virtual Tape Library (VTL)
• USB Devices
Standalone
A standalone tape drive has no robotic media changer and no internal storage slots. Multiple standalone drives
controlled by the same MediaAgent can be pooled together to support multi-stream jobs or cascade of a single stream
job without having to respond to media handling requests. Media used by a Standalone library can be pre-stamped or
new, and will be prompted for, by backup or restore jobs as necessary.
Blind
A ‘blind’ library has no barcode reader and is supported by the Commvault® software maintaining the map/inventory
externally in the CommServe® server metadata.
Dedicated
A static configuration where the drives and media changer are connected to only one MediaAgent.
Static Shared
A static configuration where the drives and media changer are connected to only one of several MediaAgent hosts.
Example: In a library with four tape drives, one MediaAgent may have control of the media changer and two drives
within the library while another MediaAgent may have control over the other two tape drives. A drive connected to one
MediaAgent host is not accessible from the other MediaAgent hosts. Should the MediaAgent component having media
changer control fail, no further loading/unloading of media can occur until that MediaAgent is active again. Shared
libraries in today’s world of Storage Area Networks (SAN) are not common.
Dynamic Shared
In a Dynamic Shared Library, the library drives and media changer are on a SAN and can be accessed by multiple
MediaAgent hosts. Drives not being used by one MediaAgent can be assigned to and used by another MediaAgent. If
the MediaAgent with control of the media changer fails, the control can be automatically passed to another
MediaAgent. The primary advantage of a Dynamic Drive library is the use of multiple MediaAgents for processing
reads/writes. Dynamic Drive capability is referred to as GridStor ® Technology. GridStor technology is an option that
enables load balancing and failover of data protection jobs.
A tape library is added directly from the CommCell ® browser by right-clicking libraries or using the expert storage
configuration applet in the storage ribbon menu. The expert storage configuration applet provides advanced
configuration options that are not available when using the CommCell browser add method.
Once completed, if the library is dedicated to a single MediaAgent, it is configured directly in the CommCell® browser.
If it is shared, it must be configured using the Expert Storage Configuration wizard.
Even if another MediaAgent requires writing data, the library operation is always conducted by the active controller.
Failover candidates can be defined to replace the active controller, should the active controller become unavailable.
When defining virtual mail slots, a starting port number must be defined, as well as the order for additional media.
The order can go up or down. For instance, an administrator could define to start with port number one and to go up
for additional media. Every day, the exported media will be ordered starting with port number one, and will go up for
as many slots are required.
When using virtual mail slots, ensure to leave at least one empty slot in the library, as it is required
by the system when re-ordering tapes.
Auto-Cleaning
Over time when using tape drives, dirt can accumulate on the drive read/write heads. When it happens it’s important
to clean the drives using a cleaning media. This process is usually automated but can be executed manually.
• Hardware Controlled Cleaning – The cleaning of the drives is handled by the library itself. In this scenario,
auto-cleaning must be disabled in Commvault® software and must be enabled on the library by using either
the administration web portal or its control panel. The library configures dedicated cleaning slots where
cleaning media are stored. The dedicated cleaning slots and cleaning media are not visible in Commvault
software and cannot be used. Manual cleaning operations must be initiated from the library web page.
• Software Controlled Cleaning – The cleaning of drives is handled by Commvault software. In this scenario,
auto-cleaning must be disabled on the library and must be enabled in Commvault software. The library does
not reserve any dedicated cleaning slots and Commvault software is aware of the cleaning media. Cleaning
must be initiated from Commvault® software.
Both cleaning methods are equally effective since both use hardware sense code and/or cleaning thresholds. The
preferred method can be determined based on the manufacturer’s recommendations.
Even if both cleaning methods are as effective, it cannot be used concurrently. A choice must be
made, and a single method used.
If the software receives a sense code and cleans the tape drive heads, but the drive still encounters errors, it is not a
dirt issue and probably is a hardware malfunction that should be investigated. In this situation, to avoid having the
system trying to clean the drive again, a minimal number of days since the last cleaning can be set before a new
cleaning attempt is conducted. The default value is 3 days, which ensures that even on a long week-end, the
administrator will notice that there is a cleaning issue before additional unnecessary cleanings are attempted.
Otherwise, it could result in using all cleaning media in a single night.
When a drive status is set to dirty and the system cannot clean the drive, such as when there are no cleaning media
available in the library, Commvault® software stops using that drive completely for both backups and restores. This
prevents damaging the media or corruption when writing data to the media a using dirty tape drive. If resources are
limited and a restore requires a tape drive, the ‘Continue using drive even if it needs cleaning, during the restore’
option is used. It would allow using the drive, but as mentioned by the option, only during restores.
If you use cleaning thresholds, some adjustments to the threshold values might be preferable. By default, the
threshold to retire a bad media is five read/write errors. But the threshold to clean a dirty drive is ten read/write
errors, which means that up to two tapes could be retired before the drive gets cleaned. And these media are
probably good media. To avoid this situation, you can slightly increase the tape threshold or decrease the drive
threshold or both, to ensure that the drive is cleaned before the media is retired, (i.e., you could increase the tape
threshold to seven and lower the drive threshold to six).
Physical media management refers to any action performed that will physically cause actions within the library, such
as:
Logical management of media focuses on the media group and the state of tape, which is represented by a media
icon. Logical actions include:
• Retire a tape
Media Icons
All tapes within a library are associated with a media status icon. These icons are used to quickly identify key
attributes of a tape.
Note that colors representing the different icons may not be accurately shown in print.
Icons: Tape
Libraries
Media Status Spare Cleaning Assigned Media Assigned Bad Media Retired Prevent
Media Media (active) Media (full) Media Export
Media Status Undiscovered Media with Media from a Appendable Aged Media Aged
Media duplicate different library Media Retired
barcodes Media
Media Lifecycle
The lifecycle of a tape is tracked from the time of initial discovery to the time of its logical destruction. The logical
lifecycle of a tape is different than the physical life. Logical management of tapes are managed with tapes in or
outside the library.
The Logical management of tape media is organized in the following media groups:
2. The library scans the slots, discovers the tape and marks it as spare media in the Scratch group.
3. The tape is selected for a job where it becomes assigned media in the Active media group.
4. The tape is then appendable; where additional jobs can be written to the media until the tape is marked full.
5. Once all jobs are aged from the tape, the tape is marked as aged and placed back into the Scratch group.
6. Tapes can also be marked as locked down where they will be placed in the Overwrite Protect group.
7. Once a tape exceeds its usage thresholds, it is marked bad and placed in the Retired Media group.
8. Cleaning media is designated as such and placed in the Cleaning Media group.
Tape Groups
Commvault® software logically organizes all tapes into media groups. Each media group identifies all tapes associated
with the group. Since all tapes are logically managed in the Commvault® environment it is important to understand
each media group and group management.
Scratch groups hold all new or recycled media. Multiple scratch groups can be used to define which tapes a job uses
when it executes. When a job requires a spare tape, the tape is pulled from a defined scratch group. The storage
policy copy’s data path is used to determine which scratch group the tape is selected from.
• The terms: Scratch Pool, Scratch Group, Spare Media Group or Spare Media Pool are used interchangeably
throughout Commvault documentation and the CommCell console
• All new and recycled tapes are placed in scratch groups
• Once a job is written to a tape it is moved out of the scratch group and into the assigned media group
• Multiple scratch groups can be created and assigned to storage policy copies. When a job for the policy copy
runs, it automatically picks a tape from the assigned scratch group
Default Scratch Group and Media Icons Associated with Scratch Tapes
When using multiple scratch groups, different storage policy copies define which scratch group tapes are pulled from.
Also, tapes can be manually moved to other scratch groups or automatically assigned to different groups. Assigning
tapes is based on barcode patterns or high watermark thresholds and scratch group priority. This allows different job
types to be placed on specific media, which simplifies tape management outside the library.
High and low watermarks are assigned to each scratch group in a tape library. Low watermarks are used to alert
administrators when the library is running out of spare tapes. The high watermark is used to limit the number of tapes
placed in a scratch group.
• Low watermarks – Reports events in the Event Viewer when scratch tapes fall below the defined number.
Alerts can also be configured to alert administrators when low watermarks are reached.
• High watermarks – Limits the number of tapes that will be assigned to a scratch group. This is useful when
multiple scratch groups and custom barcode pattern definitions are not being used. Scratch groups can be
assigned high, medium and low priorities. When tapes are discovered or recycled, they are placed in the high
priority scratch group until it reaches the upper watermark. The medium group is filled next followed by the
low priority group. If there are still additional tapes available, they are placed in the Default Scratch group
designated in the General tab of the library properties.
High and Low Tape Watermarks
Tip: Using multiple scratch groups to ensure available media for backup operations
You are managing a CommCell® environment and running backup jobs directly to tape each night. During the
day, you run auxiliary copy jobs to tape to be sent off site. You are concerned that auxiliary copy jobs may
use too many tapes and there will not be enough media for backup operations.
Solution: Create an additional scratch group and name it Auxiliary Copies. Set the priority to Medium.
Configure a High Watermark in the Default Scratch group to be a greater number than the number of tapes
required to perform nightly backups. Set the Default Scratch group priority to high.
In the secondary copy of each storage policy, use the Scratch Pool drop-down box to assign the Auxiliary
Copies scratch pool to the copy.
The Cleaning Media group manages all cleaning tapes for a library. Tape drives are cleaned based on drive counter
usage tracked by Commvault® software and/or sense codes reported from the library. Drive cleaning settings are
configured in the library properties under the Drive tab.
Best practice guidelines are to configure drive cleaning based on the library manufacturer’s
recommendations.
• Commvault software should automatically detect and move cleaning tapes to the cleaning media group when
the tapes are discovered.
• If cleaning tapes are incorrectly identified and moved to a scratch pool, you can manually move the tapes or
use custom barcode definitions to associate cleaning tapes with the cleaning media pool.
• Low watermarks can be defined to trigger events and optional alerts when the number of spare cleaning
media reaches the low threshold.
The Retired Media group is a holding area for all tapes that have exceeded tape error thresholds or are manually
marked bad. Tapes in the Retired Media group will remain in the group until they are manually marked good or
deleted. Any tapes in the Retired Media group will NOT be written to. If a tape is in the Assigned Media group and is
marked bad, it will NOT be moved to the Retired Media group until all jobs have aged from the tape.
• Only tapes that are not currently retaining job data are placed in the retired media group. If a tape is marked
bad, but is currently retaining data, it will still appear in the Assigned Media group. Once all jobs have aged
from the tape it is moved to the Retired Media group.
• Tape counters are tracked for the life of a tape from initial discovery to deletion.
• By default, manufacturer recommended thresholds are used for all tapes. These settings can be modified in
the Control Panel | Hardware Maintenance applet | Media Retirement tab. It is NOT recommended to increase
the threshold values.
• For as long as a tape is in the Retired Media group it will NOT be written to.
• Tapes can be moved out of the Retired Media group using the following methods:
• Delete – Deletes the existence of the tape from the CommServe® server database. The tape can then
be rediscovered and reused. The tape is treated as a brand-new tape and all counters are reset. If
there are any existing aged jobs on the tape they will not be recoverable.
• Mark Media Good – Is recommended if the tape has existing jobs that have aged but may still need
to be retained. If this is the case after marking the tape good, move it to the Overwrite Protect Media
group.
Tapes should be left in the Retired Media group until they are physically disposed of. This prevents a bad tape from
accidentally being discovered and reused. If a bad tape is disposed of and is replaced with a new tape with the same
barcode, delete the tape from the Retired Media group before putting the new tape in the library.
Sometimes tapes can be incorrectly marked bad due to drive problems that result in tape errors. If there is a
sudden increase in bad tapes this may be an indication of drive problems. However, do NOT discount the
possibility that the tapes are bad. There have been situations where bulk orders of brand new tapes are
legitimately bad. If you do not know what manufacturing, delivery, or storage methods are being used; then it
is critical to act on the side of caution.
The Foreign Media group manages all media from different CommCell® environments or tapes from a different backup
vendor.
• Tapes from one CommCell® environment cannot be directly restored into another. When a tape is loaded and
the OML (On Media Label) header is read, if the CommCell ID is different than the CommCell ® environment
reading the tape, the tape is moved to the Foreign Media group.
• Commvault software will not write to tapes when the OML header is not recognized as a Commvault header
and the tape is moved to the Foreign Media group.
The Overwrite Protect Media group logically locks down a tape, so it will NOT be written to or recycled. Tapes must be
manually moved to the Overwrite Protect Media group and remain there indefinitely until they are moved out of the
group.
• By default, an Overwrite Protect Media group is automatically created. Additional overwrite protect media
groups can be added.
• Tapes are moved to the Overwrite Protect Media group using the following methods:
• For active tapes in the Assigned Media group – Right-click on the tape and select Prevent Reuse. The
tape appears in the Assigned Media and the Overwrite Protect Media groups.
• For tapes in scratch groups – Right-click on the tape and select Move. For Media Group Type select
Overwrite Protect Media group and then select the overwrite group.
Moving a tape to the Overwrite Protect Media group is just one way Commvault® software can prevent data
from being overwritten. Data can also be locked down at the job level.
Consider a job that spanned multiple tapes. Manually moving tapes to the Overwrite Protect Media group
requires you to know every tape the job was written to. The job and all tapes can also be locked down
through the storage policy copy. In the job history of the policy copy, right-click on the job and select Retain
Job. You are then prompted to select a date to hold the job, or infinitely retain the job.
The Catalog Media group is used to hold all tapes that are actively being cataloged or are marked for catalog. A
catalog operation is used to catalog job metadata from a tape and enter the metadata back into the CommServe®
server database. You can perform this operation if the CommServe server database had to be restored to a point-in-
time prior to the jobs on a tape finishing. This situation can arise in cases of disaster, database corruption, or if the
CommServe server metadata backups are not properly managed.
A tape can be individually picked for catalog or multiple tapes can be picked and marked for catalog. When tapes are
picked for catalog they are moved to the Catalog Media group.
All tapes that are actively retaining data are located in the Assigned Media group. Within a library, there is only one
assigned media group. Tapes remain in the group until ALL jobs on the tape have exceeded retention and are marked
as aged. During the data aging operation, the tape is then recycled back into a scratch pool.
• Tapes in the Assigned Media group cannot be deleted. Delete is considered a non-destructive operation.
• Delete Contents can be performed on a tape which is considered a destructive operation. To delete contents
of multiple tapes, use the Shift or Ctrl keys to select multiple media. Note that this recycles the tape and the
jobs are marked aged.
Assigned Media Icons
Since tapes are associated with storage policy copies it is important to properly configure and manage storage policies.
Tip: Consequences of having too many storage policies using tape media
Your environment has 25 storage policies. This results in at least 25 storage policy primary copies. If all
primary copies are defining tape library data paths, then at least 25 tapes must be in the library to
accommodate all potential jobs. If the storage policies also have secondary copies using a tape data path,
then additional tapes must also be present to meet the media needs of the secondary copies.
In some cases, backup configurations such as Start New Media and Mark Media Full can complicate things
more. It is important to understand your environment’s needs and how Commvault software manages media.
Use global secondary copies to consolidate data from multiple storage policy secondary copies to
the same tape sets.
Action Description
Verify Media Physically verify the OML header information to CommCell® tape metadata and the
barcode label. (Except cleaning media pool)
Delete Tape Logically delete the existence of a tape from the CommServe® server database. (Except
assigned media group)
Delete Contents Logically delete contents by marking all jobs as aged and recycling the tape back into a
scratch pool. (Only in assigned media group)
Erase Media Physically erase data by writing a new OML header to the tape.
Mark Media Bad Logically mark a tape bad to prevent it from being reused.
Media Refresh Refresh active jobs on existing tapes by writing the jobs to new tapes.
Exporting Media
Exporting tapes is a physical operation that sends commands to the library to eject tapes to the import/export mail
slots.
To view tapes in the import/export slots, use the I/E Ports view in the library tree.
Individual Export
Select the tape | Right-click | All Tasks | Export
Tapes can be individually exported from any location within the library tree.
Bulk Export
Select the library | Export | Select media to export
Multiple tapes can be selected together to perform a bulk export. This is considered a library level operation, so the
bulk export is conducted by right-clicking on the library and selecting Export Media. Optionally, use the Shift or Ctrl
keys to select multiple tapes. A bulk export exports tapes until the import/export slots are full. Once tapes are
removed from the slots, the export operation continues until all tapes have been exported.
4. Optionally, an Export location can be selected for where the tapes will be located.
5. Click Finish and the export process will export all selected media.
When tapes are exported out of the library they are logically associated with an export location. There are two types
of locations that can be defined: Stationary and Transit.
Viewing Contents
Click the Assigned Media Pool | Right-click the tape| View | View Contents
Job contents and details can be viewed for individual tapes. To view the contents of a job, right-click on the tape and
select View Contents. The view contents option only appears if jobs are on the tape.
• Jobs on the tape that are actively being retained will appear in black print
• Jobs on the tape that have exceeded retention will appear in gray print
• The view contents option will appear for any tape that has active or aged jobs
One of the most common issues that may arise regarding media management is when tapes are not properly
recycling. Using the View Contents of a tape lets the administrator view which jobs are causing the tape not to
recycle. This information can then be used to track down problems that are causing the jobs to remain active
within the environment.
Another method to assist in predicting when tapes will recycle, is using the Data Retention Forecast and
Compliance report. This report lists all tapes, their expected aging date, and specific reasons why the tapes
have not aged. Each of the reasons are hyperlinks to the Commvault documentation site which will provide
more details on the explanations.
Barcode patterns are defined in the Media Management applet from the Storage menu. Once barcode patterns are
defined, the patterns can be associated with different scratch or cleaning pools. When media is discovered or recycled,
they are automatically placed in the appropriate scratch or cleaning pool that corresponds to the barcode pattern.
Once a barcode pattern has been defined, it can be associated with a scratch or cleaning pool. Once the association is
made, whenever tapes are discovered or recycled (all jobs age from the tape), the tape is moved to the proper scratch
or cleaning pool based on the barcode pattern matching.
Storage Policies
There are three primary rules that every storage policy copy must have defined:
• MediaAgent
• Library
• Drive pool (tape library)
• Scratch pool (tape library)
MediaAgent
MediaAgents are the workhorses that move data from production servers to the backup environment. They supply the
processing power to receive data, arrange it in chunk format, and send it to the library. MediaAgents can also be
responsible for encryption, compression, or deduplication processing.
Library
Libraries are logically defined and are categorized as stationary or removable media libraries. Stationary libraries
define a path to a disk location such as a drive letter or UNC path. They are considered stationary since these paths do
not change once defined. Removable media libraries are generally thought of as tape libraries, but they can also be
magnetic optical or USB storage devices.
Drive pools are a MediaAgent’s view of allocated drives within a tape library. Use of drive pools gives the MediaAgent
the flexibility of drive choice and usage within a library. Without drive pools, assigning and sending a data protection
job to a specific drive would fail if the drive was broken or offline. Having a pool of drives to choose from gives the
job the best chance of success. It also isolates resources of different technologies (i.e., LTO6 and LTO7 drives), which
allows an administrator to easily direct specific jobs to the desired set of drives, and with scratch pool definition, a
different set of tapes.
Scratch pools allow new and re-usable media within the library to be logically grouped based on media type and
intended usage. At least one default scratch pool exists for every tape library. Master drive pools can be assigned their
own default scratch pools. Additional user-defined scratch pools can be created, media assigned manually or
automatically and assigned to a storage policy copy’s data path.
• Primary snap copy (used only with IntelliSnap® feature, block-level and VSA Application Aware backups)
• Primary backup copy, also known as primary classic
• Secondary synchronous copy
• Secondary selective copy
• Secondary snap copy
Primary Copy
A storage policy primary copy sets the primary rules for protected data. Each storage policy can have two primary
copies:
• Primary snap copy – manages protected data using the Commvault IntelliSnap® feature, any agents
configured to run block-level backups or the Virtual Server Agent (VSA) using Application Aware backups.
• Primary classic copy – manages traditional agent-based data protection jobs. Most rules defined during the
policy creation process are modified after it has been created.
Secondary Copies
There are three types of secondary copies:
• Secondary Synchronous
• Secondary Selective
• Secondary snap copy
Synchronous Copy
A synchronous copy defines a secondary copy to synchronize protected data with a source copy. All valid data (jobs
that completed successfully) written to the source copy are copied to the synchronous copy via an update process
called an auxiliary copy operation. This means that all full, incremental, differential, transaction log, or archive jobs
from a source copy are also managed by the synchronous copy. Synchronous copies are useful when you want a
consistent point-in-time copy at any point within the cycle of all protected data available for restore.
• Provides consistent point-in-time copies of data required to restore data to a specific point-in-time within a
cycle.
• Provides copies that are required to be sent off-site daily.
• Provides the ability to restore multiple versions of an object from a secondary copy within a cycle.
Synchronous secondary copy concept
Selective Copy
A selective copy allows automatic selection of specific full backups or manual selection of any backup for additional
protection. Selective copy options allow the time-based automatic selection of ‘all,’ ‘weekly,’ ‘monthly,’ ‘quarterly,’ ‘half-
year,’ and/or ‘yearly full.’
Advanced options allow you to generate selective copies based on a frequency of ‘number of cycles,’ ‘days,’ ‘weeks,’ or
‘months.’ You can also choose the ‘Do Not Automatically Select Jobs’ option which allows you to use auxiliary copy
schedules to determine when copies of full backups are made.
Storage policies are used for CommServe® Disaster Recovery Backups or standard data protection. CommServe
disaster recovery storage policies are only used for protecting the CommServe® server metadata database, the
CommServe registry, configuration files, and specified log files. No standard data can be protected by a CommServe
DR policy. Standard data protection policies are used for protecting all production data within an environment.
The name of the storage policy is defined at the time of creation and later modified in the Storage Policy Properties.
The name should be descriptive and reflect what is being protected.
Creating a Storage Policy with Primary Copy using a Global Deduplication Policy
Multiplexing
Primary Copy
When writing multiple streams to a tape library, multiplexing is used to improve write performance by multiplexing
multiple Job Streams into a Device Stream. Multiplexing improves backup performance but can have a negative effect
on restore performance.
Consult with the Commvault Online Documentation for more information on the proper settings for
multiplexing.
When writing the primary copy to a disk library, there are no advantages in enabling multiplexing. The disk library
already receives multiple streams concurrently from subclients, and if available, leverages multiple mount paths.
Unless using a tape library, multiplexing should not be used.
Secondary Copy
If the source location is a disk library with multiple mount paths, this option can be used to improve read performance
from the disks when using the ‘Combine to Streams’ option.
Combine to Streams
A storage policy is configured to allow the use of multiple streams for primary copy backup. Multi-streaming of backup
data is done to improve backup performance. Normally, each stream used for the primary copy requires a
corresponding stream on each secondary copy. In the case of tape media for a secondary copy, multi-stream storage
policies consume multiple media. The ‘Combine to streams’ option is used to consolidate multiple streams from source
data on to fewer media when secondary copies are run. This allows for better media management and the grouping of
like data onto media for storage.
You back up home folders subclient to a disk library using three streams to maximize performance. The total
size of protected data is 600GB. You want to consolidate those three streams onto a single 800GB capacity
tape for off-site storage.
Solution: By creating a secondary copy and setting the ‘Combine to streams’ setting to 1 you will serially place
each stream onto the media.
In some cases, using the ‘Combine to streams’ option may not be the best method to manage data. Multi-streaming
backup data is done to improve performance. When those streams are consolidated to the same media set, they can
only be recovered in a single stream operation. Though combining to streams has a media consolidation benefit, it will
have a negative effect on the restore performance.
Another reason not to use the ‘Combine to streams’ option is for multi-streamed backups of SQL, DB2, and Sybase
subclients. When these agents use a single subclient with multi-streaming enabled, the streams must be restored in
the same sequence they were backed up in. If the streams are combined to the same tape, they must be pre-staged
to disk before they can be recovered. In this case, not enabling ‘Combine to streams’ and placing each stream on
separate media bypasses the pre-staging of the data and allows multiple streams to be restored concurrently, making
the restore process considerably faster. Note that this only applies to subclients that have been multi-streamed. If
multiple subclients have been single streamed and combined to media, they will NOT have to be pre-staged prior to
recovery.
Job Management
Jobs can be managed by viewing the job history for a storage policy copy.
1. In the storage policy properties view the Associations tab to ensure no subclients are associated with the
policy. A storage policy cannot be deleted if subclients are associated with the policy.
2. On the storage policy, right-click | select View | Jobs. De-select the option to Specify Time Range then click
OK. This step displays all jobs managed by all copies of the storage policy. Ensure that there are no jobs being
managed by the policy that require to be kept and then exit from the job history.
3. Right-click on the storage policy | Select All Tasks | Delete. Read the warning dialog box then click OK. Type
‘erase and reuse media’ then click OK.
Subclient Association
Subclient Properties
To protect a subclient it must be associated with a storage policy. During an agent install, a storage policy is selected
for the default subclient. The policy defined to manage the subclient is configured in the Storage Device tab – Data
Storage Policy sub tab. Use the storage policy drop-down box to associate the subclient with a policy.
To configure and use a Global Secondary Copy, the Global Secondary Copy Policy first needs to be created. Then, in
every storage policy for which you want to use it, a secondary copy associated to the Global Secondary Copy Policy
must be created.
When configuring storage policy copy data paths, by default, the first data path defined becomes the ‘Default Data
Path.’ If multiple data paths are defined, the ‘Default Data Path’ is the first one to be used. This path can be modified
at a later time.
When the library is shared amongst multiple MediaAgents, additional data paths can be configured. These additional
paths are referred to as ‘Alternate Data Paths.’ There are two available configurations to use with this type of path:
• Failover
• Round-Robin
This Commvault® software feature is called GridStorTM technology. For more information, about GridStorTM features,
refer to the Commvault® Online Documentation.
The ‘Alternate Data Path’ configuration is achieved from the Storage Policy Copy Properties pages. From here you can
configure the path as failover or round-robin.
Retention
Retention Overview
A data retention strategy is important for managing storage in your CommCell® environment. With Commvault®
software, you can define retention for multiple copies of data with each copy having different retention requirements.
Additionally, retention may be required at the object-level and not just the data protection operation. Commvault
software makes this strategy straight forward to implement by using storage policy copies, subclient object-level
retention, and Exchange configuration retention policies.
• Job based retention – Configured at the storage policy copy level, job schedule level, or manually by
selecting jobs or media to retain, and applying different retention.
• Subclient object based retention – Configured at the subclient level, it applies retention-based on the
deletion point of an object. Object-based retention is based on the retention setting in the subclient properties
plus the storage policy copy retention settings.
• Configuration policies – Currently used for Exchange mailbox protection. These policies include archive,
retention, cleanup, and journaling. Configuration policies provide the ability to define complete retention and
destruction policies, including the capability of deleting messages from the production Exchange environment.
Retention Basics
Commvault® software provides extensive retention control for protected data. For basic retention requirements, follow
the general guidelines and best practices for retention configuration.
• Disk storage:
• Leave the Cycles retention set at the default of two
• Use the Days retention to govern retention policies for each copy
• Never use extended retention rules when using Commvault deduplication
• Tape storage:
• Set the Cycles retention based on the number of complete sets of tape copies you want to retain. For
example, if you want 30 days of data stored off-site, which includes at least four full backups and all
dependent jobs (incremental or differential), for complete recovery from any tape set, set the Cycles
retention to four.
• Set the Days retention based on standard retention requirements.
Policy-based retention settings are configured in the storage policy copy Retention tab. The settings for backup data
are Days and Cycles. For archive data, the retention is configured in Days. Retention is also set through schedules or
applied retroactively to a job in a storage policy copy.
Days
A day is a 24-hour time period defined by the start time of the job. Each 24-hour time period is complete whether a
backup runs or not. This way, a day is considered a constant.
Cycles
A cycle is defined as all backup jobs required to restore a system to a specific point-in-time. Traditionally, cycles are
defined as a complete full backup, all dependent incremental backups, differential backups, or log backups; up to, but
not including the subsequent full backup. A cycle is referenced as Active or Complete, which means that as soon as a
full backup completes successfully it starts a new cycle which is the active cycle. The previous active cycle is marked
as a complete cycle.
An active cycle is marked complete only if a new full backup finishes successfully. If a scheduled full backup does not
complete successfully, the active cycle remains active until such time that a full backup does complete. On the other
hand, a new active cycle begins and the previous active cycle is marked complete when a full backup completes
successfully regardless of scheduling. In this way, a cycle can be thought of as a variable value based on the
successful completion or failure of a full backup. This also helps to break away from the traditional thought of a cycle
being a week long, or even a specified period of time.
When setting retention in the policy copy, base it on the primary reason data is being protected. If it is for disaster
recovery, ensure the proper number of cycles are set to guarantee a minimum number of backup sets for full backup
restore. If you are retaining data for data recovery, then set the days to the required length of time determined by
retention policies. If the data recovery policy is for three months, 12 cycles and 90 days or 1 cycle and 90 days will still
meet the retention requirements.
With the release of Commvault Version 11 software, the default retention for a storage policy
primary copy is 15 days and 2 cycles. A secondary copy’s default retention is 30 days and 4 cycles.
• Both Days and Cycles criteria must be met for aging to occur
• Data is aged in complete cycles
• Days criteria is not dependent on jobs running on a given day
Example: Retention for a storage policy copy is set to 3 days and 2 cycles. This is not a typical example, but it’s used
to logically prove the statement that both days and cycles criteria must be met for data to age. By Monday 3 full
backups have been performed. If Friday’s full backup is aged, there would be 2 full backups left meeting our criteria of
2 cycles. However, the days criteria calls for 3 days, and if the Friday full backup was aged, only 2 days would be
counted. The Friday full backup would therefore age on Tuesday.
Monday at 12 PM the data aging operation runs and determines no data can be marked aged
Tuesday at 12 PM the data aging operation runs and determines the Friday full backup can be marked aged
Example: This is another retention example used to prove the rule. Retention is configured for 7 days and 2 cycles.
Full backups are being performed on Fridays and Mondays, and incremental backups on all other days. On Saturday
the cycles criteria of 2 has been met since there are 3 full backups. If a cycle is removed there would be 2 left, a
complete cycle (Monday – Thursday) and the full backup on Friday night. However, since we prune entire cycles we
would have to age the Friday full backup and the incremental backups from Saturday and Sunday. This results in only
5 days, which does not meet our day’s retention requirements of 7. So on Monday when the data aging operation runs
(default 12PM daily) there will now be 7 days and 2 cycles which will allow the first cycle to be aged.
Retention has been defined for 7 Days and 2 Cycles. When the data aging operation runs on Saturday, the cycles
CRITERIA HAS BEEN MET BUT NOT THE DAYS CRITERIA
Retention has been defined for 7 Days and 2 Cycles. When the data aging operation runs on Monday both Cycles and
Days criteria have been met and the first cycle will be marked as aged
Example: Defining retention in both days and cycles is very important. For example, during a Friday night backup
power is lost in the building. Power is restored on Sunday resulting in two days elapsing and counting towards
retention. Note that since the Friday full backup failed, the cycle continues into the next scheduled full (following
Friday).
A FAILURE OF A FULL BACKUP ON FRIDAY DUE TO A POWER OUTAGE RESULTS IN A CYCLE CONTINUING UNTIL A VALID FULL IS COMPLETED
The Spool Copy option is used for fast disk read/write access and its multi-streaming capabilities – when there is
limited capacity available on the disks. A spool copy is not a retention copy. Data is spooled to disk and then copied to
a secondary copy. Once the data is successfully copied to the secondary copy, the data on disk is pruned, immediately
freeing up space for new backups.
Extended Retention
Right-click the desired storage policy copy | Click Properties | Retention tab
Standard retention allows you to define the length of time based on cycles and days that you want to retain data.
Extended retention allows you to define specific retention in days that you want to keep full backups for. It allows you
to extend the basic retention by assigning specific retention to full backups based on criteria configured in the
extended retention settings. Basically, it allows you to set a grandfather, father, son tape rotation scheme.
Extended retention rules are not designed to be used with disk storage and will have significant
negative effects on aging and pruning of deduplicated data.
Example: You want to retain backups for 4 cycles and 28 days. You also want to retain a monthly full backup for
three months, a quarterly full backup for a year, and a yearly full backup infinitely.
Extended retention rules are like selective copies in that they only apply to full backups. However, a selective copy
creates an additional copy of a full backup and assigns it a specific retention. Extended retention applies retention to
an existing full backup and does not create an additional copy. Determine which solution is more appropriate when
planning retention strategies.
Managed Disk Space is a feature used with disk libraries that do not use Commvault® deduplication. This feature
allows data to reside on the disk beyond its retention settings and increases the chance of recovering data faster from
primary storage on disk without changing retention settings. Managed data on disk is treated the same as retained
data for data recovery.
Managed data is held on the disk beyond the standard retention settings until an upper threshold is reached. A
monitoring process detects data exceeding the upper threshold and then deletes aged jobs from the media until a
lower threshold is reached.
It is important to note that only aged jobs will be pruned. If all aged jobs are pruned and the lower threshold is not
met, then pruning will not occur. As a rule of thumb, the upper threshold should be set to allow one hour of backups
to run after the threshold is reached. The lower threshold should be set so that the Managed Disk Space pruning
operation will not run more than once in a backup time-period, as the pruning operation will have a negative effect on
the performance of backups.
Managed disk thresholds are configured in the disk library properties and can be enabled in each storage policy copy.
For more information on configuring library settings for managed disk space, refer to the Commvault Online
Documentation.
It is NOT recommended to set zero cycles for a policy copy unless another copy has been
configured with at least one cycle defined.
Retention can be extended beyond the defined storage policy primary copy retention through a schedule or schedule
policy. This is done by setting the Extend Job Retention options in the Media tab of Advanced Options. You can set
schedule-based retention for a specified number of days or infinitely retain the data. Retention settings at the
schedule level cannot be shorter than the retention defined in the storage policy primary copy.
Retention for a job in a primary or secondary storage policy copy can be retroactively changed in the job history of the
copy. The job can be retained infinitely or until a specific date. The job icon changes to reflect that the job has been
‘pegged down’.
Commvault® Agents
Commvault® software provides agents for all major file system and applications. Agent support includes traditional
streaming backups, built-in content-aware deduplication, Commvault OnePass® archiving, block-level protection, and
hardware snapshot capabilities. It is important to note that these capabilities are inherent in the code and can be
enabled within the agent.
It is beyond the scope of this document to provide detailed information on all agent support.
Application agents
Application agents use application consistent point-in-time protection for application databases. Granular application
protection for Exchange, SharePoint, and Lotus Notes provides flexible data protection strategies and simplified
recovery methods.
Hypervisor agents
The Virtual Server Agent (VSA) provides complete virtual machine (VM) protection and granular recovery of virtual
machine data without the need to install traditional agents within a VM. Based on APIs provided by the hypervisor,
different protection and recovery capabilities are possible.
• Push install
• Interactive install
• Custom package
Agent deployment best practices:
• If DNS names are used, ensure the DNS is properly resolving the name forward and reverse.
• If deploying an agent on a remote site, consider using a remote software cache or transfer a custom package.
• If the client is behind a firewall blocking ports, set network configurations to tunnel communication in a port.
The CommCell® console is used to push the Commvault® software to clients. The following specific ports are used to
achieve the install:
The Admin console is used to push the Commvault® software to servers. The following specific ports are used to
achieve the install:
Interactive install
It is possible to download the desired packages on a client machine by using the download bootstrapper. Once
downloaded, start the installation by executing Setup.exe.
Interactive installation
Servers
The Admin Console uses a smaller set of components to configure the CommCell ® environment, such as solutions,
plans, pools, and storage targets. These components are translated in all CommCell® Console required configurations
which are libraries, global deduplication policies, storage policies copies, backup schedule policies and auxiliary copies.
Therefore, using the Admin Console greatly simplifies the configuration and management of the CommCell®
environment.
Solutions
The Admin Console is built on a self-service, solution-based architecture that adapts to your environment without
complicated administration. Once the initial deployment and configuration of the console are completed, solutions are
added with ease. For instance, if you are managing virtual environments, you can simply add the Virtual Server
Protection (VSP) package—the solution for virtual machine backup and recovery.
Protection Plan
You create protection plans to protect your data by defining what to back up, where to back up your data, and how
often to run the backup job. In the Admin Console, there are predefined plan types that include the data protection
parameters that apply to the type of data you want to protect. Plans are created during configuration; however
additional backup plans can later be defined.
The protection plan is the overall set of configurations required for protecting a server or an application. It is
composed of the following entities:
• RPO – The frequency at which the backups will be conducted. The system automatically translates SLA into
schedule policies and backup job types in the CommCell® Console.
• Storage target – Can be configured before or during the Store Pool configuration. Is translated as a library
in the CommCell® Console.
• Storage Pool – Configures deduplication for a disk storage target. Is translated into a global deduplication
policy in the CommCell®.
During the configuration of a plan, a primary storage pool must be selected. An option to add a
secondary storage pool is offered. This translates into storage policy copies and automatic auxiliary
copy jobs in the CommCell® Console
• Retention – Defines how long the backup data must be retained. This translates in retention configuration on
each storage policy copies in the CommCell® Console.
For example, a Windows 2008 file system uses Volume Shadow Service (VSS) to protect file data, so the Windows
agent has the option to enable VSS during backup operations. The agent then has a backup set defined. The backup
set is a complete representation of all data the agent is responsible to protect. Within the backup set, subclients are
used to define the actual data requiring protection. By default, a default subclient is used to define ALL data requiring
protection within the backup set.
Additional subclients can be created to define specific content requiring protection. When content is defined within the
user defined subclient, it automatically is excluded from the default subclient. An example for a custom subclient could
be defining a specific drive containing user data where VSS is initiated for the drive during backup jobs to ensure all
open files are protected.
The Server navigation structure defines a hierarchy for arranging various components of a server. This hierarchy
remains consistent throughout servers with different agents installed, which results in simplified administration of a
CommCell® environment.
Each component in the server navigation structure provides specific functionality to properly configure, manage and
protect production data. Each of these components has specific features and properties that are configured to provide
a comprehensive protection strategy.
• Server — identifies the client system by CommCell® server name and Host name or IP address.
• Agent — is installed on the server, or a proxy server, to protect specific data such as files, databases, emails or
virtual machines.
• Instance — is used with specific agent types such as database application or hypervisors to define specific
configuration for an instance.
• Backup Set — is a complete representation of all content the agent is responsible to protect.
• Subclient — defines the actual data within the backup set that requires protection.
• Security
• Agents
• Activity control
Accessing the client properties from the Admin Console
Many options and parameters are available for clients. These options are applied to the client, therefore applied
globally to any agent types installed on the client. Options are grouped in three categories.
• Basic options
• Advanced options
• Network options
Administrative Tasks
To help the Commvault® administrator manage servers, several tasks are available to perform on a server:
Connectivity to a client and all storage policy data paths for configured subclients, are checked and reported on at the
client level. The Check Readiness command ensures that the CommServe® server can communicate with the client
machine. It also checks data path connectivity to all MediaAgents and library paths for storage policies that are
associated with subclients.
Client schedules are viewed and edited from the client, agent, backup set and subclient levels. When a specific level in
the client tree structure is selected, all schedules at the selected level and all child object schedules are displayed.
• Schedules can be viewed at the client, agent, backup set and subclient levels.
• If a schedule is associated with a schedule policy, the policy name is listed in the first column.
• Schedules can be edited, deleted, disabled or executed by right-clicking on the job schedule.
• For clients with a large amount of scheduled jobs, consider using the Job Schedule report for a more
consolidated view.
• Though client schedules can be viewed at the CommCell® level using the Schedule icon, viewing schedules in
the client tree provides a simplified view easing administration, especially in large environments.
Job history for data protection and restore operations are viewed from the client, agent, backup set and subclient
level.
• The server or agent appears as deconfigured in the Admin console. This means that data is still retained and
can be restored (out-of-place) in protected storage, but the server cannot be backed up unless the license is
re-applied
• If the CommCell® licensing structure is agent based, the license becomes available to install on another
system
• If the CommCell licensing structure is capacity based, the size of data for the deconfigured server or agent no
longer counts against the capacity usage of the CommCell® environment
• Released licenses can be re-applied to the server by using the Reconfigure option
• To facilitate subclient retention and proper pruning of data, synthetic full jobs can be run on clients or agents
where the license has been released
Releasing a license is a logical operation that is applied to clients or agents. Releasing a license ‘grays out’ the client or
agent so data can still be recovered.
• The client or agent appears grayed out in the CommCell® console. This means that data is still retained and
can be restored (out-of-place) in protected storage, but the client cannot be backed up unless the license is
re-applied
• If the CommCell licensing structure is agent based, the license becomes available to install on another system
• If the CommCell licensing structure is capacity based, the size of data for the deconfigured client or agent no
longer counts against the capacity usage of the CommCell® environment
• Released licenses can be re-applied to the client by using the Reconfigure option
• To facilitate subclient retention and proper pruning of data, synthetic full jobs can be run on clients or agents
where the license has been released
Release license for agents installed on client
• All licenses for the server must be first released to allow the deletion
• Deleting a server is a destructive operation that purges the server’s data on the next data aging
• Consider releasing licenses instead of deleting a server if data needs to be preserved
Deleting a server
Deleting a client is a task that removes the client completely from the CommCell ® environment. It is considered a
destructive operation, since the data associated to that client is marked to be destroyed. During the next data aging
operation, which is scheduled every day at noon by default, the client’s data is physically purged from the disk
libraries. Data on tapes is also marked as aged, and tapes can be recalled as scratch media if empty.
• All licenses for the client must be first released to allow the deletion
• Deleting a client is a destructive operation that purges the client’s data on the next data aging
• Consider releasing licenses instead of deleting a client if data needs to be preserved
Deleting a client
Data protection and data recovery jobs are enabled or disabled in the Activity Control tab in the Client Properties. If
activity is disabled, an Enable after a Delay button is displayed. This is used to automatically enable the activity on a
specific date and time. Client activity control is useful when a client is offline, since any scheduled operations are, by
default, ignored if activity is disabled.
Data aging can be disabled for a client as well. This prevents any jobs that belong to the client to be aged and purged
from storage. This is useful in a situation where a server is decommissioned, and data is required to be kept until
further notice.
Update status for clients are viewed by selecting the Client Computers entity in the CommCell browser or through the
Client Properties page, in the Version tab.
The current Commvault® software version, service pack level, update status, and operating system platform are
viewed for all clients by selecting the Client Computers icon in the CommCell browser. All clients in the CommCell®
environment are displayed, providing summary information on their status.
The Version tab displays the current version of software, service pack level and status of each package installed on a
client.
• Needs Update
• Ahead of cache
• The Ahead of cache may appear in situations where updates have been applied to the system prior to their
general release
• Commvault recommends keeping all software up-to-date
View client update status for all clients
Server Groups
From the CommCell®, there are several methods for adding clients to groups:
In addition to the manual association method, automatic association rules can also be used. This efficient method
automatically associates servers to a group based on a set of predefined rules. Many criteria can be used to define the
rules.
When using the CommCell® Console, there are two association methods for client computer groups. First, computers
can be associated manually to a group. This method requires manual intervention to associate computers to the
group. This is accomplished during the agent installation, by editing the computer group or by editing the client’s
properties.
In addition to the manual association method, automatic association rules can also be used. This efficient method
automatically associates computers to a group based on a set of predefined rules. Many criteria can be used to define
the rules. Once the rules are defined, a Preview button is available to view the resulting list of computers. This ensures
that the rules are properly defined, and results are as expected.
Commvault® software provides the following features so that you can efficiently manage your data protection:
• Job monitoring
• Alerts
• Events
• Resource reports
Once configured, the file servers solution allows to create server plan to define protection settings such as the storage
targets to receive data as well as the retention of the data.
Different plans can be created based on the data type requiring protection. Available types are:
• Data Classification
• Server
• Virtualization
• Snap
The plan is configured from the Plans view of the Admin Console.
Subclient
Subclients are used to define data that is protected in a containerized format. Each subclient container manages
specific content within a backup set. Each backup set has one or more subclients.
Default Subclient
By default, most agents have a default subclient. During the initial installation of the agent software, an option to
associate agent data with a storage policy is provided. This determines the storage policy that manages the default
subclient data. All subclients must be associated with a storage policy to protect the data.
The default subclient acts as a catch-all for all data managed within a backup set. This means the default subclient
automatically detects and protects all data the agent is responsible to protect. When custom subclients are defined,
any data managed by the custom subclients is automatically excluded from the default subclient. This is the concept of
mutual exclusiveness of contents within a backup set. Data is mutually exclusive to the subclient in which it is defined,
and data cannot be defined in multiple subclients within the backup set. The concept of Commvault® software is to
‘Copy Once and Reuse Extensively (CORE).’ In other words, protect the data to the storage policy and use secondary
copies to create additional copies of data. There are situations where protecting data from the source location multiple
times may be required. To accomplish this, you can create additional backup sets.
Early in version 11, the mechanism of mutual exclusiveness of content was taken out of the product, allowing
duplicate content in different subclients within the same backup set to be defined. However, this mechanism
has been re-introduced in V11 SP5. If your subclient design strategy relies on defining exclusive content
between subclients, update the CommCell® environment to the latest service pack.
• Do NOT modify the contents of the default subclient. If only certain drives are to be protected, use the Filter
tab to exclude those drives.
• Add content by using the Browse button or manually entering it using the Add Paths button.
• Define content drives, folders, or file types such as Office documents or Media files.
• Use the Add Paths button to enter UNC paths to protect data on systems that do not have Commvault agents
installed. You will be prompted to enter a user account with proper permissions to read the data from the
shared location. This feature is only recommended when protecting small amounts of data.
• Any data defined in a custom subclient is automatically filtered from the default subclient.
• An explicit path such as f:\ cannot be defined in two separate subclients within the same backup set.
• If one subclient has the f:\users folder defined and a second subclient has f:\users\finance defined, the
f:\users\finance folder will be backed up by both subclients.
• If one subclient has the f:\ drive defined and a second subclient has content defined as *.DOCX, both
subclients will protect all DOCX files redundantly.
• If it is necessary to configure subclients using the above examples and you want to avoid protecting
redundant data, ensure paths are filtered from other subclients.
Example:
• If one subclient has the f:\users folder defined and a second subclient has f:\users\finance defined, the
f:\users\finance folder will be protected only by the f:\users\finance subclient and will automatically be
excluded from f:\users.
• If one subclient has the f:\ drive defined and a second subclient has content defined as *.DOCX, both
subclients will protect all DOCX files redundantly.
• To prevent file types from being protected in multiple subclients, filter the file type from other subclients. Filter
options include the ‘content library’ option.
Subclient Configuration
Add Subclients (Admin Console)
Administration | Servers | Agent | BackupSet | Add subclient
• When new subclients are added, the content defined in the new subclient is automatically removed from the
default subclient.
• Content within a backup set cannot be defined in more than one subclient, except for VSA subclients.
• When new subclients are added, the content defined in the new subclient is automatically removed from the
default subclient.
• Content within a backup set cannot be defined in more than one subclient, except for VSA subclients.
Filtering
Setting Subclient Filters (Admin Console)
Administration | Servers | Agent | BackupSet | Subclient | Content section | Edit
Filters are defined at subclient level to remove specific folders and objects that do not require protection.
Similar to defining content, wildcards and patterns are used to define exclusion filters and
exceptions.
Filters are defined at the global and subclient level to remove specific folders and objects that do not require
protection. Global filters defined in the Global Filters applet in Control Panel are automatically or manually associated
with subclients. If global filters are associated with a subclient, the choice to override the global filters is available.
• Global filters can be defined for Windows, UNIX, Exchange (Classic), Hadoop, IBM i, Macintosh, NAS, NFS,
OES, OpenVMS, and Virtual Servers.
• To enforce global filters to subclients, enable the ‘Use Global Filters on all Subclients’ checkbox.
• Subclient settings to inherit global filters are configured as:
o On – always use global filters.
o Off – never use global filters.
o Cell-Level Policy – only use global filters if ‘Use Global Filters on all Subclients’ checkbox has been
enabled.
• Subclient filters include exclusion and exception filter entries:
o Exclusion filters – determines which folders and/or objects will be excluded from the subclient.
o Exception – is an override for exclusion and global filters. This means any folders and/or objects
defined in the exception entry is protected by the subclient.
Similar to defining content, wildcards and patterns are used to define exclusion filters and
exceptions.
Commvault® software has several methods to help you successfully achieve data protection. Each method impacts the
Recovery Point Objective (RPO) and Recovery Time Objective (RTO) parameters, which determine the best data
backup plan. Understanding Commvault’s data protection methods, their strengths and benefits will result in an
optimal architectural design that achieves your data protection needs.
Backup Types
Commvault® software provides the following backup types for protecting data:
• Full
• Incremental
• Differential
• Synthetic Full
• DASH Full
Full
A full backup protects all data within a subclient each time it is run. This provides the most complete level of
protection by backing up all data every time the job runs. It also provides the fastest recovery time since all data is
contained within a single job.
Full backups require the most storage space and take the longest time to complete.
Incremental
Incremental backups protect all data that has been added or modified since the last successful backup operation.
Based on the average incremental rate of change and growth, the incremental backups should remain consistent in
size. An incremental backup is considered a dependent operation, as it is dependent on previous incremental backups
and the full backup which started the cycle.
For a full restore of data, the full and all incremental backups are required.
Differential
A differential job backs up all data that has been modified or added since the last full backup job. The first differential
job following a full backup just contains incremental changes since the full backup completed. As subsequent
differential backups are run, the backup size increases since all changed and added data is backed up in each
differential. As the cycle progresses and more differential backups are executed, they continually increase in size
requiring more storage until the next full backup runs. Restores are slower than a full, but faster than using
incremental jobs since only the full and most recent differential is required for a complete restore.
Another advantage of differential jobs is that modified data is being redundantly stored throughout the cycle as each
differential completes. This could potentially limit data loss if a differential job is lost or damaged.
For synthetic full backups to work properly, an initial full must be run which provides the foundation in which the
synthetic full backups will be based on. Incremental backups must be run after the initial full and subsequent synthetic
full to ensure all required objects are in protected storage. When the synthetic full runs, it copies all required objects
into a new synthesized full backup, which will then become the foundation for the next synthetic full backup.
• Synthetic full backups are useful for large volumes or Exchange mailbox backups where many objects require
protection or when the production client has very short operation windows.
• Synthetic full backups work best on non-deduplicated disk storage. When using Commvault deduplication, use
DASH Full backups instead of traditional synthetic full backups.
• Using Synthetic full backups on appliance-based deduplication devices can have a negative impact on
performance. In some cases, the performance can be slower than running regular full backups.
if using 3rd party deduplication solutions, test this option before implementing.
DASH Full
A read optimized synthetic DASH Full uses the Commvault® deduplication feature to logically perform synthesized full
backups without moving any data. This can be accomplished because Commvault deduplication tracks the location of
all blocks on disk storage. After the initial base full is run and subsequent incremental jobs are run, all block data
required for the synthetic full is already present in the deduplicated disk storage location. Since deduplication only
stores a unique block once in storage, the DASH Full operation only makes references to the blocks in storage and not
actually copies them. The DASH Full operation generates a new index file signifying that a full backup was run and
updates the Deduplication Database (DDB) with block record data that is used for data aging purposes. DASH Full
backups are the preferred method of running full backup jobs and can dramatically reduce backup windows.
When enabling Commvault deduplication for a primary copy, the ‘Enable DASH Full’ option is
selected by default.
Auxiliary copy operation allows you to schedule, run on-demand, save a job as a script, or set an automatic copy.
There are several options to choose from when configuring Auxiliary copy operations:
DASH Copy
A DASH Copy is an optimized auxiliary copy operation which only transmits unique blocks from the source library to
the destination library. It can be thought of as an intelligent replication which is ideal for consolidating data from
remote sites to a central data center and backups to DR sites.
• DASH Copies are auxiliary copy operations, so they can be scheduled to run at optimal time periods when
network bandwidth is readily available. Traditional replication would replicate data blocks as it arrives at the
source.
• Not all data on the source disk needs to be copied to the target disk. Using the subclient associations of the
secondary copy, only the data required to be copied would be selected. Traditional replication would require
all data on the source to be replicated to the destination.
• Different retention values can be set to each copy. Traditional replication would use the same retention
settings for both the source and target.
• DASH Copy is more resilient in that if the source disk data becomes corrupt the target is still aware of all data
blocks existing on the disk. This means after the source disk is repopulated with data blocks, duplicate blocks
will not be sent to the target, only changed blocks. Traditional replication would require the entire replication
process to start over if the source data became corrupt.
Disk optimized, which is the default setting, should always be used when the source library is using Commvault
deduplication.
Disk optimized DASH Copy extracts signatures from chunk metadata during the auxiliary copy process, which reduces
the load on the source disks and the MediaAgent since blocks do not need to be read back to the MediaAgent and
signatures generated on the blocks.
Network optimized should only be used if the source library is not using Commvault deduplication. Network optimized
DASH Copy reads all blocks required for the auxiliary copy job back to the MediaAgent, which generates signatures on
each block.
To schedule an auxiliary copy job as a DASH Copy, first go to the Secondary Copy Properties Deduplication tab and,
from the Advanced sub tab, select the ‘Enable DASH Copy’ checkbox and ensure that 'Disk Optimized' is also checked.
There are several storage policy copy options that impact auxiliary copy jobs:
• Inline Copy
• Parallel Copy
• Deferred Copy
• Selective Copy
Inline Copy
Right-click the storage policy secondary copy | Click Properties | General tab
The Inline Copy option lets you create additional copies of data at the same time you are performing primary backups.
This feature is useful when two copies of data must be done quickly. Data is passed from the client to the MediaAgent
as job streams. The MediaAgent then creates two sets of device streams; each going to the appropriate library.
Although this is a quick method for creating multiple copies, there are a few caveats to consider:
• Inline Copy is not supported if Client Side Deduplication has been enabled.
• If the primary copy fails, the secondary copy also fails.
• Since both copies are made at the same time, twice as many library resources are required, which may
prevent other jobs from running.
• Since backup data is streamed, data is sent to both libraries simultaneously, which may cause overall
performance to degrade. Basically, your job runs as fast as the slowest resource.
The last point is important to understand. Consider a scenario where the primary library receiving the client streams is
a disk library, and two secondary libraries are cloud and tapes. If Inline Copy is enabled on both secondary copies, the
three copies are performing at the speed of the slowest target, in this case, let’s assume it is the WAN link to reach
the cloud library. It might result in tape drive buffers not filling up quickly enough. The tapes therefore must
constantly be paused and repositioned, also known as “shoe shinning.” This reduces the lifespan of tapes and drives
significantly.
Parallel Copy
Right-click the storage policy secondary copy | Click Properties | General tab
A parallel copy generates two secondary copy jobs concurrently when an auxiliary copy job runs. Both secondary
copies must have the ‘Enable Parallel copy’ option selected and the destination libraries must be accessible from the
same MediaAgent. Like the Inline copy option, the performance is based on the speed of the slowest target.
There is an advantage of using Parallel Copy over Inline Copy to create multiple secondary copies. A Parallel Copy is
executed by an auxiliary copy schedule and is independent of the backup job — thus not slowing up the backup
performance as an Inline copy will do.
Deferred Copy
Right-click the storage policy secondary copy | Click Properties | General tab
Deferring an auxiliary copy prevents a copy from running for a specified number of days. Setting this option results in
data not aging from the source location, regardless of the retention on the source, until the auxiliary copy is
completed. This option is traditionally used in Hierarchal Storage Management (HSM) strategies where data will remain
in a storage policy copy for a certain period. After that time, the data is copied to another storage policy copy and
deleted from the source during the next data aging job.
With Commvault® software it is recommended to copy data to multiple HSM copies to provide for
disaster recovery, as well as HSM archiving.
Job Initiation
A job can be initiated using any of the following methods:
• Manually – by launching the job from the console and selecting to execute it immediately
• Scheduled – either an independent schedule or a schedule policy
• Save as a script – to be later executed by a user or by an external automated scheduler
Initiating a job
Different types of jobs can be scheduled to run on different days. The traditional weekly full and daily incremental is
the most common schedule, but not always the best. Consider scheduling jobs based on RTO, RPO and how
frequently data should be sent off-site.
For example, if data is being backed up to disk and DASH copied to disk at a DR location, jobs can be scheduled to run
more frequently throughout the day to shrink RPOs and send data off-site faster. If the disk data needs to be
consolidated to tape on a monthly basis, a schedule of monthly full and daily incremental with the monthly full being
Aux copied to tape may be an adequate solution. Another example is when performing a backup directly to tape. A
schedule of nightly full backups or weekly full and nightly differential can be implemented. Both methods may shrink
RTO times.
Auxiliary copies are as important to properly schedule as data protection jobs. If the corporate requirement is a four
hour RPO for critical databases, and the database backup running every four hours is DASH copied only once a day, it
can lead to an SLA not being met in case of a disaster, where the primary disk library is lost.
Commvault® software uses a standard scheduler for scheduling all operations within the CommCell® environment. The
following is a list of operations that can be scheduled:
• CommServe DR backup
• Reports
• Data Aging
Schedule Frequency
Jobs can be configured to run once or repeat based on setting the following frequencies:
• Daily – jobs are set to start and end at a certain time each day. A repeat interval can be configured to have
the job run multiple times per day with an hour and minute recurrence interval.
• Weekly – jobs are configured to run on specific days of the week or set to repeat every fixed number of
weeks. Exceptions can also be added, for example if a job is to be scheduled to run every Friday except for
the last Friday of the month, set the weekly schedule to run on Friday, repeat every week, and set an
exception for the last Friday of the month.
• Monthly – jobs are configured to run based on the following criteria:
• On the first, second, third, fourth or last day of a month.
• The day of the month specified can be a day of the week, specific day, weekday or weekend day.
• Yearly – jobs are configured to run based on the following:
• Specific day of a specific month.
• On the first, second, third, fourth or last day of a specific month.
• The day of the month specified can be a day of the week, day, weekday or weekend day.
• Automatic – schedules are used for protecting end user workstations or automatically scheduling specific
database application log backups, such as SQL or Oracle.
• Continuous – is a schedule in which an interval between job is defined.
o The first job is executed
o Starting from that point on, as soon as it completes, it waits for the time interval to elapse and
launches the job again.
Independent Schedules
Right-click the subclient | Click Backup | Job Initiation section
Jobs can be scheduled to run independently. An independent schedule is created at the client level, and is used only
by that client. It is initiated from the Job Initiation section of a job. All backup or auxiliary copy options can be defined
for the job.
There are several schedule patterns that are used, but one frequently used for independent jobs is the One-Time
pattern. It executes an on-demand job later, without having to manually launch it.
Schedule Policies
Expand Policies | Right-click Schedule Policies | Click New Schedule Policy
Jobs can be scheduled to run by using schedule policies. A schedule policy is a set of rules containing one or many
schedules. This set of schedules is deployed to multiple systems or storage policies requiring a similar schedule pattern
for backups or auxiliary copies. This helps in managing and controlling the schedules in bulk.
At any time, subclients or storage policies can be added or removed from a schedule policy. It is also possible to
manually execute a schedule from the schedule policy, which simultaneously launches backups on all systems
associated with it.
Automatic Schedules
Commvault® software provides the ability to control the execution of jobs using an automatic schedule. The automatic
schedule uses criteria which, when met, triggers the execution of the task automatically. Automatic schedules are used
for three specific examples:
• Laptop backups
• Database backups
• Auxiliary Copy
Most jobs run once during the day while normal schedules can be used for auxiliary copies. The automatic
copy allows you to set a check interval for source data to be copied. This is an advantage when jobs are run
multiple times per day, or if you are unsure when the source data will become available for copy.
Example: A critical database is running transaction log backups every four hours. You want to run an
auxiliary copy of the source transaction logs to a secondary location, in this case a disk library off-site.
Solution: Schedule the transaction logs to back up every four hours. Then set the automatic auxiliary copy
option to check for source data in much shorter intervals. If source data is present, the auxiliary copy runs
creating an additional copy of the data.
The protection of mobile users’ laptops is controlled through an automatic schedule that uses criteria, such as
available bandwidth, AC power, or if the laptop is idle. This ensures that backups are executed when the computer is
under ideal condition.
When setting up the criteria for laptop backups, two values must be set for intervals between jobs, which are as
follows:
• Minimum interval between job – This is the minimal time between two consecutive backup jobs. The
default value is 15 minutes. If the laptop has been protected more than 15 minutes ago, and if it meets all
requirements, such as available bandwidth, AC power, etc., it is then protected.
• Maximum interval between job – This is the maximum acceptable time between backup jobs. The default
value is 72 hours. This means that if a laptop has been protected more than 72 hours ago, even if it does not
meet the requirements, such as available bandwidth, AC power, etc., the system attempts to back it up. This
prevents a laptop to function for a long period without being protected. An example is when the laptop’s
battery is used frequently or the bandwidth is limited.
The automatic backup schedule for databases allows to set a space usage threshold for the volume where the
database log files reside. If the space usage is reached, a backup is triggered, which implies truncating the logs at the
end of the backup. It will therefore free up space on the volume. This is a useful tool to prevent database log volumes
to fill up if backups are not scheduled to run frequently enough.
When setting up the criteria for database backups, two values must be set for interval between backup jobs, which
are as follows:
• Minimum interval between job – This is the minimal time between two consecutive backup jobs. The
default value is 15 minutes. If the database has been protected more than 15 minutes ago, and its log volume
space usage reaches the defined usage threshold, it gets protected.
• Maximum interval between job – This is the maximum acceptable time between backup jobs. The default
value is 72 hours. This means that if a database has been protected more than 72 hours ago, even if log
volume space usage has not reached the defined threshold, the system protects it.
Commvault software supports the following applications for database backups:
For the most current list of applications, refer to the Commvault ® Online Documentation.
An automatic schedule is used to run an auxiliary copy. You can create the automatic schedule if you want to perform
the auxiliary copy at a specific frequency to ensure that an additional copy of data is automatically created. If you
define a frequency of 30 minutes, this means that every 30 minutes the system checks if there are completed jobs in a
storage policy primary copy that have not yet been copied to a secondary copy. If this is the case, the auxiliary copy
job starts copying these jobs.
An automatic schedule has a significant advantage over a traditional auxiliary copy schedule that runs after the
backups. Since it executes every 30 minutes, if jobs have completed, they are copied even if other backup jobs are
still running. This provides the shortest auxiliary copy window possible.
When Commvault® software is installed, a schedule policy named System Created Autocopy schedule is implemented
by default. When a storage policy secondary copy is created, it automatically becomes associated with that schedule
policy that executes auxiliary copies every 30 minutes.
execute a failed job that encountered an issue after it was resolved. Instead of having to initiate the job through the
Servers view and select all options, it can simply be resubmitted from the Jobs view.
Status can be changed for multiple job simultaneously by using the multi-select job option.
Resubmitting a job
The Jobs view simplifies the scavenging of information from log files by concatenating it in a central window. Each
section has a header providing information on the server and log filename from which the entries were extracted.
• If activity is disabled for a parent object in the Admin Console, activity is automatically disabled for any child
objects.
• Activity can be disabled until manually enabled or set to automatically re-enable at a specific date and time.
• If activity is enabled for a parent object, activity can be enabled or disabled for any child objects.
Example: A server group representing servers for a specific location is disabled for maintenance. By disabling activity
at the group level, all servers within the group are automatically disabled.
Example: A specific server has a maintenance window scheduled. By disabling the activity for that server, no
operations will run. All other servers within the group will operate normally.
• If activity is disabled in a parent object of the CommCell tree, activity is automatically disabled for any child
objects.
• Activity can be disabled until manually enabled or set to automatically re-enable at a specific date and time.
• If activity is enabled for a parent object in the CommCell tree, activity can be enabled or disabled for any child
objects.
• When activity is disabled or enabled, the icon where the activity was set changes to reflect the current activity
state.
Example: A client computer group representing clients for a specific location is disabled for maintenance. By disabling
activity at the group level, all clients within the group are automatically disabled.
Example: A specific client has a maintenance window scheduled. By disabling the activity for that client, no
operations will run. All other clients within the group will operate normally.
Job Priorities
Commvault® software implements a robust method for configuring job priorities. There are three different number
values that make up a job priority, the job type, client, and agent. The three numbers are combined to form a three-
digit priority level.
In Commvault® software the zero value has the highest priority and the nine value has the lowest
priority.
Each job type has a specific priority value associated with it:
• CommCell administrative operations such as data aging and the CommServe DR backup have a zero-level
priority.
• Restore operations have a zero-level priority.
• Backup operations have a one level priority.
• Auxiliary copy jobs have a two-level priority.
The backup and restore priority numbers cannot be modified. It would risk breaking the system
and prevent jobs from running.
Client Priority
Client priorities are configured for individual clients in the Job Options tab in the Client Properties. The default client
priority is six.
Agent Priority
Agent priorities are configured for each agent type in the Job Management applet in the control panel and have a
default priority of six.
Example: if an Oracle backup is running with an agent priority set to four on a client with a priority set to six, the
default priority for the backup would be 164. One for the backup type, six for the client, and four for the agent type.
Dynamic Priority
A new dynamic priority mechanism is available in SP11. It is applied to concurrent jobs with the same priority number.
Dynamic priority intelligently prioritizes the backups based on machine learning.
• Strike Count - is defined as the number of failures that happened since last successful backup job of the
same backup level. Strike count is for a subclient and is computed dynamically.
• Estimated Completion Time -The Estimated completion time is forecast based on previous backup job
patterns of the same backup level on the subclient using machine learning .
Dynamic Priority Calculation:
• When jobs have equal Operation/Client/Agent priorities, the subclient with the greater Strike Count is
prioritized.
• When the jobs have equal Strike Count also, the subclient with the higher Estimated Completion Time is
prioritized.
• Backups Preempt Other Backups – when enabled, permits newly executed jobs to preempt backups jobs.
• Backups Preempt Auxiliary Copy – when enabled, permits newly executed jobs to preempt auxiliary copy
jobs.
Example: A higher priority job preempts a lower priority job. If the first job is writing to the media, a higher priority
job can take over when the first job completes its writing phase. Higher priority jobs may acquire access to the media
right after a lower priority job completes its job phase.
Enabling preemption of too many jobs could negatively affect overall performance.
Operation Windows
Operation windows allow the Commvault® administrator to designate black out windows in which selected operations
do not run. These rules can be set at the global, client computer group, client, agent, and subclient levels. Child
objects can also be configured to ignore operation windows at higher levels. Different operation windows can be
defined for data protection jobs, recovery jobs, copy jobs and administrative jobs.
Each defined operation window can have one or more ‘Do not run’ intervals defined. Different operation rules can be
specified for the same operation type to define specific time intervals for different days of the week.
• Indexed based jobs finishes writing their current chunk, then are placed in a waiting state. When the blackout
period is lifted, the job continues from the most successfully written chunk.
• Non-Indexed jobs continues writing and ignores the operation windows blackout period.
For specific client groups, clients, agents or subclients, operation windows can be configured to ignore operation rules
at higher levels. In this case if no rules are defined, and the ignore operation rules at higher levels is enabled, the jobs
will run without interruption.
Data Recovery
• Indexed
• Non-Indexed
Find* Provides the ability to enter search criteria for a file, such as myfile.txt or *.docx, file size, or access
time. This option is useful if you know the specific file(s) that needs to be restored, but do not know the
location, or when you have some information, such as a partial file name.
Browse Provides the ability to browse for all protected data using the folder hierarchal structure (like Windows®
Explorer). This method is useful when multiple files, folders or drives need to be restored.
Restore* Provides the ability to enter a drive, folder path or file path such as ‘F:\users\jdoe’ that is required for
restore. This option is useful when you know the specific location for data required for restore.
Full system restore Provides the ability to restore an entire server in case of a full system crash. This method requires that
all data on the server including ‘system state’ data has been protected. It also requires a base operating
system and Commvault® file system agent to be installed prior to the restore. This method is useful
when the operating system can be reinstalled or if base images are being deployed to servers.
1-Touch restore* Provides the ability to restore an entire server in case of a full system crash. This method uses a boot
image to boot the system with a temporary operating system. It then rebuilds the operating system
through a full system restore. This method is useful when a system needs to be recovered with
minimum administrator effort.
Restore by Job* Provides the ability to perform a non-indexed restore using one or more streams for one or more jobs.
This method is useful in disaster recovery scenarios when the index directory is not available. An indexed-
based restore would have to restore index files from media before the restore can begin. A non-indexed
restore immediately begins restoring data.
If multiple versions are restored, each version has a sequential number appended to the filename
beginning with 1 for the most recent version of the file.
With email, you can use the Find operation to search message metadata using the ‘From,’ ‘To,’ and ‘Received’ fields of
the message.
Using Browse
A Browse and Restore operation allows the administrator to browse through the folder structure to select files and
folders to restore. You can select multiple files and folders for recovery operations. If a parent object in the folder
structure is selected, then all objects within the parent folder are automatically selected for restore.
When selecting a file that was modified multiple times during a cycle, the specific version of the file or all versions can
be selected to be recovered.
• Image Browsing
• No-Image Browsing
Image Browse
Each time a backup operation is conducted an image file is generated, which represents a view of the folder structure
at the time the backup occurred. By default, when a browse and restore operation is conducted, an ‘image browse’
method is used to present the folder structure as it existed based on the browse date and time. This is done by
displaying the folder structure from the most recent image file prior to the point-in-time being browsed. So, if a
browse is being conducted on Wednesday at 2:00 PM and the most recent backup was run on Tuesday at 10:00 PM,
the image file from the 10:00 PM backup is used. This ‘image browse’ method produces a consistent structure of the
data according to the browse time. This is important since folder structures may change from day-to-day during a
cycle.
When restoring an entire folder structure, it is important that the structure represents a specific point when a backup
was conducted—and not represent data for the entire cycle. This is best explained by using temporary files as an
example. Temporary files and folders can be generated, deleted and regenerated multiple times during a cycle. Each
time a backup is run, the file folder structure is different based on which files existed at the specific point-in-time.
When a restore operation is run, you would not want every temporary file and folder to be restored, just a particular
point-in-time or day.
Although the ‘image browse’ method is beneficial for restoring file and folder structures to a particular point-in-time, it
may also result in deleted items not showing up when a Browse and Restore operation is conducted. For example, if
on Wednesday at 2:00 PM a browse operation is run using the Tuesday 10:00 PM image file, and a file or folder was
deleted on Tuesday at 2:00 PM, the deleted files will not appear in the browse results. This is because when the 10:00
PM image file was created, the deleted files were not present.
No-Image Browsing
The ‘no-image browse’ is used to retrieve data that may have been deleted at some unknown time. It browses all the
data (including deleted items) for the selected backup set according to the browse time. It is also useful for retrieving
a previous backup version and showing deleted files across cycles.
There are two options to ensure deleted items are displayed during Browse and Restore operations:
1. Select the Show Deleted Items check box - This runs a ‘no-image browse.’ In this case, the image files
are bypassed, and the browse operation returns results from the index directory, which shows all items
backed up from the point the full was run. This method is useful when recovering user data that has been
deleted but may not be a good choice when restoring an entire folder structure, especially if the folder
structure was modified during the cycle.
2. Specify date and time to browse - This runs ‘image browse.’ If you know when the data was deleted,
specify that date and time in the Browse and Restore options. So, if data was deleted at 2:00 PM on Tuesday,
and you specify Tuesday as the browse date, then the most recent image file prior to the point the browse is
being conducted would be Monday at 10:00 PM. Since the data was deleted on Tuesday it would be present in
the image file on Monday night and will show up in the browse results.
• Data can be restored in-place, which is on the same system, in the same location.
• Data can be restored out-of-place, on the same server in an alternate location or on a different server.
• Files can be restored in a remote location by providing a UNC path and the user who has access to the share.
• Data can be overwritten if it already exists in the destination location.
There are three options to locate the data to restore:
Multi-Stream Restore
Use Browse or Find to select objects to restore | Click Recover All Selected | General tab
Commvault® Version 11 Service Pack 6 introduces the ability to uses multiple streams during an index-based restore
such as Browse and Restore or Find. By default, the system traditionally uses one stream, but this number can be
increased by using the ‘Number of streams’ option in the Restore Options General tab. Multi-stream restore is only
supported for data using V2 indexing.
Copy Precedence
Click Browse or Find | Choose Advanced Options tab
Each storage policy copy within a storage policy has a copy precedence number assigned to it. By default, the primary
copy has a precedence number of one. The next secondary copy created will have a precedence number increased by
one in the order in which they are created.
When restoring data using Commvault® software, by default, it restores the data from the lowest copy precedence
number where the data is available. For example, data is stored on disks for 30 days and copied on tapes for 90 days.
If you browse for data that is earlier than 30 days, there is no need to specify the disk copy. The system automatically
goes to the disk copy since the primary copy has a precedence number one (1). If you browse for data older than 30
days, the system automatically goes to the tape copy, which has a precedence number two (2).
If it is required to restore data from a higher precedence number (even if the data is available from a lower
precedence copy), you can override the default behavior by specifying a precedence number in the Browse and
Restore or Find operations’ Advanced Options tab.
Scenario: You are backing up data to disks and copying the data to a secondary tape copy to send offsite
daily. Your manager is asking you to prove that you can restore data from tapes before sending it to the
offsite location.
Solution: To restore data from tapes and avoid the system automatically restoring from disks, specify the
tape copy precedence number during the restore.
Restore by Job
The Restore by jobs operation is a ‘non-indexed restore’ that uses multiple concurrent streams during the restore
operation. Unlike a traditional Browse and Recovery operation – which requires index files to be available in the index
directory and uses a single stream during the restore operation, the ‘Restore by Jobs’ provides a faster recovery in DR
situations when the production index directory location is not available. Since this operation does not require indexes
and it is a chunk-based restore that allows multiple streams to run simultaneously, restore operations run immediately
and provides a faster restore speed.
There is a negative aspect of the Restore by Jobs operation. The single pass restore method used with indexed-based
recoveries (where only the proper version of an object is restored based on the point of browse) is not used. This
means that in order to bring a machine back to its last state, the last full backup job would need to be recovered first
followed by each subsequent incremental job ending with the most recent.
The Restore by Job option restores an entire job and therefore does not offer any restore
granularity.
1. Build a new machine with similar hardware and same mass storage configuration.
2. Install a working Operating System of the same level as the crashed client (i.e. Windows 2008R2 SP2).
3. Install the client agent on the target system, but configured using the crashed system client name.
4. A full backup of the system including the Windows® system state must be available. The system state backup
should not have critical components filtered. Here are the required critical system state components:
o Registry
o System Protected Files
o Active Directory
o Cluster DB
o Sysvol (For 2000 or 2003 domain controllers level)
o DFSR (Windows 2008 Domain controller level)
5. Browse at the backup set and select the entire backup set.
6. Perform the restore.
7. Reboot.
1-Touch Recovery
1-Touch can achieve multiple tasks during the restore, such as:
Virtualization Solution
There are four primary methods Commvault® software can use to protect virtual environments:
Depending on the hypervisor application being used and the virtual machine’s operating system, different features and
capabilities are available. The VSA interfaces with the hypervisor’s APIs and provides capabilities inherent to the
application. As hypervisor capabilities improve, the Commvault VSA agent is enhanced to take advantage of new
capabilities.
One issue when using agents in virtual machines is when the virtual machine needs to be restored. Since the agent
protects all data at the object level, the machine will need to be restored object by object. Compare this method to
using the VSA backup process which can restore the entire virtual machine at the VM level. When protecting large
databases which are backed up as single objects, agents can be a good solution. When backing up file servers with
large amounts of smaller objects, agents within the virtual machine would not be a good solution.
With Commvault client-side deduplication, data moved over the network is dramatically reduced once the first full
backup is completed. This provides an efficient method of backing up large amounts of data and is recommended to
improve backup performance when using agents inside of VMs. It’s important to note that when using client side
deduplication in a virtual machine, all blocks will be hashed on the client. This processing will be done using the
hosting server’s resources which may negatively impact performance when too many VMs are being backed up
concurrently. Carefully consider on which VMs you want to use agents and schedule backup operations during off-peak
hours when physical hosts have adequate resources to process and protect data.
Commvault database agents provide advanced protection features that would not be available when using VSA.
Separate protection of database and logs can be performed. Options to truncate logs or replay logs to a specific
point-in-time can be used to better manage database protection. Using database agents in virtual machines provides
application consistent database protection and is a preferred protection method.
Transport Modes
VMware Transport Modes
The VMware® VADP framework provides three transport modes to protect virtual machines:
HotAdd Mode
HotAdd mode uses a virtual VSA in the VMware environment. This requires all data to be processed and moved
through the VSA proxy on the ESX server. HotAdd mode has the advantage of not requiring a physical VSA proxy and
does not require direct SAN access to storage. It works by ‘hot adding’ virtual disks to the VSA proxy and backing up
the disks and configuration files to protected storage.
A common method of using HotAdd mode is to use Commvault deduplication with client-side deduplication, DASH Full
and incremental forever protection strategy. Using Change Block Tracking (CBT), only changed blocks within the
virtual disk have signatures generated and only unique block data are protected.
This mode is also useful when there is no physical connectivity between the physical VSA proxy and the Datastore
storage preventing the use of SAN transport mode. Some examples of such scenarios are when using NFS Datastores
or using ESX hosts local disk storage to host Datastores.
NBD Mode
NBD mode uses a VSA proxy installed on a physical host. The VSA connects to VMware and snapshots will be moved
from the ESX server over the network to the VSA proxy. This method requires adequate network resources. NBD mode
is the simplest method to protect virtual machines.
Use the Admin Console's Virtualization dashboard to check the status of applications, such as VMware® or Hyper-V®.
This view shows information specifically tailored for Virtualization jobs. From here you can check the number of
Hypervisors in your environment and the number of VMs protected. The virtualization dashboard also includes backup
job summaries with details such as how many jobs were completed, failed or killed. The dashboard also provides a
status of SLAs, and lets you monitor your Storage Utilization.
• Amazon
• Google Cloud Platform
• Microsoft Azure
• Microsoft Azure Stack
• Microsoft Hyper-V
• Nutanix AHV
• VMWare (vCenter or ESX server)
• OpenStack
• Oracle Cloud
• Oracle VM
During configuration, a backup plan must be created. The plan dictates when VMs are backed up and where the data
is stored. Additional backup plans can later be defined.
VSA Filters
Virtual Machine Swap File Filtering
When backing up VMware® or Hyper-V virtual machines, by default, the VSA filters the Windows page file or Linux
swap file. To achieve this, the system maps the virtual machine disk blocks from which the page file or swap file is
made of. These blocks are skipped during the backups, significantly reducing the storage footprint and the backup
time.
It is possible to disable the skipping of page and swap files by creating the bSkipPageFileExtent additional setting
on the VSA proxy and by setting its value to 0 (zero).
Example: A database server requires protection. For shorter recovery points and more granular backup and recovery
functionality, a database agent can be used to protect application database and log files. For system drives, the virtual
server agent can be used for quick backup and recovery. Disks containing the database and logs should be filtered
from the VSA subclient. The VSA will protect system drives and the application database agent will be used to protect
database daily and log files every 15 minutes. This solution provides shorter recovery points by conducting frequent
log backups, application aware backup and restores, and protects system drives using the virtual server agent.
Subclient or backup set filters can be used to filter virtual machines or virtual machine disks for both Hyper-V and
VMware. If auto-discovery rules are used to define content, it is recommended to apply filters at the backup set level
to ensure that no subclients protect the VM.
The option to filter an entire datastore is available in the disk filters option. This means that VM disks stored in the
selected Datastore are filtered out from backups.
Application aware VSA backups inserts an ‘application plugin’ into the VM during a VSA backup and IntelliSnap®
feature. When a VM backup runs, the plugin quiesces the application using a VSS snapshot. The VSA coordinator then
communicates with the hypervisor to conduct a VM snapshot. If IntelliSnap is used, a hardware snapshot is taken on
the Datastore and then the software snapshot and VSS snap is released.
Oracle Oracle
Microsoft Microsoft Microsoft SQL
Hypervisor database for database for
Exchange SharePoint Server
Windows Linux
Amazon
(streaming)
Microsoft Hyper-
V (streaming)
Microsoft Hyper-
V (IntelliSnap
with non-
persistent snap
engines)
To enable application aware VSA backups, a user account with administrative privileges for the application must be
used. This account can be entered at the instance or subclient level. When the VSA backup runs, the system detects if
any supported agents are installed in the VM and automatically installs the application plugin. After the backup
completes, the plugin remains in the VM for subsequent backup operations. Application data recovery is conducted
using the agent in the CommCell® console, providing full agent level recovery options.
1. It validates that the MediaAgent software is installed on the VSA proxy server
2. It validates that the Snap Copy is created for the storage policy
3. It discovers if a supported application is installed in the VM
4. It pushes the application plugin
5. It protects the application
The job history of a subclient can easily be displayed in the Admin Console.
• From the VM Group view – This initiates a backup for all the VMs defined in the VM Group/subclient.
• From the virtual machine – This initiates a backup only for the selected virtual machine.
• Virtual machines must be backed up at least once before you can request a backup on demand.
• An on-demand VM backup is always an incremental backup.
• If a VM is included in multiple subclients, an on-demand backup is run for the subclient that performed the
latest backup of the VM.
• The Hyper-V host must be running on a Windows 2012 R2 server operating system
• For Windows, the destination VM must be running on Windows 2008 R2 with Service Pack 1 or later
• The destination VM must be powered on
• Ensure the latest integration services are running on the destination VM
• Enable Guest file services on the destination VM. If not enabled, the restore operation enables the services
Setup Requirements for VMware:
• For Windows, the virtual machine must have the NTFS file system
• Requires ESX 5.x, 6.0, 6.5
• The virtual machine must be powered on
• The latest release of VMware Tools must be installed and running
• You can use open-vm-tools on guest VMs running supported Linux releases; open-vm-tools must be installed
and running
• The user account that is used to browse the destination VM must have write permissions for the VM
Whether the traditional file recovery or the Live File Recovery is used, the restore screens are the same. Only the
mechanics under the hood differs to achieve the restore.
• File Recovery – indexes every virtual machine during the backup, which increases backup time but provides
quicker browse and restore.
• Live File Recovery – doesn't require indexing during backup, which improves backup performance, but
slows browse operations since the virtual machine must be mounted from the disk library.
For file servers where browse and recovery operations are frequent, consider using traditional
recovery mode, and for all other virtual machines use the default Live File Recovery.
For Linux virtual machines, the traditional file recovery only supports EXT2 and EXT3 file systems. If the VM is using
EXT4, JFS, XFS or any other advanced file systems, in such cases the block-level browse is the recommended method.
The option to backup failed VMs only can be used to create special schedules or run special backup jobs to protect
VMs that were skipped during regular VSA backups. Let’s take a subclient that has 100 VMs defined, for which the last
backup job failed two VMs. Running another backup job for the subclient would back up the entirety of the 100 VMs.
With the ‘Backup Failed VMs Only’ option enabled, the system looks for failed VMs in the previous subclient backup job
and only protects these machines.
When restoring a full virtual machine, you can restore data to the same destination host and place all disks back to
their original Datastores; this is an in-place restore. Virtual machines can also be restored to a different location than
where they existed at the time of backup; this is an out-of-place restore. For some hypervisors, such as VMware and
Hyper-V, multiple VMs can be selected for restoration.
• Different vCenter – An entire virtual machine can be restored to a different vCenter. This feature provides
the flexibility to distribute restored virtual machines to a location with greater space and resource availability.
This is useful when the virtual machine's original location does not provide optimal space and resources.
• Different ESX server – By default, a virtual machine is restored to the ESX server where it was at the time
of backup, but you can choose to restore to a different ESX server. Specifying a different location provides the
flexibility to distribute restored virtual machines to a location with greater space and resource availability.
• Different Datastore – You can specify a different Datastore on the destination host. After the restore, the
virtual machine is automatically associated to the new Datastore.
• Different Resource Pool or vApp – By default, a virtual machine is restored to its original resource pool,
but you can select a specific resource pool or vApp on the host. vApp is a customized form of a resource pool.
• Different VM name – By default, a virtual machine is restored with the original name used at the time of
backup, but you can change the name when the VM is restored. It can be seen as a VM clone operation.
• Restore to a VM folder – Select a destination folder on the datacenter.
• Restore with different network adapter – Select a network adapter that is available for the destination.
It is possible to restore the virtual machine files, such as vmdk, vmx, and any other VM configuration files, as flat files.
A VM Files recovery operation does register the VM to any vCenter server. It is useful in cases where files can be
restored in a location to be imported in another virtualization software such VMware Player.
During the restoration, disks can also be converted to another supported hypervisor format, such as VMware vmdk to
Hyper-V vhdx.
A virtual machine disk is restored and attached to an existing virtual machine. The disk is restored, and the VSA
communicates with the vCenter server to reconfigure an existing VM to which the disk is then attached. It is useful
when a larger virtual machine with many disks crashes. If one of the disks is critical, it is quickly restored and attached
to another VM.
The Virtual Server Agent for some hypervisors, such as Microsoft Hyper-V and VMware, supports agentless restores of
files and folders into a virtual machine, without requiring the installation of the File System Agent on the destination
VM. Using this feature simplifies deployment and reduces the impact of backup and restore operations for virtual
machines.
• The Hyper-V host must be running on a Windows 2012 R2 server operating system
• For Windows, the destination VM must be running on Windows 2008 R2 with Service Pack 1 or later
• The destination VM must be powered on
• Ensure the latest integration services are running on the destination VM
• Enable Guest file services on the destination VM. If not enabled, the restore operation enables the services
Setup Requirements for VMware:
• For Windows, the virtual machine must have the NTFS file system
• Requires ESX 5.x, 6.0, 6.5
• The virtual machine must be powered on
• The latest release of VMware Tools must be installed and running
• You can use open-vm-tools on guest VMs running supported Linux releases; open-vm-tools must be installed
and running
• The user account that is used to browse the destination VM must have write permissions for the VM
Whether the traditional file recovery or the Live File Recovery is used, the restore screens are the same. Only the
mechanics under the hood differs to achieve the restore.
VM Conversion
Right-click the desired subclient or backup set | Click All Tasks | Browse and Restore | Virtual Server tab
For some hypervisor, it is possible to convert the VM to another hypervisor format. During the restore, simply select
the target supported hypervisor. This useful feature can be used as a migration tool, or to provide recovery capability
between two datacenters using different hypervisors.
Data Security
Ransomware Prevention
Ransomware
Millions of computers have been infected with ransomware type malware, such as WannaCry, CryptoLocker, CBT-
Locker, Tesla Crypt, and more. Ransomware malware takes a user system hostage by encrypting popular file types,
such as Office Files, image and video files that are present on the computer. The criminal organization behind the
malware then asks the user for money to receive the key that allows decrypting the data, thus the name,
ransomware.
This type of virus spreads itself very quickly when it can infiltrate a company network; making it hard to limit the
damages when systems are not properly monitored. Therefore, a system administrator must find ways to be alerted as
soon as possible to react to an infection. To help detect such attacks, Commvault ® software provides various tools to
combat ransomware.
Prevention
Prevention is a critical part of a sound protection strategy. It must combine human intervention and software systems.
Detection
Commvault® software includes built-in coded mechanisms that can be enabled to monitor clients and detect potential
ransomware attacks. Once an attack is detected, an event is triggered in the Event Viewer, and an alert notification
can optionally be configured to notify administrators to react as quickly as possible.
A workflow could be created to be used by the alert to take the infected system offline, to stop the spread.
This monitoring method is enabled via an additional setting that is pushed to client systems. The setting can be
applied to a client, or a client computer group in the CommCell Console. The check frequency value is set in minutes.
Enabling the ‘honey pot’ monitoring for a client or client computer group
Optionally, a report named File Activity Anomaly Report can be viewed from the Reports section of the Admin Console.
• Role-based Security – used for administrators who need permissions on multiple entities. To use role-based
security, you must create a security association between users or user groups, a role, and entities.
• Owner Security – used for end-users who need permissions on very few client entities. For example, a user
needs permission to restore files to a laptop.
Security has evolved significantly in Commvault® software. The following sections describe the
differences in security management between Version 10 and Version 11.
V10 Security
In CommCell® V10, security works by assigning local users and domain user groups to CommCell user groups. These
CommCell groups are then associated with entities in the CommCell® browser to grant access to specific areas within
the CommCell® console.
Each CommCell® user has their own login with a full or restricted set of capabilities to view entities and/or perform
tasks.
• CommCell Users – defined internally within the CommCell® software or enabled externally through
Microsoft’s Active Directory or IBM’s Domino Directory Server. The ability of a user to view entities and
perform tasks within a CommCell group collectively is managed exclusively via membership to a CommCell
User Group.
• CommCell User Group(s) – associated with CommCell entities (e.g., clients, libraries, storage policies)
allowing the member users to perform authorized tasks on those entities only.
A CommCell user is a member of any number of CommCell User Groups. The user’s ability to
perform tasks on a CommCell entity is determined by the combined capabilities of the CommCell User Groups
that are associated with that entity.
A list of tasks and required capabilities are found in the Commvault Online Documentation.
• External Users/Groups – Microsoft’s Active Directory or IBM’s Domino Directory Service can be associated
with CommCell User Groups. Associated external group members login to the CommCell ® console using their
external credentials. Single Sign-on can be enabled to allow external users, who are already logged into the
domain, access to the CommCell console without re-entering their password.
To use role-based security, you must create a ‘security association’ between users or user groups, a role, and entities:
• User(s) – defined by using a CommCell user, an external domain user, a CommCell user group, or an
external domain user group.
• Role – A collection of permissions that defines the level of access granted to a user or a user group.
• Entity – A CommCell® resource, such as a client computer, client computer group, library, storage policy,
schedule policy, MediaAgent, etc.
For instance, the separation of user/user group (who), role (permissions), and entity (what) allows a user or
user group to have different permissions depending on what their role is for a specific entity.
Example: A user requires backup and recovery permissions for a file server. The same user requires restore only
permissions for a mail server. The user is associated with the file server entity and assigned the backup and recovery
role. The same user is assigned to the mail server entity with the recovery role.
Manage Users
Two sets of users can be used to define security associations in Commvault ® software; CommCell® users and external
domain users. You can create CommCell and external domain users and control the features they have access to by
making security associations between the user(s), a role, and entities. CommCell and domain users can also be
combined in the same security association. Once CommCell users are created and external users are added, they both
are available in the CommCell® console.
To use domain accounts, a connection to the directory services must be configured prior to
configuring the security association.
During the creation of a user, the password can be generated by the system. The user receives an email prompting
him to connect for the first time with the generated password. At this point, the system prompts the user to change it.
Commvault recommends that you create at least another account with all permissions. This can be used by
the main backup administrator to ensure that in case of a disaster (e.g., directory services are unavailable),
the administrator can still use his or her account to execute restores.
Commvault recommends that you create at least another account with all permissions. This can be used by
the main backup administrator to ensure that in case of a disaster (e.g., directory services are unavailable),
the administrator can still use his or her account to execute restores.
To use external domain users, a connection to the directory services must first be created in the CommCell console.
More than one domain connection can be added if users from multiple domains are required to login to Commvault
software.
For JumpCloud directory and Apple Open Directory, an additional setting must first be configured
to allow the creation of connections.
Roles
A role is a consolidated set of permissions that is used when creating a security association. A role can be part of as
many security associations as needed to make managing permissions much easier, but each security association can
only have one role. A wide variety of associations are available to define user tasks, such as: ‘in-place recovery,’ ‘out-
of-place recovery,’ ‘tape media operation,’ ‘scheduling,’ VM operations,’ and ‘Content Search.’
Example: If User01 requires backup and restore permissions on server A and User02 requires backup and restore
permissions on server B, you can create a single role called ‘Backup and Restore’ with the appropriate permissions set.
This role can then be used on two different security associations, one for User01 and one for User02.
Create a Role
During the role creation process, you can create a security association with it. All of the users and user groups that are
a part of the security association inherit the permissions in the role.
Security Associations
To use role-based security you must create a security association between users or user groups, a role, and
CommCell® entities. The entity defines the object or group of objects on which the defined user or users can execute
tasks defined by the role. For instance, if a user needs to achieve tasks on a server, create the security association on
the desired client computer entity. If the user needs to execute tasks on several servers, a client computer group can
be leveraged on which the security association can be defined.
Encryption
Encryption Overview
Both software and hardware encryption keys are scrambled using a proprietary algorithm and maintained in the
CommServe® database. Encryption keys can optionally be written to storage media. In the event of the loss of the
CommServe database, encrypted data may be recovered using tools provided by Commvault ® Support. Encryption
keys are destroyed when the job is aged and deleted from the CommServe database. This provides complete end-to-
end encryption key management.
Data is encrypted using Commvault® encryption keys, which are stored in the CommServe® database. These keys are
encrypted using a third-party master key. The third-party system and the keys are required for any restore operations.
Inline Encryption
There are several advantages for software encryption:
• Data can be encrypted on the client during initial data protection providing complete end-to-end security.
• Different encryption ciphers are used based on security requirements.
• In certain cases, software encryption can provide a performance benefit by distributing the load of data
encryption to multiple systems as opposed to hardware encryption, where all data encryption is handled on
the tape drive.
• Data can selectively be encrypted using inline encryption by configuring encryption settings at the subclient
level. This can further improve performance by only encrypting data that requires encryption.
• Restore operations always decrypt data at the destination location.
3-DES 192
Triple Data Encryption algorithm symmetric-key block cipher. Applies cipher algorithm three times to
each block.
Advanced Encryption Standard (AES) is a symmetric block cipher which encrypts data in 128-bit blocks
and uses a key length from 128 to 256 bits.
Symmetric cipher, which divides data into 64 bits and encrypts the blocks individually. This algorithm is
available in the public domain and is fast and it is claimed to never have been compromised.
Serpent is a symmetric cipher, which encrypts data in 128-bit blocks and uses a key size between 128 to
256 bits. This algorithm is in the public domain.
The successor to Blowfish, this symmetric encryption method uses keys up to 256 bits. This algorithm is
fast and, like Blowfish, is available in the public domain.
GOST 256
Developed by Soviet and Russian government. A symmetric cipher in 64-bit blocks
using a key length of 256 bits.
AES (Rijndael) encryption is the industry standard used by hardware devices and most encryption
software. The other ciphers were AES candidates and meet all requirements. Some are faster and
some are stronger. Rijndael was selected as the most flexible.
Inline Encryption
Right-click the storage policy primary copy | Click Properties | Advanced tab
Inline encryption is used to encrypt data during primary protection operations. The encryption can take place on the
client or the MediaAgent. Encryption is enabled for Commvault ® software through the storage policy primary copy or
at the client level. Encryption can further be configured at the subclient level. Subclient level encryption provides the
flexibility of defining only that data which requires encryption. By default, when encryption is enabled on a client,
encryption is enabled on all subclients.
A storage policy primary copy is used to enable encryption on all subclients associated with the storage policy. Ensure
the clients encryption settings are configured to ‘Use Storage Policy Settings.’
1. Client Advanced properties enables encryption and provides choice of cipher, key length, and option to write a
copy of the keys on media.
2. Subclient properties provides options to encrypt on client, on MediaAgent, encrypt on client and decrypt on
MediaAgent (encrypt for transmission only), or disable encryption.
When encryption is enabled on a client, the cipher and key length must be set. The default cipher used is blowfish 128
bit. The ‘Direct Media Access’ setting determines whether encryption keys are stored on the media. The ‘Via Media
Password’ option puts the keys on the media. The ‘No Access’ option only stores the keys in the CommServe ®
database. If the keys are stored on the media, data can be recovered using Commvault ® software’s ‘catalog’ feature,
or in the case of Disaster Recovery data, the Media Explorer tool. Encryption keys are always stored in the CommServe
database.
DR Data recovery using Media Explorer requires the user to provide the Media Password used
when the data was written. The default Media Password is blank. If the Media Password is not
known, contact Commvault Support to assist in recovering the password.
When encryption is enabled for a client, the default subclient encryption setting ‘Client and MediaAgent’ encrypts all
data on the client and the data remains encrypted when written to storage.
The ‘Offline’ or ‘Copy-based’ encryption uses Commvault® software encryption to secure data during auxiliary copy
jobs. From the Data Encryption section in the storage policy copy’s Advanced tab, the ‘encryption cipher,’ ‘key lengths,’
and the option to ‘store keys on the media’ are configured.
In some cases, encrypted source data will be decrypted first then re-encrypted when storing deduplicated data or
changing encryption ciphers. By default, encrypted data is preserved during an auxiliary copy operation. The ‘Store
Plain Text’ option is selected to decrypt data during the auxiliary copy job. If ‘Store Plain Text’ option is selected, you
can still encrypt data during data transmission by selecting the option ‘Encrypt on network using selected cipher.’
Hardware Encryption
Right-click the desired storage policy tape copy | Click Properties | Select path | Click Properties | General tab
For tape drives that support hardware encryption, Commvault® software can enable or disable an encryption operation
on the drive and manage encryption keys. Keys are stored in the CommServe® database. The ‘Direct Media Access’
option ‘Via Media Password’ puts a copy of the keys on the media. The ‘No Access’ option only stores the keys in the
CommServe database.
Commvault software writes data in chunks. Tape media uses 8GB chunks for indexed-based backups and 16GB chunks
for database backups. When encryption is enabled for data protection jobs writing to tape media with ‘hardware
encryption’ enabled, each chunk has a separate encryption key seeded by a random number generator and other
factors. Generating keys at the chunk level provides an enhanced level of security and greatly reduces the potential of
data compromise.
If data has been encrypted using Commvault ® software encryption and hardware encryption is also
enabled, the data is encrypted twice.
Network Topologies
GxCVD service dynamically uses free ports for 1024 to 65535 TCP
communication during data protection and data recovery
jobs.
Since multiple ports are used, especially dynamic ports, it makes it hard to protect a computer behind a firewall if no
other mechanism is in place. If this is the case, you would have to ask your network team to open ports 1024 to
65535 between all clients in the DMZ and the internal servers. Because this scenario is not an effective solution,
Commvault® software has a set of network routes in place.
The CommCell component that establishes connection attempts to communicate with other CommCell resources when
Commvault services start. Connection attempts and ‘keep alive intervals’ are set in the Options tab of the firewall
settings.
• Client Group 1
• Client Group 2
For Proxy Network Topology
Before configuring a topology, create the required client computer groups if needed. You can then launch the network
topology configuration wizard.
Monitoring
Admin Console
The dashboard view provides an overall view of the status of the CommCell® and client protection jobs. By default, the
view displays information about all clients of all solutions. If needed, other specific solution dashboards can be
accessed.
Use your mouse to hover over and identify active areas of the dashboard that can be expanded for additional
information.
Dashboard window
• Disk Space Utilization – Provides information on space consumed and space left.
• Disk Space Savings – Provides the total amount of application data that is protected for all backups,
compared to the amount of data that is physically written in the library. By this information, the performance
of deduplication is illustrated. It also provides the amount of data that is non-deduplicated.
• Average Daily Disk Consumption – Gives the daily average of space consumption and compares it to the
daily average of space released by aging obsolete data. If more space is consumed daily than released, the
library will fill up at some point. Based on these metrics, this section estimates and displays the expected date
on which the library will be filled.
Engine Views
The Deduplication Engine view, under Storage Resources in the CommCell browser® provides information about the
Deduplication Engine performance. Information such as the amount of data protected vs. the amount of data written,
the number of unique blocks in the deduplication store, the amount of records pending deletion and the average query
and insert (Q&I) time are important to properly monitor deduplication health of a CommCell® environment.
• A Deduplication Database (DDB) with an average Q&I time over 2000 microseconds should be investigated as
it may be indicative of performance issues which could impact backup jobs, auxiliary copy jobs and data
aging.
• A DDB or DDB partition getting close to 750,000,000 unique blocks may represent a partition reaching its
capacity. Overtime, the DDB partition performances will degrade, impacting all deduplicated operations. If a
DDB is near this limit, a call should be placed with support to ensure performances are adequate.
• A DDB with an excessive number pending delete records that increases each day may be indicative of either
an underperforming DDB, or an operation window blackout period not providing enough time for the DDB to
purge obsolete records.
• Tape Drive status – online, offline, and jobs currently running and tape barcode label
Job Controller
The Job Controller in the CommCell® console is used to manage all active jobs within the CommCell® environment.
Regardless of which method is used to initiate a job (schedule, on demand or script), the job will appear in the Job
Controller. The Job Controller is the most effective tool within the CommCell ® console for managing and
troubleshooting active jobs.
Job Details
Details for specific jobs are used to provide information on job status, data path, media usage or job errors.
Event Viewer
The Event Viewer window displays events reported based on conditions within the CommCell ® environment. By
default, the event viewer displays the most recent 200 events. This number can be increased up to 1,000. The event
log maintains up to 10,000 events or 7 days of events. These default settings can be modified.
The Event Viewer can be filtered based on the available fields. Although some filters, such as ‘Date’ does not have a
practical application, other fields such as ‘Computer,’ ‘Program’ or ‘Event code’ can be used to quickly locate specific
events.
Although only 200 to 1,000 events are displayed in the event viewer, the entire event log can be searched from the
event viewer. The default total number of events retained is 10,000.
When right-clicking anywhere in the event viewer, select the option to search events. Events are searched by time
range, severity and job ID. If common searches are frequently conducted, the search criteria can be saved as a query
and run at any time.
• Enable/disable alert
• Delete an alert
• Define users or user groups to notify when triggered
Create an Alert
Several alert types are available for which a value or condition can be set. When the value is reached or the condition
met, the alert is triggered.
Notifications can be configured for an alert. In addition to showing up in the triggered alerts view, an email will be
sent to the defined recipients.
Creating an alert
• The Alert window displays all alerts configured for the CommCell® environment.
• The Alert Summary displays settings for the selected alert and provides the capability to email or save them.
Alerts inform you about certain conditions within the CommCell® environment. They can be triggered based on
failures, successes, or changes that may occur during different types of CommCell operations.
• Storage alerts are configured, and libraries are selected and monitored
Built-In Alerts
A wide range of alerts are preconfigured in the system on initial installation. Some are enable, others can be enabled if
required. These alerts monitor several components and conditions. A summary view explains what the alert is for. For
more information on the preconfigured alerts, refer to the Commvault Online Documentation.
Managing alerts
The Alert Wizard is used to configure the alert type, entities to be monitored, notification criteria and notification
method.
• Type of alert
• Entities to be monitored
• Notification criteria
• Notification method
Notification Providers
Commvault® software offers many easy-to-configure notification providers. These providers ensure that an
administrator is notified at any time should an issue arise.
• email
• SNMP
• Event Viewer
• Run Command
• Save to
• RSS Feeds
• Console Alerts
• SCOM
• Workflow
Email
Email notifications are sent to the CommCell® console or domain users by selecting them from the list. The user must
have logged in at least once to the CommCell console. Email addresses or distribution lists can be defined. If the email
server is down, the system tries to resend the email for four hours. After that time limit, if the server is still down, the
notification is discarded and will not be sent. Email notification format can be HTML or text and be modified as
needed.
To use email notification, a SMTP server must be configured using the email and Web Server applet from the
Configuration tab.
SNMP
Alerts can be sent by the CommServe® server as SNMP to any desired computer listening for SNMP traps. This
notification method is useful if an existing monitoring and/or ticketing system is in place. SNMP alerts support SNMP
Version 1 (SNMPv1) and SNMP Version 3 (SNMPv3) and require the SNMP Enabler to be installed on the CommServe
server.
Event Viewer
You can send alert notifications from the CommServe® server to the Windows Event Viewer of other computers where
it is generated as an event. For all alerts related to backup and restore operations, the following information is sent to
the Windows Event Viewer:
Run Command
The Run Command notification is used to send alert notifications from the CommServe ® server to other client
computers by executing a command script. The Run Command can be located on the CommServe server or on remote
computers but is executed only on the CommServe server. It also can be used to run a script to resolve the issue,
such as restarting some services or any desired tasks.
Save to
You can send an alert notification to a local directory, a network share, or the Cloud Services website. This is
particularly useful in obtaining a list of failed attempts in an operation. If you plan on using the Cloud Services, Cloud
Metrics Reports must first be activated for the CommCell® console.
Property Modifications:
Status: Modified
Client: winter
Agent Type: Not Applicable
Instance: Not Applicable
Backup Set: Not Applicable
Subclient: Not Applicable
Comments:
RSS Feed
It is possible to turn the CommServe® server into a RSS Feed server, which allows an alert notification to be sent as
an RSS Feed. Your favorite RSS Feed client can be configured to receive notifications by subscribing to the
CommServe server.
SCOM
The Commvault® software can send alert notifications from the CommServe® database to the Microsoft Systems
Center Operations Manager (SCOM). The Microsoft SCOM Server provides a monitoring service for critical applications
within an enterprise and sends alerts about events in these applications. An administrator can raise tickets against
these alerts and take any necessary action to resolve the problem. SCOM must first be installed and the CommServe®
server defined as a SCOM agent. For the agent to communicate with the SCOM server, firewall ports 5723 and 5724
must be open. For more information on SCOM notification configuration and prerequisites, please refer to the
Commvault Online Documentation.
Workflow
It can be useful to try to resolve an issue using automation. When an alert is triggered, a workflow notification
launches any desired workflow. Note that when configuring the alert, the workflow must be created first, then
selected.
Scenario: A backup job goes into a pending status, stating that it cannot communicate with the client.
Solution: A workflow alert could launch a script to restart the Commvault services on the client, then restart
the backup and send an email if the backup is still pending.
Console Alerts
When configuring alerts, console alerts can be selected as a notification method. Once an alert is triggered, it appears
in the Console Alerts window within the CommCell ® browser. Right-click on an alert to view details, delete, mark as
read or unread, or to insert a note. Console alerts can be pinned or deleted using the icons at the bottom of the
window.
Common Alerts
Category Type Options
Job Management Data Protection, Data Recovery Job Failed, Phase or network errors
• Job Summary
• Job Schedule
• CommCell Readiness
When the report type is selected, it is the default report in the report window. Note that any other report type can be
accessed from the window. Reports can be scheduled, saved to a specific location, or saved as report templates.
Depending on the report type selected, various report criteria are configured from the tabs in the Report Selection
window. Use the tabs to choose which resources; clients, MediaAgents, libraries, or storage policies, to include in the
report. You can also select the information to be included in the report, such as failed items, storage usage, job
information, or resource configuration.
• Job Summary report – is used to view data protection, data recovery and administrative jobs.
• CommCell® Readiness report – is used as a status report for CommCell components such as clients,
MediaAgents, library storage capacity and index directories.
• CommCell® Configuration report – provides CommCell configuration, license usage, and update status of
CommCell components.
• Job Schedule report – is used to view schedules for client computer groups, clients, and administrative jobs.
• Data Retention Forecast and Compliance report – is used to view jobs in storage, the media it is located
on, and the estimated time the data will age.
Report Outputs
When running any report, it can be formatted using HTML, text delimited, PDF, XML. A copy of the report can also be
saved on a local drive of any CommCell® client computer, on a network share by providing credentials that have
access to the share, or to an FTP site by providing login information. Language, date and time formats are selected
from drop-down lists.
Maintenance
Maintenance Modes
There are several modes that can be selected when executing a DB maintenance:
• Full - Performs a full maintenance on the database. It includes CheckDB, ReindexAll and ShrinkDB commands.
It is recommended to run on a bi-yearly basis.
• Recommended - Performs a recommended maintenance which includes ShrinkDB and
ReindexRecommended commands. It is recommended to run this maintenance mode every couple of weeks.
By default, a system created schedule will execute it on every other Sunday.
• CheckDB - Validates the consistency of the CommServe database by running an integrity check.
• ReindexRecommended - Re-indexes the largest and most frequently used tables of the database.
• ReindexAll - Re-indexes all tables of the database.
• ShrinkDB - If table re-indexing creates a significant amount of fragmentation, the ShrinkDB command will
reclaim that space by shrinking the database.
This schedule executes a workflow called DBMaintenance, which executes the maintenance based on the mode that is
selected in the schedule. The workflow also contains email components that can be modified to send a result
notification on failure or success.
CommvaultTools
Commvault Software provides several tools that are launched from the command line, or from the CommCell®
console. These tools are available in the Base folder of the Commvault software:
• Process Manager
• CVPing
• CVIPInfo
• Network Test Tool
• TapeToolGUI
Process Manager
Each operation in the CommCell® console may have processes associated with the operation. From the Process
Manager, you can view processes and information about each one, such as the CPU usage and memory usage. The
Process Manager has tabs that provide information about controlling services and log files, and has buttons to stop,
start or recycle Commvault® services.
CVPing
The CVPing tool validates network, as well as port connectivity. This tool is useful when troubleshooting issues when
implementing firewall rules.
Where:
CVIPInfo
CVIPInfo tool validates forward and reverse DNS resolution. It is important to remember that Commvault ® software
frequently uses the reverse DNS zone.
Where:
This mode is available to execute a host name lookup, which is similar to a CVIPinfo or an nslookup command.
This mode validates if Commvault® software services installed on the target machine are up-and-running and
reachable.
This mode validates network connectivity and bandwidth throughput between two servers. The servers could be
clients, MediaAgents, or the CommServe® server. It is helpful to validate the throughput between a client and its
target MediaAgent, or between two MediaAgents in preparation to implement DASH copies.
• Server mode – This is started on the target server and acts as the listener.
• Client mode – This is executed on the source machine and will connect to the target machine’s listener to
run the test.
TapeToolGUI
The TapeToolGUI tool is used to troubleshoot media in the library and run performance testing on the mounted
media. The TapeToolGui is also available as a command line interface, called TapeTool.
The TapeToolGUI tool does not connect with the Commvault ® software and does not know about the contents
of tapes. Therefore, when using the tool to perform a destructive operation on a media, such as a write,
format, or target for a copy media operation, ensure that the media that you type in the Tape Name box, is a
scratch tape and contains no data. Otherwise, the backup data can potentially be destroyed.
Tuning
Performance Benchmarks
Commvault® software is a high-performance solution for protecting all data in any environment within defined
protection windows. The software also provides many settings to improve performance. Before considering tuning
Commvault software, it is important to understand capabilities and limitations of all hardware and software deployed
within an environment.
There is no such thing as a static data center. Network infrastructures are constantly changing, new servers are
added, mission critical business systems are moving to hybrid cloud, or public cloud infrastructures. Before considering
Commvault tunables, it is first important to understand your environment including the capabilities and limitations of
the infrastructure; specifically, the ability to transfer large amounts of data of production or backup networks.
When making modifications to an environment, changes that may positively impact one aspect of the environment can
negatively affect another aspect. This is also true about Commvault settings. For example, enabling multiplexing when
writing to tape drive can improve backup speeds. However, it may have a negative impact on restores if dissimilar
data types are multiplexed to the same tape. Another example is using Commvault deduplication and setting a high
number of data streams. Since client-side deduplication is being used, there will be a low impact to the network. But if
the deduplication database needs to be sealed, the next set of backup operations may result in oversaturating the
network while re-baselining blocks in storage.
Performance Benchmarks
Benchmarks can be divided into two kinds, component and system. Component benchmarks measure the performance
of specific parts of a process, such as the network, tape or hard disk drive, while system benchmarks typically
measure the performance of the entire process end-to-end.
Establishing a benchmark focuses your performance tuning and quantifies the effects of your efforts. Building a
benchmark is made up of the following 5 steps:
You can’t document or improve something if you don’t know what’s going on. More importantly, you need to
understand what phases a job goes through and how much each phase affects the overall outcome.
For example, a backup job over a network to a tape library takes two hours to complete. You think it should take a lot
less and you spend time, effort, and money to improve your network and tape drives and parallel the movement of
data. The job now takes 1.8 hours to complete. You gained a 10% improvement.
Looking at the job in more detail we find that the scan phase of the job is taking 1.5 hours and the rest is the actual
data movement. Switching the scan method reduces the scan phase time to 12 minutes. The job now takes .4
hours. You gained a 78% improvement.
Knowing what phases a job goes through and how much each phase impacts the overall performance can help you
focus your time, effort, and money on the real problems.
Each hardware component is going to have a theoretical performance limit and a practical one. Attempting to get
improvement beyond these limits without changing the resources involved is a waste of time. Consider using newer
vs. older technologies, such as tape drives.
Large data movements are usually done during non-production hours for two reasons – one, they can degrade
production work, and two, production work can degrade the movement of data. You want to minimize competition for
resources to get a fair benchmark of what performance is achievable. In those cases, where competition cannot be
eliminated, you must accept the impact to performance or invest in more resources.
Periodic Test
A single measurement is not a benchmark. Tape devices have burst speeds that are not sustainable over the long run.
Networks have various degrees of bandwidth availability over a period of time. A single snapshot check of bandwidth
will not give you a realistic expectation. Do periodic testing over the actual usage of a resource to determine its
average performance. Try to level out the peaks and valleys - or at least try to identify what causes these variations.
Multiple measurements scattered over a day can also help in establishing if an unexpected external process is
impacting the environment. For example, if you have a database server that is slowly backing up at night, but when
you sample during the day, it is achieving expected performances, you can suspect an external process impacting the
backup, such as a database administrator dumping the database and copying it to another server at the same time in
this example.
Write it down
The hardest lessons are the ones you must learn twice. Once you’ve established your acceptable and/or expected
performance levels for each resource and end-to-end, write them down and use them as the baseline for comparing
future performance.
TCP/IP
TCP/IP is the most common network transmission protocol. Factors that can degrade TCP/IP performance are:
• Latency - Packet retransmissions over distance take longer and negatively impact overall throughput for a
transmission path.
• Concurrency - TCP/IP was intended to provide multiple users with a shared transmission media. For a single
user, it is an extremely inefficient means to move data.
• Line Quality - Transmission packet sizes are negotiated between sender/receiver based on line quality. A
poor line connection can degrade a single link’s performance.
• Duplex setting - Automatic detection of connection speed and duplex setting can result in a half-duplex
connection. Full duplex is needed for best performance.
• Switches - Each switch in the data path is a potential performance degrader if not properly configured.
• Firewalls – Firewall is the first line of defense against hackers, malware, and viruses. There are hardware
firewall appliances and software firewalls, such as operating system firewalls. Firewalls can have minor to
moderate impacts on transfer performances.
SCSI/RAID
SCSI is the most common device protocol used and provides the highest direct connection speed. An individual SCSI
drive’s speed is determined by spindle speed, access time, latency, and buffer. Overall SCSI throughput is also
dependent on how many devices are on the controller and in what type of configuration. The limitation of SCSI is the
distance between devices and the number of devices per controller.
• RAID arrays extend the single addressable capacity and random access performance of a set of disks. The
fundamental difference between reading and writing under RAID is this: when you write data in a redundant
environment, you must access every place where that data is stored; when you read the data back, you only
need to read the minimum amount of data necessary to retrieve the actual data--the redundant information
does not need to be accessed on a read. Basically – writes are slower than reads.
• RAID 0 (striping) or RAID 1 (mirror) or RAID 1+0 with narrow striping are the fastest configurations when it
comes to sequential write performance. Wider striping is better for concurrent use. A RAID 5 configured array
can have poor write performance. The tradeoff in slower write performance is redundancy should a disk fail.
Fine tuning a RAID controller for sequential read/write may be counterproductive to concurrent
read/write. If backup/archive performance is an issue, a compromise must be arranged.
iSCSI/Fibre Channel
iSCSI or Fibre Channel protocol (FCP) is essentially serial SCSI with increased distance and device support. SCSI
commands and data are assembled into packets and transmitted to devices where the SCSI command is assembled
and executed. Both protocols are more efficient than TCP/IP. FCP has slightly better statistics than iSCSI for moving
data. Performance tuning is usually setting the correct ‘Host Bus Adapter’ configuration (as recommended by the
vendor for sequential I/O) or hardware mismatch. Best performance is achieved when the hardware involved is from
the same vendor. Given that configuration and hardware is optimum, then for both iSCSI and FCP, performance is
inhibited only by available server CPU resources.
Disk I/O
Performing I/O to disks is a slow process because disks are physical devices that require time to move the heads to
the correct position on the disk before reading or writing. This re-positioning of the head is exacerbated by having
many files or having fragmented files. You can significantly improve read performance of the source data by de-
fragmenting the data on a regular basis.
Anti-Virus
Anti-viruses are intelligent software protecting a system against corrupted data by periodically scanning files systems
and ensuring that every file accessed or opened by any processes running on the system is a legitimate file (and not a
virus). You can easily imagine that when a backup runs and protects every system files, the anti-virus validation
significantly decrease backup performances. It might also access and lock Commvault files, such as log files. It is
recommended on all systems on which Commvault software is installed, to add exclusions to the anti-virus software
for Commvault® software folders, so that when Commvault related processes are in action, they do not trigger the
anti-virus validation process.
For V10 and earlier environments, any Content Store directory must be replaced by ‘Simpana.’ For
V11 environments, any Content Store directory must be replaced by ‘ContentStore.’
• For backups on Windows operating systems, ensure source disks are defragmented.
• Ensure all global and local filters are properly configured.
• If source data is on multiple physical drives increase the number of data readers to multi-stream protection jobs.
• For larger high speed disk, a maximum of two data readers can set for an individual disk. Enable ‘Allow Multiple
Data Readers within a Drive or Mount Point’ to allow multiple streams on a single disk.
• If source data is on a RAID volume, create subclient(s) for the volume and increase the number of data readers
to improve performance. Enable the ‘Allow Multiple Data Readers within a Drive or Mount Point’ option.
• Consider using synthetic full, or better, DASH Full backups over traditional full backups.
• Consider using the Commvault OnePass® agent to archive older ‘stale’ data.
• For large volumes containing millions of objects use the File System Block-Level Backup.
• Consider using multiple subclients and stagger backup operations over a weekly or even monthly time period.
• For supported hardware, consider using the Commvault IntelliSnap® feature to snap and backup volumes using
a proxy server.
• Increase the ‘Application Read Size’ from the default of 64KB to 512KB.
Data Readers
Disk I/O is the most costly, time-consuming portion of a data movement job. Using multiple data readers (also called
data streams) can improve performance.
Conditions that can degrade performance for the File System Agent:
• In some configurations, such as concurrent backups that use embedded agents on multiple virtual machines
(VMs) in a hypervisor environment, using multiple data readers for each backup might overwhelm the disk I/O
and degrade performance. In this situation, using only one data reader for each VM might achieve the best
performance.
• Internal algorithms determine the maximum number of data readers that can read concurrently from a single
physical drive. Too many data readers on a single physical drive can actually degrade performance.
• Subclient content is divided between data readers based on physical drives. Thus, the first data reader reads
from the first physical drive, the second data reader reads from the second physical drive, and so on. By
default, only one data reader is allowed per physical drive, regardless of how many data readers are
configured. Often, a data reader completes before the other data reader completes, which reduces the
performance gain of using multiple data readers.
The ‘Allow multiple data readers within a drive or mount point’ option helps you to use data readers more efficiently.
For example, if you have subclient content that spans 4 physical drives, and you configure 8 data readers. Each
physical drive gets 2 data readers. When one data reader completes its task, it assists another physical drive. This
process continues until all data is read. This process maximizes the time that multiple data streams are moving data,
which can improve performance.
Setting the number of readers and multiple readers within a drive or mount point
Values for the application read size must be in the power of 2; the minimum value is 64 KB, and the maximum value is
4,096KB (4MB).
Commvault recommends that you set the application read size at either the default value or at the
cluster size that is directed by the application.
Microsoft NTFS uses a default cluster size (allocation unit) of 4KB by default. The 4KB cluster size was established
when 2GB disks were considered large. Today, Microsoft recommends using a cluster size of 16KB or higher for NTFS
volumes on servers. Commvault recommends that you use 64KB clusters, which matches the Microsoft ReFS default
cluster size. With source data on volumes that have a 64KB cluster size, Commvault recommends using an application
read size of at least 2,048KB for NTFS and ReFS.
For information about cluster sizes, see the Microsoft support article “Default cluster size for NTFS,
FAT, and exFAT”.
• To optimize virtual environment data protection and recovery performance, contact Commvault Professional
Services for the latest guidance and assistance.
• Use the Commvault Virtual Server Agent (VSA) to protect most VMs. Specific I/O intensive VMs may require
more advanced protection methods.
• Use backup set or subclient VM filters to filter VMs that don’t require protection.
• Use subclient VM rules to group priority VMs for protection. For example, use the power state rule to set
infrequent schedules of VMs that are not powered on.
• Maximize VM backup concurrency by increasing the ‘Data Readers’ option. Use caution as setting the readers
option too high can cause performance degradation on backups and DataStores or volumes hosting the VMs.
As a general starting point, start with two VM backups per DataStore or volume.
• It is preferred to use physical VSA MediaAgent proxies versus virtual server MA proxies.
• Ensure there are enough proxies to handle data movement load.
• Use Commvault Client Side Deduplication and DASH Full backups.
• For larger VMs, consider using the Commvault OnePass® feature to archive older ‘stale’ data.
• Consider using multiple subclients and staggering schedules for when incremental and full or synthetic (DASH)
full backups run.
VMware specific guidelines
• Ensure VSA proxies can access storage using the preferred transport mode. SAN transport and HotAdd will fall
back to NBD mode if they cannot access VMs from the SAN or DataStore.
• Ensure Change Block Tracking (CBT) is enabled for all virtual machines.
When protecting applications in a virtual environment:
• Using the VSA to protect applications without the Application Aware feature or agents installed within the VM
may result in crash consistent backups.
• For low to medium I/O applications, use the Application Aware feature. Check the Commvault Online
Documentation for a list of applications supported by the VSA Application Aware feature.
• For I/O intensive applications, it is still preferred to use application agents installed in the VMs.
Commvault IntelliSnap® for VSA:
Database Agents
General Guidelines
• For large databases that are being dumped by application administrators, consider using Commvault database
agents to provide multi-streamed backup and restores.
• When using Commvault database agents for instances with multiple databases, consider creating multiple
subclients to manage databases.
• For large databases, consider increasing the number of data streams for backing up database. For multi-
streamed subclient backups of SQL and Sybase databases, the streams should not be multiplexed. During
auxiliary copy operations to tape if the streams are combined to a tape, they must be pre-staged to a secondary
disk target before they can be restored.
• For MS-SQL databases using file/folder groups, separate subclients can be configured to manage databases and
file/folder groups.
Before you modify the number of data readers, Commvault recommends recording baseline
throughput performance using the default settings, which are the recommended settings. You can
then modify the number of data readers until you achieve the fastest throughput performance.
For most Data Availability Group (DAG) environments, backup operations are performed on the passive node, and
memory usage for the application read size is not a concern. If production performance problems occur, then you can
decrease the application read size.
In a DAG environment, the stream allocation is based on the number of nodes. When the job starts, the stream logic
automatically assigns one stream to each node. If there are additional streams remaining, they are allocated based on
which node has the most databases. The stream allocation process continues in order of Exchange servers in the DAG
environment containing the most databases to fewest in a prioritized round-robin method until all streams are
allocated.
To set the number of pipeline buffers, use the ‘nNumPipelineBuffers’ additional setting.
Although the maximum value for ‘nNumPipelineBuffers’ is 1,024, if you use a value that is greater than 300, you
should consult with Commvault Support. When you increase the number of Data Pipeline buffers, the client or
MediaAgent consumes more shared memory. When available memory is low, this consumption of shared memory
might degrade the server performance for other operations.
• Internet - 30 buffers
• 100BASE - 30 buffers
• 1000BASE - 120 buffers
Network Agents
Network agents are threads or processes that transfer data to and from the network transport layer. Each network
agent spends half its time reading and half its time writing. For higher speed networks, having multiple networks
agents can improve performance.
Network agents consume both CPU and memory resources. Commvault usually recommend increasing the number of
network agents only for backup operations of large databases to LAN-free MediaAgents.
Default values and valid values for the number of network agents:
The chunk size that you specify in the additional setting overrides the values that you specify in the chunk size that
you specify for the CommCell® in the Media Management configuration.
Block Size
MediaAgents can write to media that is formatted with different block allocation sizes or file allocation sizes if the
MediaAgent operating system supports those sizes. Using a larger block size for disk library volumes can reduce
overhead and thus increase the speed of write operations to media.
Linux ext3 and Microsoft NTFS use a default block (allocation unit) of 4KB. The 4KB block size was established when
2GB disks were considered large. Today, Microsoft recommends using at least a 16KB block size or higher for NTFS
volumes. Commvault recommends that you use 64KB, which matches the Microsoft default value for the ReFS block
size.
You can increase the Linux ext3 block size only on an Itanium system. For other file systems, consult your OS vendor
documentation for your file system’s available block sizes.
To increase the speed of jobs that access the mount path, you can configure the MediaAgent to bypass the Microsoft
Windows file system buffering.
You can make this configuration for Windows MediaAgents and for disks that are mounted directly (not for UNC
paths).
• Method one – Use the GFS tool provided by most UNIX/Linux based OS. This tool sets a direct I/O flag to a
directory and all its current subdirectories and files. Once enabled, any new directory or files created will also
inherit the direct I/O attribute. It can be turned on (using the setflag parameter) or off (clearflag) as desired.
• Method two – Use the Unbuffered I/O configuration for Linux MediaAgent:
• Gfs_tool setflag inherit_directio MyDirectory
Mount the NFS filesystem using the force direct I/O flag (forcedirectio). For as long as the filesystem is
mounted, it will bypass the operating system buffer.
For more information on the GFS tool or the mount direct I/O option, refer to your operating system vendor’s
documentation.
Chunk Size for tape libraries can be modified on the data path for a specific tape library, or globally, using the Media
Management applet. Global chunk size settings are configured per agent type.
Block Size
Before changing tape block size, ensure that the following criteria are satisfied:
• Block size is supported by the MediaAgent OS, Host Bus Adapter (HBA), and the tape device.
• All the MediaAgents that are associated with a storage policy support the block size that is configured on that
storage policy. Consider the support and the compatibility of MediaAgent platforms at any disaster recovery
site.
• If you use different MediaAgents for backup operations and restore operations, and if the backup MediaAgent
has a higher block size, then ensure that the restore MediaAgent can read data that is written with a higher
block size.
Many streaming tape drives perform a read-after-write check. If the drive detects a bad block, then the drive puts a
discard token after the block, and repeats the entire buffer write. If the drive detects a discard token, then the read
cycle has corresponding logic to replace the bad block with the replacement block.
All tapes will have media defects. If you write 1,024KB blocks instead of 256KB blocks, then the chance of any block
spanning a media defect are increased by a factor of 4. Because of the larger block size, the rewrite time is 4 times as
long as well.
Increasing block size can improve the performance of writing to tape by minimizing the overhead associated with
accessing and recording each block. If you select the data path’s Use Media Type Setting option, then the data path’s
default block size for tape is 64KB. Refer to the Commvault Online Documentation: Use Media Type Setting section for
more information.
• Use caution when you select large block sizes. Large block sizes can vastly increase error rates and retries.
• Block size applies only to tape media in direct-attached libraries.
• Changes to the block size settings take effect when the next spare tape media is used.
• Ensure hardware at data center and other location, including DR sites support higher block sizes.
Stream settings are configured in various places within the CommCell® console including the storage policy,
MediaAgent, subclient, and library. The system always uses the lowest setting. If a MediaAgent is configured to
receive as many as 100 streams and one storage policy is writing through the MediaAgent and is configured to use 50
streams, then only 50 streams will be sent through the MediaAgent.
During a data protection job, streams originate at the source file or application that is being protected. One or more
read operations is used to read the source data. The number of read operations is determined by the number of
subclients and within each subclient, the number of data readers or data streams, depending on which agent is
managing the data. Once the data is read from the source it is processed by the agent and then sent to the
MediaAgent as job streams. The MediaAgent then processes the data, arranges the data into chunks and writes the
data to storage as device streams. The data is written to storage based on the number of writers, for a disk library, or
devices (tape drives) for a tape library.
Subclients • Subclients are independent jobs, meaning each subclient will have one or more streams
associated with each job.
Multi-stream • Most subclients can be multi-streamed. For subclients that do not support multiple streams,
subclients multiple subclients are used to multi-stream data protection jobs.
• Data readers are configured in the General tab of the subclient.
• Data Streams are configured in the storage device tab for MS-SQL and Oracle subclients.
Non-Subclient based • Agents such as the new Exchange Mailbox agent manage streams at the object level. For
agents Exchange, each mailbox is protected as a single stream.
• The default subclient data readers setting is still used as the primary stream governor for the
maximum number of concurrent objects that can be protected.
Job Streams • Job streams are active network streams moving from source (client or MediaAgent) to
destination (MediaAgent).
• The Job controller shows the total number of job streams currently in use in the bottom of the
window and the job stream ‘high watermark’ for the CommCell environment.
• Add the ‘Number of Readers in Use’ field in the job controller to view the number of streams
being used for each active job.
Drives • For a removable media library writing data sequentially to devices, there will be one device
stream per drive.
Writers • For a disk library where random read/write operations can be performed the number of writers
should be set to allow the maximum throughput without creating bottlenecks in your network,
MediaAgents, or disks.
Thank You
At Commvault, we have a strong focus on providing quality education. We use a 3-tier student survey process to
assess your learning experience, how the training affected your ability to be more productive using Commvault
products, and finally how the training impacted your ability to enhance and improve the impact Commvault products
have in your data management environment.
1. The initial ‘Learning Experience’ or course survey can be done as soon as your course is complete via
Education Advantage. We’ll show how to launch the survey on the next slide and take 10-15 minutes for
all to complete it.
2. The ‘Learning Effectiveness’ follow-up survey is sent to all students about 6 weeks after your course via
email. We are looking for your input on how you were able to apply the skills learned in your environment
and whether there is content we need to add to our courses to better address your skills needs (something
that may not be evident at course completion).
3. 3-6 months after completing your course you will receive the Education Value Survey via email from
Techvalidate. We use a third party to collect, audit, and validate these survey responses. This survey is used
to assess the impact training has had on your business and data management environment. Were you better
able to leverage Commvault products, with better performance, better resource usage? Were you better
skilled, reducing reliance on customer support for product usage queries over time? Finally, we ask based on
your Commvault learning experience, how likely you would be to recommend Commvault training to a friend
or colleague. This one question produces an overall learner satisfaction (or Net Promotor) score. This metric
is used to measure (at a high level) how we are doing overall.
We strive to meet your highest expectations and highest survey marks. If we fail to meet your
expectations with the learning experience, please provide specific comments on how we can
improve. We take all comments seriously and will adjust our offerings to better support your
needs.