V11 NewFeaturesWalkthroughGuide PDF
V11 NewFeaturesWalkthroughGuide PDF
P a g e 1 | 41
Legal Notices
Information in this document, including URL and other website references, represents the current view of Commvault Systems,
Inc. as of the date of publication and is subject to change without notice to you.
Descriptions or references to third party products, services or websites are provided only as a convenience to you and should not
be considered an endorsement by Commvault. Commvault makes no representations or warranties, express or implied, as to any
third party products, services or websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners. Unless
otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and
events depicted herein are fictitious.
Complying with all applicable copyright laws is the responsibility of the user. This document is intended for distribution to and
use only by Commvault customers. Use or distribution of this document by any other persons is prohibited without the express
written permission of Commvault. Without limiting the rights under copyright, no part of this document may be reproduced,
stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical,
photocopying, recording, or otherwise), or for any purpose, without the express written permission of Commvault Systems, Inc.
Commvault may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject
matter in this document. Except as expressly provided in any written license agreement from Commvault, this document does
not give you any license to Commvault’s intellectual property.
COMMVAULT MAKES NO WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED, AS TO THE INFORMATION CONTAINED IN THIS
DOCUMENT.
©1999-2015 Commvault Systems, Inc. All rights reserved
Commvault, Commvault and logo, the “C” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management,
Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker,
InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, IntelliSnap, ROMS, Simpana OnePass, CommVault
Edge and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands,
products, service names, trademarks, or registered service marks are the property of and used to identify the products or
services of their respective owners. All specifications are subject to change without notice.
All right, title and intellectual property rights in and to the Manual is owned by Commvault. No rights are granted to you other
than a license to use the Manual for your personal use and information. You may not make a copy or derivative work of this
Manual. You may not sell, resell, sublicense, rent, loan or lease the Manual to another party, transfer or assign your rights to use
the Manual or otherwise exploit or use the Manual for any purpose other than for your personal use and reference. The Manual
is provided "AS IS" without a warranty of any kind and the information provided herein is subject to change without notice.
P a g e 2 | 41
Education Services V11 Walkthrough Guide October 2015
Table of Contents
Contents
Legal Notices ......................................................................................................................................................... 2
Table of Contents .................................................................................................................................................. 3
Commvault Version 11 SP1 Highlights: .................................................................................................................... 6
CommCell Security Enhancements .......................................................................................................................... 7
Role Based security ............................................................................................................................................ 7
Roles when Upgrading from Previous Commvault Versions ................................................................................ 9
User Quotas ..................................................................................................................................................... 10
Indexing ............................................................................................................................................................. 11
Traditional Indexing (V1) .................................................................................................................................. 11
Commvault V2 Indexing.................................................................................................................................... 11
How v2 Indexing Works ................................................................................................................................ 12
Index Resiliency and Protection ..................................................................................................................... 12
Multiple MediaAgents and Shared Libraries ..................................................................................................... 15
Data Aging ................................................................................................................................................... 15
Data Recovery .............................................................................................................................................. 15
Storage ............................................................................................................................................................... 16
Disk libraries .................................................................................................................................................... 16
Library Properties.......................................................................................................................................... 16
Validate Mount Paths .................................................................................................................................... 17
Tape Library .................................................................................................................................................... 18
Scratch Pool Media Selection Rules ................................................................................................................ 18
Deduplication ...................................................................................................................................................... 19
Content Aware Deduplication ............................................................................................................................ 19
Deduplication Database .................................................................................................................................... 20
In Memory Deduplication Database ................................................................................................................ 20
DDB Backup performance enhancements ....................................................................................................... 21
Deduplication Database Resiliency ................................................................................................................. 22
Deduplication Policy Creation ............................................................................................................................ 23
Data Aging and Pruning .................................................................................................................................... 25
Pruning activity control .................................................................................................................................. 25
Data Verification............................................................................................................................................... 26
Storage Policies ................................................................................................................................................... 27
P a g e 3 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 4 | 41
Education Services V11 Walkthrough Guide October 2015
This document is provided as part of the Education Services V11 Features vILT
and eLearning course and will be updated with each major service pack release.
P a g e 5 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 6 | 41
Education Services V11 Walkthrough Guide October 2015
Example: a user requires backup and recovery permissions for a file server. The same user requires restore only
permissions for a mail server. The user is associated with the file server entity and assigned the backup and recovery
role. The same user is assigned to the mail server entity with the recovery role.
P a g e 7 | 41
Education Services V11 Walkthrough Guide October 2015
Create a Role
P a g e 8 | 41
Education Services V11 Walkthrough Guide October 2015
View System Created Roles after Upgrade from Previous Commvault Version
P a g e 9 | 41
Education Services V11 Walkthrough Guide October 2015
User Quotas
Domain users can have data protection quotas enforced for file based backups. Quotas can be set at the group or user
level. If quotas are set at the group level, they can be overridden at the user level.
P a g e 10 | 41
Education Services V11 Walkthrough Guide October 2015
Indexing
Commvault® software uses a distributed indexing structure that provides for enterprise level scalability and automated
index management. This works by using the CommServe database to only retain job based metadata such as chunk
information, which keeps the database relatively small. Detailed index information such as details of protected objects
is kept on the MediaAgent managing the job.
Job summary data maintained in the CommServe database keeps track of all data chunks being written to media. As
each chunk completes it is logged in the CommServe database. This information also tracks the media used to store
the chunks.
Commvault V2 Indexing
Commvault version 11 introduces the next generation indexing called indexing V2. It provides improved performance
and resiliency, while shrinking the size of index files in cache and in storage.
V2 indexing works by using a persistent index database maintained at the backup set level. During subclient data
protection jobs, log files are generated with all protected objects. The logs will be played into the index database.
Note: Commvault V11 SP1 currently supports V2 indexing method for file system agents. New
agent installation will use V2 indexing and existing file system agents, if upgrading from a previous
software version, V1 indexing will continue to be used.
P a g e 11 | 41
Education Services V11 Walkthrough Guide October 2015
During data protection jobs, log files are generated with records of protected objects. The maximum size of a log will
be 10,000 objects or a complete chunk. Once a Log is filled or a new chunk is started, a new Log file is created and
the closed Log will be written to the index database. By writing index logs to the database while the job is still
running, the indexing operations of the job runs independent of the actual job allowing a job to complete even if log
operations are still committing information to the database.
At the conclusion of each job, the log files are written to storage along with the job. This is an important distinction
from traditional indexing which would copy the entire index to storage. By copying just logs to storage, indexes
require significantly less space in storage which is a big benefit when protecting large file servers. Since the index
database is not copied to storage at the conclusion of each job, a special IndexBackup subclient is used to protect
index databases.
The index databases are protected with a system created subclient on the MediaAgent called IndexBackup. An index
backup operation is scheduled to run every eight hours. During the backup operation, index databases are checked to
determine if they qualify for protection. The two primary criteria to determine if a database qualifies for protection is
10 million changes or 30 days since the last backup.
P a g e 12 | 41
Education Services V11 Walkthrough Guide October 2015
3. The database will be backed up to the storage policy associated with the IndexBackup subclient
Database Checkpoint
Checkpoints are used to indicate a point-in-time in which a database was backed up. Once the database is protected
to storage, any logs that are older than the checkpoint can be deleted from the index cache location.
Database Compaction
During data aging operations, deleted jobs are marked in the database as unrecoverable but objects associated with
the job remain in the database. The compaction operation deletes all aged objects and compacts the database.
Database Backup
Once the checkpoint and compaction occurs, the database will be backed up to the primary copy location of the
storage policy. Three copies of the database will be kept in storage and normal storage policy retention rules are
ignored.
During the index backup process, the database will be frozen and no browse or find operations can be run against the
database. Each database that qualifies for backup is protected sequentially minimizing the freeze time. Data protection
jobs will not be effected by the index backup.
P a g e 13 | 41
Education Services V11 Walkthrough Guide October 2015
In the event that an index database is lost or corrupt or if the entire index cache location is lost, indexes are
automatically recovered.
2. If index logs more recent than the index database checkpoint are in the cache location, they will automatically
be replayed into the index database.
3. If index logs are not in the cache location, the logs will be restored from storage and replayed into the index
database.
P a g e 14 | 41
Education Services V11 Walkthrough Guide October 2015
Data Aging
During data aging operations, information for jobs that have exceeded retention are sent to the MediaAgent. The
index database’s job summary table will mark the jobs as unrecoverable. This results in accurate browse operations
and removes the need to restore previous indexes associated with specific jobs. During compaction operations, the
detailed information of objects within the index database are deleted.
Data Recovery
V2 indexing provides fast find and browse operations. When conducting find or browse operations, the MediaAgent
hosting the index database must be available.
Using V1 and V2 Indexing
MediaAgents can manage both V1 and V2 indexes. If a browse or find operation spans a period of time where
indexing was converted from V1 to V2, two separate operations will be required. A single find or browse job cannot
span index versions.
P a g e 15 | 41
Education Services V11 Walkthrough Guide October 2015
Storage
Miscellaneous Changes:
Schedules associated with libraries can be viewed from the library level.
Disk libraries
Library Properties
Several changes have been made in the library properties view.
P a g e 16 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 17 | 41
Education Services V11 Walkthrough Guide October 2015
Tape Library
Scratch Pool Media Selection Rules
Spare media selection criteria now includes the ‘use media based on capacity’ option. The setting can be configured to
fill lower capacity tapes first or higher capacity tapes first.
P a g e 18 | 41
Education Services V11 Walkthrough Guide October 2015
Deduplication
Miscellaneous Changes:
By default, 16 archive files will be maintained in each secondary table file resulting in smaller DDB size when
using longer retention settings.
Global deduplication policies can be configured from the deduplication engine in storage resources.
‘Do not deduplicate against objects older than’ option is available in the settings tab.
Log files are constantly changing with new information added and old information truncated. Since the state of the data is
constantly changing, deduplication will provide no space saving benefits. During log backup jobs, the application agent
will detect the log backup and no signatures are generated, saving CPU and memory resources on the production system
and speeding up backups by eliminating signature lookups in the deduplication database.
P a g e 19 | 41
Education Services V11 Walkthrough Guide October 2015
Deduplication Database
In Memory Deduplication Database
In memory deduplication provides significant performance and resiliency benefits. There are several components that
make up the in memory deduplication database: In memory logs and disk log.
In Memory Logs
In memory logs are linked to portions of the deduplication database and are dynamically added to memory by the
system. There are three memory logs; one active log, which records all database changes, a pending commit log, and
merge commit log. The active log will be record changes for 20 seconds. Once the active log is closed it becomes a
pending commit log and a new active log is started. While the active log receives changes and the pending commit log
closes, a merge commit log is committed to an on disk log. This circular clogging method ensures fast performance
without requiring significant memory or CPU resources.
DiskDB Log
The DiskDB log resides in the DDB location and receives updates from the memory logs. In addition to receiving changes
from the in memory logs, it is also used to commit records to the deduplication database. In the event of a MediaAgent
unplanned shutdown, the DiskDB log is used to bring the DDB to a consistent state.
P a g e 20 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 21 | 41
Education Services V11 Walkthrough Guide October 2015
Partial Database Reconstruction – If the deduplication database is lost or corrupt, a backup copy of the database is
restored and the database is reconstructed using chunk metadata.
Full Database Reconstruction – If the deduplication database is lost and no backup copy is available, the entire
database is reconstructed from chunk metadata.
P a g e 22 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 23 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 24 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 25 | 41
Education Services V11 Walkthrough Guide October 2015
Data Verification
Verification of Existing Jobs on Disk and Deduplication Database
Full verification for disk and DDB uses checksum data to verify block integrity by reading data chunks (Sfiles),
uncompressing, and decrypting, and using CRC information to validate block integrity. This option also verifies chunk
metadata using CRC checks. Any blocks failing the check will be marked in the DDB. New blocks generating the same
signature as a block marked bad are re-written to disk and a new signature entry is written to the DDB. This verification
method also verifies chunk integrity between the DDB and disk library.
Incremental Verification
Incremental data verification verifies data integrity for new jobs added since the last verification job. This option is
available when running ‘verification of deduplication database’ or ‘verification of existing jobs on disk and deduplication
database’ options. Since this method only verifies new jobs, full verification jobs should periodically be executed.
P a g e 26 | 41
Education Services V11 Walkthrough Guide October 2015
Storage Policies
Miscellaneous Changes
Default retention for a new storage policy primary copy is 15 days and 2 cycles.
Default retention for a new secondary copy is 30 days and 4 cycles.
Media refresh includes the option to mark refreshed media appendable for new data.
Schedules associated with a storage policy can be viewed: right-click storage policy | view | schedules.
Schedules associated with a storage policy copy can be viewed: right-click storage policy copy | view | schedules.
P a g e 27 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 28 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 29 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 30 | 41
Education Services V11 Walkthrough Guide October 2015
Client Management
Miscellaneous Changes
P a g e 31 | 41
Education Services V11 Walkthrough Guide October 2015
Subclients
Subclient Miscellaneous Changes
Subclient properties advanced button hides advanced features
Save as script option to generate XML input files
Add subclient content using multi-line text box
Example: a user defined subclient called ‘F:\ drive’ is managing all data on the F:\ drive. Another subclient is created
called ‘Word Docs’ defining ‘f:\**\*.DOCX’ as content. Both subclients will protect DOCX files.
P a g e 32 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 33 | 41
Education Services V11 Walkthrough Guide October 2015
DataStore Distribution
If VMs within a subclient exist across multiple DataStores, the coordinator will assign VMs to proxies, one VM per
DataStore until the maximum stream count is reached. Each VM will be assigned to a different data mover proxy,
balancing stream loads across proxies based on proxy resources. This will distribute the load across multiple
DataStores which will improve backup performance and maintain a healthy DataStore state. In addition to the
subclient Data Readers setting, a hard limit can be set for the maximum number of concurrent VMs that can be
protected within a single DataStore using the nVolumeActivityLimit additional setting.
1. Number of proxies available to back up a VM. The fewer proxies available, the higher in the queue the
VM will be. This will also be dependent on transport mode. If the transport mode is set to Auto (default), SAN
will have highest priority, followed by HotAdd and then NDB mode. If a specific transport mode is defined in
the subclient, only proxies that are capable of protecting the VM can be used – this could affect the available
number of proxies which could result in a higher queue priority.
2. Number of virtual disks. VMs with more virtual disks will be higher in the queue.
3. Size of virtual machine. Larger VMs will be higher in the queue.
P a g e 34 | 41
Education Services V11 Walkthrough Guide October 2015
Throttling can be hard set on a per proxy basis using the following registry keys:
nStreamsPerCPU will limit the number of streams per CPU on the proxy.
nMemoryMBPerStream will set the required memory on the proxy for each stream.
nStreamLimit will set a limit on the total number of streams for a proxy.
bHardStreamLimit will set a hard stream limit across all proxies within the VSA instance.
P a g e 35 | 41
Education Services V11 Walkthrough Guide October 2015
Data Movement
Client Job Throttling
The total number of concurrent jobs for a client can be throttled using the ‘enable job throttle’ option. This setting acts
as a governor controlling the number of jobs that can run regardless of stream settings or number of subclients. Job
throttling must first be enabled for the CommCell in ‘job management’ settings and then it must be configured in the
‘job control’ tab of the client properties.
Job throttling is used to set the maximum number of concurrent data protection jobs and the number of log backups.
There is an additional option to exclude jobs that run immediately from the job throttle.
P a g e 36 | 41
Education Services V11 Walkthrough Guide October 2015
Application Protection
Miscellaneous Changes
Run synthetic full jobs for de-configured clients. This is required to ensure proper retention and aging when
using subclient level retention.
P a g e 37 | 41
Education Services V11 Walkthrough Guide October 2015
In a DAG environment, the stream allocation will be based on the number of nodes. When the job starts, the stream
logic will automatically assign one stream to each node. If there are additional streams remaining, they will be
allocated based on which node has the most databases. The stream allocation process will continue in order of
Exchange servers in the DAG environment containing the most databases to fewest in a prioritized round-robin
method until all streams are allocated.
P a g e 38 | 41
Education Services V11 Walkthrough Guide October 2015
P a g e 39 | 41
Education Services V11 Walkthrough Guide October 2015
Data Recovery
Browse Data Using Pie Chart View
The new pie chart view in the browse window provides insight into what data is stored on a client and provides a
visual method for browsing data.
P a g e 40 | 41
COMMVAULT.COM | 888.746.3849 | EA.COMMVAULT.COM