CV SYS Admin Curse-R02
CV SYS Admin Curse-R02
CV SYS Admin Curse-R02
ii Copyright 1999-2007 CommVault Systems, Inc. All rights reserved. CommVault, the CV logo, CommVault Systems, Solving Forward, SIM, Singular Information Management, Simpana, CommVault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, GridStor, Vault Tracker, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, and InnerVault are trademarks or registered trademarks of CommVault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
iii
TABLE OF CONTENTS
User Administration Module Objectives:................................................................................... 8 CommCell Security Basics ......................................................................................................... 9 CommCell Console Access Options ......................................................................................... 10 User Identification .................................................................................................................... 12 Users and User Groups ............................................................................................................. 14 Understanding Users/User Groups ........................................................................................... 15 Understanding Capabilities & Managed Objects...................................................................... 16 Command Line Access ............................................................................................................. 18 Increased Security Options ....................................................................................................... 20 Using Computer Client Groups ................................................................................................ 22 CommCell Administration Options .......................................................................................... 24 E-mail and IIS Configuration.................................................................................................... 25 System Thresholds .................................................................................................................... 26 User Preferences ....................................................................................................................... 27 Automatic Update Configuration.............................................................................................. 29 DR Backup Settings.................................................................................................................. 31 Summary ................................................................................................................................... 33 Library Management Module Objectives: ............................................................................... 36 Understanding Libraries............................................................................................................ 37 Media ........................................................................................................................................ 38 Connection ................................................................................................................................ 40 NDMP Libraries........................................................................................................................ 43 PnP Libraries............................................................................................................................. 45 Content Addressable Storage .................................................................................................... 46 Single Instanced ........................................................................................................................ 47 Configuring Media Agent Options............................................................................................ 49 Using Shared Index Cache........................................................................................................ 51 Configuring Magnetic Libraries ............................................................................................... 53 Mount Path Usage Options ....................................................................................................... 55 Mount Paths Settings ................................................................................................................ 57 Configuring Removable Media Libraries ................................................................................. 59 Library Settings......................................................................................................................... 60 Drive Settings............................................................................................................................ 63 Summary ................................................................................................................................... 65 Media Management Module Objectives:.................................................................................. 68 Understanding Media Management.......................................................................................... 69 Types of Media ......................................................................................................................... 70 How Data is Stored ................................................................................................................... 72 Removable Media States........................................................................................................... 73 Removable Media Pools ........................................................................................................... 75 Managing Media Content ......................................................................................................... 77 Job State on Media.................................................................................................................... 79 Removable Media Usage .......................................................................................................... 82
iv Media Labeling ......................................................................................................................... 84 Threshold Settings .................................................................................................................... 86 Managing Internal and External Media .................................................................................... 87 Exporting Media ....................................................................................................................... 89 Export using Vault Tracker ....................................................................................................... 91 Using Virtual Mail Slots ........................................................................................................... 92 Using Containers....................................................................................................................... 93 Vault Tracker Jobs..................................................................................................................... 94 Importing Media ....................................................................................................................... 96 Discovering new Media ............................................................................................................ 98 Verifying Existing Media........................................................................................................ 100 Summary ................................................................................................................................. 101 Storage Policies Module Objectives: ....................................................................................... 104 Understanding Storage Policies .............................................................................................. 105 Design of Storage Policies ...................................................................................................... 106 Types of Storage Policy Copies .............................................................................................. 108 Updating Secondary Copies.................................................................................................... 109 Managing Data Streams .......................................................................................................... 111 Job vs. Device Streams ........................................................................................................... 112 Parallel vs. Multiplexing......................................................................................................... 113 Combine to Streams ................................................................................................................ 115 Data Retention, Management, and Aging ............................................................................... 116 Determining Data Retention Requirements ............................................................................ 118 Managing Backup and Archive Retention .............................................................................. 120 Understanding Cycles and Days ............................................................................................. 121 Data Aging .............................................................................................................................. 122 Retention Variants ................................................................................................................... 124 Roles of Verification, Encryption and Content Indexing........................................................ 127 Inline vs. Copy-based Encryption........................................................................................... 128 Content Indexing..................................................................................................................... 130 Content Indexing and Search .................................................................................................. 131 Data Verification ..................................................................................................................... 133 Summary ................................................................................................................................. 134 Clients Module Objectives: ...................................................................................................... 136 What are Clients?.................................................................................................................... 137 Understanding the role of Agents ........................................................................................... 138 Backup and Archive Sets ........................................................................................................ 140 Understanding Backup and Archive sets ................................................................................ 141 Using On Demand BackupSets............................................................................................... 143 Data Classification for Archive and Backup........................................................................... 145 Configuring Subclients ........................................................................................................... 147 Defining Content and Content Options................................................................................... 148 Using Filters............................................................................................................................ 150 Using Pre/Post Process Commands ........................................................................................ 152 Setting Storage Options .......................................................................................................... 154 Using Subclient Policy............................................................................................................ 156
v Summary ................................................................................................................................. 157 Job Management Module Objectives: .................................................................................... 160 Configuring Data Protection Tasks ......................................................................................... 161 Types of Data Protection......................................................................................................... 162 Understanding Synthetic Full ................................................................................................. 164 Archive Concept HSM......................................................................................................... 165 Executing Data Protection Tasks ............................................................................................ 167 Job Initiation Options.............................................................................................................. 168 Using Backup/Archive Options .............................................................................................. 170 Creating a Schedule Policy ..................................................................................................... 172 Using Command Line ............................................................................................................. 173 Using Backup/Archive Options .............................................................................................. 175 Activity Controls..................................................................................................................... 178 Operation Windows ................................................................................................................ 180 Job Management Control........................................................................................................ 182 Job Controller Window ........................................................................................................... 184 Summary ................................................................................................................................. 186 Restore Module Objectives: ..................................................................................................... 188 Image/No-Image Browse ........................................................................................................ 190 List Media & List Media (Precise) ......................................................................................... 192 Exact Index ............................................................................................................................. 194 Find ......................................................................................................................................... 195 Understanding Restore and Recovery..................................................................................... 197 Restore/Recall Options ........................................................................................................... 198 Restore by Job......................................................................................................................... 199 Mapped Restores..................................................................................................................... 201 Recall Archived Objects ......................................................................................................... 203 Copy Precedence..................................................................................................................... 205 Restore From Anywhere ......................................................................................................... 207 Out-of-Place Restores ............................................................................................................. 208 Restores to Non-client hosts ................................................................................................... 209 Working with stub files ........................................................................................................... 210 Full System Restore ................................................................................................................ 212 Summary ................................................................................................................................. 214 Monitoring Module Objectives:............................................................................................... 216 CommCell Explorer ................................................................................................................ 218 CommNet................................................................................................................................ 220 Job Controller.......................................................................................................................... 223 Resource View ........................................................................................................................ 225 Event Viewer........................................................................................................................... 226 Understanding Alerts .............................................................................................................. 229 Alert Output Options............................................................................................................... 230 Recommended Alerts .............................................................................................................. 231 Available Reports.................................................................................................................... 234 Recommended Reports ........................................................................................................... 235 Summary ................................................................................................................................. 238
vi Appendix.................................................................................................................................... 239 Vault Tracker vs. Vault Tracker Enterprise............................................................................ 240 CommCell ............................................................................................................................... 241 Training Environment............................................................................................................. 242
User Administration - 7
User Administration
www.commvault.com/training
8 User Administration
Overview
CommCell Security Basics Users and User Groups Increased Security Options Using Client Computer Groups CommCell Administration Options
User Administration - 9
10 User Administration
Web-Based Console
z
Web-based During the installation of the CommServe component - if the Internet Information Server (IIS) is installed and running on the same host - you are offered an option to configure the CommServe for web administration. Note that CommServe web administration is only possible with IIS. If the CommServe computer does not have IIS installed, or was not configured for web administration during installation, you can at anytime enable local web administration or
User Administration - 11 configure web administration via an alternate IIS host. Note that whatever IIS host is used it must have the CommCell Console installed. If using an IIS host different than the CommServe the CommCell Consoles e-mail and IIS Server applet must be configured to use the alternate IIS host. Once configured for web administration, any java enabled browser can be used to access the Java applet and perform remote administration. The web-based version of the CommCell Console has the same appearance and functionality as the installable CommCell Console Java application. Note that all login sessions where a password is required are done using Secure Socket Layer (SSL) protection.
12 User Administration
User Identification
CommCell User
z z
SMTP Address (for Alerts & Notifications) Password Aging Associated Active Directory Group Single Sign On capable Advanced message recovery
External User
z z
Outlook User
z
User Identification
CommCell User A CommCell User has access to the CommCell Console only. There are no host or network privileges assigned. A CommCell users ability to view CommCell Console objects and perform tasks is based on CommCell User Group membership. Each CommCell User account should have a SMTP e-mail address which is used for receiving reports and alerts. Passwords assigned to the CommCell User account can be aged to provide for periodic password changes to prevent long-term brute force hacks. External user An External user is a member of a Windows Active Directory domain group that has been associated with a CommCell User Group that has, at a minimum, Browse capability. External users must enter their name in the CommCell Login prompt window in the following format: <domain name>\<user name>. When a username is entered with a domain name, the CommServe Server automatically recognizes that the password information must be authenticated by the external domain server.
User Administration - 13
When enabled for an associated AD domain. External users can use the Single Sign On (SSO) feature to automatically login to the CommServe using their Active Directory user-account credentials. To bypass Single Sign On in order to login as a different user, click Cancel at the CommCell Login prompt window and enter the alternate username and password. Outlook User CommCell authentication is required for end-users using the DataArchiver Outlook Add-In to perform advanced message recovery operations such as find recoveries and browse recoveries from Outlook. The Single Sign On (SSO) feature allows Exchange administrators to establish a CommCell User Group for Outlook Add-In end-users to perform these functions using their existing Windows user accounts and passwords residing in the Active Directory domain. To configure this feature: Create and enable the nUseCommonSSOUser registry key. Once the key is enabled, refresh the CommCell Console. This will automatically create a user group called Common Outlook Add-In Default Group and a user account called Default Outlook User. This group is automatically created with associations to all CommCell objects. If necessary, change this configuration in the User Group Properties (General) dialog box. Once Single Sign-On has been configured, then Outlook users may perform find and browse recoveries of archived messages without the need to enter CommCell authentication credentials. When users select the Outlook Add-In option to Find and Recover Messages, their Windows user accounts are automatically granted rights to access the CommServe to perform this function as part of a CommCell User Group.
14 User Administration
Understanding User and User Groups Understanding Capabilities & Managed Objects Command Line Access Adding CommCell Users and Groups Enabling External Users/Groups
Using this approach, a CommCell administrator can provide users with the exact capabilities they are required. These requirements can vary, depending on the tasks each user needs to perform. A CommCell administrator can also restrict the CommCell objects that a user can view, by restricting the CommCell objects that a user's member user group has an association with.
User Administration - 15
16 User Administration
Performs
Has To Perform
TASK
To/with
Associated to
User Administration - 17 Client Computer Group Client Computer Agent Backup set Subclient Media Agent Library Storage policy
Each of these objects support specific functions within the CommCell. Two default and two permanent groups exist. These are the Master groups which have All Capabilities for All Associations, and the View All group whose members have All Association level access in the CommCell Console for viewing privileges only. Users who are not members of the View All group can only see those objects, jobs, and events to which their CommCell User Group(s) have been associated. By default, new users are automatically added to the View All group. This default characteristic can be changed in the Security tab of the CommCell level properties page.
18 User Administration
Supported on any CommCell component host Single Login/Logout required Encrypted Password required
User Administration - 19 Encrypted password Starting a command line session requires an encrypted password. The -p argument of the qlogin command provides for this purpose. You can obtain this encrypted value by saving any supported operation (i.e., a backup or restore job) as a script through the CommCell Console. This creates the qlogin string and encrypted password for the user that is currently logged on to the CommCell Console. You can then copy and reuse the encrypted password from that script in other scripts. Be sure you restrict read/execute privileges to scripts containing the qlogin password.
20 User Administration
User Administration - 21 and/or even changing the port to a lesser known address. It can be changed in the services file of the Windows/system32/drivers/etc directory. Once access to CommServe is allowed, CommCell security limits task execution and resource visibility of validated CommCell Users via CommCell User Groups with assigned functional capabilities and associated object management.
22 User Administration
View job history details Install updates Create new schedule policies and reports Set the activity control for data protection and/or data recovery operations within the CommCell Set the security parameters for users within the CommCell
A task configured for a client computer group will affect all the clients within the client computer group, e.g., the View Job History task will display the details for all of the clients within the client computer group. Client Computer Groups allow an administrator to scale down the CommCell Browser navigational requirements and manage multiple clients at the same time. A client can be a member of multiple groups. For example, an Exchange server may be a member of the Chicago site group and also a member of the Exchange Server groups.
User Administration - 23
On the security side, CommCell User Groups can be associated with a Client Computer Group for management. All clients within the CommCell User group and any clients added to the group can be managed by member users of the CommCell User Group. CommCell User Groups also add another level of Activity Control. Administrators can control multiple clients in a logical grouping with a single activity control. The greatest advantage to Client Computer Groups lies in the area of reporting and alerts. Scheduled reports and alerts can be created for each client group. As clients are added or removed from the CommCell or group, there is no need to revise the report or alert.
24 User Administration
User Administration - 25
E-mail Server
z
SMTP Capable
IIS
Alternate for Reports z Alternate for Books Online
z
The support for separate IIS hosts for Books Online and Reports is based upon expressed customer security requirements for distinct access ability and separation. Web servers other than Microsofts IIS are not supported. Alternate JAVA capable web browsers for viewing reports are supported.
26 User Administration
System Thresholds
Event Viewer
z
Default -10,000 Events retained Improve response time Warning on disk space availability
GUI Timeout
z
Database Checks
z
System Thresholds
Event Viewer The event viewer provides a real-time display of all relevant events in the CommCell. By default, 10,000 events are logged and the most recent 200 are displayed. A max of 1000 events can be configured for display in the User Preference applet. However, the more events selected for display, the longer the CommCell Console takes to initiate, and the more memory is required to maintain the CommCell Console session. Note that during Event Log search action, all 10,000 events are included regardless of how many events are initially displayed. GUI Timeout To improve individual response time of each CommCell Console you should keep the amount of CommCell Consoles that are concurrently opened to a minimum. Obviously, updating many CommCell Consoles displays can impact performance. To assist you in this effort, you can enable the GUI timeout feature in the Control Panels System control options to disconnect when a console session is inactive for a certain period of time. Database Checks The Database Space Check thresholds generate events and alerts if there is insufficient disk space for the CommServe Database to grow. While the database growth rate is low, the impact of having insufficient disk space can be severe. Events are generated and sent to the Event viewer when the database disk space is below the defined Information, Minor, Major, and Critical thresholds. The Disk Space Low alert, if configured, is only generated when the CommServe Database disk space falls below the defined Critical threshold. Changes to the Database Space Check Interval take effect immediately.
User Administration - 27
User Preferences
Default Browser for Reports GUI Look & Feel Job/Event Filters Window Settings
User Preferences
User preferences are configurable for each user login session and are retained for subsequent sessions. Reports Microsoft Internet Explorer is the default browser used to view reports. You are able to change the browser to any other preferred browser (i.e., Mozilla, Firefox). GUI Look & Feel You can configure the CommCell Console display to use one of the following styles: The Metal style, resembling a Java application environment The CDE/Motif style, resembling a UNIX environment The Windows style, resembling the Microsoft Windows environment Job Filter When a job completes, the Job Controller window continues to show the job and its final status for a configurable number of seconds (300 by default). In an environment where most jobs are performed off hours and alerts or reports are used for monitoring the results, reducing the display time can improve GUI performance.
28 User Administration Event Filter Events can be filtered to display by their severity levels. In an active environment, limiting the displayed events to major and critical, or just critical can clear the display of unnecessary information and improve GUI response times. Window Settings (7.0) Within the CommCell Console you are able to make adjustments to the size, shape, and content of any of the task windows. The configuration of the three primary windows CommCell Browser, Job Controller, and Event Viewer can be saved for subsequent sessions.
User Administration - 29
Updates
z
Service Packs & Certified Patches FTP or HTTP Proxy Local Supports all OS types Shared CIFS/NFS locations Able to Schedule
Update Cache/Share
z z
Install Updates
z
30 User Administration inaccurate. A CommCell Summary report that includes Update status will query all components and resolve and update status inconsistencies. FTP Download The FTP download operation downloads the updates from the FTP source site to the CommServe cache directory. This server name can be changed if your FTP site for downloading updates differs from the default location or if you would like to reroute the FTP transfer through an FTP proxy. If necessary you can also configure an HTTP proxy server to connect to the FTP server to download the updates. The FTP download can be scheduled or run on-demand. The software also provides you with the option to schedule FTP download operations during the install of the CommServe software. FTP download operation will download updates based on the operating system of the clients currently available in the CommCell. The Cache Directory The CommServe cache directory is configured to serve as a holding area for update packages. This can either be a local drive on the CommServe or a shared network directory. If the CommServe cache area resides on a local drive, the appropriate CommServe service is used to copy and then install updates locally on the appropriate CommCell computers. If the CommServe cache directory resides on a share, this must be a shared location with permissions set to write to the directory. Client computers that are not in the same domain as the domain in which the CommServe cache is located, must have bidirectional trust in place. Windows computers that have permissions to access the share can install updates directly from it, without the need to first copy them. UNIX computers can use this directory for installation after copying the updates to a local directory. However, if you wish to configure UNIX computers to install updates directly from a cache, a separate UNIX cache (i.e., an NFS Mount Path) must be created. If a UNIX share is configured, and an automatic download of UNIX updates from the FTP site has occurred, a copy of each UNIX update is put in the CommServe cache and then in the UNIX cache.
User Administration - 31
DR Backup Settings
Protects
z z z z
CommServe Database CommServe Registry Firewall settings Log Files Backup Magnetic Export Removable Media
Destination
z z
DR Backup Settings
The software stores all information for the CommCell in a SQL database, and in the Windows 2000 and Windows 2003 registries. It is critical to be able to retrieve this information in the case of a disaster or system failure. This metadata and Windows registry data are backed up during a Disaster Recovery (DR) Backup. This data can be browsed and then restored using the CommServe Recovery Tool. Determine the Number of Disaster Recovery Backups to Retain on Disk When the metadata is backed up and archived, it remains valid (available for restoration) for a period of time determined by the number of backup sets to retain. The retention rule is determined by the number of successful full backup cycles to be maintained. The number of Disaster Recovery backup sets is preset to five full backups during installation. This can be changed from the DR Backup Settings dialog box. This retention count refers to the number of backup sets created during the Export phase of the Disaster Recovery Backup. Retention time for copies written to media drives during the Backup phase are determined by the characteristics of the Disaster Recovery Backup storage policy. A DR Backup has two possible phases. Phase I should be to a magnetic share as far away as practical from the CommServe Host. This is your primary source of recovery and it should be located where a disaster that befalls the CommServe does not affect the Phase I backup copy. A Phase II DR Backup is assigned to a DR Storage Policy or a Standard Storage Policy and should be written to a removable media library. As with Phase I, the intent is to place a copy of the CommServe Metadata offsite for disaster recovery.
32 User Administration
DR Backups can be configured to run at user defined times. Right click on the CommServe in the CommCell browser; select all tasks and Disaster Recovery backup. CommServe DR jobs can be scheduled or run on-demand. A best practice for managing your CommCell environment would be to perform a DR backup before making any major modifications to the backup environment. This would include modifying or deleting storage policies, redefining subclient content, and adding/removing libraries. Disaster Recovery Backups Content Disaster Recovery backs up the following: Metadata - Metadata includes the Microsoft SQL Server database that holds information about all CommCell database components (including clients and media configuration) Windows registry - The Windows registry is a central resource from which the Windows operating system obtains many of the systems operating parameters. Firewall Configuration Files The firewall configuration files (FwPeers.txt, FwHosts.txt and FwPorts.txt) are also included in the Disaster Recovery backup. If necessary, the entries associated with Clients/Media Agents on the other side of the firewall from the CommServe can be restored in the event of a CommServe re-build. Note that a restore of the disaster recovery backup does not automatically restore the firewall files. To restore these files, manually select these files for restore from the CommserveDisasterRecoveryGUITool.
User Administration - 33
Module Summary
Key points to remember
Summary
Creation of Users & User Groups (External Security (AD integration, SSO), Users with just SMTP address for alerts and reports) Creating Client Computer Groups (consolidate security, reporting, activity control, updates) What are Capabilities? (where to apply levels CC, MA, CL, DA, SP,SUB, LIB, Client Comp Group, BackupSet) Where can I go to access CommCell Configuration Options? (e-mail, IIS, DR backup settings) Configuring Auto Updates (D/L from FTP, Cache area, client install options, MA update options)
34 User Administration
Library Management
www.commvault.com/training
Overview
Understanding Libraries Configuring Media Agents Configuring Magnetic Libraries Configuring Removable Media Libraries
Understanding Libraries
Consistent term for all protected storage devices Types of Libraries
Understanding Libraries
Libraries is a consistent term used by the software to refer to any managed storage device and its associated attributes. Even standalone drives are referred to as libraries. Libraries are managed by Media Agents. In some cases a single Media Agent may manage multiple libraries and multiple Media Agents may have access to the same library. All data written to, or read from a library must transit through a Media Agent.
Media
Magnetic Disk
z z
Removable Media
z z z
Media
Magnetic Disk Magnetic disk - as a protected storage library is supported on any OS supported file system. Disks, disk partitions, or logical volumes are addressed as mounted file systems or by network protocol (CIFS/NFS). Each addressable storage unit is referred to as a Mount Path. A library can have any number of Mount Path. Access to data stored on Mount Paths can be shared with other Media Agents. The primary advantage of Magnetic Disk is its ability to conduct multiple simultaneous read/write operations. The second advantage is write speed. Normally, magnetic disk is used as read/write media. However, in archive situations, Write Once Read Many (WORM) media may be required and is supported. Removable Media Removable Media is supported in three forms Tape, Magnetic Optical (R/W and WORM), and USB storage devices. The most common of the three is tape. Tape media has its advantages in being a low cost storage media that is easily transportable for offsite protection and disaster recovery. Tape media has its disadvantages in that it is a sequential access medium with a potential for wasting capacity if the tape becomes unavailable or removed before data has been written to the end of the tape. In some cases, a tape is deliberately not filled in order to meet data protection (e.g. removal) or operational needs (e.g. business boundaries).
Library Management Module - 39 Magnetic Optical medias advantage is its durability and reliability. Its primary disadvantages are cost and capacity. USB Storage Devices provide a highly compatible and portable storage device at about the same cost as tape. However, USB Storage Device support is restricted to the equivalent of a standalone tape library (i.e. no robotics or internal storage device for multiple drives)
Connection
Media Changer
MediaAgent
Static Dynamic
MediaAgent
IP-based
MediaAgent
MediaAgent
Connection
Direct Attached Direct attached Libraries are dedicated to a single Media Agent. This is the simplest form of library configuration. Shared A Shared library is one where its media devices are accessible by two or more Media Agents. In its basic form, each media device is accessible by only one Media Agent. For example: a Tape library with 4 tapes drives and exposed SCSI connectors and share its drives by having each Media Agent connect independently to a drive. Media Changer/control is managed by a Media Agent. A Static library is a magnetic library using Shared Disk Devices. These devices can mounted on the Media Agent and/or shared via network protocol or shared file system (e.g. GFS Sistina, PolyServe). Devices can be enable for read/write or read-only. Additionally, a mount path can be cloned via hardware or software and made available for read-only. This is called a Replica Static library. A Dynamic Library is a library where the read/write devices are available to two or more Media Agents. Dynamic libraries required a SAN environment. Devices are dynamically assigned/released from a Media Agent using SCSI-2 or SCSI-3 commands via Fibre/iSCSI protocol. Dynamic libraries are usually removable media libraries, but can be magnetic disks. In
Library Management Module - 41 either case, only the controlling Media Agent can read/write from the device at any one time. Media Changer/control is managed by a Media Agent. IP-Based An IP-Based library is one where the Media Changer/controller is managed by 3rd party software. CommVault currently support StroageTek Libraries controlled by ACSLS (Solaris) or Library Manager (Windows) software and ADIC library controlled by the Scalar Distributed Library Controller (SDLC)/Windows. Drives within the library can be direct-attached, shared independently or dynamically shared.
Special
NDMP Libraries
NAS Filer NAS Filer
Full Library Control And DDS or Standalone NDMP Drives SAN or Directattached Full Library Control And/Or DDS Drives NON-NDMP Data Path Media Agent NDMP Restore Enabler Other Clients
NDMP Libraries
Network Data Management Protocol (NDMP) is an open standard protocol for enterprisewide backup of heterogeneous network-attached storage. NDMP is used by Network Attached Storage (NAS) devices where the proprietary nature of the operating system precludes installation of an Agent. If a library is attached to a NAS filer, a NDMP backup can be sent directly to the library. Other NAS filers can also send NDMP backups to the NAS filer with the direct attached library. A SAN based NAS filer library and drives can be shared with other NAS Filers and Media Agents. The first configured host will have library control. Drives can be dynamically or independently shared. Non-NDMP data can use a NAS filer library even if the library is only attached to the NAS filer. However, non-NDMP data must pass through a Media Agent in route to the NAS Filer/library. NDMP Remote Server (NRS) is an optional software component from CommVault that allows data from a NAS filer to be backed up to a magnetic library or a NDMP drive pool on a tape library attached to a Media Agent. Installation requires that an appropriate license be available on the CommServe. Encryption and multiplexing of NDMP data is supported on NRS capable Media Agents. Auxiliary copy from any Media Agent managed library to a NAS filer library is not supported. NAS NDMP load balancing is supported with multiple NRS capable Media Agents.
NDMP data can be restored back to a NAS filer or to a Windows File System Agent with NDMP Restore Enabler software component installed. Direct Access Restore (DAR) optimizes the restore operation by allowing the NDMP client to directly access backed up data anywhere in a tape set without having to traverse the tape set sequentially. In a normal restore operation, a large portion of the data from the backup that included the file must be read. With DAR, only the portion of the tape which contains the data to be restored is read. This option is available for each of the NAS NDMP restore types and can save significant time in the restore operation.
PnP Libraries
One PnP Library per Windows Media Agent Uses removable USB device as media Similar functionality to a Standalone Tape Library USB Device must be detected with drive letter assigned
PnP Libraries
PnP (Plug and Play) Libraries can be added for any Windows Media Agent. Each PnP library has a Drive holder that acquires and can use a USB storage device as removable media. USB media is detected and an On Media Label written to identify it. A minimum size can be assigned to prevent detection use of small USB Drives. Additionally, a restriction that only blank USB drives. Once configured, PnP Disk Libraries are similar to a stand-alone drive and all the features, including the following are supported by these libraries: Automatic Labeling Schemes Manual Stamping of Media
USB Media is handled similar to Magnetic Optical media. When all jobs on a USB media are aged, the media will be assigned to the scratch media pool, the CV_MAGNETIC folder will be erased and the disk re-used. In the 7.0 release of the software, only one PnP Library can be added/configured per Media Agent. While multiple PnP Drives can be added/configured, only one data stream should be used to a PnP Library. With multiple USB drives, a job can spill over from one drive to the next. Multiple streams and additional capabilities are planned.
Magnetic Library Data is written in uniquely addressable blocks Duplicate blocks are logged but not written
Single Instanced
CommVault
z z
Others
z z
Single Instanced
The need for storage has been growing steadily for businesses and are based on a number of needs, such as data retention, compliance requirements, data-intensive applications, etc. This explosion necessitates the need for a smarter way of storing data. Single Instance Storage addresses this problem by identifying the duplicate items in a data protection operation and maintaining references for the duplicate items. When a data protection operation is performed for the first time all the data is stored; if the same data is subsequently identified in another data protection operation, reference counters are incremented and the data itself is physically not stored in the storage media. Single Instance Storage is currently only supported for magnetic libraries. Note that there are appliances that support single instancing of data (e.g., Centera, Data Domain, HDS DRU). Such hardware can be configured as a magnetic library with the option to write the data in a single-instance format enabled. Single Instance Storage is supported by both backup and data archival products, specifically for the file system and email attachments, in order to provide optimization in storage when copies of the same data is backed up and stored. This feature requires a Feature License to be available in the CommServe and needs to be enabled on the magnetic library.
48 Library Management Module Single Instance Single Instance Storage addresses the problem of storing duplicate items in a data protection operation and maintaining references for the duplicate items. When a data protection operation is performed for the first time, all the data is stored. On subsequent operations, if the same data is encountered a reference is made to the first stored item and the duplicate items are not written to the media. Single instancing is currently supported only on magnetic disk media.
50 Library Management Module Locating the Index Cache The Index Cache is used as a temporary working space for index-based data protection and restore jobs. Indexes located in the cache keep track of where individual objects are written on protected media. Indexes are created, updated, and written to protected media with each data protection job. Indexes are retained in the Index Cache by settings on the Media Agent for subsequent use. A Browse operation will consolidate and read indexes in the Index Cache as required by the scope of the browse. Indexes required, but not found in the Index Cache are automatically restored. The Index Cache should be located on a separate disk and it should be the fastest disk available to the Media Agent.
Required for Data path Failover and Round Robin CIFS or NFS supported Read & Write permissions required
Index Cache
MA
In order to share an index cache between two or more Media Agents, one Media Agent in the group must be setup first to share its index cache. If the location of a shared index cache is to be changed, then every Media Agent that required a new access path or user account for the new index cache location must be updated, otherwise their index cache access paths will eventually go offline.
52 Library Management Module Un-sharing an index cache is only possible if there are no storage policy copy failover data paths defined that rely on that index cache being shared. Note that when un-sharing an index cache, a copy of the index is not performed and you will have to rely on index cache restores in order to browse backup data.
54 Library Management Module Managed Disk Space Managed disk space is configured in the Magnetic Library Properties and enabled on a per Storage Policy Copy basis. Data Aging operations still need to be run on a regular basis for Storage policies using Managed disk space. Data Aging will flag the eligible data as "Managed" (an internal flag). Managed data is included in restore browse operations along with retained data without the need to include "Aged" data. The periodic Update Free Space operation will check for thresholds being exceeded and, if so, erase the "Managed" data starting with the oldest until either all managed data is erased, or the lower threshold level reached. You can increase or decrease the configuration parameter for Interval (Minutes) between magnetic space updates (default :30 minutes) located in the Service Configuration tab of the Control Panels Media Management control. If a job exceeds the high watermark and fills the available disk space before managed data can be erased, the job will go into a waiting state due to "Insufficient media". Once the erase operation is completed and space becomes available, the job will resume. Best Practice - Set your high watermark level to leave at least sufficient disk space for what your hourly throughput rate can fill. This will give some breathing room. Allocation Policy Allocation Policies are used to optimize performance and throughput to each storage device by balancing write demands against available resource. Each mount path has an allocation policy which defines the maximum number of concurrent writers for that mount path. A magnetic writer is the equivalent of a tape device. The default maximum number of writers for a mount path is 5. This can be changed for each mount path via the Allocation Policy tab of the Mount path Properties window. Magnetic libraries also have a Library Allocation Policy. This policy is used to determine the maximum number of writers allowed for the entire library. By default the Library Allocation Policy is set to the Maximum Allowed Writers which is a summation of writers from all mount paths. For example; if you have 4 mount paths with 2 writers each, the Maximum Allowed writers to the library would be 8. You can set the number of writers allowed for a library to a number less than maximum. You might want to do this if the Media Agent or data path have insufficient resources or bandwidth to support the maximum number of writers for all mount paths. There is no similar control/policy for readers. You can have any number of readers for a mount path. However, such as for writers, a practical limit bases on available resources should be used.
1 4
2 5
Media Agent
56 Library Management Module will be used to create/write the New Job. If writers are multiplexed, then all available writers will be used first before another multiplex stream is used.
58 Library Management Module This could also happen if 2 concurrent single-streamed data protection operation is initiated to the same mount path, using 2 different storage policies. Similarly, the system may consume more than the specified maximum space. Consider the following example: When a data protection job is initiated, the system checks for the available disk space and verifies whether it is less than the specified maximum space. For example if you have specified 1 GB as the maximum space and if you have used .75 GB, the data protection job will be initiated using the mount path. However, if the size of the data is .5 GB, the system will write the entire data. In such a situation the space consumed will be more than the specified maximum space. Note that a data protection operation will generally write to the mount path until the minimum reserve space is reached, before spanning to an alternate path. Fragmentation This option sets the pre-allocation mode meaning that this a reserve ahead feature. It ensures that an adequate number of contiguous chunk space is available so that all the chunks in a data protection job can be kept together on a write from a Windows Media Agent. When chunks from jobs are pruned there is less fragmentation. Writer Allocation Policy The mount path allocation policy allows you to establish the maximum number of concurrent writers or the maximum number of simultaneous data protection operations on the mount path. If necessary you can also disable the mount path for write operation. This is useful in situations where you wish to retire or phase-out a mount path. This operation is supported by all magnetic libraries.
Library Settings
Active Library Controller In a SAN environment, the library controller feature can be used to configure the active Media Agent to automatically switch over to a failover Media Agent, in the event of a failure in the active Media Agent. SCSI Reservation When this option is enabled, the Media Agent uses the drive exclusively during data protection and other operations, using SCSI reservation. This option is useful in the SAN environment where multiple computers may try to access the same drive, resulting in data corruption. Refer to the hardware manufacturer's documentation to see if this operation is supported. If this option is enabled and the hardware does not support this type of operation, subsequent data protection jobs may fail. Both SCSS-2 and SCSI-3 reservations are supported. Before enabling this option, ensure the following: Verify the target device to see whether SCSI-2 or SCSI-3 command set for persistent reservation is supported. Most tape drives are known to support either of these command sets. Inter-connecting hardware, such as storage routers or bridge, which connects standard SCSI devices into FC fabric, should also support this command. If you encounter any differences in behavior with this command, then identify the differences in the
Library Management Module - 61 interconnecting components from the working host to non-working host. (Refer to the hardware manufacturer's documentation to see if this operation is supported.) When using the SCSI-3 command set, make sure that the device drivers for the target device do not perform an implicit reservation. Some of drivers in Windows as well as on Unix are known to perform SCSI-2 reservation implicitly when the device handle is opened by an application. In this case the SCSI-3 reservation will fail with Reservation conflict, as SCSI-2 reservation is already active in the target. (Check the drivers behavior using the documentation provided for the driver.) Notably, some IBM LTO family drivers and STK 9x40 family drivers are known to perform SCSI-2 reservation on Windows. On Unix, there are hooks to enable implicit reservation using a known interface through native drivers.
Auto Cleaning You must clean each drive periodically or when necessary to remove the oxides that accumulate on the read-write heads. The drive cleaning operation includes the task of mounting the cleaning media into the drive, cleaning the read-write head and unmounting the cleaning media. When Auto Cleaning is enabled, the cleaning tape is automatically mounted and a cleaning operation is initiated on the drive, whenever the selected options indicate that the drive requires cleaning. The CommVault system automatically mounts the cleaning tape and cleans the drive if the Enable Auto-Cleaning options are enabled for the library. This options is not enabled by default. If you enable this option, verify and ensure that you have a good cleaning media in the Cleaning Media pool. When a drive is successfully cleaned, a message is displayed in the Event Viewer, and the drive cleaning parameters in the Odometers tab of Drive Properties are reset. Keep in mind that if these options are not enabled, the system does not automatically clean the drive when the hardware indicates that the drive requires cleaning. Hence subsequent mount operations in the drive may fail. This option is not available for stand-alone drives and optical libraries. Auto Discovery Before using a new media, the Media Agent must collect certain information about it through a process known as discovery. When a media has been discovered its information is entered into the CommServe database. The media information is permanently retained; media does not have to be rediscovered if it is exported from the library and re-imported. If new media are imported through a librarys mail slot, the import operation triggers a discover operation. This is dependent on whether you have enabled or disabled the Enable AutoDiscover option for the library.
62 Library Management Module If the automatic discovery option is not enabled, the system will prompt you to provide the necessary details for the media. If the automatic discovery option is enabled, the system discovers the media during a subsequent inventory update triggered by a job from the CommCell. If the automatic discovery option is not enabled for the library and if you have some undiscovered media from a previous import, or if you import new media by opening the library door and inserting them, you must initiate a discover operation. Media can be discovered from both the Library and Drive Configuration window and the CommCell Browser.
Drive Settings
Drive Allocation Active Drive Controller Drive Verification
Drive Settings
Drive Allocation simply sets the maximum number of drives that any drive pool can use at any given time in a shared library configuration. Setting allocation ensures that there are adequate drives available for all drive pools during data protection operations. A Drive Pool is a set of drives which are attached or accessible to a Media Agent. Like the Master Drive Pools, drive pools are also logical entities used to facilitate the sharing of a librarys drives between multiple Media Agents. You can modify the following Drive Pool properties: Drive Pool Name This can be used to uniquely identify a drive pool. Enable/Disable Drive Pool Use this option to activate or deactivate the drive pool. Allocate drives Use these options to assign all or some of the drives in the library to the drive pool.
Active Drive Controller The active drive controller can be viewed by right clicking on the drive in question, selecting properties, and then selecting the Drive controller tab. Alternately (And the best method) you can go to the properties of a Master Drive Pool and view all of the properties of all drives from there.
64 Library Management Module Drive Verification When selected, the drive serial number and access path is verified before reading or writing to the media. It is strongly recommended that this option be enabled at all times to prevent the overwriting of data, when the drive access path is changed due to hardware configuration changes. This option is enabled by default.
Module Summary
Summary
Libraries can be configured for Magnetic disk or removable media. These libraries can be direct attached or shared among multiple Media Agents. Support for SAN is easily configured. Single instancing is supported for libraries within the software as well as specifically designed hardware. Media Agents can be multiple platforms and can work together in a single CommCell. Sharing the Index Cache offers the added convenience of Multiple Data Paths and MediaAgent failover, as well as load balancing.
Media Management
www.commvault.com/training
Overview
Understanding Media Management Using Media Options Managing Internal and External Media
Understanding Media
Management
Types of Media How data is stored Removable Media States Removable Media Pools Managing Media Content Job States on Media
Types of Media
Stationary
z
Removable
Tape z Magnetic Optical z USB Storage Drive
z
Types of Media
Stationary Magnetic mount paths Magnetic mount paths are Media Agent addressable file systems residing on volumes hosted by individual disks or disk arrays. Data is written to the mount paths using the supporting host operating system. Volumes used as mount paths can be directly mounted on the Media Agent or accessed via network protocol (CIFS/NFS) or 3rd party protocol (Centera, HDS DRU). Note that the maximum size of a volume that can be used as a mount path can be a maximum of 4,096 Terabytes. The primary advantage of magnetic media is random/concurrent access. Multiple data streams can be used to read/write data on magnetic volumes. Data can be randomly erased and the freed capacity repurposed. In most cases, magnetic media will also be the fastest read/write storage. The primary disadvantage of magnetic media is that it is stationary. Offsite protection requirements using magnetic media can only be supported through Wide Area Networks. Removable Tape Tape media is the most common media in current use for protected storage. Its relatively inexpensive, portable, and for large volume reads or writes, it can be very fast. It meets all the requirements for protected storage media. The primary disadvantage to tape is its sequential
Media Management Module - 71 access and its all-or-nothing limitation for re-use. Additionally, of all protected media options, tape media has the smallest life span. Tape media is also the only data storage media that has physical content with the read/write device and is constantly stressed by winding/rewinding and physical movement. Tape drives are in constant need of cleaning to prevent read/write errors. Additionally, the use of tapes in varied locations demands available drive compatible. Magnetic Optical Magnetic Optical (MO) media has almost all the advantages of tape and magnetic except cost. MO media is rugged, secure, durable, and has a shelf life of 100 years. Besides cost, the other disadvantages to MO is capacity and the access to that capacity. MO platters are dual sided with only one side at a time available for reading/writing. USB Storage Device Removable USB Storage Devices can be used in a manner similar to a standalone tape drive except none of the disadvantages of tape. The USB drives are durable, portable, and compatible for offsite storage and use. Costs and capacity are comparable to tape media. The primary disadvantage is the softwares current limited use of USB libraries. This will change in the future.
Media ID
OML File marker
Bytes offset
chunk
Magnetic Media Data stored on magnetic mount paths are written in 1GB chunks (files) in uniquely numbered folders (e.g. VOL_123). A new folder will be created:
For every active writer stream When Start New Media option is selected for a data protection job For subsequent job if the previous jobs Mark Media Full after successful job completion option is selected Every 8 hours to ensure a folder does not have too many files (chunks)
Spare
Appendable
74 Media Management Module Full/Bad Full or bad media is media that can be read from for restores, but is not writable. A full state is achieved through writing data to the end of the media (EOT) or by a user/job marking the media full. A bad state is achieved through error or usage levels exceeding set threshold limits or by a user marking the media bad. Full media will become spare media when all jobs on the media are logically pruned and the media is not overwrite protected. Bad media will become retired media when all jobs on the media are logically pruned and the media is not overwrite protected.
Physical
Media In Library z Exported
z
Logical
Scratch z Cleaning z Assigned
z
76 Media Management Module manually, or automatically by barcode range. As with spare media, cleaning media is critical to successful data protection operations. Each cleaning pool has media management settings similar to scratch pools. A low watermark for each cleaning and scratch pool can also be established. This parameter represents the minimum number of media that should be available inside the library for that media pool at all times. If the number of available media falls below the low watermark, the system logs a message in the Event Viewer and , if configured, generates an alert. Non-retired media with retained data can be found in the Assigned media pool. This is the normal pool for full, appendable, bad, and active media. Media in the Overwrite Protect Media pool can be placed there manually from the Scratch pool by an administrator or can be sent there automatically from the Assigned media pool, by enabling the media option to Prevent Re-use. The Overwrite Protect Media pool status exempts the data on the media from being re-used in a normal fashion. Media must be manually moved out of the Overwrite Protect Media Pool by the administrator. The Foreign media pool is the logical repository for media that has been used by another application and is not available for use by this CommCell. Media not-recognized or not usable (e.g. media with different format or from another CommCell that optionally is not used) will be placed in this pool. The administrator can also manually move media to this pool. The Retired media pool is for media that has exceeded the vendor recommended threshold values for usage and errors. Media that has exceeded thresholds but still has retained data will not be moved to the retired media pool until all such data has been aged off.
Deleting Contents The delete contents option can be used to delete the contents of a media and move it to a specified scratch pool. This option can be used to make media available to complete an important data protection job when there are no spare media available in the library.
78 Media Management Module Writing over Contents When all the data ages on a piece of media that tape/platter is returned to the scratch pool. Once that media is in the scratch pool it is eligible to be over-written. Until that time comes the aged data is still available for recovery. Erasing Media The erase spare media operation ensures that the old data from removable media (tapes and optical platters) are not recoverable once the media is recycled. This is done as follows: On tapes, the On-Media-Label (OML) is over-written by a new OML to indicate the erased status. On Optical media, the platter is formatted and a new OML is written. Only spare, retired and recycled media from tape and optical libraries can be erased. Note, however, that the erase media operation cannot be performed on magnetic and stand-alone drives. Once a media is erased by this operation, data cannot be retrieved using Media Explorer.
Note that the Erase Spare Media operation will not erase the following data: Data associated with another CommCell, if the option to overwrite media from another CommCell is disabled in the Media tab of the Library Properties dialog box. Data written by other applications, if the option to overwrite media when content verification fails is disabled in the Media tab of the Library Properties dialog box.
The Erase Media operation is a low priority job, and is displayed in the Job Controller window. It can be killed, if necessary.
In either of the above cases, a Data Aging operation must be run to mark the job records as pruned in the CommServe database. Since the data is on tape and not actually deleted, aged jobs are available for browse or Restore by Job operations if explicitly included in these operations. This inclusion can be set CommCell wide by default or job specific. Managed Managed jobs are backup jobs in a managed magnetic library that have exceeded all retention criteria and had a Data Aging operation run to mark the job records as managed. Managed jobs are treated the same as retained jobs for browse and restore operations. An administrative process periodically checks the amount of free space available in a library and compares that against the high watermark for managed data. If the high water mark is exceeded, the process erases managed jobs until the low water mark for managed data is reached. Jobs are erased in
80 Media Management Module chronological order starting with the oldest. Use of managed jobs on magnetic libraries maximizes the use of the library capacity for data availability. Failed Failed jobs are jobs that, for some reason, did not successfully write all required data to protected storage. Indexes for failed jobs are not retained in cache or written to the protected media. However, successfully written chunks from failed jobs can be explicitly included in Restore by Job operations. Killed Killed jobs are jobs that are prematurely terminated by the software or a user before successfully writing all required data to protected storage. Indexes for killed jobs are not retained in cache or written to the protected media. However, successfully written chunks from killed jobs can be explicitly included in Restore by Job operations.
Note: The order in which spare media is used is selectable within the Scratch Media pool. The options are to Use new media first (default) or Use recycled media first. In using recycled media, the media with the least amount of usage and/or errors will be used first. Appendable Media Appendable Media will be used if: The host library option to Use Appendable media is enabled. The appendable media is assigned to the same Storage Policy copy and stream. The last write to the appendable media is within the specified time limit. There is available capacity on the appendable media. The user did not use the Start New Media option. This will use a tape from the scratch pool vice appendable media.
Media Management Module - 83 The new media requirement is not in support of a synthetic full backup. This will use a tape from the scratch pool vice appendable media since the appendable media may contain data that is required to create the synthetic full.
Overwrite Media Media will be overwritten if it is: Newly discovered and the appropriate Overwrite Media option for the library is selected. In a standalone drive and the Overwrite Media in Drive option is selected and the jobs on the media meet the overwrite condition. A previously used media in a scratch media pool and is requested by a data protection job.
The media is required by a Data recovery operation The media is required by an Auxiliary Copy operation The media is required by a Verification operation The media is required by a Content Indexing Operation The media is required by a Synthetic Full The media is required by a Data protection AND the library option to Start New Media When required media is exported is cleared
Bad Media Bad media can be used for data recovery operations only. Bad media will remain in the assigned media pool until such time as all usable data has been aged off. The bad media will then be moved to the retired Media pool awaiting ultimate deletion and removable from the library. Mixed Retention By design, storage policy copies use separate media largely to prevent mixing different retention requirements on the same piece of media. Should this happen, the media is retained for the longest retained job on the media and may not recycle back for use as expected. This condition can be caused if you use different backup and archive retention setting or use extended retention without separating the jobs to different media. CommVault warns you if different backup and archive retention requirements are set. It does not warn you if extended retention is used.
Media Labeling
Auto Stamping Barcode Patterns Updating Barcodes
Media Labeling
Media labels are used to externally identify and track removable media. Exported media is recalled using the external label. Any removable media loaded into a drive will validate the On Media Label to prevent a mis-labeled tape from being used. Auto Stamping Media in Standalone and PnP libraries should be labeled for manual handling. If the option to Auto Stamp Media in Drive is selected and an unidentified (or new) media is loaded into the drive, the system automatically stamps a new label using either the specified Barcode Labeling Scheme or a default label, which is the media creation date and time. If the option is cleared, the unidentified (or new) media must be manually stamped with an appropriate label. Media discovered in blind libraries will automatically be labeled. A default labeling scheme is used unless the user specifies an alternate barcode labeling scheme. Care must be exercised while generating the labeling scheme to ensure that each media label is unique. If the specified labeling scheme is not unique, and if duplicate labels are found, the system automatically appends a media ID to make it unique. Barcode Patterns When you use one or more standard barcode pattern(s) in the libraries in the CommCell, you can define different Barcode Patterns to automate the distribution of media when you have specific purposes for the media in the library. For example:
Media Management Module - 85 When you have a library with different drive types, we recommend you have specific barcodes for media associated with each drive type. This will help you to easily manage and administer the media in the library. When you have media that are not used by the Media Agents in this CommCell (referred to as Foreign media in the software) stored in the library and use Vault Tracker to manage the movement of such media. If you already manage media using barcode patterns, you can use this method to automate the process.
Scratch, Cleaning, and Foreign Media Pools support the use of Barcode patterns for automatic distribution of new media. Updating Barcodes Barcodes labels can wear out, fall out and be assigned different barcodes. Media from libraries with different labeling requirements may be required for use in alternate libraries. (For example, a auto stamped media from a standalone library can be assigned a barcode and used by a barcode capable library) Additionally, some libraries recognize a 6 character barcode while others recognize 8 characters. A firmware upgrade may affect all your barcode labels in your existing library. CommVault provides the means to update media barcodes in all of the previously cited conditions. Individual media labels can be edited and scalable tools allow you to add/remove prefix/suffix for all media with the library.
Threshold Settings
Usage and errors tracked for each removable media Media evaluated before each write Media that exceeds thresholds is Deprecated and moved to Retired Media Pool Storage Resources thresholds provided by vendors
Threshold Settings
Media that exceeds usage or error thresholds jeopardize data protection and should be removed from use. When the CommVault software evaluates scratch pool media for use, media that has exceeded thresholds will be automatically moved into the retired media pool the software tracks statistics for each tape and optical platter which can be seen in the odometer tab of the medias property page. The Storage Resources properties list the vendor recommended thresholds for cleaning and expiring media. These thresholds can be changed, but any change should be verified with the media vendor to ensure support and reliability of the media in recording or reading data correctly.
If the library involved is a blind library, then an inventory will be required when media is imported in order to discover and track the media within the library. When media is imported/discovered, the system assigns it in one of the following ways:
88 Media Management Module If it is a media associated with the CommCell, which was previously exported, it will be assigned to the scratch pool or a storage policy copy to which it was assigned before it was exported. If the media is a new media the system assigns it to a scratch pool based on its barcode if barcode ranges are assigned to scratch pools. If not, then the new media will go to the default scratch pool. It can then be manually moved to another scratch pool if needed. If a data protection operation requests an exported media, depending on the options selected for the library in the Media tab of the Library Properties dialog box, the following operation will be performed:
Automatically mark the exported media as full, and use a new media for the data protection operation. The job will remain in the waiting state, with the Reason for job delay stating that the media is outside the library. However, if a restore or auxiliary copy job requires data from an exported media, the system will prompt you to import the media in order to complete the operation. Information about exported media will be retained in the system; they do not have to be rediscovered if they are re-imported.
Exporting Media
Setting Export Locations Export by List Export using Vault Tracker
Exporting Media
Exporting is the process by which you physically remove one or more media from a library. If a media is reserved by a job for a read or write operation, the media cannot be exported. Media that is left in the mail slot after an export operation will be treated as an exported media. Pop-up messages associated with media outside the library and the media information in the CommCell Console will indicate that the media is in the IEPort associated with the mail slot. You can use Vault Tracker and Media Repository in Vault Tracker to manage media residing outside the library. There are two ways to remove the media from the library: You can remove the media through the librarys mail slot (if available and supported by the library). You can open the library door and remove media from the storage slots by hand.
Removing media through a mail slot offers the following advantage: The inventory update that is triggered by a mail slot export is much less time-consuming than the full inventory operation that is triggered when you close the library door. However, under certain circumstances you may want to open the library door even though a mail slot is available. For example, if you want to remove many media from a library at once, it may be faster to open the door than to use the mail slot.
90 Media Management Module Setting Export Locations When a media is exported an optional entry to specify the storage location for the media is provided. This information, can be viewed or updated in the Media Properties dialog box. This feature helps you to keep track of the exported media. The exported media are displayed with the location in the Exported Media pool in the CommCell Console. Export by List Media can be selected from a list of media and exported. This is useful when you wish to export several media at the same time and know either the medias barcode, or slot number in which the media resides. This task can be performed from the following levels in the CommCell Console: Library Media in Library Media List dialog box which appears when you select the View Media option. This dialog box can be accessed by right-clicking a Storage Policy copy. Media List dialog box which appears when you select the Change Data Path option. This dialog box can be accessed by right-clicking a Storage Policy copy. Media List dialog box which appears when you select the Media Not Copied option. This dialog box can be accessed by right-clicking a Storage Policy and Storage Policy copy.
Export using Vault Tracker Media can be exported automatically using Vault Tracker. Vault Tracker jobs can be initiated by a Tracking Policy, Data Protection Job or Auxiliary copy job.
Consolidates exported media to a specified block of internal storage slots for ease of removal
Virtual Mail Slots
Exported Media
Library
Library
Using Containers
Pre-configured or dynamically created Assigned via Vault Tracker or Media Properties Contents viewed from Vault Tracker
Using Containers
Containers are used to logically group exported media into a single addressable unit. This reduces the number of objects to track and facilitates movement of a set of media in to, or out of, third party offsite storage when a cost is assigned to each unit. Hence, ten media placed in a single container for storage would be a single unit cost in both directions. Some container facts: Containers can be pre-defined by the user or dynamically defined by a Vault Tracker Job. Each container will report the volume of data it contains. Exported Media Properties will report which container the media is in. The contents of each container can be viewed. List Media will identify the required media, container, and location.
Production Library
Container
Transport
Export Location
Picked PickedUp Up
Identifies media that must be sent off-site for storage or brought back from off site locations. Automatically moves the media in sequence in the library and provides a pick-up list for the operators. Facility to identify and track media during transit. Facility to record and track the movement of media not used by the Media Agents in this CommCell or Foreign Media.
By a Tracking Policy initiated on demand or scheduled By the Advanced Options of a Data Protection job By the Vault Tracker Options of an Auxiliary Copy job
The source location is other than the library the media was last written to.
The media movement is not directly related to a particular auxiliary copy or data protection job. The media to be moved comes from multiple sources.
Exported media monitored by a Vault Tracker job is reported as being in one of three states:
At Source Media is in a static location or library At VMS Media is in the Virtual Media Slots awaiting removal In Transit Media has been removed from a library or has been marked as Picked up from a location other than a library.
Departure and arrival of media for a library is automatically recorded by the librarys slot inventory mechanism. Hence media removed from a library source is automatically assumed to be in transit (if a transit location is specified) or at the specified export location. Media departing an export location must be manually flagged as removed from that location. It can be marked Picked Up to put it In Transit (if a transit location is specified) or optimistically at its destination (new source). In Transit media, or exported media that has not been marked Picked Up because you want to confirm its arrival at its intended export location (new source), can be marked as Reached Destination. Individual media in a transit state between two export locations that, for one reason or another, requires the return of the media to the originating source can be marked Return to Source. Alternatively, a Vault Tracker job that moved media between to export locations can be rolled back to indicate return of all of its media to the source location. Exported media that can be re-used as spare media can be tracked/scheduled with a Due Back tracking policy. When creating a Due Back tracking policy you can include all media that will be recyclable within a specific number of days. For example; you can create a weekly scheduled Due Back tracking policy that includes all media that can be recycled within the week (7 days).
Importing Media
Discovering new Media Verifying Existing Media
Importing Media
Importing is the process by which you move media that are outside a library into storage slots within the library. There are two ways of importing media:
You can import media through the librarys mail slot (if available) You can open the library door and manually insert media into storage slots in the library
Import Media through the Library Mail Slot Importing media through a mail slot offers an advantage - the inventory update that is triggered by a mail slot import is much less time-consuming than the full inventory operation that is triggered when you open and then close the library door. Directly Insert Media by Opening the Library Door Under certain circumstances you may want to open the library door even though a mail slot is available. For example, if you want to add many media to a library at once, it may be faster to open the door than to use the mail slot. Once again, the media will be automatically discovered, if the Enable Auto-Discover option for the library is enabled.
Media Management Module - 97 If this option is disabled, the media will be displayed (with a '?' icon) in the Media in Library pool in the CommCell Browser. You must subsequently discover the media, in order to use the media in the library.
Media Management Module - 99 The discover media operation mounts the media into a drive and if a valid OML is not found, it writes the On-Media-Label (OML) on the media. Once unmounted, the system also keeps track of the media location. A discover inventory job is displayed in the Job Controller window and can be killed if necessary. Also note that both the full and quick inventory processes will automatically discover new media whenever a new media is found. For a Standalone Library there is no discovery process for media. A tape is loaded into the drive and an OML is written only when a job is initiated. The media label can be automatically generated or manually input. Media labels can also be pre-configured by the Stamp Media Operation. If the media being imported was previously used with another application CommVault will not automatically detect that in a barcoded library. The media will be placed in the scratch pool and be available for jobs. Once that media is selected for a job, the system loads the tape and checks for a header. If we do not recognize the format or header information as one of our own the media is placed in the Foreign Media pool for the Administrator to deal with. In order to use this media the tape has to be either formatted or the option to Overwrite Media When Content Verification Fails must be selected from the library properties.
Module Summary
Summary
Data is stored in Chunks, 1GB for all data on Magnetic media, 4GB for indexed data to tape and 16GB for Database data to tape. Magnetic Media is faster for writing and recovering data. Tape media has its advantages in being a low cost storage media that is easily transportable for offsite protection and disaster recovery. Parallel streams are written to media simultaneously to separate device streams or media, while multiplexed streams interlace the data into a single device stream or single media. Pros and Cons of mixing retention on a single media. Using basic retention vs. extended retention. Appendable media allows for most efficient use of media. Vault Tracker gives the ability to manage media outside of the library.
Storage Policies
www.commvault.com/training
Overview
Understanding Storage Policies Data Retention, Management, and Aging Roles of Verification, Encryption and Content Indexing
Storage Policies Module Objectives: List and describe the function and role of Storage Policies in managing protected data. List and describe the common settings and options available for a Storage Policy and Storage Policy copy. Describe how data is retained, managed, and aged within a Storage Policy. List and describe the options for performing Verification, Encryption and Content Indexing on managed data.
Copies
Storage Policies Module - 107 Define Content Indexing engine, options, and source for offline content indexing.
The only permitted relationship between storage polices is through a linkage called Incremental Storage Policy. When a standard storage policy has an incremental storage policy enabled, full data protection jobs are stored and managed by the standard (or parent) storage policy, while all non-full data protection jobs are stored and managed by the associated incremental storage policy. This configuration can effectively use each media types concurrency, capacity, and associated costs to store data. For example; you can direct large, less frequent full backups to tape media and smaller more frequent non-full backups to magnetic media. Any standard storage policy can have, or be an incremental storage policy. A single storage policy can be used as an incremental storage policy for multiple standard storage policies. Philosophy of Storage Policies Conceivably, a single storage policy can be used to manage all data for a company. Different data retention and media storage requirements can be accounted for via configurable characteristics of secondary copies. However, in a practical sense, using a single storage policy for complex data management requirements is not recommended. The primary reason for storage policies and the reason for using different storage policies is ease of data management. Data boundaries can be identified along the lines of data type, data source, and management requirement. Additionally, business, legal, and security requirements need to be considered when designing a durable storage policy schema. Taking the time to map out a storage plan will yield both more efficient and easier management for your data.
Primary Copy
Synchronous z Selective
z
Secondary Copy
Primary
MediaAgent
Auxiliary Copy
Secondary
110 Storage Policies Module operation, they are done so in the chronological order in which they were added to the storage policy. An auxiliary copy operation copies only Valid Data it does not mirror media (i.e. tape for tape). Valid data is defined as data from a data protection job that has not failed or been killed. If a data protection job should not (or can not) be copied, it must be manually marked as Do Not Copy in the source copy Job View. Since a source copy can contain data from failed or killed data protection jobs, an auxiliary copy operation may result in consolidation of the copied data on fewer medium. In fact, auxiliary copy can be used as a consolidation method to move only valid data to another copy in order to delete the less efficient source copy. If the source copy has multiple streams, they are, by default, auxiliary copied serially - one stream at a time. If there are multiple streams on the source, you can enable an auxiliary copy job option to copy a number of those streams in parallel. The parallel copy option enables the use of maximum possible or a limited number of data streams as defined by available resources. While performing auxiliary copy operations, priority is given to using LAN-free read data paths. This means that if multiple source and destination Media Agents are involved, the Media Agent with the most LAN-free read paths will be used.
Client
Media Agent
Tape Library
10mb/ sec
10mb/ sec
Parallel
Multiplexing
Best Practices It is recommended that you keep the following in mind when performing data multiplexing: CommVault System Administration Course R02
Use different storage policies for file system and database type data before performing data multiplexing. Therefore, there wont be differences in the chunk sizes of the different types of data. If possible use the Restore by Jobs option to restore multiplexed data, especially when restoring large amount of data. This will provide the optimum performance during the restore operation as there are fewer tape rewinds to secure the data. It is recommended that you perform data multiplexing for jobs that have similar speeds (i.e. two database jobs), instead of mixing faster jobs (i.e. file systems) with slower jobs (i.e. databases). Mixing faster and slower jobs results in data stored on media that is not uniform. Hence, data recovery operations of slower clients will have added performance penalty. Multiplexing is recommended if you are planning to recover: Individual items, files and folders. Entire computers or databases. It is not recommended under the following conditions: If you are planning to recover scattered folders, since multiplexing will further scatter the data. Also it adds to up to extra tape mounts and rewinding/forwarding on the media. Clients which undergo very frequent restore requests. The multiplexing factor is determined based on the ratio of how fast the tape drive is compared to the disk. For example, consider the following ratios: Tape write speed = 75 GB per hour Disk read speed (backup) = 25 GB per hour Tape read speed = 100 GB per hour Disk write speed (restore) = 50 GB per hour Tape write speed/disk read speed (backup) = 100/25 = 4 Multiplexing Factor Tape read speed/disk write speed (restore) = 10060 = 2 Multiplexing Factor It is recommended that the lower of the two ratios(2 Multiplexing Factor) should be used as the multiplexing factor if you want no-penalty data recovery operations.
Combine Streams
Auxiliary Copy A B C Auxiliary Copy A B C
A B C A B C
Magnetic Library
A B C
Magnetic Library
A, B, C
Combine to Streams
Whether copying in serial or parallel fashion, an auxiliary copy operation normally copies each source data stream to a separate destination stream. This means if there were four data streams on the source copy, then the auxiliary copy operation would use four data streams on the destination library. Each stream requires its own destination media. In some cases such as copying from a multiple stream magnetic library to a limited device tape library, this may not be the most efficient or desired method of copying data. As such, the destination storage policy copy has an option to Combined to <n> Streams. Combining streams is done serially and is used to improve media storage efficiency not throughput. The Combine to <n>streams options is considered a media management tool and is configurable only on secondary copies. Example: 4 subclients with 2 data readers configured for each subclient will have a total of 8 streams. Setting the Combine to Streams option to 2 will result in a 4 to 1 consolidation and the 8 streams will be combined onto 2 tapes.
Spool Copy (No Retention): sets the retention rule on a primary copy to 0 days and 0 cycles.
Storage Policies Module - 117 Basic Retention Rules for Data/Compliance Archiver Data The amount of time that you want to retain the data that is archived via this storage policy copy. Select from the following options: Infinite Retains the data for an infinite amount of time Retain for n days
Allows you to set the length of the retention period. Extended Retention Rules for Full Backups For Infinite or n Days Keep: All Fulls - All full backups Weekly Full - The first or last full backup of every week (each week starts on Friday) Monthly Full - The first or last full backup of every month (each month starts on the first of the month) Quarterly Full - The first or last full backup of every quarter (each quarter starts on January 1, April 1, July 1, and October 1.) Half Yearly Full - The first or last full backup of every half year (each half year starts January 1, and July 1) Yearly Full - The first or last full backup of each year (each year starts on January 1)
Customer Confidence
Company Requirements
Risk Assessment
Retention Policy
Reputation
Government Requirements
Industry Requirements
Cost
You should be aware of the regulations governing your particular industry. Ignorance of the law will never pass as an alibi. Companies run the risk of paying hefty penalties for noncompliance for reasons including: Lack of awareness De-prioritizing non income-generating projects
Although experts continue to discuss and translate the ramifications of recent rulings, certain concepts are clear: CEOs and CFOs assume authority and accountability for accuracy of records, and CIOs or IT divisions ensure effective implementation and technical compliance. The responsibility to "get it right" falls largely on technology professionals.
Backup
Cycles z Days
z
Archive
z
Days
Example
Cycle 1
12 noon 12noon
Basic Retention set for 2 Cycles/14 Days Full backups scheduled every Sunday Incremental backups scheduled all other days
Data Aging
Frees storage media for re-use For data that exceeds retention
z z
Erase data on magnetic storage (default) Use managed disk space to preserve data in magnetic libraries Flags data on removable storage as Aged Data until overwritten or erased
Data Aging
Unless you have an unlimited storage budget, youll want to delete all protected data that is no longer required and free up media for re-use. This is accomplished through the Data Aging operation. Retention and conditional criteria is evaluated and those jobs or cycles that no longer meet this criteria can be Pruned. Pruning will either flag or remove data pointers from the CommServe database and, as appropriate, erase (magnetic) or reclassify (removable) media to spare status. Note that reclassifying tape media to spare status can only be done when all the data on a specific media is pruned off. While retention criterion is the main factor evaluated by a Data Aging operation, there are many conditional criterion that may exclude a job from being pruned. These include such things as transaction log jobs where the associated database is still retained and data that has not been auxiliary copied to a secondary copy as required. Consult Books Online for conditions that affect each data type. These conditions are not intended to impede re-use of media, but rather ensure complete restoration can be accomplished as expected. Data aging is performed on all active storage policy copies within the CommCell. Storage policy copies can be excluded from data aging by clearing the Enable Data Aging option on the Retention tab of the copy properties page. Different data aging schedules for each Storage Policy copy can be configured in the Aging Options tab of the CommCells Data Aging task configuration dialog window.
Storage Policies Module - 123 Data on a magnetic library is classified as Managed when it exceeds retention requirements, but storage usage does not exceed the managed disk space threshold limit. Managed data is treated the same as retained data for browse and restore. Disk space is periodically checked for used capacity. When the managed disk space threshold is exceeded, the managed data is erased oldest first until the lower threshold limit is reached. Tape media is serial vs. random access and cannot be overwritten until all jobs on the media have been flagged as Pruned in the CommServe database. Until the tape media has either been specifically erased, or is overwritten through re-use, the Aged jobs on the tape can be browsed and restored. Unlike Managed data, the ability to browse or restore Aged data must be specifically enabled or selected. Aged data is accessible in a Restore by Jobs option and can be browsed for individual objects if the Browse/Recovery control option to Show aged data during browse and recovery is selected. To extend the availability of aged jobs on tape, you can mark a specific tape as Prevent Re-use which will move the tape to the Overwrite Protect Media pool
Retention Variants
Data not copied to secondary storage Transactional log data Failed jobs Manually retained/disabled/pruned Jobs De-configured or re-associated subclients Incremental Storage Policy
Retention Variants
There are variants to the Retention/Aging rules. These variants exist to ensure every reasonable expectation for recovering data is met. Some of the more prevalent variants are: Data not copied to secondary storage If an active secondary copy exists within a storage policy, the expectation is that the administrator wants to have a copy of the source. Hence, until a copy is made, the source data cannot be aged. If this is a problem, a solution would be to inactivate the copy. However, doing so will allow the source data to be pruned without a copy being made. Transactional log data Transaction\archive\logical log (log) backups are not considered part of the backup cycle. Therefore, storage policy cycle retention parameters do not apply to them and have the following unique data aging rules: Logs older than the oldest non-prunable data are prunable regardless of the retention rules of the copy. For logs that are more recent than the oldest non-prunable data, these logs are not prunable if: o They have not exceeded the retention days criteria. o They reside on a primary copy and have not yet been copied to a secondary copy.
Storage Policies Module - 125 o They reside on the copy that has the longest days retention criteria amongst the other synchronous copies within the storage policy. The log data can be valid, partial, or disabled. If the status is partial or disabled, the valid copy of the log is kept on the copy with the longest days retention criteria among the remaining copies. For each backup copy of the database at least one copy of associated logs will be retained. Those logs will be retained on the copy with the longest retention value setting. Logs are retained beyond that copy's retention settings as long as the associated backup copy of the database exists. Logs on lesser retention value copies are pruned in accordance with the set retention value. Restoring a database with transaction logs requires that you either (a) Do not select a copy precedence for restore, or (b) you select a copy precedence that has the transaction logs. In the case of (a), the restore process will automatically determine which copy holds the required logs and restore those logs. Failed Job data Failed job data on magnetic libraries will be deleted by the Data Aging operation. Failed job data on removable media is immediately eligible for aging, but since it cannot be deleted, it can be restored using the Restore by Job option or restored with the Media Explorer tool. Manually Retained/Disabled/Pruned jobs Individual jobs can be selected for manual retention, disabled for copy, or pruned for data aging. Manually retained jobs are excluded from normal data aging. If a removable media is being prevented from moving to a scratch pool by a manually retained job, it will be indicated as such in the Data Aging and Media Forecast report. Jobs may be disabled for copy by the administrator if an archive copy job fails because the job has corrupt data. Jobs can be pruned manually before their normal data aging event. Pruning a Full backup job will include all dependent incremental or differential jobs. Pruning jobs manually is usually done to free up media. De-configured or re-associated Subclients Subclients who have been de-configured or re-associated to another storage policy can no longer generate full backups to the original storage policy. Still, the data may be required for recover. Galaxy tracks when a storage policy association is changed and knows where the pertinent data is kept. However, the retention criterion of cycles on the original storage policy is ignored and only retention days are used to determine the data aging eligibility. Incremental Storage Policy If data aging is performed on a storage policy that has an incremental storage policy enabled, the data aging operation counts backup cycles across both full and incremental storage policies. The
126 Storage Policies Module data on a full policy is pruned based on the retention of the full policy, and data on the incremental policy is pruned based on the retention rules of the incremental policy. If the incremental storage policy is also being used as a regular storage policy (and has full backups), the full backups will be also pruned according to any basic and extended retention rules that are set. It is recommended that the retention rule for the full storage policy be greater than the incremental storage policy. Data on incremental policy will be pruned earlier if it has shorter retention comparing with full policy. If the incremental storage policy has longer retention than a full storage policy, this may result in dangling incremental jobs.
Encryption
Inline
Subclient A C B
A,B
Copy-based
Subclient A B C
A,B C
Magnetic
Encrypt C
Magnetic
If you are concerned that media may be misplaced, data can be encrypted before writing it to the media and the keys stored only in the CommServe database. In this way, recovery of the data without the CommServe is impossible - not even with CommVaults Media Explorer. This No Access mode is also completely transparent. Once enabled, it will work requiring no additional activity on your part. If you need only network security, the encryption keys are randomly chosen for every session. Data is encrypted on the Client and is decrypted on the Media Agent and the keys are discarded at the end. The entire process is completely transparent. All you have to do is to enable encryption, and select the cipher and key. Encryption keys can also be protected with your own pass-phrase (RSA algorithm with 1024-bit keys) before being stored in the database. If the database is accessed by unauthorized users, and the media is stolen, the data will still not be recoverable without the pass-phrase. This highest level of security comes at the price of having to enter the pass-phrase for every recovery operation and not being able to run synthetic full backups. You can also export a file that contains the scrambled pass-phrase of the client computer to a dedicated directory on another computer, the system can recover the clients data to that (and only that) computer without prompting you for the pass-phrase. Explicitly enabling synthetic full backups in the GUI will create a copy of unlocked encryption keys in the database, which will be accessible only to synthetic full data protection operations. In this case the regular data recovery operations will still prompt you for a pass-phrase, but synthetic full data protection operations will not.
Content Indexing
Production (online) Copy-based (offline)
Content Indexing
Overview Content indexing adds the ability to find files and messages by searching their contents for a keyword or phrase. Before you can search by content, a content index must be created. Content indexing is not installed or activated by default in order to ensure efficient use of your resources. Content indexing can be configured on your Production server (online) or on your protected storage copy (offline). Production (online) The online content indexing module is used to index the live data on Windows computers. The Content Indexing Agent and Data Classification Enabler software must be installed on the computers in the CommCell you wish to content index and search. Once installed the content index function must be enabled and scheduled.
Copy-based (offline) Offline content indexing module is used to index the storage data secured by various data protection operations. For this reason the configuration of the offline content indexing is associated with a storage policy. The Offline Content Indexing is installed along with the Content Indexing Engine and does not require separate installation.
ONLINE
Client OCI Agent CI Store
OFFLINE
Storage Policy Copy Media Agent
Disk Tape
Objects Files Messages Documents
DCE Metadata
Search/Find
CommServe
Web Search
Index
132 Storage Policies Module partition. A client with an OCI agent installed must have network access to the CIS conducting the index. Search Content Indexes can be used by the CommCell Consoles Find and Search features, CommVaults Outlook Add-in and Archive Mail Browser or via the web using CommVaults Web Search Server. The Find functionality is used without content indexes to locate protected storage objects by name across multiple storage location indexes. The CommCell Consoles search feature is available at the Backup/Archive Set, Agent, Client Computer, Client Computer Group, and CommCell level. The level you choose to conduct the search defines the initial scope of the content index to be used. Search options allow you to precisely define your search criterion and scope. CommVaults Outlook Add-in has a search function that uses content indexes for Exchange messages. This search function is also available as a standalone feature called Archive Mail Browser for hosts without Microsoft Outlook. If CommVaults Web Search Server (WSS) is installed and configured, two more search capabilities are added which are accessible via the web. End User Search can be used to search for both offline and online data owned or read accessible by the logged in user. Compliance Search can be used by legal or corporate IT to conduct a search of all indexed data online and offline regardless of who owns the data. Web Search queries and result set can be saved in a SQL database for later use. Objects returned by a key word search are displayed with either dynamic or static teasers. A dynamic teaser is the text data located near the found keyword. A Static teaser is the first few lines of text from the object itself. Objects identified during an Online search can be opened or saved if a share name path is configured and available to the data. Objects identified during an Offline search must be saved as a result set and then selected for Prepare for Viewing to initiate a restore operation. Selected objects are restored to a temporary cache where they can then be selected for view or saved.
Data Verification
Verifies that all selected data is valid for restoring and for being auxiliary copied Data Verification can be scheduled or executed on demand Verification options can be set on Storage Policy copy User can also select individual backups for verification and/or select individual media for verification
Data Verification
During a data verification operation, protected data is checked to see that it is available and readable. CommCell metadata is used to determine where the data is stored, what chunks are used and whether the media and chunks can be located and successfully read. Short of restoring the data, this is the best way to verify accessibility of protected data. You can verify all or selected jobs on a specific copy or media. Verification is a schedulable job and can be run serially or in parallel for multiple stream copies. Data Verification is accomplished by selecting a storage policy. All copies in the storage policy can be verified or only a selected copy. Data verification can be run to check if valid data can be successfully auxiliary copied. On the other hand, if an auxiliary copy operation is successfully run, the data can be considered verified.
Module Summary
Key points to remember
Summary
Purpose for Storage Policy Copies - Primary (backup), Sync (Specify by client, takes all FID), Selective (specify by client fulls only, good for remote sites with little resources) Retention can be defined as Cycles and Days, Days only, Basic (cycle and days, Infinite, Spool) Extended (weekly, monthly, quarterly, yearly). Content Indexing can be configured at the Primary storage (production data) or Secondary storage (protected data). Data Aging - When does it happen, Why does it happen, What does it do? There are two types of Streams - Job based (from client), Device which is defined at the Storage Policy.
Clients
www.commvault.com/training
Overview
What are Clients? Understanding the role of Agents Backup and Archive Sets Configuring Subclients
Host system
Physical machines z Clustered Systems and Applications z Virtual machines
z
Properties
Agent Roles
File System z Application z Archive
z
Clients Module - 139 information to locate the archived object in protected storage. Native or client mechanisms are provided to facilitate the automatic and transparent recall of object when the stub is accessed by a user or application. Indexed or Non-indexed Based An index based agent provides granular recovery of individual objects based on an index maintained by CommVault (e.g. Files, System State objects, E-mail messages, Documents). Non-indexed based agents are ones where the degree of granularity is insignificant and the need to quickly protect or restore the entire data set for consistency purposes is paramount (e.g. Database Instances, Image level agents).
Logical grouping of all subclients Provides for Distinct Management Defines Data access and/or Collection Method
Backup/Archive set
Subclient Content
User-defined backup sets provide multiple management options for the same set of data. When a new user defined BackupSet is created, it becomes a complete collection of data on the client for the specific Agent.
142 Clients Module The advantage to using multiple backup sets is when customized backups need to be performed that do not conform to the current subclient design within the default backup set. Additional backup sets can be created and scheduling can be set up to only backup these subclients when needed. Note that duplicate data within separate subclients in separate backup sets can not be backed up simultaneously. Conflicting operations can cause jobs to go into a pending state until running jobs are completed. Caution should be used when creating multiple BackupSets. Improper configuration and scheduling can result in failed jobs or data being backed up multiple times. All Archiver Agents support the ability to create additional archive sets, i.e., user-defined archive sets. This feature is useful for archiving client data using different archive options or schedules, and for workload balancing. If there is an overlap in the data archived between various Archive Sets on the same client, data must be retained on all the storage policies used by various subclients in all Archive Sets on the client in order to recover data.
Dynamic content defined at backup time Supports only one subclient Can be executed from command line Not used in Full System Restore
The On Demand Instance is a logical entity that allows Oracle data to be protected on demand, with content specified in a RMAN script that is run through the Command Line Interface. The primary difference between an On Demand Instance and a traditional Instance, is that data protection operations for the subclient associated with an On Demand Instance only supports On Demand Data Protection operations.
*.txt >1G
146 Clients Module very conducive to categorizing data for classification purposes. By eschewing traditional folder/file boundaries and by searching for common attributes, the Data Classification Enabler provides a much more powerful treatment of data. The Data Classification Enabler also provides the following capabilities: It can be installed independently of any agent that uses it It minimizes the amount of scan time needed to search for data by querying the meta databases that have been created by the Data Classification services. Data Classification is highly fault-tolerant. For example, if a meta database is deleted while the Data Classification services are running a scan of this database, the services can run another scan of the database. Also, if you disable Data Classification during the initial data scan, the database will restart the initial scan. Finally, if you add a new volume after the Data Classification Enabler is installed, the new volume is automatically detected, and a new database is created.
Support for the Windows File System Agent The Windows File System Agent can use the Data Classification Enabler to scan data before backups. The availability of the Data Classification Enabler for data scanning depends on a number of factors including the data type. If the enabler is not available for data scanning, another scan method is used. To use a Data Classification Enabler scan, you must ensure that the Data Classification Enabler software is installed on the client. Data Classification uses the traditional rules established for the File Archiver for Windows Agent. In addition, it provides several unique rules. These rules are available only when Data Classification is enabled for the supported agent. All the rules for Data Classification are configurable from the DataClassSet subclient properties Rules tab of the File Archiver for Windows Agent. The unique Data Classification rules include the following: Folders/Files Owned By - Allows you to select and exclude files belonging to specific users and user groups File Paths - Allows you to select and exclude files based on file location and specific file characteristics (e.g., file extensions) SQL Query Strings - Allow you to use SQL query-like commands to define more complex rules based on your requirements
Whenever a DataClassSet subclient is created, a default set of rules is established. These rules are reflected in the various rules tabs. Also, the SQL query string, which has its own tab, is automatically disabled; however, the default set of rules is formulated into the SQL query string. At this point, you can start changing rules either from the various rules tabs or within the SQL query string.
Configuring Subclients
Defining Content Options Using Filters Pre/Post Process Commands Setting Storage Options Using Subclient Policy
Configuring Subclients
If all data had exactly the same protection management requirements there would be no need for subclients. But all data is not created equal. Some data needs to be retained longer while other data needs to protected more frequently. Some data may need to go directly to tape media for immediate offsite storage and others - not. The functionality of subclients allow you separate content and define its management options as required.
Content is mutually exclusive within a backup set Can use wildcards in defining content
z
Archive Content
Using Wildcards to Define Content Regular expressions (or wildcard characters) can be used to define content in user-defined subclients. Wildcards expressions are characters such as * or ?. Regular expressions include patterns such as [a-f] or *. [l-n]df. The terms are interchangeable. The use of regular expressions in defining content is limited to the last level in the data path. For example: F:\Users\[A-L]* is an acceptable use of regular expressions in defining content, whereas *.pst is not an acceptable use. For File System subclients, the use of regular expressions requires the explicit enabling of the subclient option to Treat characters as regular expressions. This is required for some operating systems (for example: MAC OS), because certain characters such as * or ? can be part of a legitimate file path or name. Keep in mind that this option does not appear in the default subclient since use of regular expressions in the default subclient is not supported.
Using Filters
Global Filters
z
Local Filtering
z z
Using Filters
Filtering unnecessary data from data protection operations can reduce backup time, storage space, and recovery time. Most, but not all Agents include some filtering capability at the subclient level. File systems' subclient filters can be defined for a path, directory, or file level. The exception is NAS NDMP Agents which only support file names or directory names. Some application Agents such as Exchange Mailbox, Lotus Notes Database, and Lotus Notes Document also allow filters to be defined. There are four basic types of filters that can be used: Exclusion filters can be defined at the subclient level and exclude data from being unnecessarily protected. Exception filters can only be defined at the subclient level for supported agents. An Exception filter allows you to define directory or file exceptions to a filter defined in the exclusion section. For example, you can "exclude" the C:\Temp directory with the "exception" of C:\Temp\log directory. In some cases, wildcards are supported in the Exception filter. Subclient Policy filters are made up of Exclusion and Exception filters and applied by a subclient Policy. You can elect to not include subclient Policy filters in a
Clients Module - 151 subclient and thereby define your own Exclusion and Exception filter for each subclient. Global Filters are made up of Exclusion filters that can be defined for each operating system such as Windows, and Netware, and for the Exchange application. Both Linux and MAC OS fall under the UNIX global filter. Filters defined at global level can be included or excluded in a subclient filter definition.
Exception: Netware
No timeout limit
Keep in mind that there are a couple of scenarios where a post process will not be executed for all attempts to run the job phase, even if this option is selected: when a user suspends/stops the job during the respective phase, or if the job is interrupted during that phase due to Operation Window rules. If you encounter either of these conditions and your post process is designed to bring a database online, release a snapshot or execute some other critical process, you may want to manually perform these tasks depending on the situation. NetWare will not wait for pre/post processing commands to complete, unless a command delay is configured for that process. If a command delay is not configured, NetWare will immediately move to the next phase of the backup job after launching the command. Also, command failure will not prevent the next phase of the backup job.
Single Instancing Single Instancing identifies the duplicate items in a data protection operation and maintains references for the duplicate items. When a data protection operation is performed for the first time all the data is stored; if the same data is subsequently identified in another data protection operation, reference counters are incremented and the data itself is physically not stored in the storage media.
Single Instance Storage is currently supported for magnetic libraries. Note that there are appliances that support single instancing of data (e.g., Centera). Such hardware can be configured as a magnetic library with the option to write the data in a singleinstance format enabled. Single Instance Storage is supported by both backup and data archival products, specifically for the file system and email attachments, in order to provide optimization in storage when copies of the same data is backed up and stored.
Module Summary
Summary
Clients are systems that contain data to be protected and managed. Clients must communicate with CommServe and MediaAgent to be protected. Backupsets and Archivesets are used to logically group subclient content making data recovery easy. You can define data for various user types independently of where the data is located, by using the Data Classification Enabler. Some common property settings for subclients include defining content, establishing content filters, using pre/post processing commands, and selecting storage options. Subclient policies allow you to define multiple properties in a common template and attach this policy to appropriate clients.
Job Management
www.commvault.com/training
Overview
Configuring Data Protection Tasks Executing Data Protection Tasks Task Management Tools
Backup
z z z z
Job Management Module - 163 When a normal backup operation is executed, an image file is generated. This image file contains a list of all objects within the scope of the subclients defined contents excluding filtered files. These objects are collected from data already in protected storage and written back to protected storage as an equivalent full data protection job. Actual scheduled protection is not achieved until an incremental backup from the client is completed. The synthetic full task includes an option to run an incremental backup just before or after the synthetic full is run. A synthetic full may be used when a normal full backup cannot be reasonably performed due to client or network resource limitation. Archive Archiving has no associated concept of Full, differential, or Incremental collection of a subclients content. Defined content is evaluated against a set of rules and those objects which satisfy the rules are archived. Archiving has the two major functions of freeing up production disk space by moving less frequently accessed objects to less expensive storage and managing the accessibility of data in alternate storage over extended periods of time.
Automated System Recovery (ASR) Automated System Recovery (ASR) is a tool that speeds the recovery of Windows XP and Server 2003 client computers by automatically reinstalling the operating system and restoring the hardware configuration settings, system state, program files as well as the Documents and Settings folder. The hardware configuration of the client computer is restored directly from a user-created ASR floppy. The other backed up data is restored from tape media. This procedure does not completely restore the File System, other applications, or application data, which should be restored with the appropriate Agent(s).
Full
Inc Tape 1
Inc
Inc
Primary Storage
Secondary Storage
Tape Storage
166 Job Management Module data for audit and compliance. Search can be completed by keyword, phrase, or by searching in the To, From, CC, and BCC fields. The second component is Migration Archiving. There are agents currently available for Microsoft Exchange Server Mailbox and Public Folders, Microsoft SharePoint, Novell Netware File System, Unix/Linux File System, Windows File System and File Shares. The purpose of Migration Archiving is to take unused files and remove them from the primary storage, freeing space and decreasing backup times and space. While Compliance Archiving for Exchange merely creates an archived copy of an email, Migration Archiver actually removes the e-mail or file from the server, and leaves a stub in its place. The stub contains the location of the file in storage, and instructions on how to retrieve it. Recalling migrated files becomes a very simple process. The file or e-mail can be recalled seamlessly by the user by simply opening the file or email. This triggers an immediate recall of the file from storage. Files may also be recalled by an authorized user via browse and recovery from the CommCell console. Even though the original file is being replaced by another file (Stub), space is made available because the stubs are only 4-6 kb in size. The available space is reclaimed immediately.
Schedule Policies
Job Management Module - 169 If you have a large number of clients/backup sets/subclients or storage policies in your CommCell that require the same backup or auxiliary copy schedule, you can manage their schedules through a Schedule policy. Schedule policies allow you to associate a schedule or groups of schedules to any number of clients/backupsets/subclients or storage policies within your CommCell. There are two types of schedule policies, Data Protection and Auxiliary Copy. During the CommServe install the System Created schedule policy covers all supported agent types. Additional schedules can be created to support individual agent or all agent types. A schedule policy allows you to define a maximum of six schedules, and associate them by agent type to any number of client computer groups, client computers, backupsets, or subclients. You also can define the type of schedule such as Data Protection operation or Auxiliary Copy operation, set up an alert, as well as configure Advanced Options as described in the Advanced Backup/Archive options section. In the case of an Auxiliary Copy operation, the Schedule Policy association is to the Storage Policy. Decoupling is used to remove a subclient from its association from the Schedule Policy, but does not delete the scheduled actions. Decoupled scheduled actions are treated as individual actions and can be edited or deleted as such. Command Line The Command Line is useful for incorporating Data Protection and Recovery operations into your own scripts and scheduling programs. Using the Command Line interface you can create an action using a single command with arguments or create scripts that include an answer file. An easier method may be to use the CommCell Browser to generate a script and answer file using the options you select. The commands are integrated with the base component installed with all agents and are therefore available on all computers which have any CommServe, Media Agent, or Agent software installed. In order for the commands to function, the Galaxy Commands Manager service should be up and running on the CommServe. Automatic Automatic auxiliary copy operations are performed every 30 minutes to copy data to the automatic copy. These operations will only occur when new data that needs to be copied is found on the primary copy (source). You can change the interval at which the automatic auxiliary copy operation is performed on the job initiation tab of the auxiliary copy dialog box or by editing the Automatic Copy Schedule.
Job Retry Options Use the Total Running Time option to configure the maximum time a job should run before it is killed by the system. This is to prevent hung or delayed jobs from impacting future jobs. If the job is in a running state at the max running time set, it will be allowed to continue to run to completion. If at anytime after the max running time set a job phase fails and the job state changes to pending, the job will be killed. To specify the total running time option to include jobs in a running state as well, select the option Kill Running Jobs When Total Running Time Expires. Using this feature to set a total running time will ensure that current jobs will not conflict with upcoming scheduled jobs. Configure the total running time to kill jobs before the next nights backups are scheduled to start. Media Options Depending on the type of operation being performed (Full/incremental/differential), different options will be available on the Media tab. When performing full backup operations, the option to Create New Index will be available. By default this option is selected. If cleared the previous full index will continue to be used. Over time, the Index files can grow significantly larger requiring more space to backup and slowing down browse operations. If manually creating a new index, consider scheduling a monthly full job that will create a new index, and for each Full task during the month. This will allow you to browse up to the last monthly full, but will keep the Index Cache from growing unmanageably large. When configuring a backup operation the Start New Media and Mark Media Full options can be used to determine which jobs will be placed on a specific media. With careful planning and scheduling a set of data protection operations can be scheduled to run and all of these jobs can be placed on a single media. Use the Start New Media for the first job and Mark Media Full for the last job creating keystone operations. These settings are also useful when performing Auxiliary Copy operations. In 6.1 an option was added to Allow other Schedule to use Media set. This option is enabled by default, and can be disabled only if both Start New Media and Mark Media Full options are selected. Thus, if Start New Media is also selected, all jobs within the same schedule policy event will be allowed to use the same active media started by the first job. If the option to Mark Media Full is enabled, the system will wait 10 minutes after job completion to actually mark the media full. Any job (schedule policy or not) requesting to write to the media within the 10 minute time frame will be allowed to do so. Jobs within the same schedule policy event will not have that limit enforced since the Allow other Schedule to use Media Set option is enabled. Clearing the Allow other Schedule to use Media set option and enabling Start New Media and Mark Media Full will isolate each job on its own media as you should expect.
Up to 5 schedules can be included in a Schedule Policy so a reasonably complex protection schedule policy can be created.
Synchronous (wait for completion status) z Asynchronous (verify job submission only)
z
174 Job Management Module remain valid until you explicitly log out using the Qlogout command. This is true for all command sessions, not just the one for which the login took place. Saving a Job as a Script Data Protection, Recovery, Auxiliary Copy, Disaster Recovery Backup, and Data Aging operations and their selected options can be saved as scripted files, which can later be executed from the command line. This method of script generation will create 2 files, the first is the command file that configures the environment and executes the commands, and the second text based file maintains the responses also known as an argument file. These scripts can be saved in a Shell or Batch script format and include error checking. Script Considerations Consider the following before you save a job as a script: When you use the CommCell Console to save a script, only the options that are available for the corresponding QCommand are saved. For example, the Job Retry option is available in the CommCell Console for several operations such as backup and restore. However, this option is not available in the corresponding QCommand. Therefore, if you save a Script with Job Retry options selected, they will be ignored when the script is generated. When you use the CommCell Console to save a script, certain characters (for example, the left bracket [ and number sign #) in front of an object name (for example, #csmacs as the subclient name) may cause the script to not work correctly when it is run from the command line. If your script does not work from the command line, check it for these characters. You may need to rename the object by removing the characters, and then rerun your script. When you use the CommCell Console to save a backup script for a Microsoft Share Point 2003 Document Agent and run it from the command line, the script will only back up the latest version of a document. This will happen even when you choose the "Backup All Versions" option when generating the script.
176 Job Management Module whose timing cannot be pre-determined or detected electronically. When more than one job is in suspended state, the backup administrator can decide which operations should begin first. Job Retry Options Use the Total Running Time option to configure the maximum time a job should run before it is killed by the system. This is to prevent hung or delayed jobs from impacting future jobs. If the job is in a running state at the max running time set, it will be allowed to continue to run to completion. If at anytime after the max running time set a job phase fails and the job state changes to pending, the job will be killed. To specify the total running time option to include jobs in a running state as well, select the option Kill Running Jobs When Total Running Time Expires. Using this feature to set a total running time will ensure that current jobs will not conflict with upcoming scheduled jobs. Configure the total running time to kill jobs before the next nights backups are scheduled to start. Media Options Depending on the type of operation being performed (Full/incremental/differential), different options will be available on the Media tab. When performing full backup operations, the option to Create New Index will be available. By default this option is selected. If cleared the previous full index will continue to be used. Over time, the Index files can grow significantly larger requiring more space to backup and slowing down browse operations. If manually creating a new index, consider scheduling a monthly full job that will create a new index, and for each Full task during the month. This will allow you to browse up to the last monthly full, but will keep the Index Cache from growing unmanageably large. When configuring a backup operation the Start New Media and Mark Media Full options can be used to determine which jobs will be placed on a specific media. With careful planning and scheduling a set of data protection operations can be scheduled to run and all of these jobs can be placed on a single media. Use the Start New Media for the first job and Mark Media Full for the last job creating keystone operations. These settings are also useful when performing Auxiliary Copy operations. In 6.1 an option was added to Allow other Schedule to use Media set. This option is enabled by default, and can be disabled only if both Start New Media and Mark Media Full options are selected. Thus, if Start New Media is also selected, all jobs within the same schedule policy event will be allowed to use the same active media started by the first job. If the option to Mark Media Full is enabled, the system will wait 10 minutes after job completion to actually mark the media full. Any job (schedule policy or not) requesting to write to the media within the 10 minute time frame will be allowed to do so. Jobs within the same schedule policy event will not have that limit enforced since the Allow other Schedule to use Media Set option is enabled. Clearing the Allow other Schedule to use Media set option and enabling Start New Media and Mark Media Full will isolate each job on its own media as you should expect.
Activity Control
CommCell Client Level Agent Subclient
Activity Controls
Use the Activity Control tool to enable or disable all activity, data protection, data recovery, auxiliary copy, or data aging operations within the CommCell (regardless of the client computer from which they originate). If you want to control these operations that originate from a specific client computer or agent, use the Activity Control tab of the property sheet of the specific client computer or agent. The Activity Control feature allows you to enable or disable operations including scheduled operations at the following levels in the CommCell hierarchy: CommCell - Allows you to enable/disable all activity, data protection, data recovery, auxiliary copy, data aging and/or content indexing operations for all client computers within the CommCell. Client Computer Group - Allows you to enable/disable all data protection, data recovery and/or online content indexing operations on all client computers that are members of a client computer group. Client Level - Allows you to enable/disable all data protection, data recovery and/or online content indexing operations on a specific client computer.
Job Management Module - 179 Agent - Allows you to enable/disable the data protection and/or data recovery operations of a specific agent on a specific client computer. Online Content Indexing jobs can be disabled for Online Content Indexing Agents. Subclient - Allows you to enable/disable the data protection of a specific subclient. Online Content Indexing jobs can be disabled for Online Content Indexing subclients.
When disabling operations, the CommCell level has the highest precedence while a subclient has the lowest precedence. For example, if you disable data protection operations at the CommCell level, then all data protection operations throughout the CommCell are disabled regardless of the corresponding settings of the individual client computer groups, client computers, agents, and subclients. If, however, a data protection operation is enabled at the CommCell level, you can still disable data protection operations at the client computer groups, client computer, agent, subclient levels. By default, all operations are enabled at all levels of the CommCell hierarchy.
Operation Windows
By default, all operations can run for 24 hours. To prevent certain operation types such as Data Protection or Auxiliary Copy from running during certain time periods within the day, you can define operation rules using Operation Windows so that these operations are suspended during those times. The main purpose of this feature is to help you prevent an unexpected, time consuming operation from disrupting normal operations. Operation Window Rules can be looked at as Blackout Windows Operation Window rules are defined at both the CommServe and agent levels. Rules established at the CommServe level apply globally across the entire CommCell. Operation rules established at the agent level apply only to the specified agent. When an operation rule is defined at both the CommServe and agent levels, the job will run outside of the total time frame of both levels. Note that at the agent level: Not all operations are available to be assigned an operation rule, such as administration and synthetic full operations. You can also elect to ignore the operation rules set at the CommServe level from the Operation Window dialog box. The client time zone is displayed in the Operation Window Rules Details dialog box.
Job Management Module - 181 When using Operation Window, consider the following, there are two types of Agents Indexed base and Non-Indexed base. Simply stated, Index Based agents use the index cache and the Non Index Based Agents will not. An easy example of each might be index based types might include Agents managing granular objects within a job like a file system backup or mail system messages. Non Indexed Agents usually are database applications. A listing of these type can be found within Books Online. Jobs that are started at any time within the Operation Window rules or Blackout period will still be added to the Job Manager window in a queued state. Once the do not run interval has passed, these queued jobs will resume automatically. Jobs that are started before an operation rule can run to completion if the Allow running jobs to complete past the operation window option is enabled from the General tab of the Job Management dialog box. Jobs that are not interruptible (such as Non-Indexed based jobs) will not be terminated if they fall within the time an operation rule is defined.
Job Management Module - 183 job with a priority of 066 will run before a job with a priority of 366 if a resource conflict is an issue. Job Operation(s) Data Aging, Export Media, Inventory Data Recovery, Disaster Recovery Backups Data Protection Auxiliary Copy Data Verification Erase Media Assigned Priority 0 (Not applicable) 66 (leading zeros do not appear) 166 266 366 466
Restarts When a job phase fails it will normally go into a pending state awaiting an automatic restart. The frequency interval and number of restarts can be configured. The default is every 20 minutes and 144 times (48 hours). You can also set the option on some job types as to whether they are restart-able or preempt-able. Job Status Updates Job status information is periodically updated in the Job Controller window. The period of the update is based on time, or the amount of data moved whichever is less. You can reduce the period if you want more frequent status updates. However, this can have a negative performance effect on the job. Managing chunk size, found in the CommCell Console Control Panel under Media Management can also affect job performance.
Manage
Data Protection operations z Data Recovery operations z Administration operations
z
Job Management Module - 185 The state of the job reflected in the job controller window are self explanatory (e.g. running, waiting, suspended, completed). Listed below are the less frequently encountered but equally important: Dangling Cleanup -The job has been terminated by the job manager, and the job manager is waiting for the completion of associated processors before killing the job. Pending - The Job Manager has suspended the job and will restart it without user intervention. A pending job may be waiting. Queued - The Job Manager has queued the job and will restart the job only if the condition which caused the job to queue has cleared. Waiting - The job is active, waiting for resources (e.g. media or drive) to become available or for internal processes to start. The job many also be started in the waiting state if a queue option was selected from the General tab of the Job Management dialog box.
Module Summary
Summary
There are many different options for Data Protection depending on the needs of the organization. Archiving uses hierarchical storage management concepts to move unused data between different tiers of storage. There are many options for initiating jobs within Galaxy. Global options for job queuing, job pre-emption and job priorities can be configured. The Job Controller window of the CommCell Console offers the ability to dynamically suspend, resume, or kill jobs using the Multi-Job feature. The SQL meta data for the CommServe is protected by at least two backup locations with one operation. Data Aging will purge data that has exceeded retention. Data is aged differently depending on what type of media it was written to.
Restore
www.commvault.com/training
Overview
Image/No-Image Browse
D:\DIR 1/1/06 1/2/06 1/3/06 1/4/06 No Image A.txt B.txt C.txt D.txt E.txt F.txt G.txt H.txt FileName A.txt B.txt C.txt D.txt E.txt F.txt G.txt H.txt 1 2 3 4 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9
Image/No-Image Browse
Image Browse Each indexed backup generates an image index of all files/objects scanned at the time of the backup. The image will always contain all files/objects scanned at the time of backup regardless of the backup type (e.g. full, differential, incremental). This image index is used when you: Browse backup data for a specified time Select to restore the contents of a directory or drive at a specified time Do a full system rebuild of the file system at a specified time
A backup cycle that spans a period of time will have image indexes for each backup job contained with the index. For example; a subclient doing weekly full backups with daily incremental backups will have seven (7) image indexes for the list of files/objects it backed up during the cycle one for each backup job/day. Selecting a point-in-time browse of the backup data will use the appropriate image index to show you what files/objects existed on the system/application at that time. No-Image Browse What if youre looking for a deleted file thats not in the latest backup image? How do you find it amongst the backup data? The basic answer is that you can search for the file by doing pointin-time browses of the backup data. If you are not certain of when it was last backed up, this can be a hit-or-miss exercise. Additionally, if youre guessing when it was backed up, it might not be the latest version of the file. To ensure you got the latest version would require methodically
Restore Module - 191 stepping backwards through each backup job. As you might suspect, this can take time. This is where the Image Browsing option becomes valuable. Disabling the Image Browsing option will create a browse display of all files within the index from that point back to the beginning of the index. Deleted files not in the current backup image but were captured in previous backups, would be displayed in the browse. Essentially, you are browsing all files that were backed up since the index/cycle started to the selected point-in-time. If your deleted file was backed up in this cycle, it should appear in the backup browse display. If not, you can use the Find feature to scan through multiple indexes/range of time to locate the deleted file or all versions of a file. A caution with disabling the Image Browsing option is that if you are restoring directories or drives, more files will be restored and the resultant volume may be more than the capacity of the device.
List Media
z z
Media Containing Index Media Containing Jobs within specified time range Lists media for specified objects selected for restore Lists media for specified subclient
Media prediction is useful in a variety of circumstances, including the following: To ensure that media required by an operation is available in the library, especially if you are restoring/recovering data across a firewall. In cases where data spans across several media, to identify the exact media necessary to restore/recover a file/folder/sub-section of the data. To identify and restore/recover from a copy that accesses a faster magnetic disk media rather than slower tape/optical media.
To identify media associated with an alternate copy, when the media containing data associated with a specific copy is not readily available due to the following reasons: When the media is exported from the library
List Media Associated with a Specific Backup Set, Instance or Subclient is referred to as List Media in the CommCell Console and provides the following options: Search media associated with the latest data protection cycle, starting from the latest full backup. (This is the default option.) Search media associated with data protection operations performed between a specified time range. Search for media associated with a specific storage policy copy, synchronous or selective copies, with the specified copy precedence number.
Keep in mind that when you search media from a secondary copy, the listed media may not reflect the entire instance or backup set data, unless all the storage policies associated with all the subclients have been configured for secondary copies. List Media (Precise) This operation in the CommCell Console is useful to predict precisely which media contains specific files or folders. For example: When a data protection operation spans across multiple media and you would like to know the exact media in which the files you wish to restore reside. You have a specific set of files (either a random set or a specific set, such as *.doc or *.txt) that you wish to restore and would like to know the all the media in which the files reside. You wish to restore a specific version of the file and would like to know the specific media in which the version resides.
Exact Index
Past Index Current Index Exact Index
Object A.txt B.txt C.txt D.txt E.txt F.txt M X X X X X X X X X X X X X T W X X X X X X X X X X Th F X X X S S X X Object A.txt B.txt C.txt D.txt M X X X X X X X X X X X T W X X
Restore
E.txt F.txt
(Copy on Media)
Exact Index
When you browse indexed backup data, the default index set restored/used will be the last index set generated in the backup cycle. This is true even if you are doing a point-in-time browse. This means, that no matter when within a backup cycle you want to browse, the last index set generated in that backup cycle will be used. There could be circumstances where the media containing the last index set in the cycle is not available (e.g. disaster recovery, remote restore) or the index set is lost/corrupt. In this case you can select the Use Exact Index option available in the Advanced Browse options window. The Use Exact Index option is useful in situations when you want to restore data from specific media. For example; when media is off-site and you would like to maximize the chances that the necessary index information comes from the same media.
Find/Search
Find
z z
Available at backup/archive set level Spans indexes Available at all levels Uses content indexing End user or Compliance CommCell or Web-based
Search
z z z z
Find
The Find feature allows you to search the data protection archives for any file or directory name or name pattern. Because Find supports the capability of searching multiple indexes, unlike browse, you can search beyond the last full backup (or new index) as long as the data resides on an index that exists within the retention period. The Find feature is accessible as a right-click option on the browse window, and for supported agents a non-browse find is available from the All Tasks menu. Depending on the agent, the Find option is accessible from either the backup set or archive set level. The Advanced Search tab of the Find feature is accessible only to clients that support Content Indexing and have created a content index for their archived or backed up data. Advanced Search allows you to search by content (keyword or phrase). For Exchange agents, you can also search by To, From, CC, BCC and attachment name. Advanced searches with multiple criteria can be performed. In addition, Exchange agents that support Content Indexing can also narrow search results by specifying a received time range for messages on the Find (Receive Time Range) tab. When using Find, be sure to specify a time range to prevent an unbounded search through all past data. End User Search The main concept behind End-User Searches is the ability for users to search all data objects that were created by them, or that is accessible to them. Because End-User Searches require a
196 Restore Module Windows Security ID (SID) to uniquely identify whether a user created a data object, or has the access rights to the data object, End-User Searches are not currently available for Unix-based agents and Unix-created data objects residing on NFS shares managed by Data Archiver. However, Data Archiver data residing on CIFS shares do support End-User Searches. Note that the search function from the CommCell Console does not provide end user search capability. You must use the Web-based Search Console or Data Archivers Outlook Add-in for end user search criteria. Compliance Search The Compliance Search capability allows compliance officers full access to all computers and supported applications for searching, regardless of ownership/access attributes for the piece of data. CommCell Console Search If Content Indexing is installed and available, the Search function is available at all levels of the CommCell from the backup/archive set and up. Web-based Search Console The licensed Web Search Server has features that enables both compliance and end-users to search and restore data based on individual security permissions. The web-based End User and Compliance Search Consoles have an easy-to-use search interface modeled after popular search engines. The web-based Search Console requires that an Active Directory External User Group be associated and configured before is used.
Restore/Recall Options
Restore by Job Mapped Restores Recall Archived Objects Copy Precedence Restore from anywhere Out-of-Place Restores Restore to Non-Client hosts
Restore/Recall Options
The simplest form of restore is to copy an object back to its original location. However, not all restores are simple. The process is intricate, by requirements to filter, redirect, alternate source, alternate path, parallel stream and otherwise manipulate the data being restored. Flexibility in restore, recall, or recovery of data is an essential feature in any data protection product.
Restore by Job
Accessible from BackupSet level Multiple jobs can be selected Newer files overwrite older files For CommServe DR use DR Restore option
Restore by Job
The Restore by Job feature provides the facility to select a specific backup job or set of jobs to be restored. This method of restoring data is faster under some circumstances as it reads continuously from the tape and retrieves the data independently rather than using the indexing subsystem which does individual seek offsets on the media. A key advantage of restore by jobs is restoration of any successfully written data in failed or killed jobs. If multiple job streams were used in parallel for the data protection job, then Restore by Job can use those same multiple streams in parallel for the restore. The degree of parallelism would be a factor of the degree of multiplexing for the jobs involved. The higher the multiplexing, the more parallel the restore. Optimally, youd like each job stream assigned to a device stream on a 1-to-1 basis. Multiple stream parallel restores in these cases would be as expected. On the other hand, if the job streams are multiplexed to the same tape, then a single device/job stream to the client will be used. The parallel restore occurs at the client, where multiple threads will be generated to de-mux the single stream and restore the data accordingly. A standard browse/index-based restore for more granular restore may be more appropriate at times and can be faster by not restoring unnecessary objects. However, an index-based restore is
200 Restore Module single stream only, regardless of how many streams may have been used in parallel by the data protection job. Restore by Job cannot be used to restore the Windows System State data.
Mapped Restores
Mapped Restores
When performing normal restore of the selected data youre given the option to designate one specific destination. If the restored data needs to go to multiple locations (for example youre using a restore to restructure your primary storage) you would need to perform multiple restore jobs. Using a map file provides the ability to restore individual files and folders within a single job to different locations. The map files contains a list of files to be restored with their corresponding restore paths. In addition the mapping options also provide the ability to specify whether the unmapped files that were selected for restore should be restored to the selected restore destination. The map file restore options are also supported by Restore by Job feature and restores from the command line. Map file restores are supported by the Macintosh, Unix and Windows Agents. The map file should be a CSV (Comma Separated Value) file. On Windows, the map file can be of any recognizable type, such as .txt, .doc, .xls, .rtf, etc. On Unix, use an editor such as VI to create the map file. The format of the mapping should be as follows: "<source path>","<destination path> Points to Remember While Creating the Mapping File Consider the following while creating a map file:
The specified paths for both the source and destination should be absolute paths within the same client computer. If the specified destination folder is not available, the necessary destination folder will be created during the restore. The source and destination type should be the same. For example, a folder must map to another folder and a file to another file. (The results may be unpredictable if this is not followed.) Computer names or UNC paths should not be used in the source and destination names. Wildcards in source or destination path is not supported. To filter a file, add the source folder and file and then add an empty double quotes as the destination. For example: "C:\dir5","" Use a mapping file only when there is a large number of files to be restored with multiple destination paths. Consider the following examples: You wish to restore 10 files. Within these 10 files, 8 files must be restored to the same location, while 2 files must be restored to separate destinations - in this case select these 10 file in the Browse window, specify the destination path for the 8 files to the common location and only include the 2 files in the map file. As much as possible filter the files from the Browse window, rather than filtering the files using a map file.
Stubbed, non-browsed restores can only be recalled in-place to the items previous location. However, for the Exchange Mailbox Archiver Agent, the restore destination can be the same mailbox, a different mailbox, or a PST file on another Exchange Server within the same organization and site with a compatible Exchange Mailbox Archiver Agent installed and operational. Important Notes: Prior to performing a PST migration archiving operation or a PST recovery operation with the Exchange 2003/2007 Mailbox Archiver Agent, ensure that the MAPI32.dll file has been copied into the <Software Installation Path>\Base installation directory. After completing a PST archiving or recovery operation, ensure that the MAPI32.dll file is removed from the <Software Installation Path>\Base installation directory on the Client, otherwise non-PST restore/recovery/retrieve jobs for Exchange may go into a pending state.
Copy Precedence
iDA
1
Client Magnetic Library
MA
2
Tape Library
The Each If a copy option Storage precedence to specify Policy Copy a is has a restore specified, specific copy only precedence precedence that copy number. will for restore be used If no may for specific restore be used to precedence whether bypass missing or not is used, or more suspect Galaxy will use each relative backup data datacopy or exists to in force on use precedence other of an alternate copiesorder within data to the path find the restore. Storage for dataPolicy to restore.
Remote Site
iDA
3
Tape Library
Client
MA
Copy Precedence
Storage Policy Copy precedence is a priority sequence used by a restore job to determine which storage policy copy to use when multiple copies are involved. Copy precedence is initially assigned in the order in which a storage policy copy is created (e.g. 1, 2, 3 ). As such, the primary copy is by default (and normally) the first precedence (#1) copy. However, the primary status of a copy and its copy precedence are not related. While data from the client is always written to the primary copy, there is no requirement that the primary copy also be the first copy used for restore. The precedence sequence of a copy can be changed in the Copy Precedence tab of the Storage Policy Properties dialog window. For a restore request, where the copy precedence is not specified (default), the software assumes any copy will do and will check each copy precedence in sequence until it finds one available to satisfy the restore request. When you manually specify copy precedence in the restore request you turn off the automatic copy selection and force the software to use the copy you selected regardless of whether it has the most recent data or no relevant data at all. Additionally, when manually specifying copy precedence, you should select the copy precedence before browsing the backup data set. Browsing one copy and restoring from another may yield unexpected results. For example; a restored directory size and content might not be the same as you viewed in the browse.
206 Restore Module Manually specifying copy precedence for restore is useful if: The primary copy is no longer available for a data recovery operation due to a hardware failure. You know that the media containing the data from data protection operations for a particular copy have been removed from the storage library. In this case, you can choose to browse/restore/recover from a copy whose media are inside the library. You want to browse/restore/recover from a copy that accesses faster magnetic disk media rather than slower tape media. You know that the media drives used by a particular copy are busy with another operation and want to browse/restore/recover from a different copy to avoid resource conflicts. You want to perform backups to a primary copy in one location and do restores from a secondary copy in another location. (This can be pre-set for scheduled restores or command line restore (Oracle RMAN) by altering the precedence of the secondary copy on the Storage Policy Properties dialog window.
?
Client Tape Library
MA
Export
? !
Import
Tape Library
A restore Data The media fromrequest a can client then that is be written to exported requires data from managed from the the librarywill media library and check imported . By library into managed another inventory managed to we see mean first library an inventory within who has the the is same maintained media, then by the library. CommCell. restore the data.
Out-of-Place Restores
Cross-application
z z
Exchange Mailboxes on any version All Databases Exact version only* Use Browse Options Restore to different client is supported
System State
z z
Out-of-Place Restores
By default, data is restored to the same computer from which it was backed up. You can restore to a different parent directory path by clearing the Restore to same paths option. All files restored will have the new path pre-pended to their current path. Hence a file with a fully qualified path name of D:\user\data\document\file.txt restored to a path of F:\Restored would result in a restored file of F:\Restored\user\data\document\file.txt. Multiple files with different path names would have the same new path pre-pended. The Preserve source path option can be used to keep or remove levels of the restored path and file name. This option would best be used in re-organizing directory structures in conjunction with the restore. Files can also be renamed by appending a suffix to the filename itself not the extension. Maintaining the extension keeps the application association consistent. To change the destination computer, you can select one from a list of established clients within the CommCell, with Operating Systems/Applications that support the out-of-place, crossplatform, or cross-application restore operation from this client. If the client name does not appear, the cross-platform OS or application does not support the out-of-place restore.
MA
Report.doc
Restore Module - 211 time to open because the system has to recall it from the remote location. Immediately reaccessing the file will not generate multiple recall events nor will it make the file recall faster. The user may see a pop-up notification informing them that the system is retrieving the file. This pop-up is, by default, turned off in Windows and on in Network Appliance. The administrator has the option to change this setting in the registry key. Stub Retention By default, stub retention length is decided by the retention of the archived messages/items associated with the stub. After the messages/items on the media (all the copies) have expired, the stub will be pruned in the next migration archiving operation. Users can also specify a value in days for stub retention time, from the subclient properties Archiving Rules tab. Keep in mind that the stub could be pruned before this value, if the storage policy retention time is smaller than this value. Stub Rules The following stub rules allow you to manage the creation and pruning of stubs: Preserve stubs until the media has been pruned Prune stub only after n days Archive files only, do not create a stub
Restore Module - 213 Application Restores An application is made up of its managed data set and supporting executable and configuration files. The application must be functional in order to restore its managed data. In some cases this may require using a File System restore to get necessary supporting files in place. Additionally, the application must be in a recovery state in order for a restore and full recovery to work successfully. For example; With Exchange the System Attendant Service must be running in order to restore databases. The System Attendant Service will run without the presence of the databases. However, if the databases do exist they must be enabled for overwrite by a restore. This can be done using Exchange admin tools, or it can be done automatically by selecting the appropriate option during an Exchange database restore. On the other SQL Server service will not run without the master, model, msdb and temp databases. In order to do a full application recovery of SQL Server you must build suitable replacement databases in order to start the SQL Service - and then restore your original databases from protected storage. Microsoft provides utilities to rebuild the necessary databases. The key thing to remember is that restoring an application to full functionality requires cooperation between many parts. Understanding and practicing application recovery is essential to successful recovery.
Module Summary
Summary
Finding the data to restore is simple, understanding the index cache is helpful. Finding the media is simple and that information can be obtained at various levels within the GUI. CommVault also offers search capabilities for data, but should be used with date restrictions. Data can be restored to many different locations from any library. Full system restores are available for extreme situations where the entire system needs to be recovered.
Monitoring
www.commvault.com/training
Overview
Monitoring Options Alerts Reports
Monitoring Options
CommCell Explorer CommNet Job Controller Resource View Event Viewer
CommCell Explorer
Client Views Job Views Media Views Configuration Views Database Views
CommCell Explorer
The CommCell Explorer views provide a way to query information on the CommCell components directly from the SQL database. These views are provided in addition to the CommCell Console Report Selection feature. You can use the views provided with CommCell Explorer or customize them to reflect the data in any manner appropriate to your organization. You can query the database with Microsoft SQL Mgmt Console or any SQL database query tool. Query results can be displayed through Explorer or you can use products such as Microsoft Excel or Crystal Reports to format your query output. If you modify a view or create a new view, you must reapply them after each new release. The following views are included with CommCell Explorer CommCellAdminSchedule CommCellAuxCopyInfo CommCellBackupInfo CommCellBkSchedule CommCellClientConfig CommCellClientFSFilters CommCellJobControllerCount CommCellLibraryReservInfo
Monitoring Module - 219 CommCellLicense CommCellMediaClient CommCellMediaInDrives CommCellMediaInfo CommCellRestoreInfo CommCellStoragePolicy CommCellSubClientConfig CommCellUpdateInfo
CommNet
CommNet
This management tool has been designed to manage and administer CommCells and QSMCells efficiently and in a timely manner in order to minimize administration costs. Additionally, you can analyze concise summaries and comprehensive reports showing various aspects of secondary and primary storage in order to resolve problems proactively rather than reactively. The software also provides features such as remote administration capabilities, alert mechanisms, user administration and scheduling. Alerts Alert mechanism provides the capability to notify users of critical conditions using E-mail, Pager and SNMP traps. This feature also provides the mechanism to escalate the conditions if it has not been addressed for a specified amount of time or if the condition worsens. The CommNet Server now includes the Administrative Alert, Cell Synchronization Failure Over Time, which enables users to be notified when a specified CommNet entity has encountered cell synchronization failure(s) within a specified time range. Reports The software contains various reports that help you to manage your CommNet domain. Some of the important reports are described below: Data Protection Coverage -Data Protection reports can help you to understand the whether or not a particular client has data protection coverage on a given day, week,
Monitoring Module - 221 month, or year. The coverage report also displays the status of the coverage on a copy basis to determine if the coverage is from a primary or an auxiliary copy. In addition, you can also view valuable trending analysis such as job success rate, media consumption and primary and secondary storage growth. The software provides flexibility in defining the Data Protection Windows for the CommCells for reporting purposes. You can use these customized reports to fine tune various parameters on a CommCell in order to meet the data protection window. A Window Utilization report is also provided, which can be used to determine the Clients, Agents, and Data Protection operations that are not meeting the specified window within a specified amount of time. Window Utilization - Data Protection Window Utilization Report can help you determine how well the data protection window is being utilized in the CommCells. This report provides a summary of data protection operations relative to the data protection window specified in the CommCell, including the number of jobs completing outside the window. Data Recovery Coverage - Data Recovery reports can help you to understand the details of the recovery operations performed in the CommCells, including information on the restore destination and the job successes associated with the recovery operations. Media Management Performance - Media Management related information, such as Media Agent, library and drive performance are provided in a concise manner in various Media Management reports. The software also generates performance reports for a given Media Agent, library and drive, to help analyze media management aspects such as throughput, data size transferred, number of data protection operations handled, and library and drive usage times. Prediction Capabilities - The software provides the capability to predict the following, by looking at past consumption and usage: Data growth for each CommCell as well as the sub clients in a CommCell. Media Prediction to forecast media usage (for tape/optical libraries) and capacity usage (for magnetic libraries) in the individual CommCells.
Q-Factor - Q-Factor is a scale to measure how well the data protections operations are performed in a CommCell. This is determined based on the short-term and longterm coverage. In order to customize the calculation based on your specific environment, facility to define the weights for several aspects in the short-term and long-term coverage and an acceptable Q-Factor is also provided. Primary Storage and Media Based Costing - Media based costing mechanism provides the facility to determine the cost associated with hosting an application and its protected data, based on the type of media in which the data resides. This feature includes the facility to define cost categories and the media associated with each cost category, and billable entities, such as departments within a company. A comprehensive Billable Charge Back report can be generated to view the cost, related expense, storage size, and ranking information for all vital primary and
222 Monitoring Module secondary storage entities. The report can be generated for clients or billable entities. The costing model can be defined centrally and automatically distributed to all CommCells and QSMCells within the CommNet domain. Dashboard The Dashboard displays a pictorial view of a CommNet entitys, the CommNet domain's or a CommCell's, last seven days. It contains the details of successful jobs, data protection coverage and data growth for the CommNet entity specified by the user. This is extremely useful for those needing to quickly obtain the status of their CommNet domain environment and for troubleshooting. The dashboard can display up to six (6) reports at one time. By default, it displays the following reports: Last 7 Days Job Success This bar graph depicts the data protection job success rate for the last seven days, according to the CommCell Browsers local time. It displays the total number of data protection jobs executed during this time, as well as those that have completed, were killed and failed. If viewing the last seven days of a CommCell rather than the entire CommNet domain, you can click on the graph to see a detailed report of the data protection jobs within this time period. Last 7 Days DP Coverage This bar graph depicts the data protection coverage per subclient for the last seven days, according to the CommCell Browsers local time. It displays the total number of subclients with data protection coverage as well as those with no activity, or with coverage that failed or was killed. If viewing the last seven days of a CommCell rather than the entire CommNet domain, you can click on the graph to see a detailed report of the data protection jobs per subclient within this time period. Last 7 Days Data Growth - Combined This bar graph depicts the data growth for both incremental and full data protection jobs for the last seven days, according to the CommCell Browsers local time. It displays the total growth for both incremental and full data protection jobs per day in GB. If viewing the last seven days of a CommCell rather than the entire CommNet domain, you can click on the graph to see a detailed report of the data growth within this time period. Other reports available for display in the Dashboard are:
Last 7 Days Data Growth - Incremental Last 7 Days Data Growth - Full Last 7 Days DP Activity - Data Size Last 7 Days DP Activity - Job Detail
Job Controller
Job status updates every n minutes Displays job information and any errors Filter job Optional display by job
z z z
Job Controller
The Job Controller allows you to monitor the following types of jobs: Data protection operations Data recovery operations Administration operations
You can view detailed information about these jobs as well as job events and the media used for each job. Information about a job is continually updated and available in the Job Controller window. When a job is finished, the job stays in the Job Controller for five minutes. This time can be changed by opening the control panel and selecting the Display control Job Filter tab. Once a job is finished, more information about that job is obtainable using the Job History. Job Updates By default, the system updates the information in the Job Controller every 5 minutes or 2GBs of data transferred. These values can be changed by opening the Control Panel and selecting the Job Management control Advanced tab. Job Information You can view the following information about a job in the Job Controller: Job Status Job Details
Job Status Levels A job in the Job Controller window may have one of the following status levels: Completed - The job has completed. Completed With One or More Errors -The job has completed with errors. This is relevant only to Exchange 2000 Database, Oracle, and CommServe Express Recovery, and File System backup jobs. Dangling Cleanup - The job has been terminated by the job manager, and the job manager is waiting for the completion of associated processors before killing the job. Failed - The job has failed due to errors or the job has been terminated by the job manager. Interrupt Pending - The job manager is waiting for the completion of associated processors before interrupting the job due to resource contention with jobs that have a higher priority, etc. Kill Pending - The job has been terminated by the user using the kill option, and the job manager is waiting for the completion of associated processors before killing the job. Killed - The job is terminated by the user using the kill option. Pending - The job manager has suspended the job and will restart it without user intervention. A pending job may be waiting. Running - The job is active and has access to the resources it needs. Stop Pending - A job is suspended by a user using the Suspend option, and the Job Manager is waiting for the completion of associated processes before stopping the job. Stopped - A running, waiting, or pending job has been manually stopped by a user using the Suspend option. The job will not complete until it is restarted using the resume option. Waiting - The job is active, waiting for resources (e.g. media or drive) to become available or for internal processes to start. Queued - The job conflicted with other currently running jobs (such as multiple data protection operations for the same subclient), and the Queue jobs if other conflicting jobs are active option was enabled from the General tab of the Job Management dialog box. The Job Manager will automatically resume the job only if the condition that caused the job to queue has cleared.
Resource View
Drives in Library Media currently used
Resource View
The resource view displays all the drives in the library and the media that is currently used in the drive at any point in time. The media information includes the barcode\identifier of the media and Job ID associated with the job currently using the media. If the media is not used by a job the resource view displays the Job ID as Cache Mounted. The Resource View includes information on the operation phase to show the specific actions that are occurring on the media in the drive such as writing/verifying/reading OML, loading, unloading, etc. From the drive level, you can release the reservation on resources, if the reservation is not released when a job has failed or killed in the Job Manager. In addition, for regular libraries the resource view also displays the slot information in the library and information on media that is currently in the process of being exported. For Magnetic libraries, the status and free space information for the mount path is displayed. The Resource can be viewed from the drive list in the right-pane of the CommCell Browser when you click Master Drive Pool or Drive Pool in the tree.
Event Viewer
Severity Levels # of Events available Time Range Severity Job ID Character pattern
Event Viewer
The Event Viewer allows you to monitor activities that are occurring within the CommCell. This information is useful for troubleshooting and informational purposes. For example, you may want to know if your system is experiencing hardware problems, or what jobs have started or completed. Some events will also generate an alert if an alert for that event is configured within the CommCell. The Event Viewer Window Events are displayed in the Event Viewer with the severity levels of Information, Minor, Major, and Critical, along with additional information, such as the subsystem that generated the event. By default, the maximum number of events displayed in the Event Viewer is 200. You can modify this number by opening the control panel and selecting the Display control Event Filter. You can modify the types of events to be displayed based on the severity level. The default maximum number of events retained in the Event log is 10,000. This number can be changed by opening the Control Panel and selecting System. The Event log is maintained in the metadata database.
Monitoring Module - 227 View the Details of an Event If a particular event is not entirely visible in the Event Viewer, the complete description can be viewed in the Event Details dialog box. Event Search Queries To view only selected events from the Event Viewer, a search query can be created to display only those events that correspond to a time range and severity level, or a Job ID. The results of your query are displayed in the All Found Events window. A further search using the Find feature can be run in the window to search for word/phrase patterns. Some search queries can be saved and selected at any time from the Select from this search query field. A search query based on a particular Job ID cannot be saved.
Alerts
Understanding Alerts Alert Output Options Recommended Alerts
Understanding Alerts
Understanding Alerts
CommVaults Alert feature provides real time notification on events occurring within a CommCell. These conditions can range from minor occurrences that do not require intervention to severe occurrences that need immediate intervention. The system detects conditions within two minutes of the occurrence. Alert coverage is hierarchical and can be defined for each alert created. Alerts can be configured for any object that can be seen by the user creating the alert. Best Practices When possible, configure alerts to cover Client computer groups vice individual clients and when using the E-mail output option, use group e-mail addressing vice individual users. Groups are more consistent and wont require you to modify alerts when individual Clients or users are removed or added.
Methods
z z z z
Recommended Alerts
Insufficient Storage Phase or Network Failure <Device> went Offline
Recommended Alerts
Insufficient Storage Media Management->Library Management The most common reason for data protection job failure is no spare media. You run out of magnetic disk space or have no spare media inside the library. This is particularly prevalent during weekends when large data protection jobs are usually scheduled and no one is about to monitor the status of spare media. The Insufficient Storage alert is dependent upon the Low Watermark threshold value assigned to the magnetic library, Scratch, and Cleaning Media Pools. Ensure you set these values high enough to allow the alert to give you ample warning to replenish the media before jobs start to fail. Phase or Start Job Failure Job Management->Data Protection A Data Protection job consists of two or more phases and requires network connection in order to initiate the job on the Client. If a phase fails, or the Client cant be reached the job will enter a pending state and retry. If the condition that caused the failure is transitory, a retry may succeed. If not, the entire job will eventually fail if nothing is done to overcome the reason for the network or phase failure. You can be pro-active in setting alerts for repetitive phase and/or network failure to take appropriate action before the entire job fails.
232 Monitoring Module <Device> Offline Media Management->Device Status Library, mount paths, or tape devices that are taken offline intentionally, or by themselves due to some problem can have significant impact on your ability to complete jobs in a timely manner. CommVaults CommCell Readiness Report checks for device status and should be run before each data protection window. However, for more real time notification we recommend you set an alert for offline status of key devices.
Reports
Understanding Reports Recommended Reports
TEXT XML
Metadata
Job
Output Methods
E-mailed
Storage
VaultTracker
CommCell Explorer
User Group
Saved
Folder/File Template
Available Reports
A variety of reports can be created, each tailored to a particular aspect of data management. Through filter criteria, you can customize each report to include only the data that is required. Report templates can be customized and are available to you to save, run, schedule, edit and view. CommNet CommVaults CommNet or QNet is a hierarchical reporting and management application for one or more CommCells. CommNet reports can include data from multiple CommCells and have user selectable graphing capability. CommCell Explorer CommCell Explorer is a set of database views that can be used to create customized reports.
Recommended Reports
CommCell Summary Report Job Summary Report Data Retention Forecast and Compliance Report Media Information Report CommCell Readiness Report
Recommended Reports
CommCell Summary Report The CommCell Summary Report provides the properties of the CommServe, Media Agents, clients, agents, subclients, and storage policies within the CommCell based on the selected filter criteria. This report is useful if you require: The number of computers installed in the CommCell. The type of licenses available in the CommCell, installation date for each license type consumed, as well as the expiration date of each license. Administrative schedules. Data interface pairs information. The alerts that are configured in the CommCell. CommCell user and user group information. Service Pack information for each module. Media Agent and library information. Client information such as enabled and/or disabled operations, defined operation rules, installed clients, and subclient contents and filters. Scheduled data protection information for each subclient. Retrieving all Update information from the registry of each machine in the CommCell and updating the CommServe Database with this information now includes NetWare Agents.
236 Monitoring Module A listing of mailboxes per subclient for common mailbox agents. Information that the use of QSnap or VSS was selected through subclient properties. This is indicated by a superscript Q in the subclient column of the report. The remaining index cache space, size of existing index cache, and disk size on each Media Agent. Subclient policy information through an optional checkbox. The number of streams for a subclient. All Updates which have been applied to a machine as well as the components to which they were applied the list of mailboxes per subclient for the Exchange Mailbox and Archiver Agents. The option to retrieve all Update information from the registry of each machine in the CommCell and update the CommServe Database with this information now includes NetWare Agents.
Job Summary Report The Job summary report can be filtered to display the following: All Jobs - All Data Protection, Data Recovery, and Administrative Jobs are included in the report. Data Protection Jobs - Can be filtered to include: Calendar Backup Job Summary, Backup Job Summary, Calendar Migration/Archive Job summary, QR Volume Creation Job Summary or ContinuousDataReplicator Job Summary. Data Recovery Jobs Can be filtered to include: Restore Job Summary, Recovery/Retrieve Job Summary, QR Recovery Job Summary, ContinousDataReplicator Copyback Job Summary and/or Stub Recall Summary. Administrative Jobs Can be filtered to include: Disaster Recovery Backup Job Summary, Auxiliary Copy Job Summary, Data Aging Job Summary, Data Verification Job Summary, Download Updates Job Summary and/or Install Updates Job Summary.
This report is useful if you require: To view all data protection, data recovery, and administration jobs during a specified time period. To view the failure reason and associated events for all data protection, data recovery, and administrative jobs.
Data and Aging Forecast Report The Data on Media and Aging Forecast Report provides a history of data protection jobs, associated media, data aging forecast and recyclable media based on the selected filter criteria. Note that regardless of the filter options selected for this report, the report always checks retention in days. This report is useful if you require:
A list of data protection jobs that are on each media or magnetic volume. A list of data protection jobs that either exceed their retention criteria or need to be retained. A list of media that can be recycled and a list of media that can not be recycled based on the copy retention. The amount of magnetic space that will be freed and kept for each storage policy copy. An estimated date on which a job is due to be aged.
Media Information Report The Media Information Report provides detailed information about the media that is associated with storage libraries, the media repositories and/or locations, and the media that has been written to or reserved by storage policy copies, all within a specified time range. This report is useful if you require: Media status and the reason for the media status, such as if the media is full, bad, expired, foreign, or undiscovered. A list of media that has errors. A list of empty slots in a library. A list of data that has been aged from media. Media repository information if media repositories have been configured for the media. The barcodes and locations of the media. The storage policies and storage policy copies associated with the media. An alert of all the media about to be marked expired Information on which media is exportable. This is indicated by a superscript E in the Barcode column of the report. CommCell Readiness Report The CommCell Readiness Report provides you with vital information about the condition of your CommCell that can serve as a tool to warn you about potential problems that can impact your operations before they occur. This report is useful if you require: To check for connectivity between your CommServe, Media Agents, and clients. To check the capacity of your library. To verify that your library controllers, libraries, master drive pools, drive pools, and drives are online. To verify the existence of the index cache location and to check the index cache label's contents against the selected Media Agent's index cache.
Module Summary
Summary
Alerts are configured to notify users of various occurrences within a CommCell. Many different reports are available with an abundance of options to configure. They can be scheduled for email, or saved to a file among other options. The CommCell Explorer views provide a way to query information on the CommCell components directly from the SQL database. The Job Controller window offers the availability to monitor and manage running jobs. The Event Viewer allows you to monitor all activities that are occurring within the CommCell.
Appendix - 239
Appendix
www.commvault.com/training
Appendix
240 - Appendix
X X X X X X
X X X X X VaultTracker Enterprise X X
Appendix - 241
CommCell
242 - Appendix
Training Environment