Symantec NetBackup Deduplication Guide

UNIX, Windows, Linux

Release 7.1

21159706

Symantec NetBackup Deduplication Guide

The software described in this book is furnished under a license agreement and may be used only in accordance with the terms of the agreement. Documentation version: 7.1 PN: 21159706

Legal Notice
Copyright 2011 Symantec Corporation. All rights reserved. Symantec, the Symantec Logo, and NetBackup are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. This Symantec product may contain third party software for which Symantec is required to provide attribution to the third party (Third Party Programs). Some of the Third Party Programs are available under open source or free software licenses. The License Agreement accompanying the Software does not alter any rights or obligations you may have under those open source or free software licenses. Please see the Third Party Legal Notice Appendix to this Documentation or TPIP ReadMe File accompanying this Symantec product for more information on the Third Party Programs. The product described in this document is distributed under licenses restricting its use, copying, distribution, and decompilation/reverse engineering. No part of this document may be reproduced in any form by any means without prior written authorization of Symantec Corporation and its licensors, if any. THE DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. SYMANTEC CORPORATION SHALL NOT BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES IN CONNECTION WITH THE FURNISHING, PERFORMANCE, OR USE OF THIS DOCUMENTATION. THE INFORMATION CONTAINED IN THIS DOCUMENTATION IS SUBJECT TO CHANGE WITHOUT NOTICE. The Licensed Software and Documentation are deemed to be commercial computer software as defined in FAR 12.212 and subject to restricted rights as defined in FAR Section 52.227-19 "Commercial Computer Software - Restricted Rights" and DFARS 227.7202, "Rights in Commercial Computer Software or Commercial Computer Software Documentation", as applicable, and any successor regulations. Any use, modification, reproduction release, performance, display or disclosure of the Licensed Software and Documentation by the U.S. Government shall be solely in accordance with the terms of this Agreement.

Symantec Corporation 350 Ellis Street Mountain View, CA 94043 http://www.symantec.com Printed in the United States of America. 10 9 8 7 6 5 4 3 2 1

Technical Support
Symantec Technical Support maintains support centers globally. Technical Supports primary role is to respond to specific queries about product features and functionality. The Technical Support group also creates content for our online Knowledge Base. The Technical Support group works collaboratively with the other functional areas within Symantec to answer your questions in a timely fashion. For example, the Technical Support group works with Product Engineering and Symantec Security Response to provide alerting services and virus definition updates. Symantecs support offerings include the following:

A range of support options that give you the flexibility to select the right amount of service for any size organization Telephone and/or Web-based support that provides rapid response and up-to-the-minute information Upgrade assurance that delivers software upgrades Global support purchased on a regional business hours or 24 hours a day, 7 days a week basis Premium service offerings that include Account Management Services

For information about Symantecs support offerings, you can visit our Web site at the following URL: www.symantec.com/business/support/ All support services will be delivered in accordance with your support agreement and the then-current enterprise technical support policy.

Contacting Technical Support

Customers with a current support agreement may access Technical Support information at the following URL: www.symantec.com/business/support/ Before contacting Technical Support, make sure you have satisfied the system requirements that are listed in your product documentation. Also, you should be at the computer on which the problem occurred, in case it is necessary to replicate the problem. When you contact Technical Support, please have the following information available:

Product release level

Hardware information Available memory, disk space, and NIC information Operating system Version and patch level Network topology Router, gateway, and IP address information Problem description:

Error messages and log files Troubleshooting that was performed before contacting Symantec Recent software configuration changes and network changes

Licensing and registration

If your Symantec product requires registration or a license key, access our technical support Web page at the following URL: www.symantec.com/business/support/

Customer service
Customer service information is available at the following URL: www.symantec.com/business/support/ Customer Service is available to assist with non-technical questions, such as the following types of issues:

Questions regarding product licensing or serialization Product registration updates, such as address or name changes General product information (features, language availability, local dealers) Latest information about product updates and upgrades Information about upgrade assurance and support contracts Information about the Symantec Buying Programs Advice about Symantec's technical support options Nontechnical presales questions Issues that are related to CD-ROMs or manuals

Support agreement resources

If you want to contact Symantec regarding an existing support agreement, please contact the support agreement administration team for your region as follows:
Asia-Pacific and Japan Europe, Middle-East, and Africa North America and Latin America [email protected] [email protected] [email protected]

Contents

Technical Support ............................................................................................... 4 Chapter 1 Introducing NetBackup deduplication ............................ 13

About NetBackup deduplication ...................................................... 13 About NetBackup deduplication options ..................................... 14 How deduplication works ........................................................ 15

Chapter 2

Planning your deployment ................................................ 17

Planning your deduplication deployment ......................................... About the deduplication tech note ................................................... About the deduplication storage destination ..................................... About the NetBackup Media Server Deduplication Option .................... About deduplication servers ..................................................... About deduplication nodes ....................................................... About deduplication server requirements ................................... About media server deduplication limitations ............................. About NetBackup Client Deduplication ............................................. About client deduplication requirements and limitations .............. About NetBackup Deduplication Engine credentials ........................... About the network interface for deduplication ................................... About deduplication port usage ...................................................... About deduplication compression ................................................... About dedupliction encryption ....................................................... About optimized synthetic backups and deduplication ........................ About optimized duplication of deduplicated data .............................. About optimized MSDP duplication within the same domain ........................................................................... About duplicating MSDP images to a remote NetBackup master server ............................................................................ About deduplication and SAN Client ................................................ About deduplication performance ................................................... How file size may affect the deduplication rate ............................ About deduplication stream handlers .............................................. About iSCSI, CIFS, and NFS ............................................................ Deployment best practices ............................................................. 18 19 20 21 22 23 24 25 26 27 28 28 29 29 30 30 31 32 42 43 43 45 45 45 45

Contents

About fully qualified domain names .......................................... About scaling deduplication ..................................................... Send initial full backups to the storage server ............................. Increase the number of jobs gradually ........................................ Introduce load balancing servers gradually ................................. Implement client deduplication gradually ................................... Use deduplication compression and encryption ........................... About the optimal number of backup streams .............................. About storage unit groups for deduplication ............................... About protecting the deduplicated data ...................................... Save the deduplication storage server configuration ..................... Plan for disk write caching ....................................................... How deduplication restores work .............................................. Replacing the PureDisk Deduplication Option with Media Server Deduplication on the same host ................................................ Migrating from PureDisk to the NetBackup Media Server Deduplication option .............................................................. Migrating from another storage type to deduplication ........................

46 46 47 47 47 48 48 48 49 49 50 50 50 51 52 53

Chapter 3

Provisioning the storage .................................................... 55

About provisioning the storage ....................................................... About deduplication storage requirements ....................................... About deduplication storage capacity .............................................. About the deduplication storage paths ............................................. About adding additional storage ..................................................... 55 55 56 56 57

Chapter 4

Licensing deduplication ..................................................... 59

About licensing deduplication ........................................................ 59 About the deduplication license key ................................................. 60 Licensing NetBackup deduplication ................................................. 60

Chapter 5

Configuring deduplication ................................................. 61

Configuring NetBackup deduplication .............................................. Configuring a deduplication storage server ....................................... About deduplication pools ............................................................. Configuring a deduplication pool .................................................... Media server deduplication pool properties ................................. Configuring a deduplication storage unit .......................................... Deduplication storage unit properties ........................................ Deduplication storage unit recommendations .............................. Enabling client deduplication ......................................................... 62 63 64 64 65 67 67 68 70

Contents

Enabling deduplication encryption .................................................. About backup policy configuration .................................................. Configuring optimized synthetic backups for deduplication ................. Configuring optimized duplication copy behavior .............................. Configuring optimized duplication of deduplicated data ...................... Throttling optimized duplication traffic ........................................... Adding a NetBackup load balancing server ........................................ About the deduplication pd.conf file ................................................ NetBackup deduplication pd.conf file settings ............................. Editing the deduplication pd.conf file ............................................... About the deduplication storage server configuration file .................... Getting the deduplication storage server configuration ....................... Editing a deduplication storage server configuration file ..................... Setting the deduplication storage server configuration ....................... About the deduplication host configuration file ................................. Deleting a deduplication host configuration file ................................. Resetting the deduplication registry ................................................ Configuring deduplication log file timestamps on Windows ................. Configuring a replication target for MSDP duplication to another domain ................................................................................. Setting NetBackup configuration options by using bpsetconfig ............. Reconfiguring the deduplication storage server and storage paths ...................................................................................

70 72 72 73 75 78 78 80 80 83 84 85 86 87 88 88 89 90 90 92 92

Chapter 6

Monitoring deduplication activity .................................... 95

Monitoring the deduplication rate ................................................... 95 Monitoring deduplication storage capacity and usage ......................... 97 About deduplication capacity and usage reporting ....................... 97 About deduplication container files ........................................... 99 Viewing capacity within deduplication container files ................. 100 Viewing disk reports ................................................................... 100 Monitoring deduplication processes ............................................... 102

Chapter 7

Managing deduplication .................................................. 103

Managing deduplication servers .................................................... Changing deduplication storage server properties ...................... Setting deduplication storage server attributes .......................... Clearing deduplication storage server attributes ........................ Deleting a deduplication storage server .................................... Determining the deduplication storage server state .................... Removing a load balancing server ............................................ Viewing deduplication storage servers ..................................... 103 104 105 105 106 106 107 108

10

Contents

Viewing deduplication storage server attributes ......................... Managing NetBackup Deduplication Engine credentials .................... Adding NetBackup Deduplication Engine credentials .................. Changing NetBackup Deduplication Engine credentials .............. Deleting credentials from a load balancing server ...................... Determining which media servers have deduplication credentials ................................................................................... Managing deduplication pools ...................................................... Changing deduplication disk pool properties ............................. Setting media server deduplication pool attributes ..................... Clearing deduplication pool attributes ...................................... Changing the deduplication pool state ...................................... Changing the deduplication disk volume state ........................... Deleting a deduplication pool .................................................. Determining the deduplication pool state .................................. Determining the deduplication disk volume state ....................... Viewing deduplication pools ................................................... Viewing media server deduplication pool attributes .................... Deleting backup images ............................................................... About maintenance processing ..................................................... Performing deduplication maintenance manually ............................ Resizing the storage partition ....................................................... Restoring files at a remote site ...................................................... Specifying the restore server ........................................................

109 110 110 110 111 111 111 112 113 113 114 114 115 115 116 116 117 117 118 119 119 120 121

Chapter 8

Troubleshooting ................................................................. 123

About deduplication logs ............................................................. About VxUL logs for deduplication .......................................... Troubleshooting installation issues ............................................... Installation on SUSE Linux fails .............................................. Troubleshooting configuration issues ............................................ Database system error (220) ................................................... Server not found error ........................................................... License information failure during configuration ....................... The disk pool wizard does not display a volume .......................... Troubleshooting operational issues ............................................... Verify that the server has sufficient memory ............................. Backup jobs fail .................................................................... Client deduplication fails ....................................................... Volume state changes to DOWN when volume is unmounted ................................................................... Errors, delayed response, hangs .............................................. 123 125 126 127 127 127 128 128 129 130 130 130 131 131 132

Contents

11

Deduplication fails after services are restarted or a domain controller is restarted ..................................................... Cannot delete a disk pool ....................................................... Media open error (83) ............................................................ Media write error (84) ........................................................... Storage full conditions .......................................................... Viewing disk errors and events ..................................................... Deduplication event codes and messages ........................................

132 133 134 135 136 137 137

Chapter 9

Host replacement, recovery, and uninstallation ......... 143

Replacing the deduplication storage server host computer ................. Recovering from a deduplication storage server disk failure ............... Recovering from a permanent deduplication storage server failure ................................................................................ Recovering the storage server after NetBackup catalog recovery ......... About uninstalling media server deduplication ................................ Deactivating media server deduplication ........................................ 143 145 147 149 149 150

Chapter 10

Deduplication architecture .............................................. 153

Deduplication server components .................................................. Media server deduplication process ............................................... Deduplication client components .................................................. Deduplication client backup process .............................................. About deduplication fingerprinting ............................................... Data removal process .................................................................. 153 155 158 158 161 162

Index ................................................................................................................... 165

12

Contents

Chapter

Introducing NetBackup deduplication

This chapter includes the following topics:

About NetBackup deduplication

About NetBackup deduplication

The proprietary Symantec PureDisk deduplication technology powers NetBackup integrated deduplication. Symantec packaged PureDisk into modular components. The components plug-in to NetBackup through the NetBackup OpenStorage framework. With these components, Symantec NetBackup provides the deduplication options that let you deduplicate data everywhere, as close to the source of data as you require. Deduplication everywhere provides significant return on investment, as follows.

Reduce the amount of data that is stored. Reduce backup bandwidth. Reduced bandwidth can be especially important when you want to limit the amount of data that a client sends over the network. Over the network can be to a backup server or for image duplication between remote locations. Reduce backup windows. Reduce infrastructure.

14

Introducing NetBackup deduplication About NetBackup deduplication

About NetBackup deduplication options

Deduplication everywhere lets you choose at which point in the backup process to perform deduplication. NetBackup can manage your deduplication wherever you implement it in the backup stream. Figure 1-1 shows the options for deduplication. Table 1-1 describes the options for deduplication. Figure 1-1
NetBackup client-side deduplication, including clients at remote offices that send the backups to a central data center

NetBackup deduplication

NetBackup media server deduplication

Disk appliance deduplication by using the OpenStorage option

PureDisk deduplication without using NetBackup

Introducing NetBackup deduplication About NetBackup deduplication

15

Table 1-1 Type

NetBackup deduplication options Description

With NetBackup client-side deduplication, clients deduplicate their backup data and then send it directly to the storage destination. A media server does not deduplicate the data. See About NetBackup Client Deduplication on page 26.

NetBackup Client Deduplication Option

NetBackup Media Server NetBackup clients send their backups to a NetBackup media Deduplication Option server, which deduplicates the backup data. A NetBackup media server hosts the NetBackup Deduplication Engine, which writes the data to the storage and manages the deduplicated data. See About the NetBackup Media Server Deduplication Option on page 21. Appliance deduplication The NetBackup OpenStorage option lets third-party vendor appliances function as disk storage for NetBackup. The disk appliance provides the storage and it manages the storage. A disk appliance may provide deduplication functionality. NetBackup backs up and restores client data and manages the life cycles of the data. PureDisk deduplication NetBackup PureDisk is a deduplication solution that is not part of the NetBackup distribution. PureDisk provides bandwidth-optimized backups of data in remote offices. You use PureDisk interfaces to install, configure, and manage the PureDisk servers, storage pools, and client backups. You do not use NetBackup to configure or manage the storage or backups. PureDisk has its own documentation set. See the NetBackup PureDisk Getting Started Guide. A PureDisk storage pool can be a storage destination for both the NetBackup Client Deduplication Option and the NetBackup Media Server Deduplication Option.

How deduplication works

Deduplication is a method of retaining only one unique instance of backup data on storage media. Redundant data is replaced with a pointer to the unique data copy. Deduplication occurs on both a file level and a file segment level. When two or more files are identical, deduplication stores only one copy of the file. When two or more files share identical content, deduplication breaks the files into segments and stores only one copy of each unique file segment.

16

Introducing NetBackup deduplication About NetBackup deduplication

Deduplication significantly reduces the amount of storage space that is required for the NetBackup backup images. Figure 1-2 is a diagram of file segments that are deduplicated. Figure 1-2 File deduplication
File 1 Client files to back up
A B C D E A B

File 2
Q D L

Data written to storage

The following list describes how NetBackup derives unique segments to store:

The deduplication engine breaks file 1 into segments A, B, C, D, and E. The deduplication engine breaks file 2 into segments A, B, Q, D, and L. The deduplication engine stores file segments A, B, C, D, and E from file 1 and file segments Q, and L from file 2. The deduplication engine does not store file segments A, B, and D from file 2. Instead, it points to the unique data copies of file segments A, B, and D that were already written from file 1.

More detailed information is available. See Media server deduplication process on page 155.

Chapter

Planning your deployment

This chapter includes the following topics:

Planning your deduplication deployment About the deduplication tech note About the deduplication storage destination About the NetBackup Media Server Deduplication Option About NetBackup Client Deduplication About NetBackup Deduplication Engine credentials About the network interface for deduplication About deduplication port usage About deduplication compression About dedupliction encryption About optimized synthetic backups and deduplication About optimized duplication of deduplicated data About deduplication and SAN Client About deduplication performance About deduplication stream handlers About iSCSI, CIFS, and NFS Deployment best practices Replacing the PureDisk Deduplication Option with Media Server Deduplication on the same host

18

Planning your deployment Planning your deduplication deployment

Migrating from PureDisk to the NetBackup Media Server Deduplication option Migrating from another storage type to deduplication

Planning your deduplication deployment

Table 2-1 provides an overview of planning your deployment of NetBackup deduplication. Table 2-1 Deployment task
Read the deduplication tech note

Deployment overview Where to find the information

See About the deduplication tech note on page 19.

Determine the storage destination See About the deduplication storage destination on page 20. Determine which type of deduplication to use See About the NetBackup Media Server Deduplication Option on page 21. See About NetBackup Client Deduplication on page 26. Determine the requirements for deduplication hosts See About deduplication servers on page 22. See About deduplication server requirements on page 24. See About client deduplication requirements and limitations on page 27. See About the network interface for deduplication on page 28. See About deduplication port usage on page 29. See About scaling deduplication on page 46. See About deduplication performance on page 43. Determine the credentials for deduplication Read about compression and encryption Read about optimized synthetic backups See About NetBackup Deduplication Engine credentials on page 28. See About deduplication compression on page 29. See About dedupliction encryption on page 30. See About optimized synthetic backups and deduplication on page 30.

Planning your deployment About the deduplication tech note

19

Table 2-1 Deployment task

Deployment overview (continued) Where to find the information

See About optimized duplication of deduplicated data on page 31. See About deduplication stream handlers on page 45. See Deployment best practices on page 45.

Read about optimized duplication

Read about stream handlers

Read about best practices for implementation Determine the storage requirements and provision the storage

See About provisioning the storage on page 55. See About deduplication storage requirements on page 55. See About deduplication storage capacity on page 56. See About the deduplication storage paths on page 56.

Replace a PDDO host or migrate from PDDO to NetBackup deduplication

See Replacing the PureDisk Deduplication Option with Media Server Deduplication on the same host on page 51. See Migrating from PureDisk to the NetBackup Media Server Deduplication option on page 52.

Migrate from other storage to NetBackup deduplication

See Migrating from another storage type to deduplication on page 53.

About the deduplication tech note

Symantec provides a tech note that includes the following:

Currently supported systems Media server and client sizing information Configuration, operational, and troubleshooting updates And more

See the following link:

http://symantec.com/docs/TECH77575

20

Planning your deployment About the deduplication storage destination

About the deduplication storage destination

In NetBackup 7.0 and later, several destinations exist for the deduplication storage, as follows:

A Media Server Deduplication Pool represents the disk storage that is attached to a NetBackup media server. If you use this destination, use this guide to plan, implement, configure, and manage deduplication and the storage. When you configure the storage server, select Media Server Deduplication Pool as the storage type. For a Media Server Deduplication Pool storage destination, all hosts that are used for the deduplication must be NetBackup 7.0 or later. Hosts include the master server, the media servers, and the clients that deduplicate their own data. Integrated deduplication means that the components installed with NetBackup perform deduplication. A PureDisk Deduplication Pool represents a PureDisk storage pool. If you use a PureDisk Deduplication Pool, use the PureDisk documentation to plan, implement, configure, and manage the storage. A PureDisk Deduplication Pool destination requires that PureDisk be at release 6.6 or later. See the NetBackup PureDisk Getting Started Guide. After you configure the storage, use this guide to configure backups and deduplication in NetBackup. When you configure the storage server, select PureDisk Deduplication Pool as the storage type. For a PureDisk Deduplication Pool storage destination, all hosts that are used for the deduplication must be NetBackup 7.0 or later. Hosts include the master server, the media servers, and the clients that deduplicate their own data. Integrated deduplication means that the components installed with NetBackup perform deduplication. A PureDisk storage pool destination that is not represented by a NetBackup construct. The required PureDisk Deduplication Option agent presents the PureDisk storage pool to NetBackup. In PDDO, the PureDisk Storage Pool Authority provides the PureDisk Deduplication Option agent that is installed on the NetBackup media servers. If you use a PureDisk storage pool, use the PureDisk documentation to plan, implement, configure, and manage the storage. For a PureDisk storage pool destination, you can use NetBackup 6.5 or 7.0 NetBackup hosts. Hosts include the master server and the media servers. See the NetBackup PureDisk Deduplication Option Guide.

Table 2-2 shows the storage destinations supported by the two NetBackup releases.

Planning your deployment About the NetBackup Media Server Deduplication Option

21

Table 2-2

NetBackup storage destination support Media Server Deduplication Pool

No Yes

NetBackup release
6.5 7.0

PureDisk storage pool

Yes Yes

About the NetBackup Media Server Deduplication Option

The NetBackup Media Server Deduplication Option exists in the Symantec OpenStorage framework. A storage server writes data to the storage and reads data from the storage; the storage server must be a NetBackup media server. The storage server hosts the core components of deduplication. The storage server also deduplicates the backup data. It is known as a deduplication storage server. For a backup, the NetBackup client software creates the image of backed up files as for a normal backup. The client sends the backup image to the deduplication storage server, which deduplicates the data. The deduplication storage server writes the data to disk. See About deduplication servers on page 22. The NetBackup Media Server Deduplication Option is integrated into NetBackup. It uses the NetBackup administration interfaces, commands, and processes for configuring and executing backups and for configuring and managing the storage. Deduplication occurs when NetBackup backs up a client to a deduplication storage destination. You do not have to use the separate PureDisk interfaces to configure and use deduplication. The NetBackup Media Server Deduplication Option integrates with NetBackup application agents that are optimized for the client stream format. Agents include but are not limited to Microsoft Exchange and Microsoft SharePoint Agents. Figure 2-1 shows NetBackup media server deduplication. The deduplication storage server is a media server on which the deduplication core components are enabled. The storage destination is a Media Server Deduplication Pool.

22

Planning your deployment About the NetBackup Media Server Deduplication Option

Figure 2-1

NetBackup media server deduplication

NetBackup client

NetBackup client

NetBackup client

Load balancing servers

PureDisk

PureDisk

Plug-in

Plug-in

PureDisk

Plug-in

NetBackup Deduplication Engine Deduplication storage server

Media server deduplication pool

A PureDisk storage pool may also be the storage destination. See About the deduplication storage destination on page 20. More detailed information is available. See Deduplication server components on page 153. See Media server deduplication process on page 155.

About deduplication servers

Table 2-3 describes the servers that are used for NetBackup deduplication.

Planning your deployment About the NetBackup Media Server Deduplication Option

23

Table 2-3 Host

NetBackup deduplication servers Description

Deduplication storage One host functions as the storage server for a deduplication node; server that host must be a NetBackup media server. The storage server does the following:

Writes the data to and reads data from the disk storage. Manages that storage.

The storage server also deduplicates data. Therefore, one host both deduplicates the data and manages the storage. Only one storage server exists for each NetBackup deduplication node. You can use NetBackup deduplication with one media server host only: the media server that is configured as the deduplication storage server. See About deduplication nodes on page 23. Load balancing server You can configure other NetBackup media servers to help deduplicate data. They perform file fingerprint calculations for deduplication, and they send the unique results to the storage server. These helper media servers are called load balancing servers. See About deduplication fingerprinting on page 161. A NetBackup media server becomes a load balancing server when two things occur: You enable the media server for deduplication load balancing duties. You do so when you configure the storage server or later by modifying the storage server properties. You select it in the storage unit for the deduplication pool.

See Introduce load balancing servers gradually on page 47. Load balancing servers also perform restore and duplication jobs. Load balancing servers can be any supported server type for deduplication. They do not have to be the same type as the storage server.

About deduplication nodes

A media server deduplication node is a deduplication storage server, load balancing servers (if any), the clients that are backed up, and the storage. Each node manages

24

Planning your deployment About the NetBackup Media Server Deduplication Option

its own storage. Deduplication within each node is supported; deduplication between nodes is not supported. Multiple media server deduplication nodes can exist. Nodes cannot share servers, storage, or clients.

About deduplication server requirements

The host computers CPU and memory constrain how many jobs can run concurrently. The storage server requires enough capability for deduplication and for storage management unless you offload some of the deduplication to load balancing servers. Table 2-4 shows the minimum requirements for deduplication servers. NetBackup deduplication servers are always NetBackup media servers. Processors for deduplication should have a high clock rate and high floating point performance. Furthermore, high throughput per core is desirable. Each backup stream uses a separate core. Intel and AMD have similar performance and perform well on single core throughput. Newer SPARC processors, such as the SPARC64 VII, provide the single core throughput that is similar to AMD and Intel. Alternatively, UltraSPARC T1 and T2 single core performance does not approach that of the AMD and Intel processors. Tests show that the UltraSPARC processors can achieve high aggregate throughput. However, they require eight times as many backup streams as AMD and Intel processors to do so. Table 2-4 Deduplication server minimum requirements Load balancing server or PureDisk Deduplication Option host

Component Storage server

CPU

Symantec recommends at least a Symantec recommends at least a 2.2-GHz clock rate. A 64-bit processor 2.2-GHz clock rate. A 64-bit processor is required. is required. At least four cores are required. Symantec recommends eight cores. See About deduplication performance on page 43. See About maintenance processing on page 118. At least two cores are required. Depending on throughput requirements, more cores may be helpful.

Planning your deployment About the NetBackup Media Server Deduplication Option

25

Table 2-4

Deduplication server minimum requirements (continued) Load balancing server or PureDisk Deduplication Option host
4 GBs.

Component Storage server

RAM 4 GBs to 32 GBs. If your storage exceeds 4 TBs, Symantec recommends at least 1 GB more of memory for every terabyte of additional storage. For example, 10 TBs of back-end data require 10 GBs of RAM, 32 TBs require 32 GBs of RAM, and so on. Operating system The operating system must be a supported 64-bit operating system.

The operating system must be a supported 64-bit operating system.

Note: In some environments, a single host can function as both a NetBackup master server and as a deduplication server. Such environments typically run fewer than 100 total backup jobs a day. (Total backup jobs means backups to any storage destination, including deduplication and nondeduplication storage.) If you perform more than 100 backups a day, deduplication operations may affect master server operations.

Note: If you use an existing media server for deduplication, performance may be inferior to the equipment that meets the deduplication server minimum requirement guidelines.

About media server deduplication limitations

NetBackup media server deduplication and Symantec Backup Exec deduplication cannot reside on the same host. If you use both NetBackup and Backup Exec deduplication, each product must reside on a separate host. NetBackup deduplication components cannot reside on the same host as the PureDisk Deduplication Option (PDDO) agent that is installed from the PureDisk distribution. You cannot upgrade to NetBackup 7.0 or later a NetBackup media server that hosts a PDDO agent. If the NetBackup 7.0 installation detects the PDDO agent, the installation fails. To upgrade a NetBackup media server that hosts a PDDO agent, you must first remove the PDDO agent. You then can use that host as a front end for your PureDisk Storage Pool Authority. (The host must be a host type that is supported for NetBackup deduplication.)

26

Planning your deployment About NetBackup Client Deduplication

See Replacing the PureDisk Deduplication Option with Media Server Deduplication on the same host on page 51. See the NetBackup PureDisk Deduplication Option (PDDO) Guide. Deduplication within each media server deduplication node is supported; global deduplication between nodes is not supported.

About NetBackup Client Deduplication

With normal deduplication, the client sends the full backup data stream to the media server. The deduplication engine on the media server processes the stream, saving only the unique segments. With NetBackup Client Deduplication, the client hosts the PureDisk plug-in that duplicates the backup data. The NetBackup client software creates the image of backed up files as for a normal backup. Next, the PureDisk plug-in breaks the backup image into segments and compares them to all of the segments that are stored in that deduplication node. The plug-in then sends only the unique segments to the NetBackup Deduplication Engine on the storage server. The engine writes the data to a media server deduplication pool. Client deduplication does the following:

Reduces network traffic. The client sends only unique file segments to the storage server. Duplicate data is not sent over the network. Distributes some deduplication processing load from the storage server to clients. (NetBackup does not balance load between clients; each client deduplicates its own data.)

NetBackup Client Deduplication is a solution for the following cases:

Remote office or branch office backups to the data center. LAN connected file server Virtual machine backups.

Client-side deduplication is also a useful solution if a client host has unused CPU cycles or if the storage server or load balancing servers are overloaded. Figure 2-2 shows client deduplication. The deduplication storage server is a media server on which the deduplication core components are enabled. The storage destination is a Media Server Deduplication Pool

Planning your deployment About NetBackup Client Deduplication

27

Figure 2-2

NetBackup client deduplication

NetBackup deduplication client PureDisk NetBackup deduplication client PureDisk

Plug-in

Plug-in

PureDisk

Plug-in NetBackup PureDisk Deduplication Engine Deduplication storage server Media server deduplication pool

A PureDisk storage pool may also be the storage destination. See About the deduplication storage destination on page 20. More information is available. See Deduplication client components on page 158. See Deduplication client backup process on page 158.

About client deduplication requirements and limitations

All hosts that are used for client deduplication must be NetBackup 7.0 or later. For supported systems, see the NetBackup Release Notes. Client deduplication does not support multiple copies per job. For the jobs that specify multiple copies, the backup images are sent to the storage server and may be deduplicated there.

28

Planning your deployment About NetBackup Deduplication Engine credentials

About NetBackup Deduplication Engine credentials

The NetBackup Deduplication Engine requires credentials. The deduplication components use the credentials when they communicate with the NetBackup Deduplication Engine. The credentials are for the engine, not for the host on which it runs. You enter the NetBackup Deduplication Engine credentials when you configure the storage server. The following are the rules for the credentials:

For user names and passwords, you can use characters in the printable ASCII range (0x20-0x7E) except for the following characters:

Asterisk (*) Backward slash (\) and forward slash (/) Double quote (") Left parenthesis [(] and right parenthesis [)]

The user name can be up to 127 characters in length. The password can be up to 100 characters in length. Leading and trailing spaces and quotes are ignored. The user name and password cannot be empty or all spaces.

Record and save the credentials in case you need them in the future. Caution: You cannot change the NetBackup Deduplication Engine credentials after you enter them. Therefore, carefully choose and enter your credentials. If you must change the credentials, contact your Symantec support representative.

About the network interface for deduplication

If the server host has more than one network interface, by default the host operating system determines which network interface to use. However, you can specify which interface NetBackup should use for the deduplication traffic. To use a specific interface, enter that interface name when you configure the deduplication storage server. Caution: You cannot change the network interface after NetBackup configures the deduplication storage server. Therefore, carefully enter the properties.

Planning your deployment About deduplication port usage

29

The NetBackup REQUIRED_INTERFACE setting has no affect on deduplication processes.

About deduplication port usage

The following table shows the ports that are used for NetBackup deduplication. If firewalls exist between the various deduplication hosts, open the indicated ports on the deduplication hosts. Deduplication hosts are the deduplication storage server, the load balancing servers, and the clients that deduplicate their own data. If you have only a storage server and no load balancing servers or clients that deduplicate their own data, you do not have to open firewall ports. Table 2-5 Port
10082

Deduplication ports Usage

The NetBackup Deduplication Engine (spoold). Open this port between the hosts that deduplicate data. The deduplication database (postgres). The connection is internal to the storage server, from spad to spoold. You do not have to open this port. The NetBackup Deduplication Manager (spad). Open this port between the hosts that deduplicate data.

10085

10102

About deduplication compression

NetBackup provides compression for the deduplicated data. It is separate from and different than NetBackup policy-based compression. By default, deduplication compression is enabled. Symantec recommends that you use the deduplication compression. Note: Do not enable compression by selecting the Compression setting on the Attributes tab of the Policy dialog box. If you do, NetBackup compresses the data before it reaches the PureDisk plug-in that deduplicates it. Consequently, deduplication rates are very low. See Use deduplication compression and encryption on page 48. See About the deduplication pd.conf file on page 80. See NetBackup deduplication pd.conf file settings on page 80.

30

Planning your deployment About dedupliction encryption

About dedupliction encryption

NetBackup provides encryption for the deduplicated data. It is separate from and different than NetBackup policy-based encryption. By default, deduplication encryption is disabled. Symantec recommends that you use deduplication encryption. The following is the behavior for the encryption that occurs during the deduplication process:

If you enable encryption on a client that deduplicates its own data, the client encrypts the data before it sends it to the storage server. The data remains encrypted on the storage. Data also is transferred from the client over a Secure Sockets Layer to the server regardless of whether or not the data is encrypted. Therefore, data transfer from the clients that do not deduplicate their own data is also protected. If you enable encryption on a load balancing server, the load balancing server encrypts the data. It remains encrypted on storage. If you enable encryption on the storage server, the storage server encrypts the data. It remains encrypted on storage. If the data is already encrypted, the storage server does not encrypt it.

Deduplication uses the Blowfish algorithm for encryption. Note: Do not enable encryption by selecting the Encryption setting on the Attributes tab of the Policy dialog box. If you do, NetBackup encrypts the data before it reaches the PureDisk plug-in that deduplicates it. Consequently, deduplication rates are very low. See Use deduplication compression and encryption on page 48. See Enabling deduplication encryption on page 70. See About the deduplication pd.conf file on page 80. See NetBackup deduplication pd.conf file settings on page 80.

About optimized synthetic backups and deduplication

Optimized synthetic backups are a more efficient form of synthetic backup. A media server uses messages to instruct the storage server which full and incremental backup images to use to create the synthetic backup. The storage

Planning your deployment About optimized duplication of deduplicated data

31

server constructs (or synthesizes) the backup image directly on the disk storage. Optimized synthetic backups require no data movement across the network.. Optimized synthetic backups are faster than a synthetic backup. Regular synthetic backups are constructed on the media server. They are moved across the network from the storage server to the media server and synthesized into one image. The synthetic image is then moved back to the storage server. The target storage unit's deduplication pool must be the same deduplication pool on which the source images reside. In NetBackup, the Optimizedlmage attribute enables optimized synthetic backup. It applies to both storage servers and deduplication pools. Beginning with NetBackup 7.1, the Optimizedlmage attribute is enabled by default on storage servers and media server deduplication pools. See Configuring optimized synthetic backups for deduplication on page 72. If NetBackup cannot produce the optimized synthetic backup, NetBackup creates the more data-movement intensive synthetic backup. See Setting deduplication storage server attributes on page 105. See Setting media server deduplication pool attributes on page 113.

About optimized duplication of deduplicated data

NetBackup supports several methods for optimized duplication of deduplicated data. The optimized duplication operation is more efficient that normal duplication. Only the unique, deduplicated data segments are transferred. Optimized duplication reduces the amount of data that is transmitted over your network. The following table lists the duplication methods in NetBackup supportes for optimized duplication between media server deduplication pools. Table 2-6 NetBackup OpenStorage optimized duplication methods Description
See About optimized MSDP duplication within the same domain on page 32. See About duplicating MSDP images to a remote NetBackup master server on page 42.

Optimized duplication method

Within the same NetBackup domain

To a remote NetBackup domain

32

Planning your deployment About optimized duplication of deduplicated data

About optimized MSDP duplication within the same domain

Optimized duplication within the same domain a copies the backup images from one Media Server Deduplication Pool to a Media Server Deduplication Pool in another domain. The destination for optimized duplication must be part of the same NetBackup domain. That is, the source and the destination must use the same NetBackup master server. Optimized duplication is a good method to copy your backup images off-site for disaster recovery.

Optimized MSDP duplication within the same domain requirements

The following are the requirements for optimized duplication within the same NetBackup domain:

If the source images reside on a NetBackup Media Server Deduplication Pool, the destination can be another Media Server Deduplication Pool or a PureDisk Deduplication Pool. (In NetBackup, a PureDisk storage pool is configured as a PureDisk Deduplication Pool.) If the destination is a PureDisk storage pool, the PureDisk environment must be at release level 6.6 or later. If the source images reside on a PureDisk storage pool, the destination must be another PureDisk storage pool. Both PureDisk environments must be at release level 6.6 or later. The source storage and the destination storage must have at least one media server in common. See About the media servers for optimized MSDP duplication within the same domain on page 33. In the storage unit you use for the destination for the optimized duplication, you must select only the common media server or media servers. If you select more than one, NetBackup assigns the duplication job to the least busy media server. If you select a media server or servers that are not common, the optimized duplication job fails. For more information about media server load balancing, see the NetBackup Administrator's Guide for UNIX and Linux, Volume I or the NetBackup Administrator's Guide for Windows, Volume I. The destination storage unit cannot be the same as the source storage unit.

Planning your deployment About optimized duplication of deduplicated data

33

Optimized MSDP duplication within the same domain limitations

The following are limitations for optimized duplication within the same NetBackup domain:

You cannot use optimized duplication from a PureDisk storage pool (a PureDisk Deduplication Pool) to a Media Server Deduplication Pool. If an optimized duplication job fails after the configured number of retries, NetBackup does not run the job again. By default, NetBackup retries an optimized duplication job three times. You can change the number of retries. See Configuring optimized duplication copy behavior on page 73. Optimized duplication does not work with storage unit groups. If you use a storage unit group as a destination for optimized duplication, NetBackup uses regular duplication. Optimized duplication does not support multiple copies during an optimized duplication job. If NetBackup is configured to make multiple new copies from the (source) copy of the backup image, the following occurs:

In a storage lifecycle policy, one duplication job creates one optimized duplication copy. If multiple optimized duplication destinations exist, a separate job exists for each destination. This behavior assumes that the device for the optimized duplication destination is compatible with the device on which the source image resides. If multiple remaining copies are configured to go to devices that are not optimized duplication capable, NetBackup uses normal duplication. One duplication job creates those multiple copies. For other duplication methods, NetBackup uses normal duplication. One duplication job creates all of the copies simultaneously. The other duplication methods include the following: NetBackup Vault, the bpduplicate command line, and the duplication option of the Catalog utility in the NetBackup Administration Console.

See Optimized MSDP duplication within the same domain requirements on page 32.

About the media servers for optimized MSDP duplication within the same domain
For optimized Media Server Deduplication Pool duplication within the same domain, the source storage and the destination storage must have at least one media server in common. The common server initiates, monitors, and verifies the

34

Planning your deployment About optimized duplication of deduplicated data

copy operation. The common server requires credentials for both the source storage and the destination storage. (For deduplication, the credentials are for the NetBackup Deduplication Engine, not for the host on which it runs.) Usually, you would use one of the storage servers as the common server, but you can use another media server to control the optimized duplication. It must have the credentials and the connectivity for both the source storage and the destination storage. (You configure the credentials when you configure the storage server or you can add credentials to a media server later.) Which server initiates the optimized duplication determines if the duplication is a push duplication or a pull duplication. Technically, no advantage exists with a push duplication or a pull duplication. However, the media server that initiates the duplication operation also becomes the write host for the new image copies.

About MSDP push duplication within the same domain

Figure 2-3 shows a push configuration for optmized within the same domain. Deduplication node A contains normal backups; deduplication node B is the destination for the optimized duplication copies. Host A has credentials for both nodes; it is the common server. Figure 2-3
Deduplication node A (normal backups) Host A PureDisk Credentials: Host A Host B

Push duplication
Deduplication node B (duplicates) Host B PureDisk Credentials: Host B

Plug-in

Plug-in

NetBackup Deduplication Engine Storage server A

Get ready, here comes data

NetBackup Deduplication Engine Storage server B

Host B, please verify that the data arrived

For Figure 2-3, the following screen shot shows the settings for the storage unit for the normal backups. The disk pool is the Media Server Deduplication Pool in the local environment. Because host A has credentials for both nodes, host A

Planning your deployment About optimized duplication of deduplicated data

35

appears in the storage units for both of the nodes. For node A normal backups, you do not want a remote host deduplicating data, so only host A is selected.

For optimized duplication for Figure 2-3, the following screen shot shows the storage unit settings. The destination is the Media Server Deduplication Pool in the remote environment. You must select the common server, so only host A is selected.

36

Planning your deployment About optimized duplication of deduplicated data

If you use node B for backups also, select host B and not host A in the storage unit for the node B backups. If you select host A, it becomes a load balancing server for the node B deduplication pool. Figure 2-4 shows a push deduplication from a Media Server Deduplication Pool to a PureDisk storage pool. (In NetBackup, a PureDisk storage pool is configured as a PureDiskDeduplicationPool.) The MediaServerDeduplicationPool contains normal backups; the PureDisk Deduplication Pool is the destination for the optimized duplication copies. The local media server has credentials for both environments; it is the common server.

Planning your deployment About optimized duplication of deduplicated data

37

Figure 2-4
Deduplication node A (normal backups) Local_MediaServer PureDisk

Push duplication to a PureDisk storage pool

Remote_MediaServer

Credentials: Local_MediaServer Remote_MediaServer

PureDisk

Plug-in

Plug-in PureDisk content router and storage pool

NetBackup Deduplication Engine Storage server A

Get ready, here comes data

Please verify that the data arrived

PureDisk deduplication pool

Media server deduplication pool (normal backups)

For Figure 2-4, the following screen shot shows the settings for the storage unit for the normal backups. The disk pool is the Media Server Deduplication Pool in the local environment. For normal backups, you do not want a remote host deduplicating data, so only the local host is selected.

38

Planning your deployment About optimized duplication of deduplicated data

For optimized duplication for Figure 2-4, the following screen shot shows the storage unit settings. The disk pool is the PureDisk Deduplication Pool in the remote environment. You must select the common server, so only the local media server is selected. If this configuration were a pull configuration, the remote host would be selected in the storage unit.

Planning your deployment About optimized duplication of deduplicated data

39

Figure 2-5 shows optimized duplication between two PureDisk storage pools. NetBackup media server A has credentials for both storage pools; it initiates, monitors, and verifies the optimized duplication. In the destination storage unit, the common server (media server A) is selected. This configuration is a push configuration. For a PureDisk Deduplication Pool (that is, a PureDisk storage pool), the PureDisk content router functions as the storage server.

40

Planning your deployment About optimized duplication of deduplicated data

Figure 2-5

Storage pool duplication

NetBackup media server A

PureDisk
Storage pool A, send your data to storage pool B

Plug-in

Credentials: Content router A Content router B

Storage pool B, please verify that the data arrived

PureDisk content router and storage pool A

PureDisk content router and storage pool B

You can use a load balancing server when you duplicate between two NetBackup deduplication pools. However, it is more common between two PureDisk storage pools.

About MSDP pull duplication within the same domain

Figure 2-6 shows a pull configuration for optimized duplication within the same domain. Deduplication node A contains normal backups; deduplication node B is the destination for the optimized duplication copies. Host B has credentials for both nodes; it is the common server.

Planning your deployment About optimized duplication of deduplicated data

41

Figure 2-6

Pull duplication
Deduplication node B (duplicates) Credentials: Host A Host B Host B PureDisk
Please verify that the data arrived

Deduplication node A (normal backups) Credentials: Host A Host A PureDisk

Plug-in

Plug-in

Host A, send me data please

NetBackup Deduplication Engine Storage server A

NetBackup Deduplication Engine Storage server B

For Figure 2-6, the storage unit settings for the normal backups are the same as for the push duplication example. For the duplication destination, the following screen shot shows the storage unit settings. They are similar to the push example except host B is selected. Host B is the common server, so it must be selected in the storage unit.

42

Planning your deployment About optimized duplication of deduplicated data

If you use node B for backups also, select host B and not host A in the storage unit for the node B backups. If you select host A, it becomes a load balancing server for the node B deduplication pool.

About duplicating MSDP images to a remote NetBackup master server

Duplication to a remote master server is a new feature in NetBackup 7.1. With this feature, you can duplicate backup images to a different master server domain. The ability to duplicate backups to storage in other NetBackup domains, often across various geographical sites, helps facilitate disaster recovery. Both domains must have deduplication configured and working in their domains. You must use a storage lifecycle policy to copy the backup images to another NetBackup domain. For information about how to configure duplication to a remote master server, see NetBackup Administrator's Guide, Volume I One step in the process is to configure the duplication target in the local storage server's properties.

Planning your deployment About deduplication and SAN Client

43

See Configuring a replication target for MSDP duplication to another domain on page 90. Duplicating images to another NetBackup domain is different than NetBackup optimized duplication. NetBackup optimized duplication copies images to different storage in the same NetBackup domain. See About optimized MSDP duplication within the same domain on page 32.

About deduplication and SAN Client

SAN Client is a NetBackup optional feature that provides high speed backups and restores of NetBackup clients. Fibre Transport is the name of the NetBackup high-speed data transport method that is part of the SAN Client feature. The backup and restore traffic occurs over a SAN. SAN clients can be used with the deduplication option; however, the deduplication must occur on the media server, not the client. Configure the media server to be both a deduplication storage server (or load balancing server) and an FT media server. The SAN client backups are then sent over the SAN to the deduplication server/FT media server host. At that media server, the backup stream is deduplicated. Do not enable client deduplication on SAN Clients. The data processing for deduplication is incompatible with the high-speed transport method of Fibre Transport. Client-side deduplication relies on two-way communication over the LAN with the media server. A SAN client streams the data to the FT media server at a high rate over the SAN.

About deduplication performance

Many factors affect performance, especially the server hardware and the network capacity. Table 2-7 provides information about performance during backup jobs for a deduplication storage server. The deduplication storage server conforms to the minimum host requirements. Client deduplication or load balancing servers are not used. See About deduplication server requirements on page 24.

44

Planning your deployment About deduplication performance

Table 2-7

Deduplication job load performance for a deduplication storage server Description

Initial seeding is when all clients are first backed up. Approximately 15 to 20 jobs can run concurrently under the following conditions: The hardware meets minimum requirements. (More capable hardware improves performance.) No compression. If data is compressed, the CPU usage increases quickly, which reduces the number of concurrent jobs that can be handled. The deduplication rate is between 50% to 100%. The deduplication rate is the percentage of data already stored so it is not stored again. The amount of data that is stored is less than 30% of the capacity of the storage.

When
Initial seeding

Normal operation

Normal operation is when all clients have been backed up once. Approximately 15 to 20 jobs can run concurrently and with high performance under the following conditions: The hardware meets minimum requirements. (More capable hardware improves performance.) No compression. If data is compressed, the CPU usage increases quickly, which reduces the number of concurrent jobs that can be handled. The deduplication rate is between 10% and 50%. The deduplication rate is the percentage of data already stored so it is not stored again. The amount of data that is stored is between 30% to 90% of the capacity of the storage.

Clean up periods

Clean up is when the NetBackup Deduplication Engine performs maintenance such as deleting expired backup image data segments. NetBackup maintains the same number of concurrent backup jobs as during normal operation. However, the average time to complete the jobs increases significantly.

Storage NetBackup maintains the same number of concurrent backup jobs as during approaches normal operation under the following conditions: full capacity The hardware meets minimum requirements. (More capable hardware improves performance.) The amount of data that is stored is between 85% to 90% of the capacity of the storage. However, the average time to complete the jobs increases significantly.

Planning your deployment About deduplication stream handlers

45

How file size may affect the deduplication rate

The small file sizes that are combined with large file segment sizes may result in low initial deduplication rates. However, after the deduplication engine performs file fingerprint processing, deduplication rates improve. For example, a second backup of a client shortly after the first does not show high deduplication rates. But the deduplication rate improves if the second backup occurs after the file fingerprint processing. How long it takes the NetBackup Deduplication Engine to process the file fingerprints varies.

About deduplication stream handlers

NetBackup provides the stream handlers that process various backup data stream types. Stream handlers improve backup deduplication rates by processing the underlying data stream. This release includes the following stream handlers:

Backup Exec Hyper-V NDMP VMWare vmdk VMWare vSphere backup types Windows System State

Symantec continues to develop additional stream handlers to improve backup deduplication performance.

About iSCSI, CIFS, and NFS

NetBackup supports storage area networks (SANs), direct-attached storage (DAS), or internal disks for the deduplication storage. NetBackup deduplication does not support iSCSI, CIFS, or NFS for the deduplication storage.

Deployment best practices

Symantec recommends that you consider the following practices when you implement NetBackup deduplication.

46

Planning your deployment Deployment best practices

About fully qualified domain names

Symantec recommends that you use fully qualified domain names for your NetBackup servers (and by extension, your deduplication servers). Fully qualified domain names can help to avoid host name resolution problems, especially if you use client-side deduplication. Deduplication servers include the storage server and the load balancing servers (if any). See Media write error (84) on page 135.

About scaling deduplication

You can scale deduplication processing to improve performance by using load balancing servers or client deduplication or both. If you configure load balancing servers, those servers also perform deduplication. The deduplication storage server still functions as both a deduplication server and as a storage server. NetBackup uses standard load balancing criteria to select a load balancing server for each job. However, deduplication fingerprint calculations are not part of the load balancing criteria. To completely remove the deduplication storage server from deduplication duties, do the following for every storage unit that uses the deduplication disk pool:

Select Only use the following media servers. Select all of the load balancing servers but do not select the deduplication storage server.

The deduplication storage server performs storage server tasks only: storing and managing the deduplicated data, file deletion, and optimized duplication. If you configure client deduplication, the clients deduplicate their own data. Some of the deduplication load is removed from the deduplication storage server and loading balancing servers. Symantec recommends the following strategies to scale deduplication:

For the initial full backups of your clients, use the deduplication storage server. For subsequent backups, use load balancing servers. Enable client-side deduplication gradually. If a client cannot tolerate the deduplication processing workload, be prepared to move the deduplication processing back to a server.

Planning your deployment Deployment best practices

47

Send initial full backups to the storage server

If you intend to use load balancing servers or client deduplication, use the storage server for the initial full backups of the clients. Then, send subsequent backups through the load balancing servers or use client deduplication for the backups. Deduplication uses the same file fingerprint list regardless of which host performs the deduplication. So you can deduplicate data on the storage server first, and then subsequent backups by another host use the same fingerprint list. If the deduplication plug-in can identify the last full backup for the client and the policy combination, it retrieves the fingerprint list from the server. The list is placed in the fingerprint cache for the new backup. See About deduplication fingerprinting on page 161. Symantec also recommends that you implement load balancing servers and client deduplication gradually. Therefore, it may be beneficial to use the storage server for backups while you implement deduplication on other hosts.

Increase the number of jobs gradually

Symantec recommends that you increase the Maximum concurrent jobs value gradually. (The Maximum concurrent jobs is a storage unit setting.) The initial backup jobs (also known as initial seeding) require more CPU and memory than successive jobs. After initial seeding, the storage server can process more jobs concurrently. Gradually increase the jobs value over time. Testing shows that the upper limit for a storage server with 8 GB of memory and 4 GB of swap space is 50 concurrent jobs.

Introduce load balancing servers gradually

Symantec recommends that you add load balancing servers only after the storage server reaches maximum CPU utilization. Then, introduce load balancing servers one at a time. It may be easier to evaluate how your environment handles traffic and easier to troubleshoot any problems with fewer hosts added for deduplication. Many factors affect deduplication server performance. Because of the various factors, Symantec recommends that you maintain realistic expectations about using multiple servers for deduplication. If you add one media server as a load balancing server, overall throughput should be faster. However, adding one load balancing server may not double the overall throughput rate, adding two load balancing servers may not triple the throughput rate, and so on. If all of the following apply to your deduplication environment, your environment may be a good candidate for load balancing servers:

48

Planning your deployment Deployment best practices

The deduplication storage server is CPU limited on any core. Memory resources are available on the storage server. Network bandwidth is available on the storage server. Back-end I/O bandwidth to the deduplication pool is available. Other NetBackup media servers have CPU available for deduplication.

Gigabit Ethernet should provide sufficient performance in many environments. If your performance objective is the fastest throughput possible with load balancing servers, you should consider 10 Gigabit Ethernet.

Implement client deduplication gradually

If you configure clients to deduplicate their own data, do not enable all of those clients at the same time. Implement client deduplication gradually, as follows:

Use the storage server for the initial backup of the clients. Enable deduplication on only a few clients at a time. It may be easier to evaluate how your environment handles traffic and easier to troubleshoot any problems with fewer hosts added for deduplication.

If a client cannot tolerate the deduplication processing workload, be prepared to move the deduplication processing back to the storage server.

Use deduplication compression and encryption

Do not use compression or encryption in a NetBackup policy; rather, use the compression or the encryption that is part of the deduplication process. See About deduplication compression on page 29.

About the optimal number of backup streams

A backup stream appears as a separate job in the NetBackup Activity Monitor. Various methods exist to produce streams. In NetBackup, you can use backup policy settings to configure multiple streams. The NetBackup for Oracle agent lets you configure multiple streams; also for Oracle the RMAN utilities can provide multiple backup channels. For client deduplication, the optimal number of backup streams is two. Media server deduplication can process multiple streams on multiple cores simultaneously. For large datasets in applications such as Oracle, media server deduplication leverages multiple cores and multiple streams. Therefore, media

Planning your deployment Deployment best practices

49

server deduplication may be a better solution when the application can provide multiple streams or channels. See Planning your deduplication deployment on page 18.

About storage unit groups for deduplication

You can use a storage unit group as a backup destination when the data is to be deduplicated. If you configure more than one deduplication destination, you can put the associated storage units in a storage unit group. Then, you can select the storage unit group as a destination for your backup jobs. This configuration avoids a single point of failure that can interrupt backup service. The best storage savings occur when a backup policy stores its data in the same deduplication destination disk pool instead of across multiple disk pools. For this reason, the Failover method for the Storage unit selection provides the best deduplication performance. All of the other methods are designed to use different storage every time the backup runs. Optimized duplication does not work with storage unit groups. If you use a storage unit group as a destination for optimized duplication, NetBackup uses regular duplication.

About protecting the deduplicated data

Symantec recommends the following methods to protect the deduplicated backup data and the deduplication database:

Use NetBackup optimized duplication to copy the images to another deduplication node off-site location. Optimized duplication copies the primary backup data to another deduplication pool. It provides the easiest, most efficient method to copy data off-site yet remain in the same NetBackup domain. You then can recover from a disaster that destroys the storage on which the primary copies reside by retrieving images from the other deduplication pool. See About optimized MSDP duplication within the same domain on page 32. See Configuring optimized duplication of deduplicated data on page 75. For the primary deduplication storage, use a SAN volume with resilient storage methodologies to replicate the data to a remote site. If the deduplication database is on a different SAN volume, replicate that volume to the remote site also.

The preceding methods help to make your data highly-available.

50

Planning your deployment Deployment best practices

Also, you can use NetBackup to back up the deduplication storage server system or program disks. If the disk on which NetBackup resides fails and you have to replace it, you can use NetBackup to restore the media server.

Save the deduplication storage server configuration

Symantec recommends that you save the storage server configuration. Getting and saving the configuration can help you with recovery of your environment. For disaster recovery, you may need to set the storage server configuration by using a saved configuration file. If you save the storage server configuration, you must edit it so that it includes only the information that is required for recovery. See About the deduplication storage server configuration file on page 84. See Getting the deduplication storage server configuration on page 85. See Editing a deduplication storage server configuration file on page 86.

Plan for disk write caching

Storage components may use hardware caches to improve read and write performance. Among the storage components that may use a cache are disk arrays, RAID controllers, or the hard disk drives themselves. If your storage components use caches for disk write operations, ensure that the caches are protected from power fluctuations or power failure. If you do not protect against power fluctuations or failure, data corruption or data loss may occur. Protection can include the following:

A battery backup unit that supplies power to the cache memory so write operations can continue if power is restored within sufficient time. An uninterruptible power supply that allows the components to complete their write operations.

If your devices that have caches are not protected, Symantec recommends that you disable the hardware caches. Read and write performance may decline, but you help to avoid data loss.

How deduplication restores work

The data is first reassembled on a media server before it is restored, even for the clients that deduplicate their own data. The media server that performs the restore always is a deduplication server (that is, hosts the PureDisk plug-in).

Planning your deployment Replacing the PureDisk Deduplication Option with Media Server Deduplication on the same host

51

The backup server may not be the server that performs the restore. If another server has credentials for the NetBackup Deduplication Engine (that is, for the storage server), NetBackup may use that server for the restore job. NetBackup chooses the least busy server for the restore. The following other servers can have credentials for the NetBackup Deduplication Engine:

A load balancing server in the same deduplication node. A deduplication server in a different deduplication node that is the target of optimized duplication. Optimized duplication requires a server in common between the two deduplication nodes. See About the media servers for optimized MSDP duplication within the same domain on page 33.

You can specify the server to use for restores. See Specifying the restore server on page 121.

Replacing the PureDisk Deduplication Option with Media Server Deduplication on the same host
The PureDisk Deduplication Option provides deduplication of NetBackup backups for NetBackup release 6.5. The destination storage for PDDO is a PureDisk storage pool. The PureDisk agent that performs the deduplication is installed from the PureDisk software distribution, not from the NetBackup distribution. PDDO is not the same as integrated NetBackup deduplication. You can upgrade to 7.0 a NetBackup media server that hosts a PDDO agent and use that server for integrated NetBackup deduplication. The storage can remain the PureDisk storage pool, and NetBackup maintains access to all of the valid backup images in the PureDisk storage pool. If you perform this procedure, the NetBackup PureDisk plug-in replaces the PureDisk agent on the media server. The NetBackup PureDisk plug-in can deduplicate data for either integrated NetBackup deduplication or for a PureDisk storage pool. The PDDO agent can deduplicate data only for a PureDisk storage pool. Note: To use the NetBackup PureDisk plug-in with a PureDisk storage pool, the PureDisk storage pool must be part of a PureDisk 6.6 or later environment.

52

Planning your deployment Migrating from PureDisk to the NetBackup Media Server Deduplication option

Table 2-8 Task

Replacing a PDDO host with a media server deduplication host Procedure

Deactivate all backup policies Deactivate the policies to ensure that no activity occurs on that use the host the host. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I See the NetBackup Administrator's Guide for Windows, Volume I.. Remove the PDDO plug-in NetBackup deduplication components cannot reside on the same host as a PureDisk Deduplication Option (PDDO) agent. Therefore, remove the PDDO agent from the host. See the NetBackup PureDisk Deduplication Option Guide. Upgrade the media server to If the media server runs a version of NetBackup earlier than 7.0 or later 7.0, upgrade that server to NetBackup 7.0 or later. See the NetBackup Installation Guide for UNIX and Linux. See the NetBackup Installation Guide for Windows. Configure the host In the Storage Server Configuration Wizard, select PureDisk Deduplication Pool and enter the name of the Storage Pool Authority. See Configuring a deduplication storage server on page 63. Activate your backup policies See the NetBackup Administrator's Guide for UNIX and Linux, Volume I See the NetBackup Administrator's Guide for Windows, Volume I..

Migrating from PureDisk to the NetBackup Media Server Deduplication option

NetBackup cannot use the storage hardware while PureDisk uses it for storage. The structure of the PureDisk storage is different than the structure of the storage for integrated NetBackup deduplication. The disk systems cannot be used simultaneously by both NetBackup and PureDisk. The PureDisk images on the storage cannot be transferred to the deduplication storage server storage. Therefore, to migrate from NetBackup PureDisk to the NetBackup Media Server Deduplication Option, Symantec recommends that you age the PureDisk storage pool backups until they expire.

Planning your deployment Migrating from another storage type to deduplication

53

Table 2-9 Task

To migrate from PureDisk to NetBackup deduplication Procedure

See the NetBackup Installation Guide for UNIX and Linux. See the NetBackup Installation Guide for Windows. See Configuring NetBackup deduplication on page 62.

Install and configure NetBackup Configure NetBackup deduplication Redirect your backup jobs

Redirect your backup jobs to the NetBackup media server deduplication pool. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I. See the NetBackup Administrator's Guide for Windows, Volume I.

Uninstall PureDisk

After the PureDisk backup images expire, uninstall PureDisk. See your NetBackup PureDisk documentation.

See Migrating from another storage type to deduplication on page 53.

Migrating from another storage type to deduplication

To migrate from another NetBackup storage type to deduplication storage, Symantec recommends that you age the backup images on the other storage until they expire. Symantec recommends that you age the backup images if you migrate from disk storage or tape storage. You should not use the same disk storage for NetBackup deduplication while you use it for other storage such as AdvancedDisk, BasicDisk, or SharedDisk. Each type manages the storage differently and each requires exclusive use of the storage. Also, the NetBackup Deduplication Engine cannot read the backup images that another NetBackup storage type created. Therefore, you should age the data so it expires before you repurpose the storage hardware. Until that data expires, two storage destinations exist: the media server deduplication pool and the other storage. After the images on the other storage expire and are deleted, you can repurpose it for other storage needs. Table 2-10 Task
Configure NetBackup deduplication

Migrating to NetBackup deduplication Procedure

See Configuring NetBackup deduplication on page 62.

54

Planning your deployment Migrating from another storage type to deduplication

Table 2-10 Task

Migrating to NetBackup deduplication (continued) Procedure

Redirect your backup jobs to the media server deduplication pool storage unit. To do so, change the backup policy storage destination to the storage unit for the deduplication pool. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I. See the NetBackup Administrator's Guide for Windows, Volume I.

Redirect your backup jobs

Repurpose the storage

After all of the backup images that are associated with the storage expire, repurpose that storage. If it is disk storage, you cannot add it to an existing media server deduplication pool. You can use it as storage for another, new deduplication node.

See Migrating from PureDisk to the NetBackup Media Server Deduplication option on page 52.

Chapter

Provisioning the storage

This chapter includes the following topics:

About provisioning the storage About deduplication storage requirements About deduplication storage capacity About the deduplication storage paths About adding additional storage

About provisioning the storage

How to provision the storage is beyond the scope of the NetBackup documentation. For help, consult the storage vendor's documentation. What you choose as your storage destination affects how you provision the storage. NetBackup requirements also may affect how you provision the storage. See About the deduplication storage destination on page 20. See Planning your deduplication deployment on page 18.

About deduplication storage requirements

The following defines the storage for the NetBackup Media Server Deduplication Option:
Storage media Connection Disk Storage area network (SAN), direct-attached storage (DAS), or internal disks

56

Provisioning the storage About deduplication storage capacity

Minimum performance

130 MB/sec minimum read and write

The storage must be configured and operational before you can configure deduplication in NetBackup. NetBackup requires exclusive use of the disk resources. If the storage is used for purposes other than backups, NetBackup cannot manage disk pool capacity or manage storage lifecycle policies correctly. Therefore, NetBackup must be the only entity that uses the storage. Local disk storage may leave you vulnerable in a disaster. SAN attached disk can be remounted at a newly provisioned server with the same name.

About deduplication storage capacity

For NetBackup 7.x releases, the maximum deduplication capacity is 32 TBs. The deduplication database and the transaction logs consume some of that storage. NetBackup uses a 90 percent threshold for data to prevent storage overload when the deduplication database capacity is factored in. Although it is unlikely that 10 percent is required for database and transaction logs, it some situations 10 percent may be consumed. For performance optimization, Symantec recommends that you use a separate disk, volume, partition, or spindle for the deduplication database. (If you use separate storage for the deduplication database, NetBackup still uses the 90 percent threshold to protect the data storage from any possible overload.) If your storage requirements exceed the capacity of a media server deduplication node, do one of the following:

Use more than one media server deduplication node. Use a PureDisk storage pool as the storage destination. A PureDisk storage pool provides larger storage capacity; PureDisk 6.6 supports up to 100 TB of deduplicated data. It also provides global deduplication. See About the deduplication storage destination on page 20.

Only one deduplication storage path can exist on a media server. You cannot add another storage path to increase capacity beyond 32 TBs.

About the deduplication storage paths

When you configure the deduplication storage server, you must enter the path name to the storage. The storage path is the directory in which NetBackup stores the raw backup data.

Provisioning the storage About adding additional storage

57

Because the storage requires a directory path, do not use only a root node (/) or drive letter (G:\) as the storage path. You also can specify a different location for the deduplication database. The database path is the directory in which NetBackup stores and maintains the structure of the stored deduplicated data. For performance optimization, Symantec recommends that you use a separate disk, volume, partition, or spindle for the deduplication database. If the directory or directories do not exist, NetBackup creates them and populates them with the necessary subdirectory structure. If the directory or directories exist, NetBackup populates them with the necessary subdirectory structure. The path names must use ASCII characters only. The NetBackup Media Server Deduplication Option does not support NFS mounted file systems. Caution: You cannot change the paths after NetBackup configures the deduplication storage server. Therefore, carefully decide during the planning phase where and how you want the deduplicated backup data stored. See About deduplication storage requirements on page 55.

About adding additional storage

The storage for a NetBackup Media Server Deduplication Pool is exposed as a single disk volume. You cannot add another volume to an existing Media Server Deduplication Pool. To increase the capacity of a Media Server Deduplication Pool, grow the existing volume. See Resizing the storage partition on page 119.

58

Provisioning the storage About adding additional storage

Chapter

Licensing deduplication
This chapter includes the following topics:

About licensing deduplication About the deduplication license key Licensing NetBackup deduplication

About licensing deduplication

The NetBackup deduplication components are installed by default on the supported host systems. However, you must enter a license key to enable deduplication. Before you try to install or upgrade to a NetBackup version that supports deduplication, be aware of the following:

NetBackup supports deduplication on specific 64-bit host operating systems. If you intend to upgrade an existing media server and use it for deduplication, that host must be supported. For the supported systems, see the NetBackup Release Notes. NetBackup deduplication components cannot reside on the same host as a PureDisk Deduplication Option agent. To use a PDDO agent host for NetBackup deduplication, first remove the PDDO agent from that host. See the NetBackup PureDisk Deduplication Option (PDDO) Guide. Then, upgrade that host to NetBackup 7.0 or later. Finally, configure that host as a deduplication storage server or as a load balancing server.

60

Licensing deduplication About the deduplication license key

About the deduplication license key

NetBackup deduplication is licensed separately from base NetBackup. The NetBackup Deduplication Option license key enables both NetBackup Media Server Deduplication and NetBackup Client Deduplication. The license is a front-end capacity license. It is based on the size of the data to be backed up, not on the size of the deduplicated data. You may have a single license key that activates both NetBackup and optional features. Alternatively, you may have one license key that activates NetBackup and another key that activates deduplication. If you remove the NetBackup Deduplication Option license key or if it expires, you cannot create new deduplication disk pools. you also cannot create the storage units that reference NetBackup deduplication pools. NetBackup does not delete the disk pools or the storage units that reference the disk pools. You can use them again if you enter a valid license key.

Licensing NetBackup deduplication

If you installed the license key for deduplication when you installed or upgraded NetBackup, you do not need to perform this procedure. Enter the license key on the NetBackup master server. The following procedure describes how to use the NetBackup Administration Console to enter the license key. To license NetBackup deduplication

1 2 3 4 5

On the Help menu of the NetBackup Administration Console, select License Keys. In the NetBackup License Keys dialog box, click New. In the Add a New License Key dialog box, enter the license key and click Add or OK. Click Close. Restart all the NetBackup services and daemons.

Chapter

Configuring deduplication
This chapter includes the following topics:

Configuring NetBackup deduplication Configuring a deduplication storage server About deduplication pools Configuring a deduplication pool Configuring a deduplication storage unit Enabling client deduplication Enabling deduplication encryption About backup policy configuration Configuring optimized synthetic backups for deduplication Configuring optimized duplication copy behavior Configuring optimized duplication of deduplicated data Throttling optimized duplication traffic Adding a NetBackup load balancing server About the deduplication pd.conf file Editing the deduplication pd.conf file About the deduplication storage server configuration file Getting the deduplication storage server configuration Editing a deduplication storage server configuration file

62

Configuring deduplication Configuring NetBackup deduplication

Setting the deduplication storage server configuration About the deduplication host configuration file Deleting a deduplication host configuration file Resetting the deduplication registry Configuring deduplication log file timestamps on Windows Configuring a replication target for MSDP duplication to another domain Setting NetBackup configuration options by using bpsetconfig Reconfiguring the deduplication storage server and storage paths

Configuring NetBackup deduplication

This topic describes how to configure deduplication in NetBackup. Table 5-1 describes the configuration tasks. The NetBackup administrator's guides describe how to configure a base NetBackup environment. See the NetBackup Administrator's Guide for Windows, Volume I. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I. Table 5-1 Task
Install the license key Configure a deduplication storage server Configure a disk pool

Deduplication configuration tasks Procedure

See Licensing NetBackup deduplication on page 60. See About deduplication servers on page 22. See Configuring a deduplication storage server on page 63. See About deduplication pools on page 64. See Configuring a deduplication pool on page 64.

Configure a storage unit

See Configuring a deduplication storage unit on page 67.

Optionally, enable client-side See Enabling client deduplication on page 70. deduplication Optionally, enable See Enabling deduplication encryption on page 70. compression and encryption

Configuring deduplication Configuring a deduplication storage server

63

Table 5-1 Task

Deduplication configuration tasks (continued) Procedure

Use the deduplication storage unit as the destination for the backup policy. See the NetBackup Administrator's Guide for Windows, Volume I. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I.

Configure a backup policy

Optionally, configure See Configuring optimized synthetic backups for optimized synthetic backups deduplication on page 72. Optionally, configure optimized duplication copy See Configuring optimized duplication of deduplicated data on page 75. See Configuring optimized duplication copy behavior on page 73. See Throttling optimized duplication traffic on page 78. Optionally, specify advanced See About the deduplication pd.conf file on page 80. deduplication settings See Editing the deduplication pd.conf file on page 83. See NetBackup deduplication pd.conf file settings on page 80.

Configuring a deduplication storage server

Configure in this context means to configure a NetBackup media server as a storage server for deduplication. See About deduplication servers on page 22. When you configure a storage server for deduplication, you specify the following:

The type of storage server. For NetBackup media server deduplication, select Media Server Deduplication Pool for the type of disk storage. For a PureDisk deduplication pool, select PureDisk Deduplication Pool for the type of disk storage. The credentials for the deduplication engine. See About NetBackup Deduplication Engine credentials on page 28. The storage paths. See About the deduplication storage paths on page 56.

64

Configuring deduplication About deduplication pools

The network interface. See About the network interface for deduplication on page 28. The load balancing servers, if any. See About deduplication servers on page 22.

When you create the storage server, the wizard lets you create a disk pool and storage unit also. To configure a deduplication storage server by using the wizard

1 2 3

In the NetBackup Administration Console, expand Media and Device Management > Configure Disk Storage Servers. Follow the wizard screens to configure a deduplication storage server. After NetBackup creates the deduplication storage server, you can click Next to continue to the Disk Pool Configuration Wizard.

About deduplication pools

Deduplication pools are the disk pools that are the storage destination for deduplicated backup data. NetBackup media servers or NetBackup clients deduplicate the backup data that is stored in a deduplication pool. NetBackup deduplication disk pools are of type PureDisk. NetBackup requires exclusive ownership of the disk resources that comprise the disk pool. If you share those resources with other users, NetBackup cannot manage disk pool capacity or storage lifecycle policies correctly.

Configuring a deduplication pool

When you create a deduplication storage server, you can launch the Disk Pool Configuration Wizard after NetBackup creates the deduplication storage server. If you created the disk pool already, you do not have to follow these instructions. When you configure a disk pool for deduplication, you specify the following:

The type of disk pool (PureDisk). The NetBackup deduplication storage server to query for the disk storage to use for the pool. The disk volume to include in the pool. NetBackup exposes the storage as a single volume. The disk pool properties. See Media server deduplication pool properties on page 65.

Configuring deduplication Configuring a deduplication pool

65

Symantec recommends that disk pool names be unique across your enterprise. To create a NetBackup disk pool

1 2

In the NetBackup Administration Console, select the Media and Device Management node. From the list of wizards in the Details pane, click Configure Disk Pool and follow the wizard instructions. For help, see the wizard help.

After NetBackup creates the deduplication pool, you have the option to create a storage unit that uses the pool.

Media server deduplication pool properties

Table 5-2 describes the disk pool properties. Table 5-2 Property
Name Storage server

Media server deduplication pool properties Description

The disk pool name. The storage server name. The storage server is the same as the NetBackup media server to which the storage is attached. For a media server deduplication pool, all disk storage is exposed as a single volume. PureDiskVolume is a virtual name for the storage that is contained within the directories you specified for the storage path and the database path.

Disk volume

Update Replication

Query the storage server for its replication capabilities. If the capabilities are different than the current NetBackup configuration, update the configuration. The amount of space available in the disk pool. The total raw size of the storage in the disk pool. A comment that is associated with the disk pool.

Available space Raw size Comment

66

Configuring deduplication Configuring a deduplication pool

Table 5-2 Property

High water mark

Media server deduplication pool properties (continued) Description

The High water mark setting is a threshold that invokes the following actions: The High water mark indicates that the PureDiskVolume is full. When the PureDiskVolume reaches the High water mark, NetBackup fails any backup jobs that are assigned to the storage unit. NetBackup also does not assign new jobs to a storage unit in which the disk pool is full. NetBackup also fails backup jobs if the PureDiskVolume does not contain enough storage for its estimated space requirement. NetBackup begins image cleanup when the PureDiskVolume reaches the High water mark; image cleanup expires the images that are no longer valid. NetBackup again assigns jobs to the storage unit when image cleanup reduces the PureDiskVolume capacity to less than the High water mark.

The default is 98%. Low water mark Limit I/O streams The Low water mark has no affect on the PureDiskVolume. Select to limit the number of read and write streams (that is, jobs) for each volume in the disk pool. A job may read backup images or write backup images. By default, there is no limit. If you select this property, also configure the number of streams to allow per volume. When the limit is reached, NetBackup chooses another volume for write operations, if available. If not available, NetBackup queues jobs until a volume is available. Too many streams may degrade performance because of disk thrashing. Disk thrashing is excessive swapping of data between RAM and a hard disk drive. Fewer streams can improve throughput, which may increase the number of jobs that complete in a specific time period. per volume Select or enter the number of read and write streams to allow per volume. Many factors affect the optimal number of streams. Factors include but are not limited to disk speed, CPU speed, and the amount of memory.

Configuring deduplication Configuring a deduplication storage unit

67

Configuring a deduplication storage unit

Create one or more storage units that reference the disk pool. The Disk Pool Configuration Wizard lets you create a storage unit; therefore, you may have created a storage unit when you created a disk pool. To determine if storage units exist for the disk pool, see the NetBackup Management > Storage > Storage Units window of the Administration Console. To configure a storage unit from the Actions menu

1 2 3

In the NetBackup Administration Console, expand NetBackup Management > Storage > Storage Units. On the Actions menu, select New > Storage Unit. Complete the fields in the New Storage Unit dialog box. For a storage unit for optimized duplication destination, select Only use the following media servers. Then select the media servers that are common between the two deduplication nodes.

See Deduplication storage unit properties on page 67. See Deduplication storage unit recommendations on page 68.

Deduplication storage unit properties

The following are the configuration options for a PureDisk disk pool storage unit. Table 5-3 Property Deduplication storage unit properties Description

Storage unit name A unique name for the new storage unit. The name can describe the type of storage. The storage unit name is the name used to specify a storage unit for policies and schedules. The storage unit name cannot be changed after creation. Storage unit type Select Disk as the storage unit type. Disk type Select PureDisk for the disk type for a media server deduplication pool, a PureDisk deduplication pool, or a PureDisk Deduplication Option storage pool. Select the disk pool that contains the storage for this storage unit. All disk pools of the specified Disk type appear in the Disk pool list. If no disk pools are configured, no disk pools appear in the list.

Disk pool

68

Configuring deduplication Configuring a deduplication storage unit

Table 5-3 Property

Media server

Deduplication storage unit properties (continued) Description

The Media server setting specifies the NetBackup media servers that can deduplicate datafor this storage unit. Only the load balancing servers appear in the media server list. Specify the media server or servers as follows: To allow any server in the media server list to deduplicate data, select Use any available media server. To use specific media servers to deduplicate the data, select Only use the following media servers. Then, select the media servers to allow.

NetBackup selects the media server to use when the policy runs. Maximum fragment size For normal backups, NetBackup breaks each backup image into fragments so it does not exceed the maximum file size that the file system allows. You can enter a value from 20 MBs to 51200 MBs. The Maximumconcurrentjobs setting specifies the maximum number of jobs that NetBackup can send to a disk storage unit at one time. (Default: one job. The job count can range from 0 to 256.) This setting corresponds to the Maximum concurrent write drives setting for a Media Manager storage unit. NetBackup queues jobs until the storage unit is available. If three backup jobs are scheduled and Maximum concurrent jobs is set to two, NetBackup starts the first two jobs and queues the third job. If a job contains multiple copies, each copy applies toward the Maximum concurrent jobs count. Maximum concurrent jobs controls the traffic for backup and duplication jobs but not restore jobs. The count applies to all servers in the storage unit, not per server. If you select multiple media servers in the storage unit and 1 for Maximum concurrent jobs, only one job runs at a time. The number to enter depends on the available disk space and the server's ability to run multiple backup processes.

Maximum concurrent jobs

Warning: A Maximum concurrent jobs setting of 0 disables the storage

unit.

Deduplication storage unit recommendations

You can use storage unit properties to control how NetBackup performs.

Configuring deduplication Configuring a deduplication storage unit

69

Configure a client-to-server ratio

For a favorable client-to-server ratio, you can use one disk pool and configure multiple storage units to separate your backup traffic. Because all storage units use the same disk pool, you do not have to partition the storage. For example, assume that you have 100 important clients, 500 regular clients, and four media servers. You can use two media servers to back up your most important clients and two media servers to back up you regular clients. The following example describes how to configure a favorable client-to-server ratio:

Configure the media servers for NetBackup deduplication and configure the storage. Configure a disk pool. Configure a storage unit for your most important clients (such as STU-GOLD). Select the disk pool. Select Only use the following media servers. Select two media servers to use for your important backups. Create a backup policy for the 100 important clients and select the STU-GOLD storage unit. The media servers that are specified in the storage unit move the client data to the deduplication storage server. Configure another storage unit (such as STU-SILVER). Select the same disk pool. Select Only use the following media servers. Select the other two media servers. Configure a backup policy for the 500 regular clients and select the STU-SILVER storage unit. The media servers that are specified in the storage unit move the client data to the deduplication storage server.

Backup traffic is routed to the wanted data movers by the storage unit settings. Note: NetBackup uses storage units for media server selection for write activity (backups and duplications) only. For restores, NetBackup chooses among all media servers that can access the disk pool.

Throttle traffic to the media servers

You can use the Maximum concurrent jobs settings on disk pool storage units to throttle the traffic to the media servers. Effectively, this setting also directs higher loads to specific media servers when you use multiple storage units for the same disk pool. A higher number of concurrent jobs means that the disk can be busier than if the number is lower.

70

Configuring deduplication Enabling client deduplication

For example, two storage units use the same set of media servers. One of the storage units (STU-GOLD) has a higher Maximum concurrent jobs setting than the other (STU-SILVER). More client backups occur for the storage unit with the higher Maximum concurrent jobs setting.

Enabling client deduplication

To enable Client Deduplication, set an attribute in the NetBackup master server Client Attributes host properties. To specify the clients that deduplicate backups

1 2 3 4 5

In the NetBackup Administration Console, expand NetBackup Management > Host Properties > Master Servers. In the details pane, select the master server. On the Actions menu, select Properties. On the Host Properties General tab, add the clients that use client direct to the Clients list. Select one of the following Deduplication Location options:

Always use the media server disables client deduplication. By default, all clients are configured with the Always use the media server option. Prefer to use client-side deduplication uses client deduplication if the PureDisk plug-in is active on the client. If it is not active, a normal backup occurs; client deduplication does not occur. Always use client-side deduplication uses client deduplication. If the deduplication backup job fails, NetBackup retries the job.

You can override the Prefer to use client-side deduplication or Always use client-side deduplication host property in the backup policies. See Disable client-side deduplication in the NetBackup Administrator's Guide for UNIX and Linux, Volume I. See Disable client-side deduplication in the NetBackup Administrator's Guide for Windows, Volume I.

Enabling deduplication encryption

Two procedures exist to enable encryption during deduplication, as follows:

You can enable encryption on all hosts that deduplicate their own data without configuring them individually.

Configuring deduplication Enabling deduplication encryption

71

Use this procedure if you want all of your clients that deduplicate their own data to encrypt that data. See To enable encryption on all hosts on page 71.

You can enable encryption on individual hosts. Use this procedure to enable compression or encryption on the storage server, on a load balancing server, or on a client that deduplicates its own data. See To enable encryption on a single host on page 71.

See About dedupliction encryption on page 30. To enable encryption on all hosts

On the storage server, open the contentrouter.cfg file in a text editor; it resides in the following directory:
storage_path/etc/puredisk/contentrouter.cfg

Add agent_crypt to the ServerOptions line of the file. The following line is an example:
ServerOptions=fast,verify_data_read,agent_crypt

If you use load balancing servers, make the same edits to the contentrouter.cfg files on those hosts.

To enable encryption on a single host

Use a text editor to open the pd.conf file on the host. The pd.conf file resides in the following directories:

(UNIX) /usr/openv/lib/ost-plugins/ (Windows) install_path\Veritas\NetBackup\bin\ost-plugins

See NetBackup deduplication pd.conf file settings on page 80.

2 3

For the line in the file that contains ENCRYPTION, remove the pound character (#) in column 1 from that line. In that line, replace the 0 (zero) with a 1. Note: The spaces to the left and right of the equal sign (=) in the file are significant. Ensure that the space characters appear in the file after you edit the file.

72

Configuring deduplication About backup policy configuration

Ensure that the LOCAL_SETTINGS parameter is set to 1. If LOCAL_SETTINGS is 0 (zero) and the ENCRYPTION setting on the storage server is 0, the client setting does not override the server setting. Consequently, the data is not encrypted on the client host.

5 6

Save and close the file. If the host is the storage server or a load balancing server, restart the NetBackup Remote Manager and Monitor Service (nbrmms) on the host.

About backup policy configuration

When you configure a backup policy, for the Policy storage select a storage unit that uses a deduplication pool. For a storage lifecycle policy, for the Storage unit select a storage unit that uses a deduplication pool. For VMware backups, select the Mapped full VM backup option when you configure a VMware backup policy. The Mapped full VM backup option provides the best deduplication rates. NetBackup deduplicates the client data that it sends to a deduplication storage unit.

Configuring optimized synthetic backups for deduplication

The following table shows the procedures that are required to configure optimized synthetic backups for a deduplication environment. See About optimized synthetic backups and deduplication on page 30. Table 5-4 Task
Set the Optimizedlmage attribute on the storage server.

To configure optimized synthetic backups for deduplication Procedure

See Setting deduplication storage server attributes on page 105.

Set the Optimizedlmage attribute on your See Setting media server deduplication pool existing deduplication pools. (Any attributes on page 113. deduplication pools that you create after you set the storage server attribute inherit the new functionality.)

Configuring deduplication Configuring optimized duplication copy behavior

73

Table 5-4

To configure optimized synthetic backups for deduplication (continued) Procedure

Task

Configure a Standard or MS-Windows See the administrator's guide for your backup policy. Select the Synthetic backup operating system: attribute on the Schedule Attribute tab of NetBackup Administrator's Guide for the backup policy. UNIX and Linux, Volume I. NetBackup Administrator's Guide for Windows, Volume I.

See Setting deduplication storage server attributes on page 105.

Configuring optimized duplication copy behavior

You can configure the following optimized duplication behaviors:
Optimized duplication failover The default NetBackup behavior is to use optimized duplication between the source and the target storage systems that support optimized duplication. If an optimized duplication job fails, NetBackup does not run the job again. You can configure NetBackup to use normal duplication if optimized duplication fails. For example, NetBackup does not support optimized duplication from PureDisk Storage Pool Authority source to a Media Server Deduplication Pool. Both entities support optimized duplication; however, not in this direction. Therefore, to pull or migrate data out of a PureDisk SPA into a Media Server Deduplication Pool, you must change the default NetBackup failover behavior for optimized duplication. Number of optimized duplication attempts The default is three attempts. You can specify the number of times NetBackup retries an optimized deduplication job before it fails the jobs.

Storage lifecycle policy retry If the optimized deduplication job is configured in a storage wait period lifecycle policy and the job fails, NetBackup retries the job three times. If the job is unsuccessful after three tries, NetBackup waits two hours and then retries the job. You can change the wait period.

74

Configuring deduplication Configuring optimized duplication copy behavior

Caution: These settings affect all optimized duplication jobs; they are not limited to optimized duplication to a Media Server Deduplication Pool or a PureDisk Deduplication Pool. To configure optimized duplication failover

On the master server, add the following configuration option:

RESUME_ORIG_DUP_ON_OPT_DUP_FAIL = TRUE

See Setting NetBackup configuration options by using bpsetconfig on page 92. Alternatively on UNIX systems, add the entry to the bp.conf file on the NetBackup master server. To configure the number of duplication attempts

On the master server, create a file named OPT_DUP_BUSY_RETRY_LIMIT that contains an integer the specifies the number of times to retry the job before NetBackup fails the job. The file must reside in the following directory (depending on the operating system):

UNIX: /usr/openv/netbackup/db/config Windows: install_path\NetBackup\db\config.

To configure the storage lifecycle policy wait period

Change the wait period for retries by adding an IMAGE_EXTENDED_RETRY_PERIOD_IN_HOURS entry to the NetBackup LIFECYCLE_PARAMETERS file. The default for this value is two hours. For example, the following entry configures NetBackup to wait four hours before NetBackup tries the job again:
IMAGE_EXTENDED_RETRY_PERIOD_IN_HOURS 4

The LIFECYCLE_PARAMETERS file resides in the following directories:

UNIX: /usr/openv/netbackup/db/config Windows: install_path\NetBackup\db\config.

Configuring deduplication Configuring optimized duplication of deduplicated data

75

Configuring optimized duplication of deduplicated data

You can configure optimized duplication of deduplicated backups. Before you begin, review the requirements. See About optimized MSDP duplication within the same domain on page 32. Table 5-5 Step Action To configure optimized duplication of deduplicated data Description

Step 1 Configure the storage servers One server must be common between the source storage and the destination storage. Which you choose depends on whether you want a push or a pull configuration. See About the media servers for optimized MSDP duplication within the same domain on page 33. For a push configuration, configure the common server as a load balancing server for the storage server for your normal backups. For a pull configuration, configure the common server as a load balancing server for the storage server for the copies at your remote site. Alternatively, you can add a server later to either environment. (A server becomes a load balancing server when you select it in the storage unit for the deduplication pool.) See Optimized MSDP duplication within the same domain requirements on page 32. See Configuring a deduplication storage server on page 63. Step 2 Configure the deduplication If you did not configure the deduplication pools when you configured the pools storage servers, use the Disk Pool Configuration Wizrd to configure them. See Configuring a deduplication pool on page 64. Step 3 Configure the backup storage In the storage unit for your backups, do the following: unit For the Disk type, select PureDisk.

For the Disk pool, select one of the following: If you back up to integrated NetBackup deduplication, select your Media Server Deduplication Pool. If you back up to a PureDisk environment, select the PureDisk Deduplication Pool.

If you use a pull configuration, do not select the common media server in the backup storage unit. If you do, NetBackup uses it to deduplicate backup data. (That is, unless you want to use it for a load balancing server for the source deduplication node.) See Configuring a deduplication storage unit on page 67.

76

Configuring deduplication Configuring optimized duplication of deduplicated data

Table 5-5 Step Action

To configure optimized duplication of deduplicated data (continued) Description

Symantec recommends that you configure a storage unit specifically to be the target for the optimized duplication. Configure the storage unit in the deduplication node that performs your normal backups. Do not configure it in the node that contains the copies. In the storage unit that is the destination for your duplicated images, do the following:

Step 4 Configure the duplication storage unit

For the Disk type, select PureDisk. For the Disk pool, the destination can be a Media Server Deduplication Pool or a PureDisk Deduplication Pool.

Note: If the backup destination is a PureDisk Deduplication Pool, the

duplication destination also must be a PureDisk Deduplication Pool. Also select Only use the following media servers. Then, select the media server or media servers that are common to both the source storage server and the destination storage server. If you select more than one, NetBackup assigns the duplication job to the least busy media server. If you select only a media server (or servers) that is not common, the optimized duplication job fails. See Configuring a deduplication storage unit on page 67. Step 5 Configure optimized duplication behaviors See Configuring optimized duplication copy behavior on page 73. See Throttling optimized duplication traffic on page 78.

Configuring deduplication Configuring optimized duplication of deduplicated data

77

Table 5-5 Step Action

To configure optimized duplication of deduplicated data (continued) Description

Step 6 Configure a storage lifecycle Configure a storage lifecycle policy only if you want to use one to duplicate policy for the duplication images. The storage lifecycle policy manages both the backup jobs and the duplication jobs. Configure the lifecycle policy in the deduplication environment that performs your normal backups. Do not configure it in the environment that contains the copies. When you configure the storage lifecycle policy, do the following: For the Backup destination, select the storage unit that is the target of your backups. That storage unit may use a Media Server Deduplication Pool or a PureDisk Deduplication Pool. These backups are the primary backup copies; they are the source images for the duplication operation. For the Duplication destination, select the storage unit for the destination deduplication pool. That pool may be a Media Server Deduplication Pool or a PureDisk Deduplication Pool. If the backup destination is a PureDisk Deduplication Pool, the duplication destination also must be a PureDisk Deduplication Pool.

See the NetBackup Administrators Guide for UNIX and Linux or the NetBackup Administrators Guide for Windows. Step 7 Configure a backup policy Configure a policy to back up your clients. Configure the backup policy in the deduplication environment that performs your normal backups. Do not configure it in the environment that contains the copies. If you use a storage lifecycle policy to manage the backup job and the duplication job: Select that storage lifecycle policy in the Policy storage field of the Policy Attributes tab. If you do not use a storage lifecycle policy to manage the backup job and the duplication job: Select the storage unit for the Media Server Deduplication Pool that contains your normal backups. These backups are the primary backup copies.

78

Configuring deduplication Throttling optimized duplication traffic

Table 5-5 Step Action

To configure optimized duplication of deduplicated data (continued) Description

Configure Vault duplication only if you use NetBackup Vault to duplicate the images. Configure Vault in the deduplication environment that performs your normal backups. Do not configure it in the environment that contains the copies. For Vault, you must configure a Vault profile and a Vault policy. Configure a Vault profile. On the Vault Profile dialog box Choose Backups tab, choose the backup images in the source Media Server Deduplication Pool. On the Profile dialog box Duplication tab, select the destination storage unit in the Destination Storage unit field. Configure a Vault policy to schedule the duplication jobs. A Vault policy is a NetBackup policy that is configured to run Vault jobs.

Step 8 Configure NetBackup Vault for the duplication

See the NetBackup Vault Administrators Guide. Step 9 Duplicate by using the bpduplicate command Use the NetBackup bpduplicate command to copy images manually. If you use a storage lifecycle policy or NetBackup Vault for optimized duplication, you do not have to use the bpduplicate command. Duplicate from the source storage to the destination storage. The destination storage may be a Media Server Deduplication Pool or a PureDisk Deduplication Pool. See NetBackup Commands Reference Guide.

Throttling optimized duplication traffic

You can control how much network bandwidth that optimized duplication consumes. The pd.conf file OPTDUP_BANDWIDTH parameter specifies the amount of data to allow. To specify the amount of bandwidth, edit the pd.conf file on the source storage server or the source PureDisk storage pool. Optimized duplication traffic to any deduplication destination is throttled. See About the deduplication pd.conf file on page 80.

Adding a NetBackup load balancing server

You can add a load balancing server to an existing media server deduplication node.

Configuring deduplication Adding a NetBackup load balancing server

79

See About deduplication servers on page 22. To add a load balancing server

1 2 3

In the NetBackup Administration Console, expand Media and Device Management > Credentials > Storage Server Select the deduplication storage server. On the Edit, select Change.

4 5

In the Change Storage Server dialog box, select the Media Servers tab Select the media server or servers that you want to use as a load balancing server. It must be a supported host. The media servers that are checked are configured as load balancing servers.

6 7

Click OK. For all storage units in which Only use the following media servers is configured, ensure that the new load balancing server is selected.

80

Configuring deduplication About the deduplication pd.conf file

About the deduplication pd.conf file

On each NetBackup host that deduplicates data, a pd.conf file contains various configuration settings that control the operation of deduplication for the host. You can edit the file to configure advanced settings for that host. If you change the pd.conf file on a host, it changes the settings for that host only. If you want the same settings for all of the hosts that deduplicate data, you must change the pd.conf file on all of the hosts. The pd.conf file resides in the following directories:

(UNIX) /usr/openv/lib/ost-plugins/ (Windows) install_path\Veritas\NetBackup\bin\ost-plugins

NetBackup deduplication pd.conf file settings

The following table describes the deduplication settings that you can configure. The parameters in this table are in alphabetical order; the parameters in a pd.conf file may not be in alphabetical order. Table 5-6 Setting Default value pd.conf file values Possible values Action

BACKUPRESTORERANGE N/A

Classless Inter-Domain Specifies the IP address or range of Routing format or addresses of the local network interface comma-separated list card (NIC) for backups and restores. of IP addresses 0 (no limit) to the Determines the maximum bandwidth that practical system limit, is allowed when backing up or restoring data between the media server and the in KBs/sec deduplication pool. The value is specified in KBytes/second. The default is no limit. 0 (off) or 1 (on) Specifies whether you want compression. By default, files are not compressed. If you want compression, change the value to 1. See About deduplication compression on page 29.

BANDWIDTH_LIMIT

COMPRESSION

Configuring deduplication About the deduplication pd.conf file

81

Table 5-6 Setting

DEBUGLOG

pd.conf file values (continued) Possible values Action

Specifies the file to which NetBackup writes the log information. NetBackup prepends a date stamp to each day's log file. On Windows, a partition identifier and slash must precede the file name. On UNIX, a slash must precede the file name. Any file extension You can specify a list of comma-separated file name extensions. Files in the backup stream that have the specified extensions are given a single segment if smaller than 16 MB. Larger files are deduplicated using the maximum 16-MB segment size. Example: DONT_SEGMENT_TYPES = mp3,avi This setting prevents NetBackup from analyzing and managing segments within the file types that do not deduplicate globally.

Default value

UNIX: Any path /var/log/puredisk/ pdplugin.log Windows: C:\pdplugin.log

DONT_SEGMENT_TYPES N/A

ENCRYPTION

0 (off) or 1 (on)

Specifies whether to encrypt the data. By default, files are not encrypted. If you want encryption, change the value to 1. If you set this parameter to 1 on all hosts, the data is encrypted during transfer and on the storage. See About dedupliction encryption on page 30.

LOCAL_SETTINGS

0 (allow override) or 1 Specifies whether to allow the pd.conf (always use local settings of the deduplication storage settings) server to override the settings in the local pd.conf file.

82

Configuring deduplication About the deduplication pd.conf file

Table 5-6 Setting

LOGLEVEL

pd.conf file values (continued) Possible values

An integer, 0 to 10 inclusive

Default value
0

Action
Specifies the amount of information that is written to the log file. The range is from 0 to 10, with 10 being the most logging.

Note: Change this value only when

directed to do so by a Symantec representative. MAX_IMG_MBSIZE 50,000 0 to 50,000, in MBs The maximum backup image fragment size in megabytes.

Note: Change this value only when

directed to do so by a Symantec representative. MAX_LOG_MBSIZE 500 0 to 50,000, in MBs The maximum size of the log file in megabytes.

OPTDUP_BANDWITH

0 (no limit) to the Determines the maximum bandwidth that practical system limit, is allowed for optimized duplication. The value is specified in KBytes/second. in KBs/sec 0 (off) or 1 (on) Specifies whether to compress optimized duplication data. By default, files are not compressed. If you want compression, change the value to 1. See About deduplication compression on page 29.

OPTDUP_COMPRESSION 1

OPTDUP_ENCRYPTION 0

0 (off) or 1 (on)

Specifies whether to encrypt the optimized duplication data. By default, files are not encrypted. If you want encryption, change the value to 1. If you set this parameter to 1 on all hosts, the data is encrypted during transfer and on the storage. See About dedupliction encryption on page 30.

OPTDUP_TIMEOUT

N/A

The value, expressed in minutes

Specifies the number of minutes before the optimized duplication times out. Indicated in minutes.

Configuring deduplication Editing the deduplication pd.conf file

83

Table 5-6 Setting

SEGKSIZE

pd.conf file values (continued) Possible values Action

Default value
128

32 to 16384 KBs, The file segment size in kilobytes. The increments of 32 only value must be a multiple of 32.

Warning: Changing this value can reduce

capacity and decrease performance. Change this value only when directed to do so by a Symantec representative. WS_RETRYCOUNT 3 Integer This parameter applies to the PureDisk Deduplication Option only. It does not affect NetBackup deduplication. The WS_RETRYCOUNT parameter allows you to configure the amount of retries that will be attempted in case the Web service fails or times out. WS_TIMEOUT 120 Integer This parameter applies to the PureDisk Deduplication Option only. It does not affect NetBackup deduplication. The WS_TIMEOUT parameter allows you to increase or decrease the timeout value for Web service calls made from NetBackup media servers to PureDisk storage units.

Editing the deduplication pd.conf file

If you change the pd.conf file on a host, it changes the settings for that host only. If you want the same settings for all of the hosts that deduplicate data, you must change the pd.conf file on all of the hosts. See About the deduplication pd.conf file on page 80. See NetBackup deduplication pd.conf file settings on page 80. To edit the pd.conf file

Use a text editor to open the pd.conf file. The pd.conf file resides in the following directories:

(UNIX) /usr/openv/lib/ost-plugins/

84

Configuring deduplication About the deduplication storage server configuration file

(Windows) install_path\Veritas\NetBackup\bin\ost-plugins

2 3

To activate a setting, remove the pound character (#) in column 1 from each line that you want to edit. To change a setting, specify a new value. Note: The spaces to the left and right of the equal sign (=) in the file are significant. Ensure that the space characters appear in the file after you edit the file.

4 5

Save and close the file. Restart the NetBackup Remote Manager and Monitor Service (nbrmms) on the host.

About the deduplication storage server configuration file

You can save your storage server settings in a text file. A saved storage server configuration file contains the configuration settings for your storage server. It also contains status information about the storage. A saved configuration file may help you with recovery of your storage server. Therefore, Symantec recommends that you get the storage server configuration and save it in a file. The file does not exist unless you create it. The following is an example of a populated configuration file:
V7.0 V7.0 V7.0 V7.0 V7.0 V7.0 V7.0 V7.0 V7.0 V7.0 V7.0 V7.0 V7.0 "storagepath" "D:\DedupeStorage" string "spalogpath" "D:\DedupeStorage\log" string "dbpath" "D:\DedupeStorage" string "required_interface" "HOSTNAME" string "spalogretention" "7" int "verboselevel" "3" int "replication_target(s)" "none" string "Storage Pool Size" "698.4GB" string "Storage Pool Used Space" "132.4GB" string "Storage Pool Available Space" "566.0GB" string "Catalog Logical Size" "287.3GB" string "Catalog files Count" "1288" string "Space Used Within Containers" "142.3GB" string

Configuring deduplication Getting the deduplication storage server configuration

85

V7.0 represents the version of the I/O format not the NetBackup release level. The version may differ on your system. If you get the storage server configuration when the server is not configured or is down and unavailable, NetBackup creates a template file. The following is an example of a template configuration file:
V7.0 V7.0 V7.0 V7.0 V7.0 V7.0 V7.0 "storagepath" " " string "spalogin" " " string "spapasswd" " " string "spalogretention" "7" int "verboselevel" "3" int "dbpath" " " string "required_interface" " " string

To use a storage server configuration file for recovery, you must edit the file so that it includes only the information that is required for recovery. See Getting the deduplication storage server configuration on page 85. See Editing a deduplication storage server configuration file on page 86. See Setting the deduplication storage server configuration on page 87.

Getting the deduplication storage server configuration

Symantec recommends that you get and save the storage server configuration file. The storage server configuration file can help with recovery. See About the deduplication storage server configuration file on page 84. See Recovering from a deduplication storage server disk failure on page 145. See Recovering from a permanent deduplication storage server failure on page 147. To get the storage server configuration

On the master server, enter the following command:

UNIX: /usr/openv/netbackup/bin/admincmd/nbdevconfig -getconfig

-storage_server sshostname -stype PureDisk -configlist file.txt

Windows: install_path\NetBackup\bin\admincmd\nbdevconfig -getconfig

-storage_server sshostname -stype PureDisk -configlist file.txt

For sshostname, use the name of the storage server. For file.txt, use a file name that indicates its purpose.

86

Configuring deduplication Editing a deduplication storage server configuration file

If you get the file when a storage server is not configured or is down and unavailable, NetBackup creates a template file.

Editing a deduplication storage server configuration file

To use a storage server configuration file for recovery, it must contain only the required information. You must remove any pointintime status information. (Status information is only in a configuration file that was saved on an active storage server.) You also must add several configuration settings that are not included in a saved configuration file or a template configuration file. Table 5-7 shows the configuration lines that are required. Table 5-7 Configuration setting
V7.0 "storagepath" " " string

Required lines for a recovery file Description

The value should be the same as the value that was used when you configured the storage server. For the spalogpath, use the storagepath value and append log to the path. For example, if the storagepath is D:\DedupeStorage, enter D:\DedupeStorage\log. If the database path is the same as the storagepath value, enter the same value for dbpath. Otherwise, enter the path to the database. A value for required_interface is required only if you configured one initially; if a specific interface is not required, leave it blank. In a saved configuration file, the required interface defaults to the computer's hostname. Do not change this value. Do not change this value.

V7.0 "spalogpath" " " string

V7.0 "dbpath" " " string

V7.0 "required_interface" " " string

V7.0 "spalogretention" "7" int V7.0 "verboselevel" "3" int

V7.0 "replication_target(s)" "none" string A value for replication_target(s) is required only if you configured optimized duplication. Otherwise, do not edit this line. V7.0 "spalogin" "username" string Replace username with the NetBackup Deduplication Engine user ID. Replace password with the password for the NetBackup Deduplication Engine user ID.

V7.0 "spapasswd" "password" string

Configuring deduplication Setting the deduplication storage server configuration

87

See About the deduplication storage server configuration file on page 84. See Recovering from a deduplication storage server disk failure on page 145. See Recovering from a permanent deduplication storage server failure on page 147. To edit the storage server configuration

If you did not save a storage server configuration file, get a storage server configuration file. See Getting the deduplication storage server configuration on page 85.

Use a text editor to enter, change, or remove values. Remove lines from and add lines to your file until only the required lines (see Table 5-7) are in the configuration file. Enter or change the values between the second set of quotation marks in each line. A template configuration file has a space character (" ") between the second set of quotation marks.

Setting the deduplication storage server configuration

You can set the storage server configuration (that is, configure the storage server) by importing the configuration from a file. Setting the configuration can help you with recovery of your environment. See Recovering from a deduplication storage server disk failure on page 145. See Recovering from a permanent deduplication storage server failure on page 147. To set the configuration, you must have an edited storage server configuration file. See About the deduplication storage server configuration file on page 84. See Getting the deduplication storage server configuration on page 85. See Editing a deduplication storage server configuration file on page 86. Note: The only time you should use the nbdevconfig command with the -setconfig option is for recovery of the host or the host disk.

88

Configuring deduplication About the deduplication host configuration file

To set the storage server configuration

On the master server, run the following command: UNIX: /usr/openv/netbackup/bin/admincmd/nbdevconfig -setconfig
-storage_server sshostname -stype PureDisk -configlist file.txt

Windows: install_path\NetBackup\bin\admincmd\nbdevconfig -setconfig

-storage_server sshostname -stype PureDisk -configlist file.txt

For sshostname, use the name of the storage server. For file.txt, use the name of the file that contains the configuration.

About the deduplication host configuration file

Each NetBackup host that is used for deduplication has a configuration file; the file name matches the name of the storage server, as follows:
storage_server_name.cfg

The storage_server_name is the fully qualified domain name if that was used to configure the storage server. For example, if the storage server name is DedupeServer.symantecs.org, the configuration file name is DedupeServer.symantecs.org.cfg. The following is the location of the file: UNIX: /usr/openv/lib/ost-plugins Windows: install_path\Veritas\NetBackup\bin\ost-plugins

Deleting a deduplication host configuration file

You may need to delete the configuration file from the deduplication hosts. For example, to reconfigure your deduplication environment or disaster recovery may require that you delete the configuration file on the servers on which it exists. See About the deduplication host configuration file on page 88. To delete the configuration file

Delete the file; it's location depends on the operating system type, as follows: UNIX: /usr/openv/lib/ost-plugins Windows: install_path\Veritas\NetBackup\bin\ost-plugins

Configuring deduplication Resetting the deduplication registry

89

Resetting the deduplication registry

If you reconfigure your deduplication environment, one of the steps is to reset the deduplication registry. See Reconfiguring the deduplication storage server and storage paths on page 92. Warning: Only follow these procedures if you are reconfiguring your storage server and storage paths. The procedure differs on UNIX and on Windows. To reset the deduplication registry file on UNIX and Linux

Enter the following commands on the storage server to reset the deduplication registry file:
rm /etc/pdregistry.cfg cp -f /usr/openv/pdde/pdconfigure/cfg/userconfigs/pdregistry.cfg /etc/pdregistry.cfg

To reset the deduplication registry on Windows

Delete the contents of the following keys in the Windows registry:

HKLM\SOFTWARE\Symantec\PureDisk\Agent\ConfigFilePath HKLM\SOFTWARE\Symantec\PureDisk\Agent\EtcPath

Warning: Editing the Windows registry may cause unforeseen results.

Delete the storage path in the following key in the Windows key. That is, delete everything after postgresql-8.3 -D in the key.
HKLM\SYSTEM\ControlSet001\Services\postgresql-8.3\ImagePath

For example, in the following example registry key, you would delete the content of the key that is in italic type:
"C:\Program Files\Veritas\pdde\pddb\bin\pg_ctl.exe" runservice -N postgresql-8.3 -D "D:\DedupeStorage\databases\pddb\data" -w

The result is as follows:

"C:\Program Files\Veritas\pdde\pddb\bin\pg_ctl.exe" runservice -N postgresql-8.3 -D

90

Configuring deduplication Configuring deduplication log file timestamps on Windows

Configuring deduplication log file timestamps on Windows

The default configuration for the PostgreSQL database does not add timestamps to log entries on Windows systems. Therefore, Symantec recommends that you edit the PostgreSQL configuration file on Windows hosts so timestamps are added to the log file. To configure log file timestamps on Windows

Use a text editor to open the following file:

dbpath\databases\pddb\data\postgresql.conf

The database path may be the same as the configured storage path.

2 3 4

In the line that begins with log_line_prefix, change the value from %%t to %t. (That is, remove one of the percent signs (%).) Save the file. Run the following command:
install_path\Veritas\pdde\pddb\bin\pg_ctl reload -D dbpath\databases\pddb\data

If the command output does not include server signaled, use Windows Computer Management to restart the PostgreSQL Server 8.3 service. See About deduplication logs on page 123.

Configuring a replication target for MSDP duplication to another domain

To duplicate images to a remote master server domain, you must specify the target storage server in the source environment's storage server's properties. Doing so establishes the relationship between the two domains. Configuring the target storage server is only one step in a process that is documented in the NetBackup adminstrator's guide. See About duplicating MSDP images to a remote NetBackup master server on page 42. Caution: Choose the target storage server or servers carefully. A target storage server must not also be a storage server for the source domain.

Configuring deduplication Configuring a replication target for MSDP duplication to another domain

91

To configure a replication target

1 2 3 4

In the NetBackup Administration Console, expand Media and Device Management > Credentials > Storage Server Select the deduplication storage server. On the Edit menu, select Change. In the Change Storage Server dialog box, select the Replication tab.

To add a replication target:

Enter the Storage Server Name Enter the Username and Password for the NetBackup Deduplication Engine on the target storage server. Click Add.

All targets are considered for duplication, depending on the rules of the storage lifecycle policies that control the duplication.

When finished adding replication targets, click OK.

92

Configuring deduplication Setting NetBackup configuration options by using bpsetconfig

Setting NetBackup configuration options by using bpsetconfig

Symantec recommends that you use the NetBackup Administration Console Host Properties to configure NetBackup. However, some configuration options cannot be set by using the Administration Console. You can set those configuration options by using the bpsetconfig command. Alternatively, on UNIX systems you can set configuration options in the bp.conf file. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I. To set configuration options by using the bpsetconfig command

On the host on which you want to set configuration options, write the current configuration to a file by running the following command: UNIX: /usr/openv/netbackup/bin/admincmd/bpgetconfig h hostname
> file.txt

Windows: install_path\NetBackup\bin\admincmd\bpgetconfig h
hostname > file.txt

Edit and save the file. You can change the values of the options that are in the file. You can add option and value pairs. Ensure that you understand the values that are allowed and the format of any new options that you add.

Upload the configuration by running the following command: UNIX: /usr/openv/netbackup/bin/admincmd/bpsetconfig h hostname
file.txt

Windows: install_path\NetBackup\bin\admincmd\bpsetconfig h
hostname file.txt

Reconfiguring the deduplication storage server and storage paths

An activated NetBackup Deduplication Engine rejects all configuration attempts that abandon its previously defined storage and backup images. Therefore, if you

Configuring deduplication Reconfiguring the deduplication storage server and storage paths

93

made a mistake during configuration, you must manually deactivate the engine and physically delete the storage directory before you can reconfigure deduplication. Similarly, if you want to change the storage path, you must deactivate the engine and delete the storage directory. Common configuration mistakes include the following:

Choosing a media server with an unsupported operating system. All NetBackup media servers appear in the Storage Server Configuration Wizard. Choosing a storage path that does not include a directory name. For example, D:\DedupeData is a valid storage path, but D:\ is not.

Two aspects to the configuration exist: the record of the deduplication storage in the EMM database and the physical presence of the storage on disk (the populated storage directory). Deleting the deduplication storage server does not alter the contents of the storage on physical disk. To protect against inadvertent data loss, NetBackup does not automatically delete the storage when you delete the storage server. Warning: Deleting valid backup images may cause data loss. Table 5-8 Task Reconfigure media server deduplication Procedure

Ensure that no deduplication Deactivate all backup policies that use deduplication storage. activity occurs See the NetBackup Administrator's Guide for UNIX and Linux, Volume I See the NetBackup Administrator's Guide for Windows, Volume I. Expire backup images Expire all backup images that reside on the deduplication disk storage. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I See the NetBackup Administrator's Guide for Windows, Volume I. Delete the storage units that See the NetBackup Administrator's Guide for UNIX and use the disk pool Linux, Volume I See the NetBackup Administrator's Guide for Windows, Volume I. Delete the disk pool See Deleting a deduplication pool on page 115.

94

Configuring deduplication Reconfiguring the deduplication storage server and storage paths

Table 5-8 Task

Reconfigure media server deduplication (continued) Procedure

Stop the NetBackup services See the NetBackup Administrator's Guide for UNIX and on the storage server Linux, Volume I. See the NetBackup Administrator's Guide for Windows, Volume I. Delete the storage directory Delete the storage directory and database directory (if you and the database directory configured a database directory). Reset the deduplication registry See Resetting the deduplication registry on page 89.

Start the NetBackup services See the NetBackup Administrator's Guide for UNIX and on the media server Linux, Volume I. See the NetBackup Administrator's Guide for Windows, Volume I. Delete the deduplication storage server Delete the storage server configuration file See Deleting a deduplication storage server on page 106.

The storage server and every load balancing server contain a deduplication configuration file. Delete that file from every server that you use for deduplication. See Deleting a deduplication host configuration file on page 88.

Reconfigure

See Configuring NetBackup deduplication on page 62.

Chapter

Monitoring deduplication activity

This chapter includes the following topics:

Monitoring the deduplication rate Monitoring deduplication storage capacity and usage Viewing disk reports Monitoring deduplication processes

Monitoring the deduplication rate

The deduplication rate is the percentage of data that was stored already. That data is not stored again. NetBackup reports the rate of deduplication as follows:

The Deduplication Rate column of the Activity Monitor Jobs tab. The Job Details dialog box. The Detailed Status tab shows detailed information, including the deduplication rate. The information depends on whether it is media server deduplication or client-side deduplication, as follows:

For media server deduplication, the Detailed Status tab shows the deduplication rate on the server that performed the deduplication. The following job details excerpt shows details for a client for which Server_A deduplicated the data (the dedup field shows the deduplication rate):

96

Monitoring deduplication activity Monitoring the deduplication rate

10/6/2010 10:02:09 AM - Info Server_A(pid=30695) StorageServer=PureDisk:Server_A; Report=PDDO Stats for (Server_A): scanned: 30126998 KB, stream rate: 162.54 MB/sec, CR sent: 1720293 KB, dedup: 94.3%, cache hits: 214717 (94.0%)

The other fields that show deduplication information are highlighted in the example. For the field descriptions, see Table 6-1.

For client-side deduplication jobs, the Detailed Status tab shows two deduplication rates. The first deduplication rate is always for the client data. The second duplication rate is for the disk image header and True Image Restore information (if applicable). That information is always deduplicated on a server; typically, deduplication rates for that information are zero or very low. The following job details example excerpt shows the two rates:
10/8/2010 11:54:21 PM - Info Server_A(pid=2220) Using OpenStorage client direct to backup from client Client_B to Server_A 10/8/2009 11:58:09 PM - Info Server_A(pid=2220) StorageServer=PureDisk:Server_A; Report=PDDO Stats for (Server_A): scanned: 3423425 KB, stream rate: 200.77 MB/sec, CR sent: 122280 KB, dedup: 96.4%, cache hits: 49672 (98.2%) 10/8/2010 11:58:09 PM - Info Server_A(pid=2220) Using the media server to write NBU data for backup Client_B_1254987197 to Server_A 10/8/2010 11:58:19 PM - Info Server_A(pid=2220) StorageServer=PureDisk:Server_A; Report=PDDO Stats for (Server_A): scanned: 17161 KB, stream rate: 1047.42 MB/sec, CR sent: 17170 KB, dedup: 0.0%, cache hits: 0 (0.0%) the requested operation was successfully completed(0)

The bpdbjobs command shows the deduplication rate if you configure a COLDREFS entry for DEDUPRATIO in the bp.conf file on the media server on which you run the command. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I.

Many factors affect deduplication performance. See About deduplication performance on page 43.

Monitoring deduplication activity Monitoring deduplication storage capacity and usage

97

Table 6-1 Field

cache hits

Deduplication activity field descriptions Description

The percentage of time that the local fingerprint cache contained a record of the segment. The deduplication plug-in did not have to query the database about the segment. The amount of data that is sent from the deduplication plug-in to the component that stores the data. (In NetBackup, the NetBackup Deduplication Engine stores the data. In PureDisk, a content router stores the data.) If the storage server deduplicates the data, it does not travel over the network. The deduplicated data travels over the network when the deduplication plug-in runs on a computer other than the storage server, as follows: On a NetBackup client that deduplicates its own data (client-side deduplication). On a fingerprinting media server that deduplicates the data. The plug-in on the fingerprinting server sends the data to the storage server, which writes it to a Media Server Deduplication Pool. On a media server that then sends it to a PureDisk environment for storage. (In NetBackup, a PureDisk Storage Pool represents the storage of a PureDisk environment.)

CR sent

dedup

The percentage of data that was stored already. That data is not stored again. The amount of data that the deduplication plug-in scanned. The speed of the scan: The kilobytes of data that are scanned divided by how long the scan takes.

scanned stream rate

Monitoring deduplication storage capacity and usage

Several options exist to monitor your deduplication storage capacity and usage. See About deduplication capacity and usage reporting on page 97. See About deduplication container files on page 99. See Viewing capacity within deduplication container files on page 100.

About deduplication capacity and usage reporting

Several factors may affect the expected NetBackup deduplication capacity and usage results, as follows:

98

Monitoring deduplication activity Monitoring deduplication storage capacity and usage

Expired backups may not change the available size and the used size. An expired backup may have no unique data segments. Therefore, the segments remain valid for other backups. NetBackup Deduplication Manager clean-up may not have run yet. The Deduplication Manager performs clean up twice a day. Until it performs clean-up, deleted image fragments remain on disk.

If you use operating system tools to examine storage space usage, their results may not match the usage reported by NetBackup, as follows:

The operating system tools cannot report usage accurately. The storage implementation uses container files. Deleted segments can leave free space in container files, but the container file sizes do not change. If other applications use the storage, NetBackup cannot report usage accurately. NetBackup requires exclusive use of the storage.

Table 6-2 describes the options for monitoring capacity and usage. Table 6-2 Option
Change Storage Server dialog box

Capacity and usage reporting Description

The Change Storage Server dialog box Properties tab displays storage capacity and usage. This dialog box displays the most current capacity usage that is available in the NetBackup Administration Console. See Changing deduplication storage server properties on page 104.

Disk Pools window

The Disk Pools window of the Administration Console displays a value that was stored when NetBackup polled the disk pools. NetBackup polls every 5 minutes; therefore, the value may not be as current as the value that is displayed in the Storage Server window. To display the window, expand Media and Device Management > Devices > Disk Pools.

Disk Pool Status report

The Disk Pool Status report displays the state of the disk pool and usage information. See Viewing disk reports on page 100.

Monitoring deduplication activity Monitoring deduplication storage capacity and usage

99

Table 6-2 Option

Disk Logs report

Capacity and usage reporting (continued) Description

The Disk Logs report displays event and message information. A useful event for monitoring capacity is event 1044; the following is the description of the even in theDisk Logs report: The usage of one or more system resources has exceeded a warning level. The threshold for this message is at 90% capacity. No more data can be stored. See Viewing disk reports on page 100. See Deduplication event codes and messages on page 137.

License Keys dialog box The summary of active capacity-based license features in the NetBackup License Keys dialog box. The summary displays the storage capacity for which you are licensed and the capacity used. It does not display the amount of physical storage space. On the Help menu in the NetBackup Administration Console, select License Keys. View container command A command that is installed with NetBackup provides a view of storage capacity and usage within the deduplication container files. See About deduplication container files on page 99. See Viewing capacity within deduplication container files on page 100. The nbdevquery command The nbdevquery command shows the state of the disk volume and its properties and attributes. It also shows capacity, usage, and percent used. See Determining the deduplication disk volume state on page 116. NetBackup OpsCenter The NetBackup OpsCenter also provides information about storage capacity and usage. See the NetBackup OpsCenter Administrator's Guide.

About deduplication container files

The deduplication storage implementation allocates container files to hold backup data. Deleted segments can leave free space in containers files, but the container

100

Monitoring deduplication activity Viewing disk reports

file sizes do not change. Segments are deleted from containers when backup images expire and the NetBackup Deduplication Manager performs clean-up.

Viewing capacity within deduplication container files

A deduplication command reports on storage usage within containers. The following is the pathname of the command:

On UNIX and Linux systems, the path name of the command is /usr/openv/pdde/pdcr/bin/crcontrol. On Windows systems, the path name of the command is install_path\Veritas\pdde\Crcontrol.exe.

The following is an example of the command usage on a Windows deduplication storage server. The command shows the data store statistics (--dsstat option).
C:\Program Files\Veritas\pdde>Crcontrol.exe --dsstat 1 ************ Data Store statistics ************ Data storage Size Used Avail Use% 68.4G 46.4G 22.0G 68% Number of containers : 67 Average container size : 187441541 bytes (178.76MB) Space allocated for containers : 12558583274 bytes (11.70GB) Space used within containers : 12551984871 bytes (11.69GB) Space available within containers: 6598403 bytes (6.29MB) Space needs compaction : 508432 bytes (0.48MB) Records marked for compaction : 3 Active records : 95755 Total records : 95758

The NetBackup Deduplication Manager periodically compacts the space available inside the container files. Therefore, space within a container is not available as soon as it is free. Various internal parameters control whether a container file is compacted. Although space may be available within a container file, the file may not be eligible for compaction. The NetBackup Deduplication Manager checks for space every 20 seconds. For help with the command options, use the --help option.

Viewing disk reports

The NetBackup disk reports include information about the disk pools, disk storage units, disk logs, images that are stored on disk media, and storage capacity.

Monitoring deduplication activity Viewing disk reports

101

Table 6-3 describes the disk reports available. Table 6-3 Report
Images on Disk

Disk reports Description

The Images on Disk report generates the image list present on the disk storage units that are connected to the media server. The report is a subset of the Images on Media report; it shows only disk-specific columns. The report provides a summary of the storage unit contents. If a disk becomes bad or if a media server crashes, this report can let you know what data is lost.

Disk Logs

The Disk Logs report displays the media errors or the informational messages that are recorded in the NetBackup error catalog. The report is a subset of the Media Logs report; it shows only disk-specific columns. Either PureDisk or Symantec Deduplication Engine in the description identifies a deduplication message. (The identifiers are generic because the deduplication engine does not know which application consumes its resources. NetBackup, Symantec Backup Exec, and NetBackup PureDisk are Symantec applications that use deduplication.

Disk Storage Unit

The Disk Storage Unit Status report displays the state of disk storage units in the current NetBackup configuration. For disk pool capacity, see the disk pools window in Media and Device Management > Devices > Disk Pools. Multiple storage units can point to the same disk pool. When the report query is by storage unit, the report counts the capacity of disk pool storage multiple times.

Disk Pool Status

The Disk Pool Status report displays the state of disk pool and usage information.

To view disk reports

1 2 3 4

In the NetBackup Administration Console, expand NetBackup Management > Reports > Disk Reports. Select the name of a disk report. In the right pane, select the report settings. Click Run Report.

102

Monitoring deduplication activity Monitoring deduplication processes

Monitoring deduplication processes

The following table shows the deduplication processes about which NetBackup reports: See Deduplication server components on page 153. Table 6-4 What
NetBackup Deduplication Engine

Where to monitor the main deduplication processes Where to monitor it

On Windows systems, in the NetBackup Administration Console Activity Monitor Services tab. On UNIX, the NetBackup Deduplication Engine appears as spoold in the Administration Console Activity Monitor Daemons tab. The NetBackupbpps command shows the spoold process.

NetBackup Deduplication Manager

On Windows systems, NetBackup Deduplication Manager in the Activity Monitor Services tab. On UNIX, the NetBackup Deduplication Manager appears as spad in the Administration Console Activity Monitor Daemons tab. The NetBackup bpps command shows the spad process.

The database On Windows systems, the postgres database processes appear processes (postgres) in the Activity Monitor Processes tab. The NetBackup bpps command shows the postgres processes.

Chapter

Managing deduplication
This chapter includes the following topics:

Managing deduplication servers Managing NetBackup Deduplication Engine credentials Managing deduplication pools Deleting backup images About maintenance processing Performing deduplication maintenance manually Resizing the storage partition Restoring files at a remote site Specifying the restore server

Managing deduplication servers

After you configure deduplication, you can perform various tasks to manage deduplication servers. See Changing deduplication storage server properties on page 104. See Setting deduplication storage server attributes on page 105. See Clearing deduplication storage server attributes on page 105. See Deleting a deduplication storage server on page 106. See Determining the deduplication storage server state on page 106. See Removing a load balancing server on page 107. See Viewing deduplication storage servers on page 108.

104

Managing deduplication Managing deduplication servers

See Viewing deduplication storage server attributes on page 109.

Changing deduplication storage server properties

You can change the retention period and logging level for the NetBackup Deduplication Manager. To change deduplication storage server properties

1 2 3 4

In the NetBackup Administration Console, expand Media and Device Management > Credentials > Storage Server Select the deduplication storage server. On the Edit menu, select Change. In the Change Storage Server dialog box, select the Properties tab.

5 6 7

For the property to change, select the value in the Value column. Change the value. Click OK.

Managing deduplication Managing deduplication servers

105

Setting deduplication storage server attributes

You may have to set storage server attributes to enable new functionality. For example, to support optimized synthetic backups, you must set an attribute. If you set an attribute on the storage server, you may have to set the same attribute on existing deduplication pools. The overview or configuration procedure for the new functionality describes the requirements. See Setting media server deduplication pool attributes on page 113. See About optimized synthetic backups and deduplication on page 30. To set a storage server attribute

The following is the command syntax to set a storage server attribute. Run the command on the master server or on the storage server.
nbdevconfig -changests -storage_server storage_server -stype PureDisk -setattribute attribute -media_server media_server

The following describe the options that require your input:

-storage_server storage_server -setattribute attribute The name of the storage server.

The attribute is the name of the argument that represents the new functionality. For example, OptimizedImage specifies that the environment supports the optimized synthetic backup method.

-media_server media_server

A NetBackup media server that connects to the storage server. The media server queries the storage server for its capabilities. Because the storage server is also a media server, use the storage server name.

The following is the path to the nbdevconfig command:

UNIX: /usr/openv/netbackup/bin/admincmd Windows: install_path\NetBackup\bin\admincmd

Clearing deduplication storage server attributes

Use the nbdevconfig command to remove storage server attributes. To clear deduplication storage server attributes

Run the following command on the NetBackup master server or on a storage server:

106

Managing deduplication Managing deduplication servers

nbdevconfig -changests -storage_server storage_server -stype PureDisk -clearattribute attribute -storage_server storage_server -setattribute attribute The name of the storage server.

The attribute is the name of the argument that represents the functionality.

The following is the path to the nbdevconfig command:

UNIX: /usr/openv/netbackup/bin/admincmd Windows: install_path\NetBackup\bin\admincmd

Deleting a deduplication storage server

If you delete a deduplication storage server, NetBackup disables the deduplication functionality and the storage server functionality on that media server. NetBackup does not delete the media server from your configuration. To delete the media server, use the NetBackup nbemmcmd command. If a disk pool is configured from the disk volume that the deduplication storage server manages, you cannot delete the deduplication storage server. Warning: Do not delete a deduplication storage server if its storage contains unexpired NetBackup images; if you do, data loss may occur. To delete a deduplication storage server

1 2 3

In the NetBackup Administration Console, expand Media and Device Management > Credentials > Storage Server On the Edit menu, select Delete. Click Yes in the confirmation dialog box.

See Reconfiguring the deduplication storage server and storage paths on page 92.

Determining the deduplication storage server state

Use the NetBackup nbdevquery command to determine the state of a deduplication storage server. The state is either UP or DOWN.

Managing deduplication Managing deduplication servers

107

To determine deduplication storage server state

Run the following command on the NetBackup master server or a deduplication storage server: UNIX: /usr/openv/netbackup/bin/admincmd/nbdevquery -liststs
-storage_server server_name -stype PureDisk U

Windows: install_path\NetBackup\bin\admincmd\nbdevquery -liststs

-storage_server server_name -stype PureDisk U

The following is example output:

Storage Server : bit Storage Server Type : PureDisk Storage Type : Formatted Disk, Network Attached State : UP

This example output is shortened; more flags may appear in actual output.

Removing a load balancing server

You can remove a load balancing server from a deduplication node. The media server no longer deduplicates client data. See About deduplication servers on page 22. After you remove the load balancing server, restart the NetBackup Enterprise Media Manager service. The NetBackup disk polling service may try to use the removed server to query for disk status. Because the server is no longer a load balancing server, it cannot query the disk storage. Consequently, NetBackup may mark the disk volume as DOWN. When the EMM service restarts, it chooses a different deduplication server to monitor the disk storage. If the host failed and is unavailable, you can use the tpconfig device configuration utility in menu mode to delete the server. However, you must run the tpconfig utility on a UNIX or Linux NetBackup server. For procedures, see the NetBackup Administrators Guide for UNIX and Linux, Volume II.

108

Managing deduplication Managing deduplication servers

To remove a media server from a deduplication node

For every storage unit that specifies the media server in Use one of the following media servers, clear the check box that specifies the media server. This step is not required if the storage unit is configured to use any available media server.

2 3

In the NetBackup Administration Console, expand Media and Device Management > Credentials > Storage Server. Select the deduplication storage server, then select Edit > Change.

4 5 6

In the Change Storage Server dialog box, select the Media Servers tab. Clear the check box of the media server you want to remove. Click OK.

Viewing deduplication storage servers

Use the NetBackup Administration Console to view a list of deduplication storage servers already configured.

Managing deduplication Managing deduplication servers

109

To view deduplication storage servers

In the NetBackup Administration Console, expand Media and Device Management > Credentials > Storage Server. The All Storage Servers pane shows all configured deduplication storage servers. deduplication storage servers show PureDisk in the Disk Type column.

Viewing deduplication storage server attributes

Use the NetBackup nbdevquery command to view the deduplication storage server attributes. The server_name you use in the nbdevquery command must match the configured name of the storage server. If the storage server name is its fully-qualified domain name, you must use that for server_name. To view deduplication storage server attributes

The following is the command syntax to set a storage server attribute. Run the command on the NetBackup master server or on the deduplication storage server: UNIX: /usr/openv/netbackup/bin/admincmd/nbdevquery -liststs
-storage_server server_name -stype PureDisk U

Windows: install_path\NetBackup\bin\admincmd\nbdevquery -liststs

-storage_server server_name -stype PureDisk U

The following is example output:

Storage Server : bit Storage Server Type : PureDisk Storage Type : Formatted Disk, Network Attached State : UP Flag : OpenStorage Flag : CopyExtents Flag : AdminUp Flag : InternalUp Flag : LifeCycle Flag : CapacityMgmt Flag : OptimizedImage Flag : FT-Transfer

This example output is shortened; more flags may appear in actual output.

110

Managing deduplication Managing NetBackup Deduplication Engine credentials

Managing NetBackup Deduplication Engine credentials

You can manage existing credentials in NetBackup. See Adding NetBackup Deduplication Engine credentials on page 110. See Changing NetBackup Deduplication Engine credentials on page 110. See Deleting credentials from a load balancing server on page 111. See Determining which media servers have deduplication credentials on page 111.

Adding NetBackup Deduplication Engine credentials

You may need to add the NetBackup Deduplication Engine credentials to an existing storage server or load balancing server. For example, disaster recovery may require that you add the credentials. Add the same credentials that you already use in your environment. Another procedure exists to add a load balancing server to your configuration. See Adding a NetBackup load balancing server on page 78. To add NetBackup Deduplication Engine credentials by using the tpconfig command

On the host to which you want to add credentials, run the following command: UNIX: /usr/openv/volmgr/bin/tpconfig -add -storage_server
sshostname -stype PureDisk -sts_user_id UserID -password PassWord

Windows: install_path\Veritas\NetBackup\Volmgr\bin\tpconfig -add

-storage_server sshostname -stype PureDisk -sts_user_id UserID -password PassWord

For sshostname, use the name of the storage server.

Changing NetBackup Deduplication Engine credentials

You cannot change the NetBackup Deduplication Engine credentials after you enter them. If you must change the credentials, contact your Symantec support representative. See About NetBackup Deduplication Engine credentials on page 28.

Managing deduplication Managing deduplication pools

111

Deleting credentials from a load balancing server

You may need to delete the NetBackup Deduplication Engine credentials from a load balancing server. For example, disaster recovery may require that you delete the credentials on a load balancing server. Another procedure exists to remove a load balancing server from a deduplication node. See Removing a load balancing server on page 107. To delete credentials from a load balancing server

On the load balancing server, run the following command: UNIX: /usr/openv/volmgr/bin/tpconfig -delete -storage_server
sshostname -stype PureDisk -sts_user_id UserID

Windows: install_path\Veritas\NetBackup\Volmgr\bin\tpconfig -delete

-storage_server sshostname -stype PureDisk -sts_user_id UserID

For sshostname, use the name of the storage server.

Determining which media servers have deduplication credentials

You can determine which media servers have credentials configured for the NetBackup Deduplication Engine. The servers with credentials are load balancing servers. To determine if NetBackup Deduplication Engine credentials exist

1 2 3

In the NetBackup Administration Console, expand Media and Device Management > Credentials > Storage Server. Select the storage server, then select Edit > Change. In the Change Storage Server dialog box, select the Media Servers tab. The media servers for which credentials are configured are checked.

Managing deduplication pools

After you configure NetBackup deduplication, you can perform various tasks to manage your deduplication disk pools. See Changing deduplication disk pool properties on page 112. See Setting media server deduplication pool attributes on page 113. See Clearing deduplication pool attributes on page 113. See Changing the deduplication pool state on page 114.

112

Managing deduplication Managing deduplication pools

See Changing the deduplication disk volume state on page 114. See Deleting a deduplication pool on page 115. See Determining the deduplication pool state on page 115. See Determining the deduplication disk volume state on page 116. See Viewing deduplication pools on page 116. See Viewing media server deduplication pool attributes on page 117.

Changing deduplication disk pool properties

You can change the properties of a deduplication disk pool. To change disk pool properties

1 2 3

In the NetBackup Administration Console, expand Media and Device Management > Devices > Disk Pools. Select the disk pool you want to change in the details pane. On the Edit menu, select Change.

In the Change Disk Pool dialog box, change properties. See Media server deduplication pool properties on page 65.

Managing deduplication Managing deduplication pools

113

Setting media server deduplication pool attributes

You may have to set attributes on your existing media server deduplication pools. For example, if you configure the storage server to support optimized synthetic backups, you must then configure the existing deduplication pools for the same support. See Setting deduplication storage server attributes on page 105. To set a Media Server Deduplication Pool attribute

The following is the command syntax to set a deduplication pool attribute. Run the command on the master server or on the storage server.
nbdevconfig -changedp -dp pool_name -stype PureDisk -setattribute attribute

The following describe the options that require your input:

-changedp pool_name -setattribute attribute The name of the disk pool.

The attribute is the name of the argument that represents the new functionality. For example, OptimizedImage specifies that the environment supports the optimized synthetic backup method.

The following is the path to the nbdevconfig command:

UNIX: /usr/openv/netbackup/bin/admincmd Windows: install_path\NetBackup\bin\admincmd

Clearing deduplication pool attributes

You may have to clear attributes on your existing media server deduplication pools. To clear a Media Server Deduplication Pool attribute

The following is the command syntax to clear a deduplication pool attribute. Run the command on the master server or on the storage server.
nbdevconfig -changedp -dp pool_name -stype PureDisk -clearattribute attribute

The following describe the options that require your input:

114

Managing deduplication Managing deduplication pools

-changedp pool_name -setattribute attribute

The name of the disk pool.

The attribute is the name of the argument that represents the new functionality.

The following is the path to the nbdevconfig command:

UNIX: /usr/openv/netbackup/bin/admincmd Windows: install_path\NetBackup\bin\admincmd

Changing the deduplication pool state

Disk pool state is UP or DOWN. To change the state to DOWN, the disk pool must not be busy. If backup jobs are assigned to the disk pool, the state change fails. Cancel the backup jobs or wait until the jobs complete. To change deduplication pool state

1 2 3 4

In the NetBackup Administration Console, expand Media and Device Management > Device Monitor. Select the Disk Pools tab. Select the disk pool. Select either Actions > Up or Actions > Down.

See About deduplication pools on page 64. See Determining the deduplication pool state on page 115. See Media server deduplication pool properties on page 65. See Configuring a deduplication pool on page 64.

Changing the deduplication disk volume state

The disk volume state is UP or DOWN. To change the state to DOWN, the disk pool in which the volume resides must not be busy. If backup jobs are assigned to the disk pool, the state change fails. Cancel the backup jobs or wait until the jobs complete. To change the deduplication disk volume state

Determine the name of the disk volume. The following command lists all volumes in the specified disk pool:

Managing deduplication Managing deduplication pools

115

nbdevquery -listdv -stype PureDisk -dp disk_pool_name

The nbdevquery and the nbdevconfig commands reside in the following directory:

UNIX: /usr/openv/NetBackup/bin/admincmd Windows: install_path\NetBackup\bin\admincmd

To display the disk volumes in all disk pools, omit the -dp option.

Change the disk volume state; the following is the command syntax:
nbdevconfig -changestate -stype PureDisk -dp disk_pool_name dv vol_name -state state

The state is either UP or DOWN.

Deleting a deduplication pool

You can delete a disk pool if it does not contain valid NetBackup backup images or image fragments. If it does, you must first expire and delete those images or fragments. If you delete a disk pool, NetBackup removes it from your configuration. If a disk pool is the storage destination of a storage unit, you must first delete the storage unit. To delete a deduplication disk pool

1 2 3 4

In the NetBackup Administration Console, expand Media and Device Management > Devices > Disk Pools. Select a disk pool On the Edit menu, select Delete. In the Delete Disk Pool dialog box, verify that the disk pool is the one you want to delete and then click OK.

Determining the deduplication pool state

The disk pool state is UP or DOWN. To determine disk pool state

1 2 3

In the NetBackup Administration Console, expand Media and Device Management > Device Monitor. Select the Disk Pools tab. The state is displayed in the Status column.

116

Managing deduplication Managing deduplication pools

Determining the deduplication disk volume state

Use the NetBackup nbdevquery command to determine the state of the volume in a deduplication disk pool. The command shows the properties and attributes of the PureDiskVolume. To determine deduplication disk volume state

Display the volume state by using the following command: UNIX: /usr/openv/netbackup/bin/admincmd/nbdevquery -listdv -stype
PureDisk -U

Windows: install_path\NetBackup\bin\admincmd\nbdevquery -listdv

-stype PureDisk -U

The state is either UP or DOWN. The following is example output

Disk Pool Name Disk Type Disk Volume Name Disk Media ID Total Capacity (GB) Free Space (GB) Use% Status Flag Flag Flag Num Read Mounts Num Write Mounts Cur Read Streams Cur Write Streams : : : : : : : : : : : : : : : PD_Disk_Pool PureDisk PureDiskVolume @aaaab 49.98 43.66 12 UP ReadOnWrite AdminUp InternalUp 0 1 0 0

Viewing deduplication pools

Use the NetBackup Administration Console to view configured disk pools. To view disk pools

In the NetBackup Administration Console, expand Media and Device Management > Devices > Disk Pools.

Managing deduplication Deleting backup images

117

Viewing media server deduplication pool attributes

Use the NetBackup nbdevquery command to view deduplication pool attributes. To view deduplication pool attributes

The following is the command syntax to view the attributes of a deduplication pool. Run the command on the NetBackup master server or on the deduplication storage server: UNIX: /usr/openv/netbackup/bin/admincmd/nbdevquery -listdp -dp
pool_name -stype PureDisk U

Windows: install_path\NetBackup\bin\admincmd\nbdevquery -listdp

-dp pool_name -stype PureDisk U

The following is example output:

Disk Pool Name Disk Pool Id Disk Type Status Flag Flag Flag Flag Flag Flag Raw Size (GB) Usable Size (GB) Num Volumes High Watermark Low Watermark Max IO Streams Storage Server : : : : : : : : : : : : : : : : : MediaServerDeduplicationPool MediaServerDeduplicationPool PureDisk UP OpenStorage AdminUp InternalUp LifeCycle CapacityMgmt OptimizedImage 235.76 235.76 1 98 80 -1 DedupeServer.symantecs.org (UP)

This example output is shortened; more flags may appear in actual output.

Deleting backup images

Image deletion may be time consuming. Therefore, if you delete images manually, Symantec recommends the following approach. See Data removal process on page 162.

118

Managing deduplication About maintenance processing

To delete backup images manually

Expire all of the images by using the bpexpdate command and the -notimmediate option. The -notimmediate option prevents bpexpdate from calling the nbdelete command, which deletes the image. Without this option, bpexpdate calls nbdelete to delete images. Each call to nbdelete creates a job in the Activity Monitor, allocates resources, and launches processes on the media server.

After you expire the last image, delete all of the images by using the nbdelete command with the allvolumes option. Only one job is created in the Activity Monitor, fewer resources are allocated, and fewer processes are started on the media servers. The entire process of expiring images and deleting images takes less time.

About maintenance processing

The following are the background maintenance processes of the NetBackup Deduplication Engine:
Queue processing Operations that require database updates accumulate in a transaction queue. Twice a day, the NetBackup Deduplication Manager directs the deduplication engine to process the queue as one batch. The schedule is frequency-based. By default, queue processing occurs every 12 hours, 20 minutes past the hour. Queue processing consumes two CPU cores. Queue processing writes to the deduplication engine storaged.log file. See About deduplication logs on page 123. Garbage collection In a few rare scenarios, some data segments may become orphaned. Garbage collection cleans these segments up by removing them. Garbage collection is an unobtrusive process; once a week the NetBackup Deduplication Manager directs the deduplication engine to collect and remove garbage.

Because maintenance processing occurs automatically, you should not need to invoke those processes manually. However, you may do so. See Performing deduplication maintenance manually on page 119.

Managing deduplication Performing deduplication maintenance manually

119

Because maintenance processing does not block any other deduplication process, rescheduling should not be necessary. Users cannot change the maintenance process schedules. However, if you must reschedule these processes, contact your Symantec support representative. See About deduplication server requirements on page 24.

Performing deduplication maintenance manually

Usually, you should not need to run the maintenance processes manually. However, you can do so; the processes are transaction queue processing and garbage collection. See About maintenance processing on page 118. A control command launches the queue processing and garbage collection. The following is the path name of the command:

On UNIX and Linux systems, /usr/openv/pdde/pdcr/bin/crcontrol. On Windows systems, install_path\Veritas\pdde\Crcontrol.exe.

To process the transaction queue manually

Run the control command with the --processqueue option. The following is an example on a Windows system:
install_path\Veritas\pdde\Crcontrol.exe --processqueue

To examine the results, run the control command with the --dsstat 1 option (number 1 not lowercase letter l). The command may run for a long time; if you omit the 1, results return more quickly but they are not as accurate. See Viewing capacity within deduplication container files on page 100.

To collect garbage manually

Run the control command with the -v -m +1,+2 --noreport options. The following is an example on a UNIX system:
/usr/openv/pdde/pdcr/bin/crcollect -v -m +1,+2 --noreport

Resizing the storage partition

If the volume that contains the deduplication storage is resized dynamically, restart the NetBackup services on the storage server. You must restart the services so that NetBackup can use the resized partition correctly. If you do not restart the services, NetBackup reports the capacity as full prematurely.

120

Managing deduplication Restoring files at a remote site

To resize the deduplication storage

1 2

Stop all NetBackup jobs on the storage on which you want to change the disk partition sizes and wait for the jobs to end. Deactivate the media server that hosts the storage server. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I or the NetBackup Administrator's Guide for Windows, Volume I.

Stop the NetBackup services on the storage server. Be sure to wait for all services to stop.

4 5 6

Use the operating system or disk manager tools to dynamically increase or decrease the deduplication storage area. Restart the NetBackup services. Activate the media server that hosts the storage server. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I or the NetBackup Administrator's Guide for Windows, Volume I.

Restart the deduplication jobs.

Restoring files at a remote site

If you use optimized duplication to copy images from a local site to a remote site, you can restore from the copies at the remote site to clients at the remote site. To do so, use a server-directed restore or a client-redirected restore, which restores files to a client other than the original client. Information about how to redirect restores is in a different guide. See Managing client restores in the NetBackup Administrator's Guide for UNIX and Linux, Volume I or the NetBackup Administrator's Guide for Windows, Volume I. You may have to configure which media server performs the restore. In optimized duplication, the media server that initiates the duplication operation becomes the write host for the new image copies. The write host restores from those image copies. If the write host is at the local site, it restores from those images at the remote site to the alternate client at the remote site. That host reads the image across the WAN and then writes the image back across the WAN to the alternate client. In this case, you can specify that the media server at the remote site as the restore server.

Managing deduplication Specifying the restore server

121

Specifying the restore server

NetBackup may not use the backup server as the restore server for deduplicated data. See How deduplication restores work on page 50. You can specify the server to use for restores. The following are the methods that specify the restore server:

Always use the backup server. Two methods exist, as follows:

Use NetBackup Host Properties to specify a Media host override server. All restore jobs for any storage unit on the original backup server use the media server you specify. Specify the same server for the Restore server as for the Original backup server. See Forcing restores to use a specific server in the NetBackup Administrator's Guide for UNIX and Linux, Volume I or the NetBackup Administrator's Guide for Windows, Volume I. This procedure sets the FORCE_RESTORE_MEDIA_SERVER option. Configuration options are stored in the bp.conf file on UNIX systems and the registry on Windows systems. Create the touch file USE_BACKUP_MEDIA_SERVER_FOR_RESTORE on the NetBackup master server in the following directory: UNIX: usr/openv/netbackup/db/config Windows: install_path\veritas\netbackup\db\config This global setting always forces restores to the server that did the backup. It applies to all NetBackup restore jobs, not just deduplication restore jobs. If this touch file exists, NetBackup ignores the FORCE_RESTORE_MEDIA_SERVER and FAILOVER_RESTORE_MEDIA_SERVER settings.

Always use a different server. Use NetBackup Host Properties to specify a Media host override server. See the previous explanation about Media host override, except: Specify the different server for the Restore server. A single restore instance. Use the bprestore command with the -disk_media_server option. Restore jobs for each instance of the command use the media server you specify. See NetBackup Commands Reference Guide.

122

Managing deduplication Specifying the restore server

Chapter

Troubleshooting
This chapter includes the following topics:

About deduplication logs Troubleshooting installation issues Troubleshooting configuration issues Troubleshooting operational issues Viewing disk errors and events Deduplication event codes and messages

About deduplication logs

The NetBackup deduplication components write information to various log files. Table 8-1 describes the log files for each component. Information about VxUL log files is available. See About VxUL logs for deduplication on page 125.

124

Troubleshooting About deduplication logs

Table 8-1 Component

Client deduplication proxy plug-in

Logs for NetBackup deduplication activity

Description
The client deduplication proxy plug-in on the media server runs under bptm, bpstsinfo, and bpbrm processes. Examine the log files for those processes for proxy plug-in activity. The strings proxy or ProxyServer embedded in the log messages identify proxy server activity. They write log files to the following directories:

For bptm: UNIX: /usr/openv/netbackup/logs/bptm Windows: install_path\Veritas\NetBackup\logs\bptm

For bpstsinfo: Windows: /usr/openv/netbackup/logs/admin UNIX: /usr/openv/netbackup/logs/bpstsinfo Windows: install_path\Veritas\NetBackup\logs\admin Windows: install_path\Veritas\NetBackup\logs\stsinfo

For bpbrm: UNIX: /usr/openv/netbackup/logs/bpbrm Windows: install_path\Veritas\NetBackup\logs\bpbrm

Client deduplication proxy server

The deduplication proxy server nbostpxy on the client writes messages to files in an eponymous directory, as follows: UNIX: /usr/openv/netbackup/logs/nbostpxy Windows: install_path\Veritas\NetBackup\logs\nbostpxy.

Deduplication configuration script

The following is the path name of the log file for the deduplication configuration script:

UNIX: storage_path/log/pdde-config.log Windows: storage_path\log\pdde-config.log

NetBackup creates this log file during the configuration process. If your configuration succeeded, you do not need to examine the log file. The only reason to look at the log file is if the configuration failed. If the configuration process failed after it created and populated the storage directory, this log file identifies when the configuration failed. Deduplication database The deduplication database log file (postgresql.log) is in the storage_path/databases/pddb directory. You can configure log parameters. For more information, see the following: http://www.postgresql.org/docs/current/static/runtime-config-logging.html See Configuring deduplication log file timestamps on Windows on page 90.

Troubleshooting About deduplication logs

125

Table 8-1 Component

NetBackup Deduplication Engine

Logs for NetBackup deduplication activity (continued)

Description
The NetBackup Deduplication Engine writes several log files, as follows:

Log files in the storage_path/log/spoold directory, as follows:

The spoold.log file is the main log file The storaged.log file is for queue processing. A log file for each connection to the engine is stored in a directory in the storage path spoold directory. The following describes the pathname to a log file for a connection: IP_address/application/TaskName/FirstDigitofSessionID/sessionID-current_time_in_seconds.log For example, the following is an example of a crcontrol connection log pathname on a UNIX system: /storage_path/log/spoold/127.0.0.1/crcontrol/Control/2/2916742631-1257956402.log

Usually, the only reason to examine these connection log files is if a Symantec support representative asks you to. A VxUL log file for the events and errors that NetBackup receives from polling. The originator ID for the deduplication engine is 364. See About VxUL logs for deduplication on page 125. NetBackup Deduplication Manager The log files are in the /storage_path/log/spad directory.

spad.log sched_GarbageCollection.log sched_QueueProcess.log SchedClass.log

You can set the log level and retention period in the Change Storage Server dialog box Properties tab. See Changing deduplication storage server properties on page 104. PureDisk plug-in You can configure the location and name of the log file and the logging level. To do so, edit the DEBUGLOG entry and the LOGLEVEL in the pd.conf file. See About the deduplication pd.conf file on page 80. See Editing the deduplication pd.conf file on page 83.

About VxUL logs for deduplication

Some NetBackup commands or processes write messages to their own log files. Other processes use Veritas unified log (VxUL) files. VxUL uses a standardized name and file format for log files. An originator ID (OID) identifies the process that writes the log messages. Table 8-2 shows the NetBackup logs for disk-related activity.

126

Troubleshooting Troubleshooting installation issues

The messages that begin with a sts_ prefix relate to the interaction with the storage vendor software plug-in. Most interaction occurs on the NetBackup media servers. To view and manage VxUL log files, you must use NetBackup log commands. For information about how to use and manage logs on NetBackup servers, see the NetBackup Troubleshooting Guide. Table 8-2 Activity
NetBackup Deduplication Engine Backups and restores

NetBackup VxUL logs VxUL OID

364

Processes that use the ID

The NetBackup Deduplication Engine that runs on the deduplication storage server.

N/A

Messages appear in the log files for the following processes:

The bpbrm backup and restore manager The bpdbm database manager The bpdm disk manager The bptm tape manager for I/O operations

Backups and restores

117

The nbjm Job Manager.

Device 111 configuration and monitoring Device 178 configuration and monitoring Device 202 configuration and monitoring Device 230 configuration and monitoring

The nbemm process.

The Disk Service Manager process that runs in the Enterprise Media Manager (EMM) process.

The storage server interface process that runs in the Remote Manager and Monitor Service. RMMS runs on media servers. The Remote Disk Service Manager interface (RDSM) that runs in the Remote Manager and Monitor Service. RMMS runs on media servers.

Troubleshooting installation issues

The following sections may help you troubleshoot installation issues. See Installation on SUSE Linux fails on page 127.

Troubleshooting Troubleshooting configuration issues

127

Installation on SUSE Linux fails

The installation trace log shows an error when you install on SUSE Linux:
....NetBackup and Media Manager are normally installed in /usr/openv. Is it OK to install in /usr/openv? [y,n] (y) Reading NetBackup files from /net/nbstore/vol/test_data/PDDE_packages/ suse/NB_FID2740_LinuxS_x86_20090713_6.6.0.27209/linuxS_x86/anb /net/nbstore/vol/test_data/PDDE_packages/suse/NB_FID2740_LinuxS_x86_ 20090713_6.6.0.27209/linuxS_x86/catalog/anb/NB.file_trans: symbol lookup error: /net/nbstore/vol/test_data/PDDE_packages/suse/ NB_FID2740_LinuxS_x86_20090713_6.6.0.27209/linuxS_x86/catalog/anb/ NB.file_trans: undefined symbol: head /net/nbstore/vol/test_data/ PDDE_packages/suse/NB_FID2740_LinuxS_x86_20090713_6.6.0.27209/ linuxS_x86/catalog/anb/NB.file_trans failed. Aborting ...

Verify that your system is at patch level 2 or later, as follows:

cat /etc/SuSE-release SUSE Linux Enterprise Server 10 (x86_64) VERSION = 10 PATCHLEVEL = 2

Troubleshooting configuration issues

The following sections may help you troubleshoot configuration issues. See About deduplication logs on page 123. See Database system error (220) on page 127. See Server not found error on page 128. See License information failure during configuration on page 128. See The disk pool wizard does not display a volume on page 129.

Database system error (220)

A database system error indicates that an error occurred in the storage initialization.
Error message ioctl() error, Database system error (220)

128

Troubleshooting Troubleshooting configuration issues

Example

RDSM has encountered an STS error: Failed to update storage server ssname, database system error

Diagnosis

The PDDE_initConfig script was invoked, but errors occurred during the storage initialization. First, examine the deduplication configuration script log file for references to the server name. See About deduplication logs on page 123.. Second, examine the tpconfig command log file errors about creating the credentials for the server name. The tpconfig command writes to the standard NetBackup administrative commands log directory.

Server not found error

The following information may help you resolve a server not found error message that may occur during configuration.
Error message Example Server not found, invalid command parameter RDSM has encountered an issue with STS where the server was not found: getStorageServerInfo Failed to create storage server ssname, invalid command parameter

Diagnosis

Possible root causes: When you configured the storage server, you selected a media server that runs an unsupported operating system. All media servers in your environment appear in the Storage Server Configuration Wizard; be sure to select only a media server that runs a supported operating system. If you used the nbdevconfig command to configure the storage server, you may have typed the host name incorrectly. Also, case matters for the storage server type, so ensure that you use PureDisk for the storage server type.

License information failure during configuration

A configuration error message about license information failure indicates that the NetBackup servers cannot communicate with each other.

Troubleshooting Troubleshooting configuration issues

129

If you cannot configure a deduplication storage server or load balancing servers, your network environment may not be configured for DNS reverse name lookup. You can edit the hosts file on the media servers that you use for deduplication. Alternatively, you can configure NetBackup so it does not use reverse name lookup. To prohibit reverse host name lookup by using the Administration Console

1 2 3 4 5

In the NetBackup Administration Console, expand NetBackup Management > Host Properties > Master Servers. In the details pane, select the master server. On the Actions menu, select Properties. In the Master Server Properties dialog box, select the Network Settings properties. Select one of the following options:

Allowed Restricted Prohibited

For a description of these options, see the NetBackup online Help or the administrator's guide. To prohibit reverse host name lookup by using the bpsetconfig command

Enter the following command on each media server that you use for deduplication:
echo REVERSE_NAME_LOOKUP = PROHIBITED | bpsetconfig -h host_name

The bpsetconfig command resides in the following directories: UNIX: /usr/openv/netbackup/bin/admincmd Windows: install_path\Veritas\NetBackup\bin\admincmd

The disk pool wizard does not display a volume

The Disk Pool Configuration Wizard does not display a disk volume for the deduplication storage server. First, restart all of the NetBackup daemons or services. The step ensures that the NetBackup Deduplication Engine is up and ready to respond to requests. Second, restart the NetBackup Administration Console. This step clears cached information from the failed attempt to display the disk volume.

130

Troubleshooting Troubleshooting operational issues

Troubleshooting operational issues

The following sections may help you troubleshoot operational issues. See Verify that the server has sufficient memory on page 130. See Backup jobs fail on page 130. See Client deduplication fails on page 131. See Volume state changes to DOWN when volume is unmounted on page 131. See Errors, delayed response, hangs on page 132. See Deduplication fails after services are restarted or a domain controller is restarted on page 132. See Cannot delete a disk pool on page 133. See Media open error (83) on page 134. See Media write error (84) on page 135. See Storage full conditions on page 136. See Specifying the restore server on page 121.

Verify that the server has sufficient memory

Insufficient memory on the storage server can cause operation problems. If you have operation issues, you should verify that your storage server has sufficient memory. See About deduplication server requirements on page 24. If the NetBackup deduplication processes do no start on Red Hat Linux, configure shared memory to be at least 128 MB (SHMMAX=128MB).

Backup jobs fail

If backup jobs fail with an Error 800: Disk Volume is Down message, examine the disk error logs to determine why the volume was marked DOWN. If the storage server is busy with jobs, it may not respond to master server disk polling requests in a timely manner. A busy load balancing server also may cause this error. Consequently, the query times out and the master server marks the volume DOWN. If the error occurs for an optimized duplication job: verify that source storage server is configured as a load balancing server for the target storage server. Also verify that the target storage server is configured as a load balancing server for the source storage server.

Troubleshooting Troubleshooting operational issues

131

See Viewing disk errors and events on page 137.

Client deduplication fails

NetBackup client-side agents (including client deduplication) depend on reverse host name look up of NetBackup server names. Conversely, regular backups depend on forward host name resolution. Therefore, the backup of a client that deduplicates it's own data may fail, while a normal backup of the client may succeed. If a client-side deduplication backup fails, verify that your Domain Name Server includes all permutations of the storage server name. Also, Symantec recommends that you use fully-qualified domain names for your NetBackup environment. See About fully qualified domain names on page 46.

Volume state changes to DOWN when volume is unmounted

If a volume becomes unmounted, NetBackup changes the volume state to DOWN. NetBackup jobs that require that volume fail. To determine the volume state

Invoke the following command on the master server or the media server that functions as the deduplication storage server: The following example output shows that the DiskPoolVolume is UP:
Disk Pool Name Disk Type Disk Volume Name Disk Media ID Total Capacity (GB) Free Space (GB) Use% Status Flag Flag Flag Num Read Mounts Num Write Mounts Cur Read Streams Cur Write Streams : : : : : : : : : : : : : : : PD_Disk_Pool PureDisk PureDiskVolume @aaaab 49.98 43.66 12 UP ReadOnWrite AdminUp InternalUp 0 1 0 0

132

Troubleshooting Troubleshooting operational issues

To change the volume state to UP

Mount the file system After a brief period of time, the volume state changes to UP. No further action is required.

Errors, delayed response, hangs

Insufficient memory or inadequate host capabilities may cause multiple errors, delayed response, and hangs. See About deduplication server requirements on page 24. For virtual machines, Symantec recommends that you do the following:

Set the memory size of each virtual machine to double the physical memory of the host. Set the minimum and the maximum values of each virtual machine to the same value (double the physical memory of the host). These memory settings prevent the virtual memory from becoming fragmented on the disk because it does not grow or shrink.

These recommendations may not be the best configuration for every virtual machine. However, Symantec recommends that you try this solution first when troubleshooting performance issues.

Deduplication fails after services are restarted or a domain controller is restarted

Windows systems only. After the NetBackup services are restarted or a domain controller is restarted, deduplication may fail on Microsoft Windows systems. The deduplication postgresql-8.3 database service may fail during the restart if the purediskdbuser account does not have the "log on as a service" right. The right may be removed inadvertently; for example, a security policy on a domain controller may remove the right after the service is running. As a result, the postgresql-8.3 service cannot start during the next restart. To work around the problem, do one of the following:

Amend the security policy to allow the purediskdbuser account to have the "log on as a service" right. Change the service manually to a new domain account or to any other account that can run services.

Troubleshooting Troubleshooting operational issues

133

Note: If you choose to switch the service to a different pre-existing account, that account must be granted full permissions to the<DBPATH>\databases\pddb\data directory.

Change the service to run as a Local System. Note: If you change the service to run as a Local System, the account that runs the postgresql-8.3 service must be granted full permissions to the<DBPATH>\databases\pddb\data directory.

Cannot delete a disk pool

If you cannot delete a disk pool that you believe contains no valid backup images, the following information may help you troubleshoot the problem.

Expired fragments remain on disk

Under some circumstances, the fragments that compose an expired backup image may remain on disk even though the images have expired. For example, if the storage server crashes, normal clean-up processes may not run. In those circumstances, you cannot delete a disk pool because image fragment records still exist. The error message may be similar to the following:
DSM has found that one or more volumes in the disk pool diskpoolname has image fragments.

To delete the disk pool, you must first delete the image fragments. The nbdelete command deletes expired image fragments from disk volumes. To delete the fragments of expired images

Run the following command on the master server:

UNIX: /usr/openv/netbackup/bin/admincmd/nbdelete -allvolumes -force Windows: install_path\NetBackup\bin\admincmd\nbdelete -allvolumes

-force

The -allvolumes option deletes expired image fragments from all volumes that contain them. The -force option removes the database entries of the image fragments even if fragment deletion fails.

134

Troubleshooting Troubleshooting operational issues

Incomplete SLP duplication jobs

Incomplete storage lifecycle policy duplication jobs may prevent disk pool deletion. You can determine if incomplete jobs exist and then cancel them. To cancel storage lifecycle policy duplication jobs

Determine if incomplete SLP duplication jobs exist by running the following command on the master server: UNIX: install_path\NetBackup\bin\admincmd\nbstlutil stlilist
-incomplete

Windows: /usr/openv/netbackup/bin/admincmd/nbstlutil stlilist

-incomplete

Cancel the incomplete jobs by running the following command for each backup ID returned by the previous command (xxxxx represents the backup ID): UNIX: install_path\NetBackup\bin\admincmd\nbstlutil cancel
-backupid xxxxx

Windows: /usr/openv/netbackup/bin/admincmd/nbstlutil cancel

-backupid xxxxx

Media open error (83)

To diagnose and resolve media open errors that may occur during backups, see the following possible causes and corrective actions:
Possible causes The NetBackup Deduplication Engine (spoold) was too busy to respond to the deduplication process in a timely manner. The NetBackup Deduplication Manager (spad) was too busy to respond to the deduplication process in a timely manner. Diagnosis Examine what caused the core media server deduplication processes (spad and spoold) to be unresponsive. Were they temporarily busy (such as queue processing in progress)? Do too many jobs run concurrently? See About deduplication performance on page 43. Status 83 is a generic error for the duplications. The NetBackup bpdm log provides additional information for determining the specific issue.

Troubleshooting Troubleshooting operational issues

135

Media write error (84)

To diagnose and resolve media write errors that may occur during backups, see the following possible causes and corrective actions:
The NetBackup Deduplication Engine (spoold) was too busy to respond. Examine the Disk Logs report for PureDisk errors. Examine the disk monitoring services log files for details from the deduplication plug-in. See Viewing disk reports on page 100. The NetBackup Deduplication Engine rebuilt the deduplication cache when the backup occurred. Data removal is running. Restarting the NetBackup Deduplication Engine causes the cache to be rebuilt. During the cache rebuild no backups are accepted. Data cannot be backed up at the same time as it is removed. See About maintenance processing on page 118. A user tampered with the storage. Users must not add files to, change files on, or delete files from the storage. If a file was added, remove it. If you grew the storage, you must restart the NetBackup services on the storage server so the new capacity is recognized. If possible, increase the storage capacity. See About adding additional storage on page 57. The deduplication pool is down. Change the state to up. See Changing the deduplication pool state on page 114. Firewall ports are not open. Ensure that ports 10082 and 10102 are open in any firewalls between the deduplication hosts.

Storage capacity was increased.

The storage is full.

136

Troubleshooting Troubleshooting operational issues

Host name resolution problems.

Client-side deduplication can fail if the client cannot resolve the host name of the server. More specifically, the error can occur if the storage server was configured with a short name and the client tries to resolve a fully qualified domain name. To determine which name the client uses for the storage server, examine the deduplication host configuration file on the client. See About the deduplication host configuration file on page 88. To fix this problem, configure your network environment so that all permutations of the storage server name resolve. Symantec recommends that you use fully qualified domain names. See About fully qualified domain names on page 46.

Storage full conditions

Operating system tools such as the UNIX df command do not report deduplication disk usage accurately. The operating system commands may report that the storage is full when it is not. NetBackup tools let you monitor storage capacity and usage more accurately. The NetBackup Deduplication Guide includes information about how to monitor the storage. See About deduplication capacity and usage reporting on page 97. See About deduplication container files on page 99. See Viewing capacity within deduplication container files on page 100. Examining the disk log reports for threshold warnings can give you an idea of when a storage full condition may occur. How NetBackup performs maintenance can affect when storage is freed up for use. See About maintenance processing on page 118. See Data removal process on page 162. Although not advised, you can reclaim free space manually. See Performing deduplication maintenance manually on page 119.

Troubleshooting Viewing disk errors and events

137

Viewing disk errors and events

You can view disk errors and events in several ways, as follows:

The Disk Logs report. See Viewing disk reports on page 100. The NetBackup bperror command with the -disk option reports on disk errors. The command resides in the following directories: UNIX: /usr/openv/netbackup/bin/admincmd Windows: install_path\Veritas\NetBackup\bin\admincmd

Deduplication event codes and messages

The following table shows the deduplication event codes and their messages. Event codes appear in the bperror command -disk output and in the disk reports in the NetBackup Administration Console. Table 8-3 Event #
1000

Deduplication event codes and messages Event Severity

2

NetBackup Message example Severity

Error Operation configload/reload failed on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. Operation configload/reload failed on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. The open file limit exceeded in server PureDisk:server1.symantecs.orgon host server1.symantecs.org. Will attempt to continue further. A connection request was denied on the server PureDisk:server1.symantecs.orgon host server1.symantecs.org. Network failure occurred in server PureDisk:server1.symantecs.org on host server1.symantecs.org.

1001

Error

1002

Warning

1003

Error

1004

Critical

138

Troubleshooting Deduplication event codes and messages

Table 8-3 Event #

1013

Deduplication event codes and messages (continued) Event Severity

1

NetBackup Message example Severity

Critical Task session start request on server PureDisk:server1.symantecs.org on host server1.symantecs.org got an unexpected error. Task Aborted; An unexpected error occurred during communication with remote system in server PureDisk:server1.symantecs.orgon host server1.symantecs.org.

1008

Error

1009

Authorization Authorization request from <IP> for user <USER> denied (<REASON>). Error Task initialization on server PureDisk:server1.symantecs.org on host server1.symantecs.org got an unexpected error. Task ended on server PureDisk:server1.symantecs.org on host server1.symantecs.org. A request for agent task was denied on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. Task session start request on server PureDisk:server1.symantecs.orgon host server1.symantecs.org got an unexpected error. Task creation failed, could not initialize task class on server PureDisk:server1.symantecs.orgon host server1.symantecs.org.

1010

1011

Error

1012

Error

1014

Critical

1015

Critical

Troubleshooting Deduplication event codes and messages

139

Table 8-3 Event #

1017

Deduplication event codes and messages (continued) Event Severity

1

NetBackup Message example Severity

Critical Service Symantec DeduplicationEngine exit on server PureDisk:server1.symantecs.org on host server1.symantecs.org. Please check the server log for the probable cause of this error. The application has terminated. Startup of Symantec DeduplicationEngine completed successfully on server1.symantecs.org. Service Symantec DeduplicationEngine restart on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. Please check the server log for the probable cause of this error. The application has restarted. Service Symantec Deduplication Engine connection manager restart failed on server PureDisk:server1.symantecs.org on host server1.symantecs.org. Please check the server log for the probable cause of this error.The application has failed to restart. Service Symantec DeduplicationEngine abort on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. Please check the server log for the probable cause of this error.The application has caught an unexpected signal. Double backend initialization failure; Could not initialize storage backend or cache failure detected on host PureDisk:server1.symantecs.org in server server1.symantecs.org.

1018

16

Info

1019

Critical

1020

Critical

1028

Critical

1029

Critical

140

Troubleshooting Deduplication event codes and messages

Table 8-3 Event #

1030

Deduplication event codes and messages (continued) Event Severity

1

NetBackup Message example Severity

Critical Operation Storage Database Initialization failed on server PureDisk:server1.symantecs.org on host server1.symantecs.org. Operation Content router context initialization failed on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. Operation log path creation/print failed on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. Operation a transaction failed on server PureDisk:server1.symantecs.org on host server1.symantecs.org. Transaction failed on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. Transaction will be retried. The usage of one or more system resources has exceeded a warning level. Operations will or could be suspended. Please take action immediately to remedy this situation. Operation Database recovery failed on server PureDisk:server1.symantecs.org on host server1.symantecs.org. Operation Storage recovery failed on server PureDisk:server1.symantecs.org on host server1.symantecs.org. CRC mismatch detected; possible corruption in server PureDisk:server1.symantecs.org on host server1.symantecs.org.

1031

Critical

1032

Critical

1036

Warning

1037

Warning

1044

multiple

multiple

1040

Error

1043

Error

1047

Error

Troubleshooting Deduplication event codes and messages

141

Table 8-3 Event #

2000

Deduplication event codes and messages (continued) Event Severity NetBackup Message example Severity
Error Low space threshold exceeded on the partition containing the storage database on server PureDisk:server1.symantecs.org on host server1.symantecs.org.

142

Troubleshooting Deduplication event codes and messages

Chapter

Host replacement, recovery, and uninstallation

This chapter includes the following topics:

Replacing the deduplication storage server host computer Recovering from a deduplication storage server disk failure Recovering from a permanent deduplication storage server failure Recovering the storage server after NetBackup catalog recovery About uninstalling media server deduplication Deactivating media server deduplication

Replacing the deduplication storage server host computer

If you replace the deduplication storage server host computer, use these instructions to install NetBackup and reconfigure the deduplication storage server. For the new host, you must use the same host name. The new host cannot host a deduplication storage server already. Reasons to replace the host include a lease swap or perhaps the current deduplication storage server host does not meet your performance requirements.

144

Host replacement, recovery, and uninstallation Replacing the deduplication storage server host computer

Warning: The new host must use the same byte order as the old host. If it does not, you cannot access the deduplicated data. In computing, endianness describes the byte order that represents data: big endian and little endian. For example, Sun SPARC processors and Intel processors use different byte orders. Therefore, you cannot replace a Solaris SPARC host with a host that has an Intel processor. Table 9-1 Task How to replace the deduplication storage server host Procedure

Change the disk volume state See Changing the deduplication disk volume state and disk pool state to DOWN on page 114. See Changing the deduplication pool state on page 114. Configure the new host so it See About deduplication servers on page 22. meets deduplication See About deduplication server requirements on page 24. requirements Move the storage to the new See the storage vendor's documentation. host. Install the NetBackup media See the NetBackup Installation Guide for UNIX and Linux. server software on the new See the NetBackup Installation Guide for Windows. host Delete the NetBackup Deduplication Engine credentials If you have load balancing servers, delete the NetBackup Deduplication Engine credentials on those media servers. On each load balancing server, run the following command: See Deleting credentials from a load balancing server on page 111. Add the credentials to the storage server Add the NetBackup Deduplication Engine credentials to the storage server. See Adding NetBackup Deduplication Engine credentials on page 110. Get a configuration file template If you did not save a storage server configuration file before the failure, get a template configuration file. See Getting the deduplication storage server configuration on page 85. Edit the configuration file See Editing a deduplication storage server configuration file on page 86.

Host replacement, recovery, and uninstallation Recovering from a deduplication storage server disk failure

145

Table 9-1 Task

How to replace the deduplication storage server host (continued) Procedure

Configure the storage server Configure the storage server by uploading the configuration from the file you edited. If you saved a configuration file before the storage server failure, use that file. See Setting the deduplication storage server configuration on page 87. Configure the load balancing If you have load balancing servers, add them to the servers configuration. See Adding a NetBackup load balancing server on page 78. Change configuration settings If you edited the deduplication configuration file, make the same changes to that file. See About the deduplication pd.conf file on page 80. See Editing the deduplication pd.conf file on page 83. Change the disk volume state See Changing the deduplication disk volume state and disk pool state to UP on page 114. See Changing the deduplication pool state on page 114. Change the disk volume state See Changing the deduplication disk volume state and disk pool state to UP on page 114. See Changing the deduplication pool state on page 114. Restart the backup jobs If any backup jobs failed, restart those jobs. Alternatively, wait until the next scheduled backup, at which time the backup jobs should succeed.

Recovering from a deduplication storage server disk failure

If recovery mechanisms do not protect the disk on which the NetBackup software resides, the deduplication storage server configuration is lost if the disk fails. This topic describes how to recover from a system disk or program disk failure where the disk was not backed up. Note: This procedure describes recovery of the disk on which the NetBackup media server software resides not the disk on which the deduplicated data resides. The disk may or may not be the system boot disk.

146

Host replacement, recovery, and uninstallation Recovering from a deduplication storage server disk failure

After recovery, your NetBackup deduplication environment should function normally. Any valid backup images on the deduplication storage should be available for restores. Symantec recommends that you use NetBackup to protect the deduplication storage server system or program disks. You then can use NetBackup to restore that media server if the disk on which NetBackup resides fails and you have to replace it. Table 9-2 Process to recover from media server disk failure

Replace the disk.

If the disk is a system boot disk, also install the operating system. See the hardware vendor and operating system documentation.

Mount the storage.

Ensure that the storage and database are mounted at the same locations. See the storage vendor's documentation.

Install and license the NetBackup media server software.

See the NetBackup Installation Guide for UNIX and Linux. See the NetBackup Installation Guide for Windows. See About the deduplication license key on page 60.

Delete the configuration file If you use load balancing servers in your environment, delete on deduplication servers the storage server configuration files on those servers. See Deleting a deduplication host configuration file on page 88. Delete the credentials on deduplication servers If you have load balancing servers, delete the NetBackup Deduplication Engine credentials on those media servers. See Deleting credentials from a load balancing server on page 111. Add the credentials to the storage server Add the NetBackup Deduplication Engine credentials to the storage server. See Adding NetBackup Deduplication Engine credentials on page 110. Get a configuration file template If you did not save a storage server configuration file before the disk failure, get a template configuration file. See Getting the deduplication storage server configuration on page 85.

Host replacement, recovery, and uninstallation Recovering from a permanent deduplication storage server failure

147

Table 9-2

Process to recover from media server disk failure (continued)

Edit the configuration file

See Editing a deduplication storage server configuration file on page 86.

Configure the storage server Configure the storage server by uploading the configuration from the file you edited. See Setting the deduplication storage server configuration on page 87. Add load balancing servers If you use load balancing servers in your environment, add them to your configuration. See Adding a NetBackup load balancing server on page 78.

Recovering from a permanent deduplication storage server failure

To recover from a permanent media server failure, use the process that is described in the following table. For the new host, you must use the same host name. Information about recovering the master server is available. See the NetBackup Troubleshooting Guide. Warning: The new host must use the same byte order as the old host. If it does not, you cannot access the deduplicated data. (In computing, endianness describes the byte order that represents data: big endian and little endian. For example, Sun SPARC processors and Intel processors use different byte orders. Therefore, you cannot replace a Solaris SPARC host with a host that has an Intel processor. Table 9-3 Task Process to recover from a permanent media server failure Procedure

Change the disk volume state See Changing the deduplication disk volume state and disk pool state to DOWN on page 114. See Changing the deduplication pool state on page 114.

148

Host replacement, recovery, and uninstallation Recovering from a permanent deduplication storage server failure

Table 9-3

Process to recover from a permanent media server failure (continued) Procedure

Task

Configure the new host so it Use the same host name as the failed server. meets deduplication See About deduplication servers on page 22. requirements See About deduplication server requirements on page 24. Move the storage to the new Ensure that the storage and database are mounted at the host. same locations. See the storage vendor's documentation. Install the NetBackup media See the NetBackup Installation Guide for UNIX and Linux. server software on the new See the NetBackup Installation Guide for Windows. host See About the deduplication license key on page 60. Delete the credentials on deduplication servers If you have load balancing servers, delete the NetBackup Deduplication Engine credentials on those media servers. See Deleting credentials from a load balancing server on page 111. Add the credentials to the storage server Add the NetBackup Deduplication Engine credentials to the storage server. See Adding NetBackup Deduplication Engine credentials on page 110. Get a configuration file template If you did not save a storage server configuration file before the failure, get a template configuration file. See Getting the deduplication storage server configuration on page 85. Edit the configuration file See Editing a deduplication storage server configuration file on page 86.

Configure the storage server Configure the storage server by uploading the configuration from the file you edited. See Setting the deduplication storage server configuration on page 87. Configure the load balancing If you have load balancing servers, add them to the servers configuration. See Adding a NetBackup load balancing server on page 78.

Host replacement, recovery, and uninstallation Recovering the storage server after NetBackup catalog recovery

149

Table 9-3

Process to recover from a permanent media server failure (continued) Procedure

If you edited the deduplication configuration file, make the same changes to that file. See About the deduplication pd.conf file on page 80. See Editing the deduplication pd.conf file on page 83.

Task
Change configuration settings

Change the disk volume state See Changing the deduplication disk volume state and disk pool state to UP on page 114. See Changing the deduplication pool state on page 114. Restart the backup jobs If any backup jobs failed, restart those jobs. Alternatively, wait until the next scheduled backup, at which time the backup jobs should succeed.

Recovering the storage server after NetBackup catalog recovery

If a disaster requires a recovery of the NetBackup catalog, you must set the storage server configuration after the NetBackup catalog is recovered. See Setting the deduplication storage server configuration on page 87. Symantec recommends that you save your storage server configuration. See Save the deduplication storage server configuration on page 50. Information about recovering the master server is available. See the NetBackup Troubleshooting Guide.

About uninstalling media server deduplication

You cannot uninstall media server deduplication components separately from NetBackup. The deduplication components are installed when you install NetBackup software, and they are uninstalled when you uninstall NetBackup software. Other topics describe related procedures, as follow:

Reconfigure an existing deduplication environment. See Reconfiguring the deduplication storage server and storage paths on page 92.

150

Host replacement, recovery, and uninstallation Deactivating media server deduplication

Deactivate deduplication and remove the configuration files and the storage files. See Deactivating media server deduplication on page 150.

Deactivating media server deduplication

The following process describes how to deactivate NetBackup media server deduplication and remove the deduplication storage files and the catalog files. The host remains a NetBackup media server. This process assumes that all backup images that reside on the deduplication disk storage have expired. Warning: If you deactivate deduplication and valid NetBackup images reside on the deduplication storage, data loss may occur. Table 9-4 Deactivate media server deduplication

Disable client deduplication Remove the clients that deduplicate their own data from the client deduplication list. Delete the storage units that See the NetBackup Administrator's Guide for UNIX and use the disk pool Linux, Volume I See the NetBackup Administrator's Guide for Windows, Volume I.. Delete the disk pool Delete the deduplication storage server Stop the services on the storage server See Deleting a deduplication pool on page 115. See Deleting a deduplication storage server on page 106.

See the NetBackup Administrator's Guide for UNIX and Linux, Volume I. See the NetBackup Administrator's Guide for Windows, Volume I.

Delete the storage directories Delete the storage directory and database directory. (Using a separate database directory was an option when you configured deduplication.) See the operating system documentation.

Host replacement, recovery, and uninstallation Deactivating media server deduplication

151

Table 9-4

Deactivate media server deduplication (continued)

Reset the deduplication registry

See Resetting the deduplication registry on page 89.

On Windows, delete accounts On Windows storage servers and load balancing servers, and files delete the following: The purediskdbuser account. The account is for the deduplication database administration. The purediskdbuser folder.

See the operating system documentation. On UNIX and Linux, remove On UNIX and Linux storage servers and load balancing files servers, remove the following files:

etc/pdregistry.cfg opt/pdag opt/pdshared

Delete the storage server configuration file

The storage server and every load balancing server contain a deduplication configuration file. Delete that file from every server that you use for deduplication. See Deleting a deduplication host configuration file on page 88.

Remove the NetBackup Deduplication license key

See the NetBackup Administrator's Guide for UNIX and Linux, Volume I. See the NetBackup Administrator's Guide for Windows, Volume I.

Start the NetBackup services See the NetBackup Administrator's Guide for UNIX and on the media server Linux, Volume I. See the NetBackup Administrator's Guide for Windows, Volume I.

152

Host replacement, recovery, and uninstallation Deactivating media server deduplication

Chapter

10

Deduplication architecture
This chapter includes the following topics:

Deduplication server components Media server deduplication process Deduplication client components Deduplication client backup process About deduplication fingerprinting Data removal process

Deduplication server components

Figure 10-1 is a diagram of the storage server components.

154

Deduplication architecture Deduplication server components

Figure 10-1

Storage server deduplication components

PureDisk plug-in

NetBackup Deduplication Engine

NetBackup Deduplication Manager

Storage path

Catalog plugin

Database application

Database path Data path Catalog metadata path Control flow

Table 10-1 describes the components. Table 10-1 Component

PureDisk plug-in

NetBackup deduplication components Description

The PureDisk plug-in is the data interface to the NetBackup Deduplication Engine on the storage server. The PureDisk plug-in does the following:

Separates the files metadata from the files content. Deduplicates the content (separates files into segments ). Controls the data stream from NetBackup to the NetBackup Deduplication Engine and vice versa.

The plug-in runs on the deduplication storage server. The plug-in also runs on load balancing servers and on the clients that deduplicate their own data. NetBackup Deduplication Engine The NetBackup Deduplication Engine is one of the storage server core components. It stores and manages deduplicated file data. The binary file name is spoold, which is short for storage pool daemon; do not confuse it with a print spooler daemon. The spoold process appears as the NetBackup Deduplication Engine in the NetBackup Administration Console.

Deduplication architecture Media server deduplication process

155

Table 10-1 Component

NetBackup Deduplication Manager

NetBackup deduplication components (continued) Description

The deduplication manager is one of the storage server core components. The deduplication manager maintains the configuration and controls internal processes, optimized duplication, security, and event escalation. The deduplication manager binary file name is spad. The spad process appears as the NetBackup Deduplication Manager in the NetBackup Administration Console.

Catalog plug-in

The catalog plug-in implements a standardized catalog API, which lets the NetBackup Deduplication Engine communicate with the back-end database process. The catalog plug-in translates deduplication engine catalog calls into the calls that are native to the back-end database. The deduplication database stores and manages the metadata of deduplicated files. The metadata includes a unique fingerprint that identifies the files content. The metadata also includes information about the file such as its owner, where it resides on a client, when it was created, and other information. NetBackup uses the PostgresSQL database for the deduplication database. You can use the NetBackup bpps command to view the database process (postgres). The deduplication database is separate from the NetBackup catalog. The NetBackup catalog maintains the usual NetBackup backup image information. On Windows systems, NetBackup creates a purediskdbuser account for database management.

Deduplication database

Media server deduplication process

Figure 10-2 shows the backup process when a media server deduplicates the backups. The destination is a media server deduplication pool. A description follows.

156

Deduplication architecture Media server deduplication process

Figure 10-2

Deduplication to a media server deduplication pool

Deduplication storage server PureDisk plug-in

Master server

nbjm bpdbm

bpbrm

bptm

NetBackup Deduplication Engine

bpbkar Control path Data path

Client

Media server deduplication pool

The following list describes the backup process when a media server deduplicates the backups and the destination is a media server deduplication pool:

The NetBackup Job Manager (nbjm) starts the Backup/Restore Manager (bpbrm) on a media server. The Backup/Restore Manager starts the bptm process on the media server and the bpbkar process on the client. The Backup/Archive Manager (bpbkar) on the client generates the backup images and moves them to the media server bptm process. The Backup/Archive Manager also sends the information about files within the image to the Backup/Restore Manager (bpbrm). The Backup/Restore Manager sends the file information to the bpdbm process on the master server for the NetBackup database.

The bptm process moves the data to the PureDisk plug-in. The PureDisk plug-in retrieves a list of fingerprints from the last full backup for the client from the NetBackup Deduplication Engine. The list is used as a cache so the plug-in does not have to request each fingerprint from the engine. The PureDisk plug-in performs file fingerprinting calculations. The PureDisk plug-in compares the file fingerprints and the segment fingerprints against the fingerprint list in its cache.

Deduplication architecture Media server deduplication process

157

The PureDisk plug-in sends only unique data segments to the NetBackup Deduplication Engine on the storage server. The NetBackup Deduplication Engine writes the data to the media server deduplication pool.

Figure 10-3 shows the backup process when a media server deduplicates the backups. The destination is a PureDisk storage pool. A description follows. Figure 10-3
Master server

Deduplication to a PureDisk storage pool

NetBackup media server PureDisk plug-in

nbjm bpdbm

bpbrm

bptm Control path Data path

bpbkar

Client PureDisk storage pool

The following list describes the backup process when a media server deduplicates the backups and the destination is a PureDisk storage pool:

The NetBackup Job Manager (nbjm) starts the Backup/Restore Manager (bpbrm) on a media server. The Backup/Restore Manager starts the bptm process on the media server and the bpbkar process on the client). The Backup/Archive Manager (bpbkar) generates the backup images and moves them to the media server bptm process. The Backup/Archive Manager also sends the information about files within the image to the Backup/Restore Manager (bpbrm). The Backup/Restore Manager sends the file information to the bpdbm process on the master server for the NetBackup database.

The bptm process moves the data to the PureDisk plug-in. The PureDisk plug-in retrieves a list of fingerprints from the last full backup for the client from the PureDisk storage pool. The list is used as a cache so the plug-in does not have to request each fingerprint from the storage pool.

158

Deduplication architecture Deduplication client components

The PureDisk plug-in compares the file fingerprints and the segment fingerprints against the fingerprint list in its cache. The PureDisk plug-in performs file fingerprinting calculations. The PureDisk plug-in sends only unique data segments to the PureDisk storage pool.

Deduplication client components

Table 10-2 describes the client deduplication components. Table 10-2 Component Client deduplication components Host Description
The PureDisk plug-in is the data interface to the NetBackup Deduplication Engine on the deduplication storage server. The PureDisk plug-in does the following:

PureDisk plug-in Client

Separates the files metadata from the files content.

Deduplicates the content (separates files into segments ). Controls the data stream from NetBackup to the NetBackup Deduplication Engine and vice versa. Proxy server Client The OpenStorage proxy server (nbostpxy) manages control communication with the media server.

Proxy plugin

Media server The proxy plug-in manages control communication with the client.

Deduplication client backup process

Figure 10-4 shows the backup process of a client that deduplicates its own data. The destination is a media server deduplication pool. A description follows.

Deduplication architecture Deduplication client backup process

159

Figure 10-4
Master server nbjm

Deduplication client backup to a media server deduplication pool

Control path Data path Deduplication client

bpdbm bpbkar PureDisk plug-in Proxy server (nbostpxy)

bpbrm

Proxy plug-in

bptm

NetBackup Deduplication Engine

Deduplication storage server Media server deduplication pool

The following list describes the backup process for a deduplication client to a media server deduplication pool:

The NetBackup Job Manager (nbjm) starts the Backup/Restore Manager (bpbrm) on a media server. The Backup/Restore Manager probes the client to determine if it is configured and ready for deduplication. If the client is ready, the Backup/Restore Manager starts the following processes: The OpenStorage proxy server (nbostpxy) on the client and the data moving processes (bpbkar) on the client and bptm on the media server). NetBackup uses the proxy plug-in on the media server to route control information from bptm to nbostpxy.

The Backup/Archive Manager (bpbkar) generates the backup images and moves them to the client nbostpxy process by shared memory. The Backup/Archive Manager also sends the information about files within the image to the Backup/Restore Manager (bpbrm). The Backup/Restore Manager sends the file information to the bpdbm process on the master server for the NetBackup database.

The client nbostpxy process moves the data to the PureDisk plug-in.

160

Deduplication architecture Deduplication client backup process

The PureDisk plug-in retrieves a list of fingerprints from the last full backup for the client from the NetBackup Deduplication Engine. The list is used as a cache so the plug-in does not have to request each fingerprint from the engine. The PureDisk plug-in performs file fingerprinting calculations. The PureDisk plug-in sends only unique data segments to the storage server, which writes the data to the media server deduplication pool.

Figure 10-5 shows the backup process of a client that deduplicates its own data. The destination is a PureDisk storage pool. A description follows. Figure 10-5
Master server nbjm bpdbm bpbkar PureDisk plug-in Proxy server (nbostpxy)

Deduplication client backup to a PureDisk storage pool

Control path Data path Deduplication client

bpbrm

Proxy plug-in

bptm PureDisk storage pool Media server

The following list describes the backup process for a deduplication client to a media server deduplication pool:

The NetBackup Job Manager (nbjm) starts the Backup/Restore Manager (bpbrm) on a media server. The Backup / Restore Manager probes the client to determine if it is configured and ready for deduplication. If the client is ready, the Backup/Restore Manager starts the following processes: The OpenStorage proxy server (nbostpxy) on the client and the data moving processes (bpbkar on the client and bptm on the media server). NetBackup uses the proxy plug-in on the media server to route control information from bptm to nbostpxy.

Deduplication architecture About deduplication fingerprinting

161

The Backup/Archive Manager (bpbkar) generates the backup images and moves them to the client nbostpxy process by shared memory. The Backup/Archive Manager also sends the information about files within the image to the Backup/Restore Manager (bpbrm). The Backup/Restore Manager sends the file information to the bpdbm process on the master server for the NetBackup database.

The client nbostpxy process moves the data to the PureDisk plug-in. The PureDisk plug-in retrieves a list of fingerprints from the last full backup for the client from the NetBackup Deduplication Engine. The list is used as a cache so the plug-in does not have to request each fingerprint from the engine. The PureDisk plug-in performs file fingerprinting calculations. The PureDisk plug-in sends only unique data segments to the PureDisk storage pool.

About deduplication fingerprinting

The NetBackup Deduplication Engine uses a unique identifier to identify each file and each file segment that is backed up. The engine identifies files inside the backup images and then processes the files. The process is known as fingerprinting. For the first deduplicated backup, the following is the process:

The PureDisk plug-in reads the backup image and separates the image into files. The plug-in separates files into segments. For each segment, the plug-in calculates the hash key (or fingerprint) that identifies each data segment. To create a hash, every byte of data in the segment is read and added to the hash. The plug-in compares its calculated fingerprints to the fingerprints that the NetBackup Deduplication Engine stores on the media server. Two segments that have the same fingerprint are duplicates of each other. The plug-in sends unique segments to the deduplication engine to be stored. A unique segment is one for which a matching fingerprint does not exist in the engine already. The first backup may have a 0% deduplication rate; however, a 0% deduplication rate is unlikely. Zero percent means that all file segments in the backup data are unique.

162

Deduplication architecture Data removal process

The NetBackup Deduplication Engine saves the fingerprint information for that backup.

For subsequent backups, the following is the process:

The PureDisk plug-in retrieves a list of fingerprints from the last full backup for the client from the NetBackup Deduplication Engine. The list is used as a cache so the plug-in does not have to request each fingerprint from the engine. The PureDisk plug-in reads the backup image and separates the image into files. The PureDisk plug-in separates files into segments and calculates the fingerprint for each file and segment. The plug-in compares each fingerprint against the local fingerprint cache. If the fingerprint is not known in the cache, the plug-in requests that the engine verify if the fingerprint already exists. If the fingerprint does not exist, the segment is sent to the engine. If the fingerprint exists, the segment is not sent.

The fingerprint calculations are based on the MD5 algorithm. However, any segments that have different content but the same MD5 hash key get different fingerprints. So NetBackup prevents MD5 collisions.

Data removal process

The following list describes the data removal process for expired backup images:

NetBackup removes the image record from the NetBackup catalog. NetBackup directs the NetBackup Deduplication Manager to remove the image. The deduplication manager immediately removes the image entry and adds a removal request for the image to the database transaction queue. From this point on, the image is no longer accessible. When the queue is next processed, the NetBackup Deduplication Engine executes the removal request. The engine also generates removal requests for underlying data segments At the successive queue processing, the NetBackup Deduplication Engine executes the removal requests for the segments.

Storage is reclaimed after two queue processing runs; that is, in one day. However, data segments of the removed image may still be in use by other images. If you manually delete an image that has expired within the previous 24 hours, the data becomes garbage. It remains on disk until removed by the next garbage collection process.

Deduplication architecture Data removal process

163

See About maintenance processing on page 118. See Deleting backup images on page 117.

164

Deduplication architecture Data removal process

Index

A
about NetBackup deduplication 13 about NetBackup deduplication options 14 about the deduplication host configuration file 88 appliance deduplication 15 attributes clearing deduplication pool 113 clearing deduplication storage server 105 OptimizedImage 31 setting deduplication pool 113 setting deduplication storage server 105 viewing deduplication pool 117 viewing deduplication storage server 109

configuring deduplication 62 container files about 100 viewing capacity within 100 CR sent field of the job details 97 credentials 28 adding NetBackup Deduplication Engine 110 changing NetBackup Deduplication Engine 110

D
data removal process for deduplication 162 database system error 127 deactivating media server deduplication 150 dedup field of the job details 97 deduplication about credentials 28 about fingerprinting 161 about the license key 60 adding credentials 110 cache hits field of the job details 97 capacity and usage reporting 97 changing credentials 110 client backup process 158 compression 29 configuration file 80 configuring 62 configuring optimized synthetic backups 72 container files 100 CR sent field of the job details 97 data removal process 162 dedup field of the job details 97 encryption 30 event codes 137 garbage collection 118 how it works 15 license key for 60 licensing 60 limitations 25 maintenance processing 118 media server process 155

B
backup client deduplication process 158 big endian 147 byte order 147

C
cache hits field of the job details 97 capacity adding storage 57 capacity and usage reporting for deduplication 97 CIFS 45 clearing deduplication pool attributes 113 client deduplication about 26 components 158 host requirements 27 limitations 27 sizing the systems 19 Common Internet File System 45 compression and deduplication 29 pd.conf file setting 80 configuring a deduplication pool 64 configuring a deduplication storage server 63 configuring a deduplication storage unit 67

166

Index

deduplication (continued) network interface 28 node 24 performance 43 planning deployment 18 requirements for optimized within the same domain 32 scaling 46 scanned field of the job details 97 storage capacity 56 storage destination 20 storage paths 56 storage requirements 55 storage unit properties 67 stream rate field of the job details 97 supported systems 19 deduplication configuration file editing 83 settings 80 deduplication database about 155 log file 124 purediskdbuser account on Windows 155 deduplication disk pool. See deduplication pool deduplication disk volume changing the state 114 determining the state 116 deduplication host configuration file about 88 deleting 88 deduplication hosts and firewalls 29 client requirements 27 load balancing server 23 server requirements 24 storage server 23 deduplication logs about 123 client deduplication proxy plug-in log 124 client deduplication proxy server log 124 configuration script 124 deduplication database 124 NetBackup Deduplication Engine 125 NetBackup Deduplication Manager 125 PureDisk plug-in log 125 VxUL deduplication logs 125 deduplication node about 24 adding a load balancing server 78

deduplication node (continued) removing a load balancing server 107 deduplication optimized synthetic backups about 31 deduplication pool about 64 changing properties 112 changing the state 114 clearing attributes 113 configuring 64 deleting 115 determining the state 115 properties 65 setting attributes 113 viewing 116 viewing attributes 117 deduplication port usage about 29 troubleshooting 135 deduplication processes do not start 130 deduplication rate how file size affects 45 monitoring 95 deduplication registry resetting 89 deduplication servers about 22 components 153 host requirements 24 deduplication storage capacity about 56 monitoring 97 viewing capacity in container files 100 deduplication storage destination 20 deduplication storage paths 56 deduplication storage requirements 55 deduplication storage server about 23 changing properties 104 clearing attributes 105 components 153 configuration failure 128 configuring 63 configuring target for remote master server duplication 90 deleting 106 determining the state 106 editing configuration file 86 getting the configuration 85

Index

167

deduplication storage server (continued) reconfiguring 93 recovery 147 replacing the host 143 setting attributes 105 setting the configuration 87 viewing 108 viewing attributes 109 deduplication storage server configuration file about 84 deduplication storage type 20 Deduplication storage unit Only use the following media servers 68 Use any available media server 68 deleting backup images 117 deleting deduplication host configuration file 88 disaster recovery protecting the data 49 recovering the storage server after catalog recovery 149 disk failure deduplication storage server 145 disk logs 101 disk logs report 99 disk pool. See deduplication pool cannot delete 133 disk pool status report 98, 101 disk storage unit report 101 Disk type 67 disk volume changing the state 114 determining the state of a deduplication 116 volume state changes to down 131 duplicating images to another NetBackup domain about 42

F
fingerprinting about deduplication 161 firewalls and deduplication hosts 29

G
garbage collection for deduplication 118 invoke manually 119

H
host requirements 24 how deduplication works 15

I
images on disk report 101 initial seeding 44 iSCSI 45

L
license information failure for deduplication 128 license key for deduplication 60 licensing deduplication 60 limitations media server deduplication 25 little endian 147 load balancing server about 23 adding to a deduplication node 78 for deduplication 23 removing from deduplication node 107 logs about deduplication 123 client deduplication proxy plug-in log 124 client deduplication proxy server log 124 deduplication configuration script log 124 deduplication database log 124 disk 101 NetBackup Deduplication Engine log 125 NetBackup Deduplication Manager log 125 PureDisk plug-in log 125 VxUL deduplication logs 125

E
encryption and deduplication 30 pd.conf file setting 81 endian big 147 little 147 event codes deduplication 137

M
maintenance processing for deduplication 118

168

Index

Mapped full VM backup 72 Maximum concurrent jobs 68 Maximum fragment size 68 media server deduplication process 155 sizing the systems 19 Media Server Deduplication Option about 21 media server deduplication pool. See deduplication pool migrating from PureDisk to NetBackup deduplication 52 migrating to NetBackup deduplication 53

optimized MSDP deduplication (continued) pull configuration within the same domain 40 push configuration within the same domain 34 requirements 32 within the same domain 32 optimized MSDP duplication about 31 optimized synthetic backups configuring for deduplication 72 deduplication 31 OptimizedImage attribute 31

P
pd.conf file about 80 editing 83 settings 80 pdde-config.log 124 performance deduplication 43 monitoring deduplication rate 95 port usage and deduplication 29 troubleshooting 135 provisioning the deduplication storage 55 PureDisk deduplication 15 PureDisk Deduplication Option replacing with media server deduplication 51 PureDisk plug-in about 154 log file 125 purediskdbuser account 155

N
NDMP 45 NetBackup Client Deduplication Option 15 NetBackup deduplication about 13 license key for 60 NetBackup Deduplication Engine about 154 about credentials 28 adding credentials 110 changing credentials 110 logs 125 NetBackup Deduplication Manager about 155 logs 125 NetBackup deduplication options 14 NetBackup Media Server Deduplication Option 15 network interface for deduplication 28 NFS 45 node deduplication 24

Q
queue processing invoke manually 119 queue processing for deduplication 118

O
OpenStorage appliance deduplication 15 optimized deduplication copy configuring 75 guidance for 34 limitations 33 push configuration 36 throttling traffic 78 optimized MSDP deduplication about the media server in common within the same domain 34

R
reconfiguring deduplication 93 recovery deduplication storage server 147 from deduplication storage server disk failure 145 Red Hat Linux deduplication processes do not start 130 replacing PDDO with NetBackup deduplication 51 replacing the deduplication storage server 143

Index

169

replication about duplicating images to another NetBackup domain 42 reports disk logs 99, 101 disk pool status 98, 101 disk storage unit 101 resetting the deduplication registry 89 restores at a remote site 120 how deduplication restores work 50 specifying the restore server 121 reverse host name lookup prohibiting 128 reverse name lookup 128

S
scaling deduplication 46 scanned field of the job details 97 seeding initial 44 server not found error 128 setting deduplication pool attributes 113 storage capacity about 56 adding 57 for deduplicaton 56 monitoring deduplication 97 viewing capacity in container files 100 storage paths for deduplication 56 reconfiguring 93 storage requirements for deduplication 55 storage server about the configuration file 84 changing properties for deduplication 104 components for deduplication 153 configuring for deduplication 63 configuring target for remote master server duplication 90 deduplication 23 deleting a deduplication 106 determining the state of a deduplication 106 editing deduplication configuration file 86 getting deduplication configuration 85 reconfiguring deduplication 93 recovery 147 replacing the deduplication host 143

storage server (continued) setting the deduplication configuration 87 viewing 108 viewing attributes 109 storage server configuration getting 85 setting 87 storage server configuration file editing 86 storage type for deduplication 20 storage unit configuring for deduplication 67 properties for deduplication 67 recommendations for deduplication 68 Storage unit name 67 Storage unit type 67 stream handlers Backup Exec 45 Hyper-V 45 NDMP 45 NetBackup 45 VMWare vmdk 45 vSphere 45 Windows System State 45 stream rate field of the job details 97 supported systems 19

T
troubleshooting database system error 127 deduplication backup jobs fail 130 deduplication processes do not start 130 general operational problems 132 host name lookup 128 installation fails on Linux 127 no volume appears in disk pool wizard 129 server not found error 128

U
uninstalling media server deduplication 149

V
viewing deduplication pool attributes 117 viewing storage server attributes 109

W
Windows System State 45