Professional Documents
Culture Documents
NetBackup8301 Dedupe Guide
NetBackup8301 Dedupe Guide
Deduplication Guide
Release 8.3.0.1
Veritas NetBackup™ Deduplication Guide
Last updated: 2020-09-03
Legal Notice
Copyright © 2020 Veritas Technologies LLC. All rights reserved.
Veritas, the Veritas Logo, and NetBackup are trademarks or registered trademarks of Veritas
Technologies LLC or its affiliates in the U.S. and other countries. Other names may be
trademarks of their respective owners.
This product may contain third-party software for which Veritas is required to provide attribution
to the third party (“Third-party Programs”). Some of the Third-party Programs are available
under open source or free software licenses. The License Agreement accompanying the
Software does not alter any rights or obligations you may have under those open source or
free software licenses. Refer to the Third-party Legal Notices document accompanying this
Veritas product or available at:
https://www.veritas.com/about/legal/license-agreements
The product described in this document is distributed under licenses restricting its use, copying,
distribution, and decompilation/reverse engineering. No part of this document may be
reproduced in any form by any means without prior written authorization of Veritas Technologies
LLC and its licensors, if any.
The Licensed Software and Documentation are deemed to be commercial computer software
as defined in FAR 12.212 and subject to restricted rights as defined in FAR Section 52.227-19
"Commercial Computer Software - Restricted Rights" and DFARS 227.7202, et seq.
"Commercial Computer Software and Commercial Computer Software Documentation," as
applicable, and any successor regulations, whether delivered by Veritas as on premises or
hosted services. Any use, modification, reproduction release, performance, display or disclosure
of the Licensed Software and Documentation by the U.S. Government shall be solely in
accordance with the terms of this Agreement.
http://www.veritas.com
Technical Support
Technical Support maintains support centers globally. All support services will be delivered
in accordance with your support agreement and the then-current enterprise technical support
policies. For information about our support offerings and how to contact Technical Support,
visit our website:
https://www.veritas.com/support
You can manage your Veritas account information at the following URL:
https://my.veritas.com
If you have questions regarding an existing support agreement, please email the support
agreement administration team for your region as follows:
Japan CustomerCare_Japan@veritas.com
Documentation
Make sure that you have the current version of the documentation. Each document displays
the date of the last update on page 2. The latest documentation is available on the Veritas
website:
https://sort.veritas.com/documents
Documentation feedback
Your feedback is important to us. Suggest improvements or report errors or omissions to the
documentation. Include the document title, document version, chapter title, and section title
of the text on which you are reporting. Send feedback to:
NB.docs@veritas.com
You can also see documentation information or ask a question on the Veritas community site:
http://www.veritas.com/community/
https://sort.veritas.com/data/support/SORT_Data_Sheet.pdf
Contents
Type Description
NetBackup appliance Veritas provides several hardware and a software solutions that
deduplication include NetBackup deduplication.
https://www.veritas.com/content/support/en_US/Appliances.html
Chapter 2
Planning your deployment
This chapter includes the following topics:
Step 1 Learn about deduplication nodes See “About MSDP deduplication nodes” on page 19.
and storage destinations
See “About the NetBackup deduplication destinations” on page 19.
Step 2 Understand the storage capacity See “About MSDP storage capacity” on page 20.
and requirements
See “About MSDP storage and connectivity requirements” on page 21.
Step 3 Determine which type of See “About NetBackup media server deduplication” on page 25.
deduplication to use
See “About NetBackup Client Direct deduplication” on page 30.
Step 4 Determine the requirements for See “About MSDP storage servers” on page 27.
deduplication hosts
See “About MSDP server requirements” on page 28.
Step 5 Determine the credentials for See “About the NetBackup Deduplication Engine credentials”
deduplication on page 34.
Step 6 Read about compression and See “About MSDP compression” on page 127.
encryption
See “About MSDP encryption” on page 129.
Step 7 Read about optimized synthetic See “About MSDP optimized synthetic backups” on page 36.
backups
Step 8 Read about deduplication and SAN See “About MSDP and SAN Client” on page 37.
Client
Step 9 Read about optimized duplication See “About MSDP optimized duplication and replication” on page 37.
and replication
Step 10 Read about stream handlers See “About MSDP stream handlers” on page 39.
Planning your deployment 18
NetBackup naming conventions
Step 11 Read about best practices for See “MSDP deployment best practices” on page 41.
implementation
Step 12 Determine the storage requirements See “About provisioning the storage for MSDP” on page 47.
and provision the storage
See “About MSDP storage and connectivity requirements” on page 21.
Step 13 License MSDP See “About the MSDP license” on page 51.
Step 14 Configure MSDP See “Configuring MSDP server-side deduplication” on page 56.
Step 15 Migrate from other storage to See “Migrating from another storage type to MSDP” on page 453.
NetBackup deduplication
The naming conventions for the NetBackup Deduplication Engine differ from these
NetBackup naming conventions.
See “About the NetBackup Deduplication Engine credentials” on page 34.
Storage server The storage server deduplicates the backups, writes the data to the
storage, and manages the storage.
Load balancing Load balancing servers assist the storage server by deduplicating
servers backups. Load balancing servers are optional.
Clients The clients may include the clients that deduplicate their own data
(Client Direct).
See “About NetBackup Client Direct deduplication” on page 30.
Multiple media server deduplication nodes can exist. Nodes cannot share servers
or storage.
Each node manages its own storage. Deduplication within each node is supported;
deduplication between nodes is not supported.
See “About NetBackup media server deduplication” on page 25.
See “About MSDP storage servers” on page 27.
Destination Description
Media Server A NetBackup Media Server Deduplication Pool represents the disk
Deduplication Pool or cloud storage that is attached to a NetBackup media server.
NetBackup deduplicates the data and hosts the storage.
If you use a NetBackup 5000 series appliance for storage, use the
following documentation:
250 TBs NetBackup supports 250 TBs of storage in a new Media Server
Deduplication Pool on the supported versions of the following
operating systems:
NetBackup reserves 4 percent of the storage space for the deduplication database
and transaction logs. Therefore, a storage full condition is triggered at a 96-percent
threshold. If you use separate storage for the deduplication database, NetBackup
still uses the 96-percent threshold to protect the data storage from any possible
overload.
If your storage requirements exceed the capacity of a Media Server Deduplication
Pool, you can use more than one media server deduplication node.
See “About MSDP deduplication nodes” on page 19.
For the operating system versions that NetBackup supports for deduplication, see
the NetBackup operating system compatibility list available through the following
URL:
http://www.netbackup.com/compatibility
Storage media
The following are the minimum requirements for single stream read or write
performance for each disk volume. Greater individual data stream capability or
aggregate capability may be required to satisfy your objectives for writing to and
reading from disk.
Local disk storage may leave you vulnerable in a disaster. SAN disk can be
remounted at a newly provisioned server with the same name.
NetBackup requires the exclusive use of the disk resources. If the storage is also
used for purposes other than backups, NetBackup cannot manage disk pool capacity
or manage storage lifecycle policies correctly. Therefore, NetBackup must be the
only entity that uses the storage.
NetBackup Media Server Deduplication Pool does not support the following
storage types for deduplication storage:
■ Network Attached Storage (that is, file based storage protocols) such as CIFS
or NFS.
■ The ZFS file system.
The NetBackup compatibility lists are the definitive source for supported operating
systems, computers, and peripherals. See the compatibility lists available at the
following website:
http://www.netbackup.com/compatibility
Planning your deployment 23
About MSDP storage and connectivity requirements
The storage must be provisioned and operational before you can configure
deduplication in NetBackup.
See “About provisioning the storage for MSDP” on page 47.
Storage connection
The storage must be direct-attached storage (DAS), internal disks, or connected
by a dedicated, low latency storage area network (Fibre Channel or iSCSI).
A storage area network should conform to the following criteria:
HBAs The storage server should have an HBA or HBAs dedicated to the storage.
Those HBAs must have enough bandwidth to satisfy your throughput
objectives.
See “Fibre Channel and iSCSI comparison for MSDP” on page 23.
Genesis Storage networking architecture that is designed Storage network protocol that is built on top of
to handle the same block storage format that TCP/IP to use the same wiring as the rest of the
storage devices use. enterprise.
Protocol FCP is a thin, single-purpose protocol that iSCSI is a multiple layer implementation that
provides lossless, in-order frame delivery and facilitates data transfers over intranets and long
low switch latency. distances. The SCSI protocol expects lossless,
in-order delivery, but iSCSI uses TCP/IP, which
experiences packet loss and out-of-order
delivery.
Host CPU load Low. Fibre Channel frame processing is Higher. Most iSCSI implementations use the
offloaded to dedicated low-latency HBAs. host processor to create, send, and interpret
storage commands. Therefore, Veritas requires
dedicated network interfaces on the storage
server to reduce storage server load and reduce
latency.
Flow control A built-in flow control mechanism that ensures No built-in flow control. Veritas recommends
data is sent to a device when it is ready to that you use the Ethernet priority-based flow
accept it. control as defined in the IEEE 802.1Qbb
standard.
Deduplication Deduplication
Load balancing
servers
plug-in plug-in
Deduplication
plug-in
Storage server
NetBackup
Deduplication
Engine
Media Server
Deduplication Pool
Note: Load balancing for Cloud Catalyst storage servers or Cloud Catalyst
appliances is supported. See the following topic for configuration considerations.
See “Configuring source control or target control optimized duplication for Cloud
Catalyst” on page 280.
Tests show that the UltraSPARC processors can achieve high aggregate throughput.
However, they require eight times as many backup streams as AMD and Intel
processors to do so.
CPU Veritas recommends at least a 2.2-GHz clock rate. Veritas recommends at least a 2.2-GHz clock rate.
A 64-bit processor is required. A 64-bit processor is required.
At least four cores are required. Veritas At least two cores are required. Depending on
recommends eight cores. throughput requirements, more cores may be
helpful.
For 64 TBs of storage, Intel x86-64 architecture
requires eight cores.
Operating The operating system must be a supported 64-bit The operating system must be a supported 64-bit
system operating system. operating system.
See the operating system compatibility list for your Load balancing for Cloud Catalyst storage servers
NetBackup release on the Veritas Support website. or Cloud Catalyst appliances is supported. See the
following topic for configuration considerations.
http://www.netbackup.com/compatibility
See “Configuring source control or target control
optimized duplication for Cloud Catalyst”
on page 280.
http://www.netbackup.com/compatibility
A Veritas tech note provides detailed information about and examples for sizing
the hosts for deduplication. Information includes the number of the NICs or the
HBAs for each server to support your performance objectives.
For more information, refer to http://veritas.com/docs/TECH77575.
Note: This page has been updated for NetBackup version 7.5.
Planning your deployment 30
About NetBackup Client Direct deduplication
Note: In some environments, a single host can function as both a NetBackup master
server and as a deduplication server. Such environments typically run fewer than
100 total backup jobs a day. (Total backup jobs means backups to any storage
destination, including deduplication and non-deduplication storage.) If you perform
more than 100 backups a day, deduplication operations may affect master server
operations.
Deduplication Deduplication
plug-in plug-in
Deduplication
plug-in
NetBackup
Deduplication
Engine
■ The resilient connections provide automatic recovery from network failures and
latency (within the parameters from which NetBackup can recover).
When you configure a resilient connection, NetBackup uses that connection for the
backups. Use the NetBackup Resilient Network host properties to configure
NetBackup to use resilient network connections.
See “Resilient Network properties” on page 200.
See “Specifying resilient connections” on page 204.
The pd.conf FILE_KEEP_ALIVE_INTERVAL parameter lets you configure the
frequency of keep-alive operations on idle sockets.
See “MSDP pd.conf file parameters” on page 210.
You can improve the performance of the first backup for a remote client.
See “About seeding the MSDP fingerprint cache for remote client deduplication”
on page 70.
Note: Record and save the credentials in case you need them in the future.
Caution: You cannot change the NetBackup Deduplication Engine credentials after
you enter them. Therefore, carefully choose and enter your credentials. If you must
change the credentials, contact your Veritas support representative.
Planning your deployment 35
About the network interface for MSDP
Configure a specific To use a specific interface, you can enter that interface name
interface when you configure the deduplication storage server. NetBackup
uses this interface for all deduplication traffic unless you also
configure a separate interface for duplication and replication.
Configure an interface You can configure a separate network interface for the duplication
for duplication and and the replication traffic. The backup and restore traffic continues
replication traffic to use the default interface or the specific configured interface.
Port Usage
10082 The NetBackup Deduplication Engine (spoold). Open this port between the
hosts that deduplicate data. Hosts include load balancing servers and the clients
that deduplicate their own data.
10102 The NetBackup Deduplication Manager (spad). Open this port between the
hosts that deduplicate data. Hosts include load balancing servers and the clients
that deduplicate their own data.
What Description
Requirements The target storage unit's deduplication pool must be the same
deduplication pool on which the source images reside.
Limitations NetBackup does not support storage unit groups as a destination for
optimized synthetic backups. If NetBackup cannot produce the optimized
synthetic backup, NetBackup creates the more data-movement intensive
synthetic backup.
Within the same NetBackup domain See “About MSDP optimized duplication within the
same domain” on page 141.
Table 2-9 MSDP job load performance for an MSDP storage server
When Description
Normal Normal operation is when all clients have been backed up once.
operation Approximately 15 to 20 jobs can run concurrently and with high performance
under the following conditions:
Table 2-9 MSDP job load performance for an MSDP storage server
(continued)
When Description
Storage NetBackup maintains the same number of concurrent backup jobs as during
approaches normal operation under the following conditions:
full capacity
■ The hardware meets minimum requirements. (More capable hardware
improves performance.)
■ The amount of data that is stored is between 85% to 90% of the capacity
of the storage.
When the Oracle stream handler is used, the FILESPERSET variable is more flexible.
You can set FILESPERSET > 1 in the policy or the bp.conf without a large decrease
in deduplication rates.
The Marker Entry file is used to override the default behavior by using the
cacontrol command utility with the --sth flag at a client and or policy level only.
Note: When you use the Oracle stream handler, it is not recommended to use
variable-length deduplication.
■ You can query the setting for the stream handler per policy.
■ You can query the setting for the stream handler per policy.
When you enable the Microsoft SQL Server stream handler, the Job Details tab
in the Administration Console displays the following:
MS-SQL stream handler enabled
load balancing server for each job. However, deduplication fingerprint calculations
are not part of the load balancing criteria.
To completely remove the deduplication storage server from deduplication duties,
do the following for every storage unit that uses the deduplication disk pool:
■ Select Only use the following media servers.
■ Select all of the load balancing servers but do not select the deduplication storage
server.
The deduplication storage server performs storage server tasks only: storing and
managing the deduplicated data, file deletion, and optimized duplication.
If you configure client deduplication, the clients deduplicate their own data. Some
of the deduplication load is removed from the deduplication storage server and
loading balancing servers.
Veritas recommends the following strategies to scale MSDP:
■ For the initial full backups of your clients, use the deduplication storage server.
For subsequent backups, use load balancing servers.
■ Enable client-side deduplication gradually.
If a client cannot tolerate the deduplication processing workload, be prepared
to move the deduplication processing back to a server.
■ Use the storage server for the initial backup of the clients.
■ Enable deduplication on only a few clients at a time.
Doing so provides information about deduplication affects the clients other jobs.
It also may be easier to evaluate how your environment handles traffic and
easier to troubleshoot any problems
If a client cannot tolerate the deduplication processing workload, be prepared to
move the deduplication processing back to the storage server.
reason, the Failover method for the Storage unit selection uses the least amount
of storage. All of the other methods are designed to use different storage every
time the backup runs. Veritas recommends that you select the Failover method for
the Storage unit selection type.
Table 2-10 MSDP requirements and limitations for storage unit groups
What Description
Requirements A group must contain storage units of one storage destination type only.
That is, a group cannot contain both Media Server Deduplication Pool
storage units and storage units with other storage types.
Limitations NetBackup does not support the following for storage unit groups:
How many storage instances you provision depends on your storage requirements
for your backups. If your requirements are greater than one deduplication node can
accommodate, you can configure more than one node.
See “About MSDP deduplication nodes” on page 19.
Optimized duplication and replication also can affect the number of nodes you
provision.
See “About MSDP optimized duplication and replication” on page 37.
Other NetBackup requirements may affect how you provision the storage.
See “About MSDP storage and connectivity requirements” on page 21.
How to provision the storage is beyond the scope of the NetBackup documentation.
Consult the storage vendor’s documentation.
Provisioning the storage 48
About provisioning the storage for MSDP
Up to 64 TBs of storage
Provision the backup storage so that it appears as a single mount point to the
operating system.
Because the storage requires a directory path, do not use only the root node (/) or
drive letter (E:\) as the storage path. (That is, do not mount the storage as a root
node (/) or a drive letter (E:\).
If you use a separate disk volume for the deduplication database, provision a 1-TB
volume on a different mount point than the backup data storage.
250-TBs of storage
NetBackup supports 250 TBs of storage in a single Media Server Deduplication
Pool on certain operating systems.
See “About MSDP storage capacity” on page 20.
Before you configure the MSDP storage server, you must provision the volumes.
Each volume must conform to the following items:
■ Formatted with a file system that NetBackup supports for MSDP. The same file
system must be used for all volumes.
■ Reside on a separate disk from the other volumes that you allocate for the MSDP
storage.
■ Mounted on a separate mount point on the computer that you want to use as
the MSDP storage server
Veritas recommends that you use a descriptive naming convention for the mount
point names.
For supported systems, see the InfoScale hardware compatibility list at the Veritas
website:
http://www.veritas.com/
Note: Although InfoScale Storage supports NFS, NetBackup does not support NFS
targets for Media Server Deduplication Pool storage. Therefore, Media Server
Deduplication Pool does not support NFS with InfoScale Storage.
Chapter 4
Licensing deduplication
This chapter includes the following topics:
■ About seeding the MSDP fingerprint cache for remote client deduplication
■ About the storage lifecycle policies required for Auto Image Replication
■ About NetBackup WORM storage support for immutable and indelible data
Configuring deduplication 56
Configuring MSDP server-side deduplication
Step 1 Install the license for See “Licensing NetBackup MSDP” on page 52.
deduplication
Step 2 Create NetBackup log file See “NetBackup MSDP log files” on page 410.
directories on the master
See “ Creating NetBackup log file directories for MSDP ” on page 409.
server and the media servers
Step 3 Configure the Deduplication The Deduplication Multi-Threaded Agent uses the default configuration values
Multi-Threaded Agent that control its behavior. You can change those values if you want to do so.
behavior
See “About the MSDP Deduplication Multi-Threaded Agent” on page 59.
Step 4 Configure the fingerprint Configuring the fingerprint cache behavior is optional.
cache behavior
See “About the MSDP fingerprint cache” on page 68.
Step 5 Enable support for 250-TB Before you configure a storage server that hosts a 250-TB Media Server
MSDP Deduplication Pool, you must enable support for that size storage.
Step 6 Configure a deduplication How many storage servers you configure depends on: your storage
storage server requirements and on whether or not you use optimized duplication or
replication. When you configure a storage server, the wizard also lets you
configure a disk pool and a storage unit.
Step 7 Configure a disk pool If you already configured a disk pool when you configured the storage server,
you can skip this step.
How many disk pools you configure depends on: your storage requirements
and on whether or not you use optimized duplication or replication.
Step 8 Create the data directories for For a 250-TB Media Server Deduplication Pool, you must create the data
250-TB support directories under the mount points for the storage directories.
See “Creating the data directories for 250-TB MSDP support” on page 118.
Step 9 Add the other volumes for For a 250-TB Media Server Deduplication Pool, you must add the second
250-TB support and third volumes to the disk pool.
Step 10 Configure a storage unit See “Configuring a Media Server Deduplication Pool storage unit” on page 121.
Step 13 Configure MSDP restore Optionally, you can configure NetBackup to bypass media servers during
behavior restores.
Step 16 Configure a backup policy Use the deduplication storage unit as the destination for the backup policy.
If you configured replication, use the storage lifecycle policy as the storage
destination.
Step 18 Protect the MSDP data and See “About protecting the MSDP data” on page 45.
catalog
See “About protecting the MSDP catalog” on page 233.
Step 1 Configure media server See “Configuring MSDP server-side deduplication” on page 56.
deduplication
Configuring deduplication 59
About the MSDP Deduplication Multi-Threaded Agent
Step 2 Learn about client See “About NetBackup Client Direct deduplication” on page 30.
deduplication
Step 4 Enable client-side See “Configuring client attributes for MSDP client-side deduplication”
deduplication on page 125.
Step 5 Configure remote client Configuring remote client fingerprint cache seeding is optional.
fingerprint cache seeding
See “Configuring MSDP fingerprint cache seeding on the client” on page 72.
See “About seeding the MSDP fingerprint cache for remote client
deduplication” on page 70.
Step 6 Configure client-direct Configuring client-direct restores is optional. If you do not do so, restores
restores travel through the NetBackup media server components.
The Deduplication Multi-Threaded Agent uses the default configuration values that
control its behavior. You can change those values if you want to do so. The following
table describes the Multi-Threaded Agent interactions and behaviors. It also provides
links to the topics that describe how to configure those interactions and behaviors.
Interaction Procedure
The clients that should use the Deduplication See “Configuring deduplication plug-in
Multi-Threaded Agent for backups interaction with the Multi-Threaded Agent”
on page 66.
The backup policies that should use the See “Configuring deduplication plug-in
Deduplication Multi-Threaded Agent interaction with the Multi-Threaded Agent”
on page 66.
Table 5-4 describes the operational notes for MSDP multithreading. If the
Multi-Threaded Agent is not used, NetBackup uses the single-threaded mode.
Item Description
Unsupported use cases NetBackup does not use the Multi-Threading Agent for the
following use cases:
Item Description
/usr/openv/pdde/pdag/bin/mtstrmd –terminate
/usr/openv/pdde/pdag/bin/mtstrmd
Configuring deduplication 62
Configuring the Deduplication Multi-Threaded Agent behavior
Logging parameters
The following table describes the logging parameters of the mtstrm.conf
configuration file.
Logging Description
Parameter
■ Windows: LogPath=install_path\Veritas\pdde\\..\netbackup\logs\pdde
■ UNIX: LogPath=/var/log/puredisk
Configuring deduplication 63
Configuring the Deduplication Multi-Threaded Agent behavior
Logging Description
Parameter
Possible values:
To enable or disable other logging information, append one of the following to the logging value,
without using spaces:
Retention How long to retain log files (in days) before NetBackup deletes them.
LogMaxSize The maximum log size (MB) before NetBackup creates a new log file. The existing log files that
are rolled over are renamed mtstrmd.log.<date/time stamp>
Possible value: 1 to the maximum operating system file size in MBs, inclusive.
Process parameters
The following table describes the process parameters of the mtstrm.conf
configuration file.
Configuring deduplication 64
Configuring the Deduplication Multi-Threaded Agent behavior
MaxConcurrentSessions The maximum number of concurrent sessions that the Multi-Threaded Agent
processes. If it receives a backup job when the MaxConcurrentSessions value
is reached, the job runs as a single-threaded job.
NetBackup configures the value for this parameter during installation or upgrade.
The value is the hardware concurrency value of the host divided by the
BackupFpThreads value (see Table 5-7). (For the purposes of this parameter,
the hardware concurrency is the number of CPUs or cores or hyperthreading units.)
On media servers, NetBackup may not use all hardware concurrency for
deduplication. Some may be reserved for other server processes.
For more information about hardware concurrency, see the pd.conf file
MTSTRM_BACKUP_ENABLED parameter description.
BackupShmBufferSize The size of the buffers (MB) for shared memory copying. This setting affects three
buffers: The shared memory buffer itself, the shared memory receive buffer in the
mtstrmd process, and the shared memory send buffer on the client process.
BackupReadBufferSize The size (MB) of the memory buffer to use per session for read operations from a
client during a backup.
BackupReadBufferCount The number of memory buffers to use per session for read operations from a client
during a backup.
BackupBatchSendEnabled Determines whether to use batch message protocols to send data to the storage
server for a backup.
FpCacheMaxMbSize The maximum amount of memory (MB) to use per session for fingerprint caching.
SessionCloseTimeout The amount of time to wait in seconds for threads to finish processing when a
session is closed before the agent times-out with an error.
SessionInactiveThreshold The number of minutes for a session to be idle before NetBackup considers it
inactive. NetBackup examines the sessions and closes inactive ones during
maintenance operations.
Threads parameters
The following table describes the threads parameters of the mtstrm.conf
configuration file.
Configuring deduplication 66
Configuring deduplication plug-in interaction with the Multi-Threaded Agent
BackupFpThreads The number of threads to use per session to fingerprint incoming data.
Default value: BackupFpThreads= (calculated by NetBackup; see the following
explanation).
NetBackup configures the value for this parameter during installation or upgrade.
The value is equal to the following hardware concurrency threshold values.
For more information about hardware concurrency, see the pd.conf file
MTSTRM_BACKUP_ENABLED parameter description.
BackupSendThreads The number of threads to use per session to send data to the storage server during
a backup operation.
■ (Windows) install_path\Veritas\NetBackup\bin\ost-plugins
2 To change a setting, specify a new value. The following are the settings that
control the interaction:
■ MTSTRM_BACKUP_CLIENTS
■ MTSTRM_BACKUP_ENABLED
■ MTSTRM_BACKUP_POLICIES
■ MTSTRM_IPC_TIMEOUT
See “About seeding the MSDP fingerprint cache for remote client deduplication”
on page 70.
Behavior Description
On the client Configure seeding on the client for one or only a few clients.
On the storage server The use case that benefits the most is many clients to seed,
and they can use the fingerprint cache from a single host.
To ensure that NetBackup uses the seeded backup images, the first backup of a
client after you configure seeding must be a full backup with a single stream.
Specifically, the following two conditions must be met in the backup policy:
■ The Attributes tab Allow multiple data streams attribute must be unchecked.
■ The backup selection cannot include anyNEW_STREAM directives.
If these two conditions are not met, NetBackup may use multiple streams. If the
Attributes tab Limit jobs per policy is set to a number less than the total number
Configuring deduplication 71
About seeding the MSDP fingerprint cache for remote client deduplication
of streams, only those streams use the seeded images to populate the cache. Any
streams that are greater than the Limit jobs per policy value do not benefit from
seeding, and their cache hit rates may be close to 0%.
After the first backup, you can restore the original backup policy parameter settings.
The following items are example of informational messages that show that seeding
occurred:
02:15:17.433[4452.4884][DEBUG][dummy][11:bptm:6340:nbmaster1][DEBUG]
PDSTS: cache_util_load_fp_cache_nbu: enter
dir_path=/nbmaster1#1/2/#pdseed/opscenter1,
t=16s, me=1024
02:15:17.449[4452.4884][DEBUG][dummy][11:bptm:6340:nbmaster1][DEBUG]
PDSTS: cache_util_load_fp_cache_nbu: adding
'nbmaster1_1420181254_C1_F1.img' to cache list
(1)
02:15:17.449[4452.4884][DEBUG][dummy][11:bptm:6340:nbmaster1][DEBUG]
PDSTS: cache_util_load_fp_cache_nbu: opening
/nbmaster1#1/2/#pdseed/opscenter1/nbmaster1_1420181254_C1_F1.img
for image cache (1/1)
02:15:29.585[4452.4884][DEBUG][dummy][11:bptm:6340:nbmaster1][DEBUG]
PDVFS: pdvfs_lib_log: soRead: segment
c32b0756d491871c45c71f811fbd73af already
present in cache.
02:15:29.601[4452.4884][DEBUG][dummy][11:bptm:6340:nbmaster1][DEBUG]
PDVFS: pdvfs_lib_log: soRead: segment
346596a699bd5f0ba5389d4335bc7429 already
present in cache.
For more information about seeding, see the following Veritas tech note:
http://www.veritas.com/docs/TECH144437
See “About the MSDP fingerprint cache” on page 68.
Information about when to use this seeding method and how to choose a client
from which to seed is available.
See “About seeding the MSDP fingerprint cache for remote client deduplication”
on page 70.
Warning: Do not use this procedure on the storage server or the load balancing
server. If you do, it affects all clients that are backed up by that host.
clienthostmachine The name of the existing similar client from which to seed
the cache.
Note: NetBackup treats long and short host names
differently, so ensure that you use the client name as it
appears in the policy that backs it up.
(By default, NetBackup uses the same path for the storage and the catalog; the
database_path and the storage_path are the same. If you configure a separate
path for the deduplication database, the paths are different.)
When a backup runs, NetBackup loads the fingerprints from the #pdseed directory
for the client. (Assuming that no fingerprints exist for that client in the usual catalog
location.)
Information about when to use this seeding method and how to choose a client
from which to seed is available.
See “About seeding the MSDP fingerprint cache for remote client deduplication”
on page 70.
To seed the fingerprint cache from the storage server
1 Before the first backup of the remote client, specify the clients and the policy
in the following format:
UNIX: /usr/openv/pdde/pdag/bin/seedutil -seed -sclient client_name
-spolicy policy_name -dclient destination_client_name
Note: NetBackup treats long and short host names differently, so ensure that
you use the client name as it appears in the policy that backs it up.
After one full backup for the client or clients, NetBackup clears the seeding
directory automatically. If the first backup fails, the seeded data remains for
successive attempts. Although NetBackup clears the seeding directory
automatically, Veritas recommends that you clear the client seeding directories
manually.
Configuring deduplication 75
Configuring MSDP fingerprint cache seeding on the storage server
-dclient destination_client_name The name of the new client for which you are
seeding the data.
-sclient source_client_name The client from which to copy the data for
seeding.
Note: NetBackup treats long and short host
names differently, so ensure that you use the
client name as it appears in the policy that
backs it up.
■ UNIX: /usr/openv/pdde/pdag/bin
■ Windows: C:\Program Files\Veritas\pdde
mkdir c:\etc
echo Windows_BYO > "c:\etc\nbapp-release"
Note: You cannot disable the MSDP KMS service once you enable it.
If the KMS service is not available for MSDP or the key in the KMS service that is
used by MSDP is not available, then MSDP waits in an infinite loop. When MSDP
goes in an infinite loop, few commands that you run might not respond.
After you configure KMS encryption or once the MSDP processes restart, check
the KMS encryption status after the first backup finishes.
The keys in the key dictionary must not be deleted, deprecated, or terminated.
You can use the following commands to get the status of the KMS mode:
■ For UNIX:
/usr/openv/pdde/pdcr/bin/crcontrol --getmode
For MSDP cloud, run the following keydictutil command to check if the LSU
is in KMS mode:
/usr/openv/pdde/pdcr/bin/keydictutil --list
■ For Windows:
<install_path>\Veritas\pdde\crcontrol.exe --getmode
Note: If you use the nbdevconfig command to add a new encrypted cloud Logical
Storage Unit (LSU) and an encrypted LSU exists in this MSDP, the keygroupname
must be the same as the keygroupname in the previous encrypted LSU.
Note: The following steps are not supported on Solaris OS. For Solaris, refer to
the following article:
Upgrade KMS encryption for MSDP on the Solaris platform
■ For Windows:
<install_path>\Veritas\NetBackup\bin\nbkms.exe -createemptydb
Enter the following parameters when you receive a prompt:
■ Enter the HMK passphrase
Enter a password that you want to set as the host master key (HMK)
passphrase. Press Enter to use a randomly generated HMK passphrase.
The passphrase is not displayed on the screen.
■ Enter HMK ID
Enter a unique ID to associate with the host master key. This ID helps
you to determine an HMK associated with any key store.
■ Enter KPK passphrase
Enter a password that you want to set as the key protection key (KPK)
passphrase. Press Enter to use a randomly generated HMK passphrase.
The passphrase is not displayed on the screen.
■ Enter KPK ID
Enter a unique ID to associate with the key protection key. This ID helps
you to determine a KPK associated with any key store.
After the operation completes successfully, run the following command on the
master server to start KMS:
■ For UNIX:
/usr/openv/netbackup/bin/nbkms
■ For Windows:
sc start NetBackup Key Management Service
2 Create a key group and an active key by entering the following commands:
■ For UNIX:
/usr/openv/netbackup/bin/admincmd/nbkmsutil -createkg -kgname
msdp
Configuring deduplication 79
About MSDP Encryption using external KMS server
■ For Windows:
<install_path>\Veritas\NetBackup\bin\admincmd\nbkmsutil.exe
-createkg -kgname msdp
<install_path>\Veritas\NetBackup\bin\admincmd\nbkmsutil.exe
-createkey -kgname msdp -keyname name -activate
■ On Windows:
<install_path>\Veritas\pdde\kms.cfg
[KMSOptions]
KMSEnable=true
KMSKeyGroupName=YourKMSKeyGroupName
KMSServerName=YourKMSServerName
KMSType=0
For KMSServerName, enter the hostname of the server where the KMS service
runs, mainly the master server hostname.
After completing the steps, you can upgrade MSDP.
The type of storage. Select Media Server Deduplication Pool for the type of disk
storage.
The credentials for the See “About the NetBackup Deduplication Engine credentials”
deduplication engine. on page 34.
The storage paths. See “MSDP storage path properties” on page 94.
The network interface. See “About the network interface for MSDP” on page 35.
The load balancing servers, See “About MSDP storage servers” on page 27.
if any.
When you configure the storage server, the wizard also lets you create a disk pool
and storage unit.
Prerequisite For a 96-TB Media Server Deduplication Pool, you must create the
required directories before you configure the storage server.
3 On the Welcome panel, select Media Server Deduplication Pool from the
drop-down menu.
The types of storage servers that you can configure depend on the options for
which you are licensed.
The following is an example of the wizard panel:
After you select Media Server Deduplication Pool, click Next. The Add
Storage Server wizard panel appears.
Configuring deduplication 82
Configuring a storage server for a Media Server Deduplication Pool
4 On the Add Storage Server panel, select or enter the appropriate information.
The following is an example of the wizard panel:
Media server Select the media server that you want to configure as the
storage server.
Username Enter the user name for the NetBackup Deduplication Engine.
5 On the Storage Server Properties panel, enter or select the properties for
the deduplication storage server.
See “MSDP storage path properties” on page 94.
See “MSDP network interface properties” on page 97.
The following is an example of the wizard panel:
After you click Next, the behavior depends on whether you have media servers
installed, as follows:
For the media servers that you select, NetBackup enables their deduplication
functionality and populates them with the NetBackup Deduplication Engine
credentials you entered on a previous panel.
When you click Next, the Storage Server Encryption panel appears next.
Note: Load balancing for Cloud Catalyst storage servers or Cloud Catalyst
appliances is supported. See the following topic for configuration considerations.
See “Configuring source control or target control optimized duplication for Cloud
Catalyst” on page 280.
7 On the Storage Server Encryption panel, you can enable encryption and
KMS for the Media Server Deduplication Pool. The Enable KMS option is
available when you select Enable Encryption.
If none of the KMS servers is configured, NetBackup KMS is configured:
Configuring deduplication 86
Configuring a storage server for a Media Server Deduplication Pool
Option Description
Host master key (HMK) passphrase Enter a password that you want to set as
the host master key (HMK) passphrase.
Configuring deduplication 87
Configuring a storage server for a Media Server Deduplication Pool
Option Description
Key protection key (KPK) passphrase Enter a password that you want to set as
the key protection key (KPK) passphrase.
This ID helps you to determine a KPK
associated with any key store.
Option Description
■ If you select Enable KMS and if external KMS is already configured on the
Master Server, you must enter the following information:
Option Description
Ensure that a key with a custom attribute set with value of key group name
already exists in the external KMS server.
For more information about KMS configuration, see the NetBackup Security
and Encryption Guide.
When you click Next, the Storage Server Configuration Summary panel
appears next.
Configuring deduplication 88
Configuring a storage server for a Media Server Deduplication Pool
9 The Storage Server Configuration Status wizard panel describes the status
of the operation.
Configure a disk pool Ensure that Create a disk pool using the storage server
that you have just created is selected and then click Next.
10 On the Select Volumes panel, select the volume for this disk pool. All of storage
in the Storage Path that you configured in the Storage Server Configuration
Wizard is exposed as a single volume. The PureDiskVolume is a virtual name
for that storage.
The following is an example of the wizard panel:
After you select the PureDiskVolume volume, click Next. The Additional Disk
Pool Information wizard panel appears.
Configuring deduplication 91
Configuring a storage server for a Media Server Deduplication Pool
11 On the Additional Disk Pool Information panel, enter the values for this disk
pool.
See “Media Server Deduplication Pool properties” on page 116.
The following is an example of the wizard panel:
After you enter the appropriate information or select the necessary options,
click Next. The Disk Pool Configuration Summary wizard panel appears.
Configuring deduplication 92
Configuring a storage server for a Media Server Deduplication Pool
12 On the Disk Pool Configuration Summary panel, verify the selections. If OK,
click Next.
The following is an example of the wizard panel:
To configure the disk pool, click Next. The Disk Pool Configuration Status
wizard panel appears.
Configuring deduplication 93
Configuring a storage server for a Media Server Deduplication Pool
13 The Disk Pool Configuration Status wizard panel describes the progress of
the operation.
The following is an example of the wizard panel:
Configure a storage unit Ensure that Create a storage unit using the disk pool that
you have just created is selected and then click Next. The
Storage Unit Creation wizard panel appears. Continue to
the next step.
After you enter the appropriate information or select the necessary options,
click Next to create the storage unit.
15 After NetBackup configures the storage unit, the Finished panel appears. Click
Finish to exit from the wizard.
Property Description
Storage path The path to the storage. The storage path is the directory in which NetBackup stores the
raw backup data. Backup data should not be stored on the system disk.
Because the storage requires a directory path, do not use only the root node (/) or drive
letter (E:\) as the storage path. (That is, do not mount the storage as a root node (/) or a
drive letter (E:\).
For a 250-TB Media Server Deduplication Pool, you must enter the path name of the
mount point for the volume that you consider the first 32-TB storage volume. The following
is an example of a volume naming convention for the mount points for the backups:
See “Creating the data directories for 250-TB MSDP support” on page 118.
You can use the following characters in the storage path name:
NetBackup requirements for the deduplication storage paths may affect how you expose
the storage.
Property Description
Use alternate path for By default, NetBackup uses the storage path for the MSDP database (that is, the MSDP
deduplication database catalog) location. The MSDP database is different than the NetBackup catalog.
Select this option to use a location other than the default for the deduplication database.
For a 250-TB Media Server Deduplication Pool, you must select this option.
For performance optimization, it is recommended that you use a separate disk volume for
the deduplication database than for the backup data.
Database path If you selected Use alternate path for deduplication database, enter the path name for
the database. The database should not be stored on the system disk.
For a 250-TB Media Server Deduplication Pool, you must enter the path name of the
partition that you created for the MSDP catalog. For example, if the naming convention for
your mount points is /msdp/volx , the following path is recommended for the MSDP
catalog directory:
/msdp/cat
For performance optimization, it is recommended that you use a separate disk volume for
the deduplication database than for the backup data.
You can use the following characters in the path name:
If the directory or directories do not exist, NetBackup creates them and populates
them with the necessary subdirectory structure. If the directory or directories exist,
NetBackup populates them with the necessary subdirectory structure.
Caution: You cannot change the paths after NetBackup configures the deduplication
storage server. Therefore, decide during the planning phase where and how you
want the deduplicated backup data to be stored and then carefully enter the paths.
Configuring deduplication 97
Configuring a storage server for a PureDisk Deduplication Pool
Caution: You cannot change the network interface after NetBackup configures the
deduplication storage server. Therefore, enter the properties carefully.
Property Description
Use specific network Select this option to specify a network interface for the
interface deduplication traffic. If you do not specify a network interface,
NetBackup uses the operating system host name value.
After you select PureDisk Deduplication Pool, click Next. The Add Storage
Server wizard panel appears.
Configuring deduplication 99
Configuring a storage server for a PureDisk Deduplication Pool
4 On the Add Storage Server panel, select or enter the appropriate information.
The following is an example of the wizard panel:
Media server Select a media server to use to query the NetBackup 5000
series appliance for its capabilities. That media server also
is used for deduplication.
Storage server name Enter the host name of the NetBackup 5000 series appliance
in the Storage server name field.
Username Enter the user name for the NetBackup 5000 series
appliance.
Password Enter the password for the NetBackup 5000 series appliance.
When you click Next, the Storage Server Configuration Summary panel
appears next.
Configuring deduplication 101
Configuring a storage server for a PureDisk Deduplication Pool
The Storage Server Creation Status panel appears. It describes the status
of the wizard actions.
Configuring deduplication 102
Configuring a storage server for a PureDisk Deduplication Pool
7 The Storage Server Configuration Status wizard panel describes the status
of the operation, as follows:
Configure a disk pool Ensure that Create a disk pool using the storage server
that you have just created is selected and then click Next.
8 On the Volume Selection panel, select the volume for this disk pool. All of
storage of the NetBackup 5000 series appliance is exposed as a single volume.
The PureDiskVolume is a virtual name for that storage.
The following is an example of the wizard panel:
After you select the PureDiskVolume volume, click Next. The Additional Disk
Pool Information wizard panel appears.
Configuring deduplication 104
Configuring a storage server for a PureDisk Deduplication Pool
9 On the Additional Disk Pool Information panel, enter the values for this disk
pool.
See “Media Server Deduplication Pool properties” on page 116.
The following is an example of the wizard panel:
After you enter the appropriate information or select the necessary options,
click Next. The Disk Pool Configuration Summary wizard panel appears.
Configuring deduplication 105
Configuring a storage server for a PureDisk Deduplication Pool
10 On the Disk Pool Configuration Summary panel, verify the selections. If OK,
click Next.
The following is an example of the wizard panel:
To configure the disk pool, click Next. The Disk Pool Configuration Status
wizard panel appears.
Configuring deduplication 106
Configuring a storage server for a PureDisk Deduplication Pool
11 The Disk Pool Configuration Status wizard panel describes the progress of
the operation.
The following is an example of the wizard panel:
Configure a storage unit Ensure that Create a storage unit using the disk pool that
you have just created is selected and then click Next. The
Storage Unit Creation wizard panel appears. Continue to
the next step.
After you enter the appropriate information or select the necessary options,
click Next to create the storage unit.
13 After NetBackup configures the storage unit, the Finished panel appears. Click
Finish to exit from the wizard.
How many deduplication pools you configure depends on your storage requirements.
It also depends on whether or not you use optimized duplication or replication, as
described in the following table:
Type Requirements
Optimized duplication within Optimized duplication in the same domain requires the following deduplication pools:
the same NetBackup domain
■ At least one for the backup storage, which is the source for the duplication
operations. The source deduplication pool is in one deduplication node.
■ Another to store the copies of the backup images, which is the target for the
duplication operations. The target deduplication pool is in a different deduplication
node.
See “About MSDP optimized duplication within the same domain” on page 141.
Auto Image Replication to a Auto Image Replication deduplication pools can be either replication source or replication
different NetBackup domain target. The replication properties denote the purpose of the deduplication pool. The
deduplication pools inherit the replication properties from their volumes.
See “About the replication topology for Auto Image Replication” on page 164.
Auto Image Replication requires the following deduplication pools:
■ The deduplication storage server to query for the disk storage to use for the
pool.
■ The disk volume to include in the pool.
NetBackup exposes the storage as a single volume.
■ The disk pool properties.
See “Media Server Deduplication Pool properties” on page 116.
Veritas recommends that disk pool names be unique across your enterprise.
To configure a deduplication disk pool by using the wizard
1 In the NetBackup Administration Console, select either NetBackup
Management or Media and Device Management.
2 From the list of wizards in the right pane, click Configure Disk Pool.
3 Click Next on the welcome panel of the wizard.
The Disk Pool panel appears.
Configuring deduplication 110
Configuring a disk pool for deduplication
After you select the disk pool Type, click Next. The Select Storage Server
wizard panel appears.
Configuring deduplication 111
Configuring a disk pool for deduplication
5 On the Select Storage Server panel, select the storage server for this disk
pool. The wizard displays the deduplication storage servers that are configured
in your environment. For a PureDisk Deduplication Pool, the storage server
is the NetBackup 5000 series appliance.
The following is an example of the wizard panel:
6 On the Volume Selection panel, select the volume for this disk pool.
The following is an example of the wizard panel:
Media Server All of storage in the Storage Path that you configured in the
Deduplication Pool Storage Server Configuration Wizard is exposed as a
single volume. The PureDiskVolume is a virtual name for
that storage.
After you select the PureDiskVolume volume, click Next. The Additional Disk
Pool Information wizard panel appears.
Configuring deduplication 113
Configuring a disk pool for deduplication
7 On the Additional Disk Pool Information panel, enter the values for this disk
pool.
See “Media Server Deduplication Pool properties” on page 116.
The following is an example of the wizard panel:
After you enter the appropriate information or select the necessary options,
click Next. The Disk Pool Configuration Summary wizard panel appears.
Configuring deduplication 114
Configuring a disk pool for deduplication
8 On the Disk Pool Configuration Summary panel, verify the selections. If OK,
click Next.
The following is an example of the wizard panel:
To configure the disk pool, click Next. The Disk Pool Configuration Status
wizard panel appears.
Configuring deduplication 115
Configuring a disk pool for deduplication
9 The Disk Pool Configuration Status wizard panel describes the progress of
the operation.
The following is an example of the wizard panel:
Configure a storage unit Ensure that Create a storage unit using the disk pool that
you have just created is selected and then click Next. The
Storage Unit Creation wizard panel appears. Continue to
the next step.
After you enter the appropriate information or select the necessary options,
click Next to create the storage unit.
11 After NetBackup configures the storage unit, the Finished panel appears. Click
Finish to exit from the wizard.
Property Description
Storage server The storage server name. The storage server is the same as the
NetBackup media server to which the storage is attached.
Configuring deduplication 117
Configuring a disk pool for deduplication
Property Description
Storage server type For a Media Server Deduplication Pool, the storage type is
PureDisk.
Disk volumes For a Media Server Deduplication Pool, all disk storage is
exposed as a single volume.
Total available space The amount of space available in the disk pool.
Total raw size The total raw size of the storage in the disk pool.
Disk Pool name The disk pool name. Enter a name that is unique across your
enterprise.
High water mark The High water mark indicates that the volume is full. When the
volume reaches the High water mark, NetBackup fails any backup
jobs that are assigned to the storage unit. NetBackup also does
not assign new jobs to a storage unit in which the deduplication
pool is full.
The High water mark includes the space that is committed to other
jobs but not already used.
Low water mark The Low water mark has no affect on the PureDiskVolume.
Limit I/O streams Select to limit the number of read and write streams (that is, jobs)
for each volume in the disk pool. A job may read backup images
or write backup images. By default, there is no limit. If you select
this property, also configure the number of streams to allow per
volume.
Property Description
per volume Select or enter the number of read and write streams to allow per
volume.
Prerequisite ■ The volumes must be formatted with the file systems that NetBackup
supports for MSDP and mounted on the storage server.
See “About provisioning the storage for MSDP” on page 47.
■ The storage server must be configured already.
See “Configuring a storage server for a Media Server Deduplication
Pool” on page 80.
The following is an example of the mount points for the three required storage
volumes:
The following two examples show how to add the pathnames for /msdp/vol1
and /msdp/vol2 volumes:
After the next polling cycle of the NetBackup Remote Manager and Monitor
Service, the NetBackup Administration Console displays the new, expanded
capacity for the disk pool.
2 On the MSDP storage server, you can use the crcontrol utility to verify that
the Media Server Deduplication Pool contains the new volumes. If the
volumes were added correctly, there should be three mount points as follows:
3 To display detailed information about the disk pool, use the crcontrol
--dsstat 3 command, as follows:
/usr/openv/pdde/pdcr/bin/crcontrol --dsstat 3
Configuring deduplication 121
Configuring a Media Server Deduplication Pool storage unit
Property Description
Storage unit A unique name for the new storage unit. The name can describe the
name type of storage. The storage unit name is the name used to specify a
storage unit for policies and schedules. The storage unit name cannot
be changed after creation.
Disk type Select PureDisk for the disk type for a Media Server Deduplication
Pool.
Disk pool Select the disk pool that contains the storage for this storage unit.
All disk pools of the specified Disk type appear in the Disk pool list.
If no disk pools are configured, no disk pools appear in the list.
Media server The Media server setting specifies the NetBackup media servers that
can deduplicate the data for this storage unit. Only the deduplication
storage server and the load balancing servers appear in the media
server list.
Specify the media server or servers as follows:
NetBackup selects the media server to use when the policy runs.
Maximum For normal backups, NetBackup breaks each backup image into
fragment size fragments so it does not exceed the maximum file size that the file
system allows. You can enter a value from 20 MBs to 51200 MBs.
For a FlashBackup policy, Veritas recommends that you use the default,
maximum fragment size to ensure optimal deduplication performance.
http://www.veritas.com/docs/DOC5332
Configuring deduplication 124
Configuring a Media Server Deduplication Pool storage unit
Property Description
Maximum The Maximum concurrent jobs setting specifies the maximum number
concurrent jobs of jobs that NetBackup can send to a disk storage unit at one time.
(Default: one job. The job count can range from 0 to 256.) This setting
corresponds to the Maximum concurrent write drives setting for a Media
Manager storage unit.
NetBackup queues jobs until the storage unit is available. If three backup
jobs are scheduled and Maximum concurrent jobs is set to two,
NetBackup starts the first two jobs and queues the third job. If a job
contains multiple copies, each copy applies toward the Maximum
concurrent jobs count.
The number to enter depends on the available disk space and the
server's ability to run multiple backup processes.
Warning: A Maximum concurrent jobs setting of 0 disables the storage
unit.
Use WORM This option is enabled for storage units that are WORM capable.
WORM is the acronym for Write Once Read Many.
Select this option if you want the backup images on this storage unit to
be immutable and indelible until the WORM Unlock Time.
■ Configure the media servers for NetBackup deduplication and configure the
storage.
■ Configure a disk pool.
■ Configure a storage unit for your most important clients (such as STU-GOLD).
Select the disk pool. Select Only use the following media servers. Select two
media servers to use for your important backups.
■ Create a backup policy for the 100 important clients and select the STU-GOLD
storage unit. The media servers that are specified in the storage unit move the
client data to the deduplication storage server.
■ Configure another storage unit (such as STU-SILVER). Select the same disk
pool. Select Only use the following media servers. Select the other two media
servers.
■ Configure a backup policy for the 500 regular clients and select the STU-SILVER
storage unit. The media servers that are specified in the storage unit move the
client data to the deduplication storage server.
Backup traffic is routed to the wanted data movers by the storage unit settings.
Note: NetBackup uses storage units for media server selection for write activity
(backups and duplications) only. For restores, NetBackup chooses among all media
servers that can access the disk pool.
Option Description
Compression for For backups, the deduplication plug-in compresses the data after it is deduplicated. The
backups data remains compressed during transfer from the plug-in to the NetBackup Deduplication
Engine on the storage server. The Deduplication Engine writes the encrypted data to the
storage. For restore jobs, the process functions in the reverse direction.
The COMPRESSION parameter in the pd.conf file on each MSDP host controls compression
and decompression for that host. By default, backup compression is enabled on all MSDP
hosts. Therefore, compression and decompression occur on the following hosts as necessary:
■ The clients that deduplicate their own data (that is, client-side deduplication).
■ The load balancing servers.
■ The storage server.
MSDP compression cannot occur on normal NetBackup clients (that is, the clients that do
not deduplicate their own data).
Note: Do not enable backup compression by selecting the Compression option on the
Attributes tab of the Policy dialog box. If you do, NetBackup compresses the data before
it reaches the plug-in that deduplicates it. Consequently, deduplication rates are very low.
Also, NetBackup does not use the Deduplication Multi-Threaded Agent if policy-based
encryption is configured.
Compression for For duplication and replication, the deduplication plug-in compresses the data for transfer.
duplication and The data remains compressed during transfer from the plug-in to the NetBackup Deduplication
replication Engine on the storage server and remains compressed on the storage.
NetBackup chooses the least busy host to initiate and manage each duplication job and
replication job. To ensure that compression occurs for all optimized duplication and replication
jobs: do not change the default setting of the OPTDUP_COMPRESSION parameter.
Option Description
Backup encryption For backups, the deduplication plug-in encrypts the data after it is deduplicated. The data
remains encrypted during transfer from the plug-in to the NetBackup Deduplication Engine
on the storage server. The Deduplication Engine writes the encrypted data to the storage.
For restore jobs, the process functions in the reverse direction.
The MSDP pd.conf file ENCRYPTION parameter controls backup encryption for individual
hosts. By default, backup encryption is disabled on all MSDP hosts. If you want backup
encryption, you must enable it on the following MSDP hosts:
■ The clients that deduplicate their own data (that is, client-side deduplication).
■ The MSDP load balancing servers.
■ The MSDP storage server.
Option Description
Duplication and For duplication and replication, the deduplication plug-in on MSDP servers encrypts the
replication encryption data for transfer. The data is encrypted during transfer from the plug-in to the NetBackup
Deduplication Engine on the target storage server and remains encrypted on the target
storage.
NetBackup chooses the least busy host to initiate and manage each duplication job and
replication job.
See “Configuring encryption for MSDP optimized duplication and replication” on page 133.
NetBackup 8.0 introduced the Advanced Encryption Standard 256 bit, CTR (AES)
encryption algorithm to Media Server Deduplication Pool (MSDP). The AES
encryption algorithm replaces the older Blowfish encryption algorithm.
See “About the rolling data conversion mechanism for MSDP” on page 134.
See “MSDP encryption behavior and compatibilities” on page 137.
ENCRYPTION 0 1 0 1 1
COMPRESSION 1 1 1 1 1
OPTDUP_ENCRYPTION 0 0 1 0 1
OPTDUP_COMPRESSION 1 0 1 1 1
Outcomes on the disk Backup MSDP: Backup MSDP: Backup MSDP: Backup MSDP: Backup MSDP:
pools
Encryption: 0 Encryption: 1 Encryption: 0 Encryption: 1 Encryption: 1
Compression: 1 Compression: 1 Compression: 1 Compression: 1 Compression: 1
Target MSDP: Target MSDP: Target MSDP: Target MSDP: Target MSDP
Notes See the following See the following See the following
note. note. note.
Note: Settings A and C have the same outcome on target storage as setting D
because of the interaction of compression and encryption for the backups. If the
backups are encrypted and compressed, they are also encrypted and compressed
during optimized duplication and replication and encrypted and compressed on
target storage. The OPTDUP_ENCRYPTION setting is ignored.
For client-side deduplication clients, a parameter on the storage server can override
the pd.conf file ENCRYPTION parameter.
See “Configuring encryption for MSDP backups” on page 132.
See “About MSDP compression” on page 127.
See “About MSDP encryption” on page 129.
See “About the MSDP pd.conf configuration file” on page 209.
Configuring deduplication 132
Configuring encryption for MSDP backups
Configure encryption for Use this procedure to configure encryption for all of your clients
all Client Direct clients that deduplicate their own data (that is, client-side deduplication).
If you use this procedure, you do not have to configure each
client-side deduplication client individually.
To ensure that encryption occurs for all backups jobs, configure it on all MSDP
hosts. MSDP hosts include the MSDP storage server, the MSDP load balancing
servers, and the NetBackup Client Direct deduplication clients.
See “About MSDP encryption” on page 129.
To configure backup encryption on a single host
1 Use a text editor to open the pd.conf file on the host.
The pd.conf file resides in the following directories:
■ (UNIX) /usr/openv/lib/ost-plugins/
■ (Windows) install_path\Veritas\NetBackup\bin\ost-plugins
See “MSDP pd.conf file parameters” on page 210.
2 For the line that begins with #ENCRYPTION, remove the pound sign ( or hash
sign, #) in column 1.
Configuring deduplication 133
Configuring encryption for MSDP optimized duplication and replication
Note: The spaces to the left and right of the equal sign (=) in the file are
significant. Ensure that the space characters appear in the file after you edit
the file.
2 Add encrypt to the ServerOptions line of the file. The following line is an
example:
ServerOptions=fast,verify_data_read,encrypt
2 For the line that begins with #OPTDUP_ENCRYPTION, remove the pound sign (
or hash sign, #) in column 1.
3 In that same line, replace the 0 (zero) with a 1.
Note: The spaces to the left and right of the equal sign (=) in the file are
significant. Ensure that the space characters appear in the file after you edit
the file.
The conversion begins automatically after an upgrade to NetBackup 8.0. You can
control some aspects of the conversion process or stop it entirely.
Rolling data conversion traverses all existing data containers. If the data is encrypted
with the Blowfish algorithm, the data is re-encrypted with the AES-256 algorithm.
Then a new SHA-512/256 fingerprint is generated. After the conversion, the data
container has an additional .map file, in addition to the .bhd and .bin files. The
.map file contains the mapping between the SHA-512/256 and the MD5-like
fingerprints. It is used for the compatibility between SHA-512/256 fingerprints and
MD5-like fingerprints. The .bhd file includes the SHA-512/256 fingerprints.
When you upgrade to NetBackup 8.1.1, there might be encrypted data that is not
encrypted using a customer key. The encrypted data must be encrypted by a
customer key and to handle the data conversion and secure the data, a new internal
task encrypts the existing data using a customer key. After the encryption and the
fingerprint rolling conversion completes, the KMS rolling conversion begins.
Note: The performance numbers shown were observed in the Veritas test
environment and are not a guarantee of performance in your environment.
Table 5-18 MSDP crcontrol command options for rolling data conversion
Option Description
Windows:
install_path\Veritas\pdde\Crcontrol.exe
--dataconverton
UNIX: /usr/openv/pdde/pdcr/bin/crcontrol
--dataconverton
Windows:
install_path\Veritas\pdde\Crcontrol.exe
--dataconvertoff
UNIX: /usr/openv/pdde/pdcr/bin/crcontrol
--dataconvertoff
Configuring deduplication 137
MSDP encryption behavior and compatibilities
Table 5-18 MSDP crcontrol command options for rolling data conversion
(continued)
Option Description
Windows:
install_path\Veritas\pdde\Crcontrol.exe
--dataconvertstate
UNIX: /usr/openv/pdde/pdcr/bin/crcontrol
--dataconvertstate
Windows:
install_path\Veritas\pdde\Crcontrol.exe
--dataconvertmode mode
UNIX: /usr/openv/pdde/pdcr/bin/crcontrol
--dataconvertmode <mode>
Client with NetBackup version earlier than AES (using inline data conversion)
8.0, using the Client Direct deduplication
Load balancing server with NetBackup AES (using inline data conversion)
version earlier than 8.0
Table 5-20 Encryption behavior for optimized duplication and Auto Image
Replication operations to a NetBackup 8.0 target server
Note: Inline data conversion takes place simultaneously while the backup,
duplication, or replication operations are in progress.
Backups and restores For backups and restores, NetBackup uses the network interface
that was configured during the storage server configuration.
Both the backup and restore traffic and the control traffic travel
over the backup network.
Duplication and For the duplication and the replication traffic, configure your host
replication operating systems to use a different network than the one you use
for backups and restores.
Both the duplication and the replication data traffic and the control
traffic travel over the duplication and replication network.
See “About MSDP optimized duplication within the same domain” on page 141.
See “About MSDP replication to a different domain” on page 156.
Configuring deduplication 140
Configuring a separate network path for MSDP duplication and replication
Veritas recommends that you always use the fully qualified domain name when
you specify hosts.
2 On the destination storage server, add the source storage servers's dedicated
network interface to the operating system hosts file. If SourceStorageServer
is the name of the source host on the network that is dedicated for duplication,
the following is an example of the hosts entry in IPv4 notation:
Veritas recommends that you always use the fully qualified domain name when
specifying hosts.
Configuring deduplication 141
About MSDP optimized duplication within the same domain
3 To force the changes to take effect immediately, flush the DNS cache. See the
operating system documentation for how to flush the DNS cache.
4 From each host, use the ping command to verify that each host resolves the
name of the other host.
If the ping command returns positive results, the hosts are configured for
duplication and replication over the separate network.
5 When you configure the target storage server, ensure that you select the host
name that represents the alternate network path.
By default, NetBackup does not retry failed optimized duplication jobs invoked by
NetBackup Vault of the bpduplicate command. You can change that behavior.
See “Configuring NetBackup optimized duplication or replication behavior”
on page 153.
You can use a separate network for the duplication traffic.
See “About a separate network path for MSDP duplication and replication”
on page 139.
See “Configuring MSDP optimized duplication within the same NetBackup domain”
on page 149.
Review the following requirements and limitations.
■ NetBackup does not support MSDP optimized duplication to storage unit groups.
If you use a storage unit group as a destination for optimized duplication,
NetBackup uses regular duplication.
■ Optimized duplication does not support multiple copies. If NetBackup is
configured to make multiple new copies from the (source) copy of the backup
image, the following occurs:
■ In a storage lifecycle policy, one duplication job creates one optimized
duplication copy. If multiple optimized duplication destinations exist, a
separate job exists for each destination. This behavior assumes that the
device for the optimized duplication destination is compatible with the device
on which the source image resides.
If multiple remaining copies are configured to go to devices that are not
optimized duplication capable, NetBackup uses normal duplication. One
duplication job creates those multiple copies.
■ For other duplication methods, NetBackup uses normal duplication. One
duplication job creates all of the copies simultaneously. The other duplication
methods include the following: NetBackup Vault, the bpduplicate command
line, and the duplication option of the Catalog utility in the NetBackup
Administration Console.
■ The copy operation uses the maximum fragment size of the source storage unit,
not the setting for the destination storage unit. The optimized duplication copies
the image fragments as is. For greater efficiency, the duplication does not resize
and reshuffle the images into a different set of fragments on the destination
storage unit.
■ If the destination is a PureDisk Deduplication Pool, that PureDisk storage
domain can be the destination of duplication from one NetBackup domain only.
If you try to duplicate from MSDP storage in more than one NetBackup domain,
optimized duplication does not function correctly.
About the media servers for MSDP optimized duplication within the
same domain
For optimized Media Server Deduplication Pool duplication within the same
domain, the source storage and the destination storage must have at least one
media server in common. The common server initiates, monitors, and verifies the
duplication operation. The common server requires credentials for both the source
storage and the destination storage. (For deduplication, the credentials are for the
NetBackup Deduplication Engine, not for the host on which it runs.)
Which media server initiates the duplication operation determines if it is a push or
a pull operation, as follows:
■ If the media server is co-located physically with the source storage server, it is
push duplication.
■ If the media server is co-located physically with the destination storage server,
it is a pull duplication.
Technically, no advantage exists with a push duplication or a pull duplication.
However, the media server that initiates the duplication operation also becomes
the write host for the new image copies.
A storage server or a load balancing server can be the common server. The common
server must have the credentials and the connectivity for both the source storage
and the destination storage.
Deduplication Deduplication
plug-in plug-in
NetBackup NetBackup
2
Deduplication Deduplication
Engine Engine
3
Please verify that the data arrived
MSDP_L
Credentials: Credentials:
Credentials:
StorageServer-L StorageServer-L StorageServer-R
StorageServer-R
Figure 5-2 shows the settings for the storage unit for the normal backups for the
local deduplication node. The disk pool is the MSDP_L in the local environment.
Because all hosts in the local node are co-located, you can use any available media
server for the backups.
Configuring deduplication 146
About MSDP optimized duplication within the same domain
Figure 5-3 shows the storage unit settings for the optimized duplication. The
destination is the MSDP_R in the remote environment. You must select the common
server, so only load balancing server LB_L2 is selected.
Configuring deduplication 147
About MSDP optimized duplication within the same domain
If you use the remote node for backups also, select StorageServer-R and load
balancing server LB_R1 in the storage unit for the remote node backups. If you
select server LB_L2, it becomes a load balancing server for the remote Media
Server Deduplication Pool . In such a case, data travels across your WAN.
You can use a load balancing server when you duplicate between two NetBackup
deduplication pools.
Please verify
that the
plug-in plug-in
data arrived
MediaServer_DedupePool_A
MediaServer_DedupePool_B
Figure 5-5 shows the storage unit settings for the duplication destination. They are
similar to the push example except host B is selected. Host B is the common server,
so it must be selected in the storage unit.
Configuring deduplication 149
Configuring MSDP optimized duplication within the same NetBackup domain
If you use node B for backups also, select host B and not host A in the storage unit
for the node B backups. If you select host A, it becomes a load balancing server
for the node B deduplication pool.
Step 1 Review optimized duplication See “About MSDP optimized duplication within the same domain”
on page 141.
Configuring deduplication 150
Configuring MSDP optimized duplication within the same NetBackup domain
Step 2 Configure the storage servers See “Configuring a storage server for a Media Server Deduplication Pool”
on page 80.
One server must be common between the source storage and the
destination storage. Which you choose depends on whether you want a
push or a pull configuration.
See “About the media servers for MSDP optimized duplication within the
same domain” on page 144.
Step 3 Configure the deduplication If you did not configure the deduplication pools when you configured the
pools storage servers, use the Disk Pool Configuration Wizard to configure
them.
Step 4 Configure the storage unit for In the storage unit for your backups, do the following:
backups
1 For the Disk type, select PureDisk.
2 For the Disk pool, select your Media Server Deduplication Pool
or PureDisk Deduplication Pool.
If you use a pull configuration, do not select the common media server in
the backup storage unit. If you do, NetBackup uses it to deduplicate backup
data. (That is, unless you want to use it for a load balancing server for the
source deduplication node.)
Step 5 Configure the storage unit for Veritas recommends that you configure a storage unit specifically to be
duplication the target for the optimized duplication. Configure the storage unit in the
deduplication node that performs your normal backups. Do not configure
it in the node that contains the copies.
Also select Only use the following media servers. Then, select the media
server or media servers that are common to both the source storage server
and the destination storage server. If you select more than one, NetBackup
assigns the duplication job to the least busy media server.
If you select only a media server (or servers) that is not common, the
optimized duplication job fails.
Step 6 Configure optimized Optionally, you can configure the bandwidth for replication.
duplication bandwidth
See “About configuring MSDP optimized duplication and replication
bandwidth” on page 191.
Step 7 Configure optimized Optionally, you can configure the optimized duplication behavior.
duplication behaviors
See “Configuring NetBackup optimized duplication or replication behavior”
on page 153.
Step 8 Configure a storage lifecycle Configure a storage lifecycle policy only if you want to use one to duplicate
policy for the duplication images. The storage lifecycle policy manages both the backup jobs and
the duplication jobs. Configure the lifecycle policy in the deduplication
environment that performs your normal backups. Do not configure it in the
environment that contains the copies.
When you configure the storage lifecycle policy, do the following:
■ The first operation must be a Backup. For the Storage for the Backup
operation, select the storage unit that is the target of your backups.
That storage unit may use a Media Server Deduplication Pool or a
PureDisk Deduplication Pool.
These backups are the primary backup copies; they are the source
images for the duplication operation.
■ For the second, child Operation, select Duplication. Then, select the
storage unit for the destination deduplication pool. That pool may be
a Media Server Deduplication Pool or a PureDisk Deduplication
Pool.
Step 9 Configure a backup policy Configure a policy to back up your clients. Configure the backup policy in
the deduplication environment that performs your normal backups. Do not
configure it in the environment that contains the copies.
■ If you use a storage lifecycle policy to manage the backup job and the
duplication job: Select that storage lifecycle policy in the Policy storage
field of the Policy Attributes tab.
■ If you do not use a storage lifecycle policy to manage the backup job
and the duplication job: Select the storage unit that contains your normal
backups. These backups are the primary backup copies.
Note: NetBackup supports PureDisk Deduplication Pool storage on
NetBackup 5000 series appliances only.
Step 10 Configure NetBackup Vault Configure Vault duplication only if you use NetBackup Vault to duplicate
for the duplication the images.
Step 11 Duplicate by using the Use the NetBackup bpduplicate command only if you want to duplicate
bpduplicate command images manually.
http://www.veritas.com/docs/DOC5332
Behavior Description
You can change the number of hours for the wait period.
If you use a storage lifecycle policy for duplication, do not configure optimized
duplication behavior for NetBackup Vault or the bpduplicate command, and vice
versa. NetBackup behavior may not be predictable.
Caution: These settings affect all optimized duplication jobs; they are not limited
to a specific NetBackup storage option.
■ Windows: install_path\NetBackup\db\config.
Configuration options are key and value pairs, as shown in the following examples:
■ CLIENT_READ_TIMEOUT = 300
■ LOCAL_CACHE = NO
■ RESUME_ORIG_DUP_ON_OPT_DUP_FAIL = TRUE
■ SERVER = server1.example.com
You can specify some options multiple times, such as the SERVER option.
Configuring deduplication 156
About MSDP replication to a different domain
/usr/openv/netbackup/bin/nbsetconfig
On a NetBackup server:
/usr/openv/netbackup/bin/admincmd/bpsetconfig
install_path\NetBackup\bin\nbsetconfig.exe
On a NetBackup server:
install_path\NetBackup\bin\admincmd\bpsetconfig.exe
2 At the command prompt, enter the key and the value pairs of the configuration
options that you want to set, one pair per line.
You can change existing key and value pairs.
You can add key and value pairs.
Ensure that you understand the values that are allowed and the format of any
new options that you add.
3 To save the configuration changes, type the following, depending on the
operating system:
Windows: Ctrl + Z Enter
UNIX: Ctrl + D Enter
Media Server Deduplication Pool A Media Server Deduplication Pool, which can be
hosted on the following systems:
Cloud Catalyst Storage Server ■ A disk pool for Cloud Catalyst storage server.
■ A Media Server Deduplication Pool
Auto Image Replication does not support replicating from a storage unit group. That
is, the source copy cannot be in a storage unit group.
If a replication job fails, NetBackup retries the replication until it succeeds or the
source images expire. You can change the retry interval behavior.
See “Configuring NetBackup optimized duplication or replication behavior”
on page 153.
If a job fails after it replicates some of the images, NetBackup does not run a
separate image cleanup job for the partially replicated images. The next time the
replication runs, that job cleans up the image fragments before it begins to replicate
the images.
You can use a separate network for the duplication traffic.
See “About a separate network path for MSDP duplication and replication”
on page 139.
See “Configuring MSDP replication to a different NetBackup domain” on page 158.
Configuring deduplication 158
Configuring MSDP replication to a different NetBackup domain
Step 1 Learn about MSDP replication See “About MSDP replication to a different domain” on page 156.
Step 3 Add the remote storage server as See “Configuring a target for MSDP replication to a remote domain”
a replication target on page 184.
Step 4 Configure a storage lifecycle The following are the options when you configure the SLP operations:
policy
■ If you configured a trust relationship with the target domains, you can
specify one of the following options:
■ All replication target storage servers (across different
NetBackup domains)
NetBackup automatically creates an import SLP in the target
domain when the replication job runs.
■ A specific Master Server. If you choose this option, you then
select Target master server and Target import SLP.
You must create an import SLP in the target domain before you
configure an SLP in the source domain.
■ If you did not configure a trust relationship with the target domains,
All replication target storage servers (across different NetBackup
domains) is selected by default. You cannot choose a specific target
storage server.
NetBackup automatically creates an import SLP in the target domain
when the replication job runs.
See “About the storage lifecycle policies required for Auto Image
Replication” on page 192.
Step 5 Configure replication bandwidth Optionally, you can configure the bandwidth for replication.
■ Many-to-one model
Remote offices in multiple domains can back up to a storage device in a single
domain.
■ Many-to-many model
Remote datacenters in multiple domains can back up multiple disaster recovery
sites.
NetBackup supports Auto Image Replication from a Media Server Deduplication
Pool in one NetBackup domain to a Media Server Deduplication Pool in another
domain.
NetBackup supports Auto Image Replication from a disk volume in a Media Server
Deduplication Pool in one NetBackup domain to a disk volume in a Media Server
Deduplication Pool in another domain.
Process Overview
Table 5-26 is an overview of the process, generally describing the events in the
originating and target domains.
NetBackup uses storage lifecycle policies in the source domain and the target
domain to manage the Auto Image Replication operations.
See “About the storage lifecycle policies required for Auto Image Replication”
on page 192.
1 Originating master server Clients are backed up according to a backup policy that indicates a storage
(Domain 1) lifecycle policy as the Policy storage selection. The SLP must include at least
one Replication operation to similar storage in the target domain.
2 Target master server The storage server in the target domain recognizes that a replication event
(Domain 2) has occurred. It notifies the NetBackup master server in the target domain.
3 Target master server NetBackup imports the image immediately, based on an SLP that contains an
(Domain 2) import operation. NetBackup can import the image quickly because the
metadata is replicated as part of the image. (This import process is not the
same as the import process available in the Catalog utility.)
4 Target master server After the image is imported into the target domain, NetBackup continues to
(Domain 2) manage the copies in that domain. Depending on the configuration, the media
server in Domain 2 can replicate the images to a media server in Domain 3.
Figure 5-6 represents the following cascading configuration across three domains.
■ The image is created in Domain 1, and then replicated to the target Domain 2.
■ The image is imported in Domain 2, and then replicated to a target Domain 3.
■ The image is then imported into Domain 3.
Domain 1
SLP (D1toD2toD3)
Backup
Replication to target master Domain 2
SLP (D1toD2toD3)
Import Import
Replication to target server Domain 3
In the cascading model, the originating master server for Domain 2 and Domain 3
is the master server in Domain 1.
Note: When the image is replicated in Domain 3, the replication notification event
indicates that the master server in Domain 2 is the originating master server.
However, after the image is imported successfully into Domain 3, NetBackup
correctly indicates that the originating master server is in Domain 1.
The cascading model presents a special case for the Import SLP that replicates
the imported copy to a target master. (This master server that is neither the first
nor the last in the string of target master servers.)
The Import SLP must include at least one operation that uses a Fixed retention
type and at least one operation that uses a Target Retention type. So that the
Import SLP can satisfy these requirements, the import operation must use a Target
Retention.
Table 5-27 shows the difference in the import operation setup.
Configuring deduplication 163
Configuring MSDP replication to a different NetBackup domain
At least one operation must use the Target Here is the difference:
retention.
To meet the criteria, the import operation
must use Target retention.
Domain 1
SLP (D1toD2toD3)
Backup
Domain 2
Replication to target master
SLP (D1toD2toD3)
Import
Import Duplication
Domain 3
Replication to target master
Caution: Choose the target storage server carefully. A target storage server must
not also be a storage server for the originating domain.
Source A source volume contains the backups of your clients. The volume is the
source for the images that are replicated to a remote NetBackup domain.
Each source volume in an originating domain has one or more replication
partner target volumes in a target domain.
Target A target volume in the remote domain is the replication partner of a source
volume in the originating domain.
The following are the options and arguments for the command:
Save the output to a file so that you can compare the current topology with the
previous topology to determine what has changed.
See “Sample volume properties output for MSDP replication” on page 166.
Trust relationship You can select a subset of your trusted domains as a target
for replication. NetBackup then replicates to the specified
domains only rather than to all configured replication targets.
This type of Auto Image Replication is known as targeted
A.I.R.
Note: The NetBackup Administration Console does not support adding a trusted
master server using an external CA-signed certificate.
See “Adding a trusted master server using external CA-signed certificate”
on page 178.
See “About the certificate to be used for adding a trusted master server” on page 171.
The following diagram illustrates the different tasks for adding trusted master servers
when NetBackup CA-signed certificate (or host ID-based certificate) is used for
establishing trust between the source and the target master servers.
Configuring deduplication 169
Configuring MSDP replication to a different NetBackup domain
Domain A Domain B
(Source) (Target)
6
Targeted
Media server Image AIR Image Media server
5 4
Replication Policy Import Policy
3
Trust
Master Server A established Master Server B
Admin Admin
1
Get CA certificate Fingerprint and Authorization token or User
credentials from remote master servers
Step 1 Administrators of both the source and target master servers To obtain the authorization tokens, use the
must obtain each other’s CA certificate fingerprint and bpnbat command to login and nbcertcmd
authorization tokens or the user credentials. This activity to get the authorization tokens.
must be performed offline.
To obtain the SHA1 fingerprint of root
Note: It is recommended to use an authentication token to certificate, use the nbcertcmd
connect to the remote master server. An authentication token -displayCACertDetail command.
provides restricted access and allows secure communication
To perform this task, see the NetBackup
between both the hosts. The use of user credentials (user
Commands Reference Guide.
name and password) may present a possible security breach.
Note: When you run the commands, keep
the target as the remote server.
Configuring deduplication 170
Configuring MSDP replication to a different NetBackup domain
Step 2 Establish trust between the source and the target domains. To perform this task in the NetBackup
Administration Console, see the following
■ On the source master server, add the target master server
topic:
as trusted server.
■ On the target master server, add the source master server See “Adding a trusted master server using a
as trusted server. NetBackup CA-signed (host ID-based)
certificate” on page 173.
Step 3 After you have added the source and target trusted servers, To understand the use of host ID-based
they have each other’s host ID-based certificates. The certificates, see the NetBackup Security and
certificates are used during each communication. Encryption Guide.
Step 3.1 Configure the source media server to get the security See “Configuring NetBackup CA and
certificates and the host ID certificates from the target master NetBackup host ID-based certificate for
server. secure communication between the source
and the target MSDP storage servers”
on page 182.
Step 4 Create an import storage lifecycle policy in the target domain. See “About storage lifecycle policies”
on page 192.
Step 5 On the source MSDP server, use the Replication tab from See “Configuring a target for MSDP
the Change Storage Server dialog box to add the credentials replication to a remote domain” on page 184.
of the target storage server.
Step 5.1 Create a replication storage lifecycle policy in the source See “About storage lifecycle policies”
domain using the specific target master server and storage on page 192.
lifecycle policy.
Step 6 The backups that are generated in one NetBackup domain See “About NetBackup Auto Image
can be replicated to storage in one or more target NetBackup Replication” on page 159.
domains. This process is referred to as Auto Image
Replication.
If your source and target trusted servers use different NetBackup versions, consider
the following.
Note: When you upgrade both the source and the target master server to version
8.1 or later, you need to update the trust relationship. Run the following command:
nbseccmd -setuptrustedmaster -update
8.1 and later 8.1 and later Add a trusted master server using
authorization token.
8.1 and later 8.0 or earlier On the source server, add the target
as the trusted master server using the
remote (target) server’s credentials.
8.0 or earlier 8.1 and later On the source server, add the target
as the trusted master server using the
remote (target) server’s credentials.
Which certificate The target master server may support external CA, NetBackup CA, or
authorities (CA) both. The following settings show the CA usage information of the
does the target master server:
master server
■ In the NetBackup Administration Console - NetBackup
support?
Management > Security Management > Global Security Settings
■ In the NetBackup web user interface - Security > Global Security
Settings > Secure Communication.
The following table lists CA support scenarios and certificate to be used to establish
trust between the source and the target master servers.
The source master See “Adding a trusted master server using external CA-signed
server can use certificate” on page 178.
NetBackup CA and
external CA for NetBackup CA NetBackup CA
communication with a See “Adding a trusted master server using a NetBackup CA-signed
remote master server (host ID-based) certificate” on page 173.
External CA and NetBackup prompts to select the CA that you want to use for trust
NetBackup CA setup
■ For remote Windows master server, if the user's domain is not same as that
of the authentication service, you must add the domain with LDAP using the
vssat addldapdomain command. See the NetBackup Commands Reference
Guide.
Also, this user must have RBAC security administrator permissions. See the
NetBackup Web UI Security Administrator's Guide.
8 Click OK.
9 Perform the same procedure on the remote master server that you added in
step 5.
To add a trusted master server, when both the source and the target server
are NetBackup version 8.0
1 Ensure that the Enable insecure communication with NetBackup 8.0 and
earlier hosts option is enabled in the global security settings.
2 In the NetBackup Administration Console, expand NetBackup Management
> Host Properties > Master Servers in the left pane.
3 In the right pane, select the master server and Actions > Properties.
4 In the properties dialog box left pane, select Servers.
5 On the Trusted Master Servers tab, click Add.
6 Enter the fully-qualified host name of the remote master server and click
Validate Certificate Authority.
Configuring deduplication 178
Configuring MSDP replication to a different NetBackup domain
7 Enter the Username and Password of the remote master server host.
8 Click OK.
More information
See “About trusted master servers for Auto Image Replication” on page 167.
See “Configuring MSDP replication to a different NetBackup domain” on page 158.
For details on usage reporting in the web UI, see the NetBackup Web UI for Backup
Administrator's Guide.
For more information on commands, see the NetBackup Commands Reference
Guide. For details on the authalias.conf, see the NetBackup Security and
Encryption Guide.
See “About the certificate to be used for adding a trusted master server” on page 171.
Note: The NetBackup Administration Console does not support adding a trusted
master server using an external certificate.
If you try to add a trusted master server with an external certificate using the
NetBackup Administration Console, an error is displayed.
Note: If either your source or the target server is on version 8.0 or earlier, follow
the procedure that is prescribed in the respective guide.
Configuring deduplication 180
Configuring MSDP replication to a different NetBackup domain
2 Delete all storage lifecycle policies (SLPs) that use the trusted master as a
destination.
Note: Before deleting a storage lifecycle policy, ensure that there are no backup
policies that indicate the SLP for the Policy storage.
Note: In case of multiple NICs, if you have established trust using more that one
host NIC and if you remove the trust relationship with any one host NIC, the trust
with all the other host NICs is broken.
nodes of the cluster. The certificates are used to establish SSL connections between
the NetBackup hosts. The inter-node authentication allows the following NetBackup
functionality:
Targeted A.I.R. (Auto Image Auto Image Replication in which a master server is in a cluster
Replication) requires inter-node authentication among the hosts in that
cluster. The NetBackup authentication certificates provide
the means to establish the proper trust relationships.
# bpnbaz -setupat
You will have to restart Netbackup services on this machine after
the command completes successfully.
Do you want to continue(y/n)y
Gathering configuration information.
Please be patient as we wait for 10 sec for the security services
to start their operation.
Generating identity for host 'bit1.remote.example.com'
Setting up security on target host: bit1.remote.example.com
nbatd is successfully configured on Netbackup Master Server.
Operation completed successfully.
Configuring deduplication 182
Configuring MSDP replication to a different NetBackup domain
Note: After you upgrade to NetBackup 8.1.2 or later, manually deploy NetBackup
CA and the NetBackup host ID-based certificate on the source MSDP server to use
the existing Auto Image Replication.
■ UNIX
/usr/openv/netbackup/bin/nbcertcmd -displayCACertDetail
2. On the source MSDP storage server, run the following command to get the
NetBackup CA from target NetBackup master server:
■ Windows
install_path\NetBackup\bin\nbcertcmd -getCACertificate -server
target_master_server
■ UNIX
/usr/openv/netbackup/bin/nbcertcmd -getCACertificate -server
target_master_server
When you accept the CA, ensure that the CA fingerprint is the same as
displayed in the previous step.
3. On the source MSDP storage server, run the following command to get a
certificate generated by target NetBackup master server:
■ Windows
Configuring deduplication 183
Configuring MSDP replication to a different NetBackup domain
■ UNIX
/usr/openv/netbackup/bin/nbcertcmd -getCertificate -server
target_master_server -token token_string
■ NetBackup Commands
■ Use the bpnbat command to log on the target NetBackup master server.
■ Use the nbcertcmd command to get the authorization tokens.
For more information on the commands, refer to the NetBackup Commands
Reference Guide.
Install Certificate. When you use the certutil tool to install the root certificate,
the store name parameter must be Root. When you use Windows explorer to install
the root certificate, the store location must be Local Machine and store name must
be Trusted Root Certification Authorities.
Note: About clustered master servers: If you add a trusted master server for
replication operations, you must enable inter-node authentication on all of the nodes
in the cluster. Enable the authentication before you begin the following procedure.
This requirement applies regardless of whether the clustered mastered server is
the source of the replication operation or the target.
See “About trusted master servers for Auto Image Replication” on page 167.
See “Enabling NetBackup clustered master server inter-node authentication”
on page 180.
Caution: Choose the target storage server or servers carefully. A target storage
server must not also be a storage server for the source domain. Also, a disk volume
must not be shared among multiple NetBackup domains.
4 In the Change Storage Server dialog box, select the Replication tab.
The following is an example of the Change Storage Server dialog box
Replication tab:
Configuring deduplication 186
Configuring MSDP replication to a different NetBackup domain
5 On the Replication tab, click Add. The Add a Replication Target Across a
Different NetBackup Domain dialog box appears.
The following is an example of the dialog box.
Add a trusted master Add a trusted master server if you want to replicate backup
server images to a subset of available targets.
3 Repeat the first two steps until you have finished adding
trusted master servers.
Select a trusted master Select a trusted master server if you want to replicate backup
server and replication images to a subset of available targets.
target
To select a trusted master server and replication target
1 In the Target master server drop-down list, select the
master server of the domain to which you want to
replicate data.
All trusted master servers are in the drop-down list.
5 Click OK.
Enter a replication target Enter a replication target, except the Cloud Catalyst storage
server, if you did not configure trusted master servers.
3 Click OK.
Option Description
Target master server All trusted master servers are in the drop-down list.
Target storage server type If a trusted master server is configured, the value
is Target storage server name.
Option Description
Target storage server name Select the Cloud Catalyst storage server name
only from the drop-down list of the trusted master
server.
Option Description
UNIX
/usr/openv/pdde/pdcr/bin/spauser -a -u <username> -p <password>
--role air
By default, bandwidthlimit=0.
The agent.cfg file resides in the following directory:
■ UNIX: storage_path/etc/puredisk
■ Windows: storage_path\etc\puredisk
By default, OPTDUP_BANDWIDTH = 0.
See “Configuring MSDP optimized duplication within the same NetBackup domain”
on page 149.
See “Configuring MSDP replication to a different NetBackup domain” on page 158.
Domain 1 The Auto Image Replication SLP in the source domain must meet the following criteria:
(Source ■ The first operation must be a Backup operation to a Media Server Deduplication Pool.
domain) Indicate the exact storage unit from the drop-down list. Do not select Any Available.
Note: The target domain must contain the same type of storage to import the image.
■ At least one operation must be a Replication operation to a Media Server Deduplication Pool
in another NetBackup domain.
You can configure multiple Replication operations in an Auto Image Replication SLP. The
Replication operation settings determine whether the backup is replicated to all replication targets
in all master server domains or only to specific replication targets.
See “About trusted master servers for Auto Image Replication” on page 167.
■ The SLP must be of the same data classification as the Import SLP in Domain 2.
Domain 2 If replicating to all targets in all domains, in each domain NetBackup automatically creates an Import
SLP that meets all the necessary criteria.
(Target
domain) Note: If replicating to specific targets, you must create the Import SLP before creating the Auto
Image Replication SLP in the originating domain.
■ The first operation in the SLP must be an Import operation. NetBackup must support the
Destination storage as a target for replication from the source storage.
Indicate the exact storage unit from the drop-down list. Do not select Any Available.
■ The SLP must contain at least one operation that has the Target retention specified.
■ The SLP must be of the same data classification as the SLP in Domain 1. Matching the data
classification keeps a consistent meaning to the classification and facilitates global reporting by
data classification.
See the following topic for more information about Replication operation configuration:
Figure 5-9 shows how the SLP in the target domain is set up to replicate the images
from the originating master server domain.
Configuring deduplication 194
Creating a storage lifecycle policy
Figure 5-9 Storage lifecycle policy pair required for Auto Image Replication
Import
Import operation
imports copies
Note: Restart nbstserv after you make changes to the underlying storage for any
operation in an SLP.
3 In the New Storage Lifecycle Policy dialog box, enter a Storage lifecycle
policy name.
See “NetBackup naming conventions” on page 18.
Configuring deduplication 196
Creating a storage lifecycle policy
4 Add one or more operations to the SLP. The operations are the instructions
for the SLP to follow and apply to the data that is specified in the backup policy.
If this is the first operation added to the SLP, click Add.
If this is not the first operation in the SLP, add an operation that is either
hierarchical or non-hierarchical:
To create a hierarchical operation, select an operation to become the source
of the next operation. Click Add. The new operation is a child of the selected
operation. The child is indented under the parent operation.
5 In the Properties tab of the New Storage Operation dialog box, select an
Operation type. If you're creating a child operation, the SLP displays only
those operations that are valid based on the parent operation that you've
selected.
The name of the operation reflects its purpose in the SLP:
■ Backup
■ Duplication
■ Import
■ Replication
See “About NetBackup Auto Image Replication” on page 159.
7 The Window tab displays for the following operation types: Backup From
Snapshot, Duplication, Import, Index From Snapshot, and Replication. If
you'd like to control when the secondary operation runs, create a window for
the operation.
8 Click the Advanced button in the Properties tab to display options about how
the window should behave if the window closes and a secondary operation is
not yet complete.
9 Click OK to create the operation.
10 Add additional operations to the SLP as needed. (See step 4.)
11 Change the hierarchy of the operations in the SLP if necessary.
12 Click OK to create the SLP. NetBackup validates the SLP when it is first created
and whenever it is changed.
13 Configure a backup policy and select a storage lifecycle policy as the Policy
storage.
See “Creating a backup policy” on page 200.
Setting Description
Storage lifecycle The Storage lifecycle policy name describes the SLP. The name cannot be modified after
policy name the SLP is created.
Data classification The Data classification defines the level or classification of data that the SLP is allowed
to process. The drop-down menu contains all of the defined classifications as well as the
Any classification, which is unique to SLPs.
The Any selection indicates to the SLP that it should preserve all images that are submitted,
regardless of their data classification. It is available for SLP configuration only and is not
available to configure a backup policy.
In an Auto Image Replication configuration where the master server domains run different
versions of NetBackup, see the following topic for special considerations:
See “About the storage lifecycle policies required for Auto Image Replication” on page 192.
One data classification can be assigned to each SLP and applies to all operations in the
SLP.
If a data classification is selected (other than Any), the SLP stores only those images from
the policies that are set up for that data classification. If no data classification is indicated,
the SLP accepts images of any classification or no classification.
The Data classification setting allows the NetBackup administrator to classify data based
on relative importance. A classification represents a set of backup requirements. When data
must meet different backup requirements, consider assigning different classifications.
For example, email backup data can be assigned to the silver data classification and financial
data backup may be assigned to the platinum classification.
A backup policy associates backup data with a data classification. Policy data can be stored
only in an SLP with the same data classification.
Once data is backed up in an SLP, the data is managed according to the SLP configuration.
The SLP defines what happens to the data from the initial backup until the last copy of the
image has expired.
Priority for secondary The Priority for secondary operations option is the priority that jobs from secondary
operations operations have in relationship to all other jobs. The priority applies to the jobs that result
from all operations except for Backup and Snapshot operations. Range: 0 (default) to
99999 (highest priority).
For example, you may want to set the Priority for secondary operations for a policy with
a gold data classification higher than for a policy with a silver data classification.
The priority of the backup job is set in the backup policy on the Attributes tab.
Configuring deduplication 199
About MSDP backup policy configuration
Setting Description
Operations Use the Add, Change, and Remove buttons to create a list of operations in the SLP. An
SLP must contain one or more operations. Multiple operations imply that multiple copies
are created.
The list also contains the columns that display information about each operation. Not all
columns display by default.
Arrows Use the arrows to indicate the indentation (or hierarchy) of the source for each copy. One
copy can be the source for many other copies.
Active The Active and Postponed options appear under State of Secondary Operation
Processing and refer to the processing of all duplication operations in the SLP.
and
Note: The Active and Postponed options apply to duplication operations that create
Postponed
tar-formatted images. For example, those created with bpduplicate. The Active and
Postponed options do not affect the images that are duplicated as a result of OpenStorage
optimized duplication, NDMP, or if one or more destination storage units are specified as
part of a storage unit group.
■ Enable Active to let secondary operations continue as soon as possible. When changed
from Postponed to Active, NetBackup continues to process the images, picking up
where it left off when secondary operations were made inactive.
■ Enable Postponed to postpone the secondary operations for the entire SLP. Postponed
does not postpone the creation of duplication jobs, it postpones the creation of images
instead. The duplication jobs continue to be created, but they are not run until secondary
operations are active again.
All secondary operations in the SLP are inactive indefinitely unless the administrator
selects Active or until the Until option is selected and an activation date is indicated.
Validate Across Click this button to see how changes to this SLP can affect the policies that are associated
Backup Policies button with this SLP. The button generates a report that displays on the Validation Report tab.
This button performs the same validation as the -conflict option performs when used
with the nbstl command.
For VMware backups, select the Enable file recovery from VM backup option
when you configure a VMware backup policy. The Enable file recovery from VM
backup option provides the best deduplication rates.
NetBackup deduplicates the client data that it sends to a deduplication storage unit.
server, the client, and the media server that processes the backup or restore job.
If the connection is interrupted or fails, the services attempt to re-establish a
connection and synchronize the data.
NetBackup protects only the network socket connections that the NetBackup Remote
Network Transport Service (nbrntd) creates. Examples of the connections that are
not supported are:
■ Clients that back up their own data (deduplication clients and SAN clients)
■ Granular Recovery Technology (GRT) for Exchange Server or SharePoint Server
■ NetBackup nbfsd process.
NetBackup protects connections only after they are established. If NetBackup cannot
create a connection because of network problems, there is nothing to protect.
Resilient connections apply between clients and NetBackup media servers, which
includes master servers when they function as media servers. Resilient connections
do not apply to master servers or media servers if they function as clients and back
up data to a media server.
Resilient connections can apply to all of the clients or to a subset of clients.
Note: If a client is in a different subdomain than the server, add the fully qualified
domain name of the server to the client’s hosts file. For example,
india.veritas.org is a different subdomain than china.veritas.org.
When a backup or restore job for a client starts, NetBackup searches the Resilient
Network list from top to bottom looking for the client. If NetBackup finds the client,
NetBackup updates the resilient network setting of the client and the media server
that runs the job. NetBackup then uses a resilient connection.
Configuring deduplication 202
Resilient Network properties
Property Description
Host Name or IP Address The Host Name or IP Address of the host. The address can
also be a range of IP addresses so you can configure more
than one client at once. You can mix IPv4 addresses and
ranges with IPv6 addresses and subnets.
Use the arrow buttons on the right side of the pane to move
up or move down an item in the list of resilient networks.
Note: The order is significant for the items in the list of resilient networks. If a client
is in the list more than once, the first match determines its resilient connection
status. For example, suppose you add a client and specify the client IP address
and specify On for Resiliency. Suppose also that you add a range of IP addresses
as Off, and the client IP address is within that range. If the client IP address appears
before the address range, the client connection is resilient. Conversely, if the IP
range appears first, the client connection is not resilient.
Note: If multiple backup streams run concurrently, the Remote Network Transport
Service writes a large amount of information to the log files. In such a scenario, it
is recommended that you set the logging level for the Remote Network Transport
Service to 2 or less. Instructions to configure unified logs are in a different guide.
See the NetBackup Logging Reference Guide.
■ More sockets are open on media servers and clients. Three open sockets are
required rather than one for a non-resilient connection. The increased number
of open sockets may cause issues on busy media servers.
■ More processes run on media servers and clients. Usually, only one more
process per host runs even if multiple connections exist.
■ The processing that is required to maintain a resilient connection may reduce
performance slightly.
4 Click Add.
Add To All If you select multiple hosts in the NetBackup Administration Console,
the entries in the Resilient Network list may appear in different colors,
as follows:
■ The entries that appear in black type are configured on all of the
hosts.
■ The entries that appear in gray type are configured on some of the
hosts only.
For the entries that are configured on some of the hosts only, you can
add them to all of the hosts. To do so, select them and click Add To All.
2 Click Change.
4 Click OK.
2 Click Remove.
2 Click the appropriate button to move up the item or move down the
item.
4 In the Change Storage Server dialog box, select the Media Servers tab
5 Select the media server or servers that you want to use as a load balancing
server. It must be a supported host.
The media servers that are checked are configured as load balancing servers.
6 Click OK.
7 For all storage units in which Only use the following media servers is
configured, ensure that the new load balancing server is selected.
If your data was modified in a shifting mode, that is, if some data was inserted in
the middle of a file, then variable-length deduplication enables you to get higher
deduplication ratios when you back up the data. Variable-length deduplication
reduces backup storage, improves the backup performance, and lowers the overall
cost that is spent on data protection.
Note: Use variable-length deduplication for data types that do not show a good
deduplication ratio with the current MSDP intelligent deduplication algorithm and
affiliated streamers. Enabling Variable-length deduplication might improve the
deduplication ratio, but consider that the CPU performance might get affected.
The following table describes the effect of variable-length deduplication on the data
backup:
■ (Windows) install_path\Veritas\NetBackup\bin\ost-plugins
See “MSDP pd.conf file parameters” on page 210.
See “Editing the MSDP pd.conf file” on page 210.
Note: Veritas recommends that you make a backup copy of the file before you edit
it.
2 To activate a setting, remove the pound character (#) in column 1 from each
line that you want to edit.
3 To change a setting, specify a new value.
Note: The spaces to the left and right of the equal sign (=) in the file are
significant. Ensure that the space characters appear in the file after you edit
the file.
The parameters in this table are in alphabetical order; the parameters in a pd.conf
file may not be in alphabetical order.
The parameters in the file in your release may differ from those that are described
in this topic.
You can edit the file to configure advanced settings for a host. If a parameter does
not exist in a pd.conf file, you can add it. During upgrades, NetBackup adds only
required parameters to existing pd.conf files.
The pd.conf file resides in the following directories:
■ (Windows) install_path\Veritas\NetBackup\bin\ost-plugins
■ (UNIX) /usr/openv/lib/ost-plugins/
Parameter Description
BACKUPRESTORERANGE On a client, specifies the IP address or range of addresses that the local
network interface card (NIC) should use for backups and restores.
Specify the value in one of two ways, as follows:
Parameter Description
CR_STATS_TIMER Specifies a time interval in seconds for retrieving statistics from the storage
server host. The default value of 0 disables caching and retrieves statistics
on demand.
Consider the following information before you change this setting:
■ If disabled (set to 0), a request for the latest storage capacity information
occurs whenever NetBackup requests it.
■ If you specify a value, a request occurs only after the specified number
of seconds since the last request. Otherwise, a cached value from the
previous request is used.
■ Enabling this setting may reduce the queries to the storage server. The
drawback is the capacity information reported by NetBackup becomes
stale. Therefore, if storage capacity is close to full, Veritas recommends
that you do not enable this option.
■ On high load systems, the load may delay the capacity information
reporting. If so, NetBackup may mark the storage unit as down.
DEBUGLOG Specifies the file to which NetBackup writes the deduplication plug-in log
information. NetBackup prepends a date stamp to each day's log file.
On Windows, a partition identifier and slash must precede the file name.
On UNIX, a slash must precede the file name.
Note: This parameter does not apply for NDMP backups from a NetApp
appliance.
Default value:
Parameter Description
DISABLE_BACKLEVEL_TLS When secure communication is established between the client and the
server, this parameter specifies whether or not to disable older TLS
versions. NetBackup version 8.0 and earlier use older TLS versions such
as SSLV2, SSLV3, TLS 1.0, and TLS 1.1.
For a standard backup, NetBackup client version 8.0 and earlier can
communicate with NetBackup server (media server or load balance server)
version 8.1 that has TLS 1.2 enabled.
Further, NetBackup 5000 series appliance does not support TLS 1.2.
Parameter Description
ENCRYPTION Specifies whether to encrypt the data during backups. By default, files are
not encrypted.
If you set this parameter to 1 on all hosts, the data is encrypted during
transfer and on the storage.
FIBRECHANNEL Enable Fibre Channel for backup and restore traffic to and from a
NetBackup series appliance.
Parameter Description
To determine the keep alive interval that NetBackup uses, examine the
deduplication plug-in log file for a message similar to the following:
For more information about the deduplication plug-in log file, see DEBUGLOG
and LOGLEVEL in this table.
Configuring deduplication 216
Editing the MSDP pd.conf file
Parameter Description
FP_CACHE_CLIENT_POLICY
Note: Veritas recommends that you use this setting on the individual
clients that back up their own data (client-side deduplication). If you use it
on a storage server or load balancing server, it affects all backup jobs.
Specifies the client, backup policy, and date from which to obtain the
fingerprint cache for the first backup of a client.
By default, the fingerprints from the previous backup are loaded. This
parameter lets you load the fingerprint cache from another, similar backup.
It can reduce the amount of time that is required for the first backup of a
client. This parameter especially useful for remote office backups to a
central datacenter in which data travels long distances over a WAN.
clienthostmachine,backuppolicy,date
The date is the last date in mm/dd/yyyy format to use the fingerprint cache
from the client you specify.
See “Configuring MSDP fingerprint cache seeding on the client” on page 72.
Because incremental backups only back up what has changed since the
last backup, cache loading has little affect on backup performance for
incremental backups.
FP_CACHE_LOCAL Specifies whether or not to use the fingerprint cache for the backup jobs
that are deduplicated on the storage server. This parameter does not apply
to load balancing servers or to clients that deduplicate their own data.
Parameter Description
FP_CACHE_MAX_COUNT Specifies the maximum number of images to load in the fingerprint cache.
FP_CACHE_MAX_MBSIZE Specifies the amount of memory in MBs to use for the fingerprint cache.
FP_CACHE_PERIOD_REBASING_THRESHOLD Specifies the threshold (MB) for periodic rebasing during backups. A
container is considered for rebasing if both of the following are true:
■ The container has not been rebased within the last three months.
■ For that backup, the data segments in the container consume less
space than the FP_CACHE_PERIOD_REBASING_THRESHOLD value.
FP_CACHE_REBASING_THRESHOLD Specifies the threshold (MB) for normal rebasing during backups. A
container is considered for rebasing if both of the following are true:
■ The container has been rebased within the last three months.
■ For that backup, the data segments in the container consume less
space than the FP_CACHE_REBASING_THRESHOLD value.
Default value:FP_CACHE_REBASING_THRESHOLD = 4
If you change this value, consider the new value carefully. If you set it too
large, all containers become eligible for rebasing. Deduplication rates are
lower for the backup jobs that perform rebasing.
Parameter Description
LOCAL_SETTINGS Specifies whether to use the pd.conf settings of the local host or to allow
the server to override the local settings. The following is the order of
precedence for local settings:
■ Local host
■ Load balancing server
■ Storage server
LOGLEVEL Specifies the amount of information that is written to the log file. The range
is from 0 to 10, with 10 being the most logging.
MAX_LOG_MBSIZE The maximum size of the log file in megabytes. NetBackup creates a new
log file when the log file reaches this limit. NetBackup prepends the date
and the ordinal number beginning with 0 to each log file, such as
120131_0_pdplugin.log, 120131_1_pdplugin.log, and so on.
Parameter Description
MTSTRM_BACKUP_CLIENTS If set, limits the use of the Multi-Threaded Agent to the backups of the
specified clients. The clients that are not specified use single-threading.
This setting does not guarantee that the specified clients use the
Multi-Threaded Agent. The MaxConcurrentSessions parameter in the
mtstrm.conf file controls the number of backups the Multi-Threaded
Agent processes concurrently. If you specify more clients than the
MaxConcurrentSessions value, some of the clients may use
single-threaded processing.
Parameter Description
MTSTRM_BACKUP_ENABLED Use the Multi-Threaded Agent in the backup stream between the
deduplication plug-in and the NetBackup Deduplication Engine.
The following items describe the values that are used for the determination
algorithm:
■ A Linux media server that has 8 CPU cores with two hyperthreading
units per core has a hardware concurrency of 16. Therefore, the
hardware concurrency value for the algorithm is 8 (for media servers,
half of the system's hardware concurrency). Eight is greater than two
(the threshold value of Windows and Linux), so multithreading is enabled
(MTSTRM_BACKUP_ENABLED = 1).
■ A Solaris client that has 2 CPU cores without hyperthreading has a
hardware concurrency of 2. The hardware concurrency value for the
algorithm is 2 (for clients, all of the system's hardware concurrency).
Two is not greater than four (the threshold value of Solaris), so
multithreading is not enabled (MTSTRM_BACKUP_ENABLED = 0).
Parameter Description
MTSTRM_BACKUP_POLICIES If set, limits the use of the Multi-Threaded Agent to the backups of the
specified policies. The clients in the policies that are not specified use
single-threading, unless the client is specified in the
MTSTRM_BACKUP_CLIENTS parameter.
This setting does not guarantee that all of the clients in the specified policies
use the Multi-Threaded Agent. The MaxConcurrentSessions parameter
in the mtstrm.conf file controls the number of backups the Multi-Threaded
Agent processes concurrently. If the policies include more clients than the
MaxConcurrentSessions value, some of the clients may use
single-threaded processing.
MTSTRM_IPC_TIMEOUT The number of seconds to wait for responses from the Multi-Threaded
Agent before the deduplication plug-in times out with an error.
OPTDUP_BANDWIDTH Determines the bandwidth that is allowed for each optimized duplication
and Auto Image Replication stream on a deduplication server.
OPTDUP_BANDWIDTH does not apply to clients. The value is specified in
KBytes/second.
Parameter Description
OPTDUP_COMPRESSION Specifies whether to compress the data during optimized duplication and
Auto Image Replication. By default, files are compressed. To disable
compression, change the value to 0. This parameter does not apply to
clients.
OPTDUP_ENCRYPTION Specifies whether to encrypt the data during optimized duplication and
replication. By default, files are not encrypted. If you want encryption,
change the value to 1 on the MSDP storage server and on the MSDP load
balancing servers. This parameter does not apply to clients.
If you set this parameter to 1 on all hosts, the data is encrypted during
transfer.
OPTDUP_TIMEOUT Specifies the number of minutes before the optimized duplication times
out.
Parameter Description
■ For Windows:
install_path\Veritas\NetBackup\bin\ost-plugins
■ For UNIX: /usr/openv/lib/ost-plugins/
After copying the root certificate file, you must rename it to
storage_server_name.pem, where storage_server_name is the
fully qualified domain name of the NetBackup 5000 series appliance.
The root certificate file on NetBackup 5000 series appliance is:
■ (1.4.4)
/Storage/var/VRTSat/.VRTSat/profile/certstore/trusted/*.0
■ (1.4.5)
/Storage/var/VxAT/root/.VRTSat/profile/certstore/trusted/*.0
PREFERRED_EXT_SEGKSIZE Specifies the file extensions and the preferred segment sizes in KB for
specific file types. File extensions are case sensitive. The following describe
the default values: edb are Exchange Server files; mdfare SQL Server
master database files, ndf are SQL Server secondary data files, and
segsize64k are Microsoft SQL streams.
Parameter Description
PREFETCH_SIZE The size in bytes to use for the data buffer for restore operations.
RESTORE_DECRYPT_LOCAL Specifies on which host to decrypt and decompress the data during restore
operations.
You can also specify the segment size for specific file types. See
PREFERRED_EXT_SEGKSIZE.
Configuring deduplication 225
Editing the MSDP pd.conf file
Parameter Description
You can also specify different maximum and minimum segment sizes with
this parameter for different NetBackup clients. If you do not specify the
segment sizes, then the default values are considered.
■ VLD_CLIENT_NAME = *
Enables variable-length deduplication for all NetBackup clients and
uses the default VLD_MIN_SEGKSIZE and VLD_MAX_SEGKSIZE values.
■ VLD_CLIENT_NAME = clientname
Enables variable-length deduplication for NetBackup client
clientname and uses the default VLD_MIN_SEGKSIZE and
VLD_MAX_SEGKSIZE values.
■ VLD_CLIENT_NAME = clientname (64, 256)
Enables variable-length deduplication for NetBackup client
clientname and uses 64 KB as the VLD_MIN_SEGKSIZE and 256
KB as the VLD_MAX_SEGKSIZE value.
VLD_MIN_SEGKSIZE The minimum size of the data segment for variable-length deduplication
in KB. The segment size must be in multiples of 4 and fall in between 4
KB to 16384 KB. The default value is 64 KB.
Parameter Description
VLD_MAX_SEGKSIZE The maximum size of the data segment for variable-length deduplication
in KB. VLD_MAX_SEGKSIZE is used to set a boundary for the data
segments. The segment size must be in multiples of 4 and fall in between
4 KB to 16384 KB. The default value is 128 KB.
You can also specify different maximum and minimum segment sizes with
this parameter for different NetBackup policies. If you do not specify the
segment sizes, then the default values are considered.
■ VLD_POLICY_NAME = *
Enables variable-length deduplication for all NetBackup policies and
uses the default VLD_MIN_SEGKSIZE and VLD_MAX_SEGKSIZE
values.
■ VLD_POLICY_NAME = policyname
Enables variable-length deduplication for NetBackup policy
policyname and uses the default VLD_MIN_SEGKSIZE and
VLD_MAX_SEGKSIZE values.
■ VLD_POLICY_NAME = policyname (64, 256)
Enables variable-length deduplication for NetBackup policy
policyname and uses 64 KB as the VLD_MIN_SEGKSIZEand 256
KB as the VLD_MAX_SEGKSIZE value.
V7.0 represents the version of the I/O format not the NetBackup release level. The
version may differ on your system.
If you get the storage server configuration when the server is not configured or is
down and unavailable, NetBackup creates a template file. The following is an
example of a template configuration file:
To use a storage server configuration file for recovery, you must edit the file so that
it includes only the information that is required for recovery.
See “Saving the MSDP storage server configuration” on page 228.
See “Editing an MSDP storage server configuration file” on page 229.
See “Setting the MSDP storage server configuration” on page 230.
For sshostname, use the name of the storage server. For file.txt, use a file name
that indicates its purpose.
Configuring deduplication 229
Editing an MSDP storage server configuration file
If you get the file when a storage server is not configured or is down and unavailable,
NetBackup creates a template file.
V7.0 "storagepath" " " string The value should be the same as the value that was used
when you configured the storage server.
V7.0 "spalogpath" " " string For the spalogpath, use the storagepath value and
append log to the path. For example, if the storagepath
is D:\DedupeStorage, enter D:\DedupeStorage\log.
V7.0 "dbpath" " " string If the database path is the same as the storagepath
value, enter the same value for dbpath. Otherwise, enter
the path to the database.
V7.0 "required_interface" " " string A value for required_interface is required only if you
configured one initially; if a specific interface is not
required, leave it blank. In a saved configuration file, the
required interface defaults to the computer's hostname.
V7.0 "spalogin" "username" string Replace username with the NetBackup Deduplication
Engine user ID.
V7.0 "spapasswd" "password" string Replace password with the password for the NetBackup
Deduplication Engine user ID.
Configuring deduplication 230
Setting the MSDP storage server configuration
V7.0 "encryption" " " int The value should be the same as the value that was used
when you configured the storage server.
V7.0 "kmsenabled" " " int The value is used to enable or disable MSDP KMS
configuration. The value should be the same as the value
that was used when you configured the storage server.
V7.0 "kmsservertype" " " int The value is KMS server type. This value should be 0.
V7.0 "kmsservername" " " string The value is NBU Key Management Server. The value
should be the same as the value that was used when you
configured the storage server.
V7.0 "keygroupname" " " string The value should be the same as the value that was used
when you configured the storage server.
See “About saving the MSDP storage server configuration” on page 227.
See “Recovering from an MSDP storage server disk failure” on page 383.
See “Recovering from an MSDP storage server failure” on page 384.
To edit the storage server configuration
1 If you did not save a storage server configuration file, get a storage server
configuration file.
See “Saving the MSDP storage server configuration” on page 228.
2 Use a text editor to enter, change, or remove values.
Remove lines from and add lines to your file until only the required lines (see
Table 5-37) are in the configuration file. Enter or change the values between
the second set of quotation marks in each line. A template configuration file
has a space character (" ") between the second set of quotation marks.
See “Recovering from an MSDP storage server disk failure” on page 383.
See “Recovering from an MSDP storage server failure” on page 384.
To set the configuration, you must have an edited storage server configuration file.
See “About saving the MSDP storage server configuration” on page 227.
See “Saving the MSDP storage server configuration” on page 228.
See “Editing an MSDP storage server configuration file” on page 229.
Note: The only time you should use the nbdevconfig command with the -setconfig
option is for recovery of the host or the host disk.
Windows: install_path\NetBackup\bin\admincmd\nbdevconfig
-setconfig -storage_server sshostname -stype PureDisk -configlist
file.txt
For sshostname, use the name of the storage server. For file.txt, use the name
of the file that contains the configuration.
The storage_server_name is the fully qualified domain name if that was used to
configure the storage server. For example, if the storage server name is
DedupeServer.example.com, the configuration file name is
DedupeServer.example.com.cfg.
Warning: Only follow these procedures if you are reconfiguring your storage server
and storage paths.
rm /etc/pdregistry.cfg
cp -f /usr/openv/pdde/pdconfigure/cfg/userconfigs/pdregistry.cfg
/etc/pdregistry.cfg
■ HKLM\SOFTWARE\Symantec\PureDisk\Agent\EtcPath
Daily shadow copies NetBackup automatically creates copies of the MSDP catalog.
Catalog backup policy Veritas provides a utility that you can use to configure a NetBackup
policy that backs up the MSDP catalog.
Warning: You can change the path only during initial MSDP configuration only. If
you change it after MSDP backups exist, data loss may occur.
By default, the NetBackup Deduplication Manager keeps five shadow copies of the
catalog. You can change the number of copies.
See “Changing the number of MSDP catalog shadow copies” on page 238.
Retention 2 weeks
Configuring deduplication 235
About protecting the MSDP catalog
UNIX:
/database_path/databases/catalogshadow
/storage_path/etc
/database_path/databases/spa
/storage_path/var
/usr/openv/lib/ost-plugins/pd.conf
/usr/openv/lib/ost-plugins/mtstrm.conf
/database_path/databases/datacheck
/msdpc/storage/databases/catalog/__info__.2
/msdpc/storage/databases/catalog/2
/msdpc/storage/data/.dataconverted
/msdpc/storage/data/.format
/msdpc/storage/data/.gcnf
/msdpc/storage/data/.identity
/msdpc/storage/data/.srcmap
/msdpc/cache/etc
/msdpc/cache/fsdb
/msdpc/cache/userdata/proc/cloud.lsu
Windows:
database_path\databases\catalogshadow
storage_path\etc
storage_path\var
install_path\Veritas\NetBackup\bin\ost-plugins\pd.con
install_path\Veritas\NetBackup\bin\ost-plugins\mtstrm
database_path\databases\spa
database_path\databases\datacheck
By default, NetBackup uses the same path for the storage and the
catalog; the database_path and the storage_path are the
same. If you configure a separate path for the deduplication database,
the paths are different. Regardless, the drcontrol utility captures the
correct paths for the catalog backup selections.
You should consider the following items carefully before you configure an MSDP
catalog backup:
■ Do not use the Media Server Deduplication Pool as the destination for the
catalog backups. Recovery of the MSDP catalog from its Media Server
Deduplication Pool is impossible.
Configuring deduplication 236
Changing the MSDP shadow catalog path
■ Use a storage unit that is attached to a NetBackup host other than the MSDP
storage server.
■ Use a separate MSDP catalog backup policy for each MSDP storage server.
The drcontrol utility does not verify that the backup selections are the same
for multiple storage servers. If the backup policy includes more than one MSDP
storage server, the backup selection is the union of the backup selections for
each host.
■ You cannot use one policy to protect MSDP storage servers on both UNIX hosts
and Windows hosts.
UNIX MSDP storage servers require a Standard backup policy and Windows
MSDP storage servers require an MS-Windows policy.
See “Configuring an MSDP catalog backup” on page 239.
See “Updating an MSDP catalog backup policy” on page 244.
Warning: You can change the shadow catalog path during initial MSDP configuration
only. If you change it after MSDP backups exist, data loss may occur.
Note: There is a period (.) in front of the file name that denotes a hidden file.
7 If an MSDP catalog backup policy exists, update the policy with the new shadow
catalog directories. To do so, invoke the following command on the MSDP
storage server:
UNIX: /usr/openv/pdde/pdcr/bin/drcontrol --update_policy --policy
policy_name
2 Edit the second section of the file (40 3 * * *). The schedule section conforms
to the UNIX crontab file convention, as follows:
40 3 * * *
┬ ┬ ┬ ┬ ┬
│ │ │ │ │
│ │ │ │ │
│ │ │ │ └───── Day of week (0 - 7, Sunday is both 0 and 7, or use
│ │ │ │ sun, mon, tue, wed, thu, fri, sat; asterisk (*) is
│ │ │ │ every day)
│ │ │ └────────── Month (1 - 12; asterisk (*) is every month)
│ │ └─────────────── Day of month (1 - 31; asterisk (*) is every
│ │ day of the month)
│ └──────────────────── Hour (0 - 23; asterisk (*) is every hour)
└───────────────────────── Minute (0 - 59; asterisk (*) is every
minute of the hour)
Descriptions of the options are available in another topic. Note: To ensure that
NetBackup activates the policy, you must specify the --residence residence
option.
See “MSDP drcontrol options” on page 240.
The utility creates a log file and displays its path in the command output.
See “NetBackup MSDP log files” on page 410.
Table 5-38 MSDP drcontrol options for catalog backup and recovery
Option Description
--auto_recover_DR Recover the MSDP catalog from the most recent backup image.
This option automatically recovers the catalog and performs all of
the actions necessary to return MSDP to full functionality.
To recover the catalog from a backup other than the most recent,
contact your Veritas Support representative.
--client host_name The client to back up (that is, the host name of the MSDP storage
server).
--cleanup Remove all of the old MSDP catalog directories during the catalog
recovery process. Those directories are renamed during the
recovery.
--delete_old_files_for_cloud_catalyst Deletes the existing deduplication catalog files before restoring them
from the last backup in the recover_last_image step. Used for
a cloud media server after the file system has been recovered in the
recover_last_cloud_catalyst_image step.
--dsid The data selection ID is the catalog directory for one of the
NetBackup domains.
--hardware machine_type The hardware type or the computer type for the host.
Default: Unknown.
Configuring deduplication 242
Configuring an MSDP catalog backup
Table 5-38 MSDP drcontrol options for catalog backup and recovery
(continued)
Option Description
--initialize_DR Performs the following actions to prepare for MSDP catalog recovery:
--list_files List the files in the most recent MSDP catalog backup.
--log_file pathname The pathname for the log file that the drcontrol utility creates. By
default, the utility writes log files to
/storage_path/log/drcontrol/.
--new_policy Create a new policy to protect the deduplication catalog on this host.
If a policy with the given name exists already, the command fails.
Note: To ensure that NetBackup activates the policy, you must
specify the --residence residence option.
Default: Dedupe_Catalog_shorthostname
Table 5-38 MSDP drcontrol options for catalog backup and recovery
(continued)
Option Description
--recover_last_image Restore the MSDP catalog from the last set of backup images (that
is, the last full plus all subsequent incrementals). The drcontrol
utility calls the NetBackup bprestore command for the restore
operation.
--refresh_shadow_catalog Deletes all existing shadow catalog copies and creates a new catalog
shadow copy.
--residence residence The name of the storage unit on which to store the MSDP catalog
backups.
Table 5-38 MSDP drcontrol options for catalog backup and recovery
(continued)
Option Description
■ If the client name (of this media server) is not in the policy’s client
list, add the client name to the policy’s client list.
■ If you specify the --OS or --hardware options, replace the
values currently in the policy with the new values.
■ Update the backup selection based on the locations of the MSDP
storage directories and configuration files. Therefore, if you modify
any of the following, you must use this option to update the
catalog backup policy:
■ Any of the following values in the spa.cfg file
(section:variable pairs):
■ StorageDatabase:CatalogShadowPath
■ StorageDatabase:Path
■ Paths:Var
■ The spa.cfg or contentrouter.cfg locations in the
pdregistry.cfg file.
This option fails if there is no policy with the given policy name. It
also fails if the existing policy type is incompatible with the operating
system of the host on which you run the command.
■ StorageDatabase:Path
■ Paths:Var
The NetBackup MSDP is now FIPS validated and can be operated in FIPS mode.
Note: You must run FIPS mode on a new installation of NetBackup 8.1.1.
Caution: Enabling MSDP FIPS mode might affect the NetBackup performance on
a server with the Solaris operating system.
Enable the FIPS mode for MSDP by running the following commands:
■ For UNIX:
/usr/openv/pdde/pdag/scripts/set_fips_mode.sh 1
For Windows:
<install_path>\Veritas\pdde\set_fips_mode.bat 1
■ /usr/openv/netbackup/bin/bp.start_all
■ For Windows:
■ <install_path>\NetBackup\bin\bpdown
■ <install_path>\NetBackup\bin\bpup
Warning: It is recommended that you do not disable the MSDP FIPS mode once
you enable it, for security reasons.
For Windows:
<install_path>\Veritas\pdde\crcontrol.exe --getmode
4 Configure the NetBackup client-side deduplication backup policy and run the
backup operation.
Configuring deduplication 248
About MSDP multi-domain support
■ UNIX
/usr/openv/pdde/pdcr/bin/spauser -a -u <username> -p <password>
To list all the MSDP users, run the following command on the MSDP storage server:
■ Windows
<install_path>\pdde\spauser -l
■ UNIX
/usr/openv/pdde/pdcr/bin/spauser -l
To use an MSDP storage server from another NetBackup domain, you must obtain
a NetBackup certificate from another NetBackup domain.
Run the following commands on every NetBackup media server or client that wants
to use an MSDP storage server from another domain:
■ Windows
install_path\NetBackup\bin\nbcertcmd -getCACertificate –server
another_master_server
install_path\NetBackup\bin\nbcertcmd -getCertificate –server
another_master_server -token token_string
■ UNIX
/usr/openv/netbackup/bin/nbcertcmd -getCACertificate –server
another_master_server
Configuring deduplication 249
About MSDP multi-domain support
■ NetBackup Commands
■ Use the bpnbat command to log on the target NetBackup master server.
■ Use the nbcertcmd command to get the authorization tokens.
For more information on the commands, refer to the NetBackup Commands
Reference Guide.
masterA masterB
mediaA1 mediaB
mediaA2
clientA
MasterA is the host name of the master server of NetBackup domain A and the
domain contains two media servers (mediaA1, mediaA2), and one client (clientA).
MasterB is the host name of the master server of NetBackup domain B and the
domain contains one media server (mediaB).
Using the following sample steps, create an MSDP storage server in domain B and
let domain A use the MSDP storage server:
1. Create an MSDP storage server on media server mediaB of NetBackup domain
B. (NetBackup Administration Console > Media and Device Management >
Configure Disk storage servers > Media Server Deduplication Pool)
2. Run the following command on mediaB to create a new MSDP user testuser1
with password as testuser1pass.
Configuring deduplication 250
About MSDP multi-domain support
To run client-direct backup from clientA to MSDP storage server mediaB, run
the following certificate command on clientA:
■ nbcertcmd -GetCACertificate -server masterB
5. After creating the MSDP OpenStorage server, create a related NetBackup disk
pool and storage unit. Use the storage unit to run all the related NetBackup
jobs.
When optimized duplication and multi-domain are used together, there is
communication between the MSDP storage servers from two different NetBackup
domains. The MSDP storage server from the other domain must have a certificate
generated by master server of the local NetBackup domain. Run the nbcertcmd
commands on source side MSDP storage server to request a certificate from the
NetBackup master server of target MSDP storage server.
When backup and restore jobs on the client and multi-domain are used together,
there is communication between the NetBackup client and MSDP storage server
from two different NetBackup domains. Run the nbcertcmd commands on the
NetBackup client to request a certificate from the NetBackup master server of MSDP
storage server.
Configuring deduplication 251
About MSDP mutli-domain VLAN Support
When one NetBackup domain uses the MSDP storage server of another NetBackup
domain, the MSDP storage server cannot be the A.I.R target of that NetBackup
domain.
If an external CA is used in the NetBackup setup, you do not need to run the
nbcertcmd –GetCACertificate and the nbcertcmd –GetCertificate commands.
If NetBackup domains A and B do not use the same external CA, synchronize the
external root CA between the two NetBackup domains for MSDP communication.
For more information of the external CA, refer to NetBackup Security and Encryption
Guide.
When one NetBackup domain uses an MSDP storage server that has multiple
network interfaces and related host names, another NetBackup domain can use
any one host name to configure the OpenStorage server. If the MSDP storage
server that has multiple host names uses an external CA, the Subject Alternative
Name field of the external certificate must contain all the host names that are used
to configure the OpenStorage server.
mediaA - (10.XX.30.2/24)
mediaA2 - (10.XX.40.2/24)
masterA is the master server of domain A and has two host names and IP
addresses. mediaA is the media server of domain A and has two host names and
IP addresses. MSDP storage server is created on media server mediaA.
Configuring deduplication 252
About MSDP mutli-domain VLAN Support
To let domain B access the MSDP storage server on mediaA of domain A, run the
following steps:
1. Create an MSDP storage server on media server mediaA of NetBackup domain
A. (NetBackup Administration Console > Media and Device Management >
Configure Disk storage servers > Media Server Deduplication Pool)
2. Run following command on mediaA to create a new MSDP user testuser1
with password testuser1pass:
spauser -a -u “testuser1” -p “testuser1pass”
The following table describes the WORM-specific options and arguments for the
catdbutil command.
Table 5-39 The options and arguments for the catdbutil command.
The spadb command line utility that lets you use the NetBackup Deduplication
Manager (spad) to set WORM for an LSU and define the WORM mode and the
interval for making the image immutable and indelible.
The Deduplication Manager reads the WORM mode from the
/etc/lockdown-mode.conf file file.
The following table describes the WORM-specific options and arguments for the
spadb command.
Table 5-40 The options and arguments for the spadb command.
spadb spadb update WORM set Use the data selection ID to configure the
${FIELD1_NAME}=xxx, following WORM properties:
Command line utility
${FIELD2_NAME}=xxxx where
that lets you use the ■ indelible_minimum_interval
id=${DSID} #
NetBackup and indelible_maximum_interval
Deduplication field names: Set the minimum and maximum interval in
Manager (spad) days for making the image indelible.
■ indeliable_minimum_interval
For example,
■ indeliable_maximum_interval
spadb -c "update WORM set
indelible_minimum_interval=1
where dsid=2"
spadb -c "update WORM set
indelible_maximum_interval=1000000
where dsid=2"
Chapter 6
Configuring deduplication
to the cloud with
NetBackup Cloud Catalyst
This chapter includes the following topics:
Figure 6-1 Flow of data from an MSDP storage server to a Cloud Catalyst
storage server
Master server
NetBackup 8.1 or later
Client data
Optimized duplication
Copy 2
This use case is preferred to backing up directly to a Cloud Catalyst storage server.
(See “Backups go directly to a Cloud Catalyst storage server” on page 259.)
See “Configuring a Cloud Catalyst storage server for deduplication to the cloud”
on page 264.
Figure 6-2 Fan-in MSDP storage servers and multiple Cloud Catalyst storage
servers
Master server
NetBackup 8.1 or later
MSDP
Client data storage servers
Cloud provider A
Cloud Catalyst
storage server
Optimized
duplication
NetBackup Cloud
Catalyst Appliance
Cloud provider B
Cloud Catalyst
storage server
Optimized duplication
Media server
RHEL 7.3
NetBackup 8.1 or later
■ If a restore is required, the data may no longer be in the local cache. If the data
has been uploaded to the cloud, the restore may be slow and expensive.
■ Client Direct is not supported. To review the limitations, See “Cloud Catalyst
requirements and limitations” on page 260.
Client data
Cloud Catalyst
storage server
Copy 1
Table 6-1 The types of media servers that can be used as Cloud Catalyst
storage servers
Host Version
The Cloud Catalyst storage server can work with a proxy server when the NTLM
authentication method is used. The proxy server must follow certain requirements
to function inside of the NetBackup system. The following items are required only
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 261
Cloud Catalyst requirements and limitations
when the NTLM authentication method is used. When the BASIC authentication
method or NONE is selected, there is no such requirement.
Requirements for a proxy server:
■ The proxy server must support both NTLMv1 and NTLMv2.
■ Cloud Catalyst is dependent on Classic Cloud Connector to get the list of buckets.
■ The Classic Cloud Connector only supports NTLMv1, and not NTLMv2.
Note: A Cloud Catalyst storage server is supported for Auto Image Replication
(A.I.R.) and Target A.I.R. as the source and the target from NetBackup 8.3.0.1 or
later.
Limitations
NetBackup Cloud Catalyst recognizes nearly all other MSDP features and
capabilities.
See the NetBackup compatibility lists for updated information on supported cloud
vendors and feature support:
http://www.netbackup.com/compatibility
The following are current exclusions or limitations:
■ Support limitations as listed in the NetBackup Cloud Administrator's Guide:
http://www.veritas.com/docs/DOC5332
■ The NetBackup media servers that you use as Cloud Catalyst storage servers
must be the same NetBackup version as the master server.
■ Load balancing for Cloud Catalyst storage servers or Cloud Catalyst appliances
is supported. See the following topic for configuration considerations.
See “Configuring source control or target control optimized duplication for Cloud
Catalyst” on page 280.
■ OpsCenter cannot report on the data that Cloud Catalyst storage servers upload
to the cloud.
■ A single media server cannot be both an MSDP storage server and a Cloud
Catalyst storage server.
■ A Cloud Catalyst storage server is supported as the target for A.I.R. when the
storage server is selected from the drop-down list of the trusted master server.
■ NetBackup Client Direct deduplication is supported in a Cloud Catalyst duplication
to cloud scenario. However, Client Direct deduplication is not supported in a
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 262
Configuring a Linux media server as a Cloud Catalyst storage server
scenario where the backup is uploaded directly to the cloud and no duplication
is configured.
■ Direct backup to Cloud Catalyst using the accelerator option is not supported
when Amazon S3 Glacier or Amazon S3 Glacier Deep Archive storage classes
are used. Backup to an MSDP with accelerator and then optimized duplication
to Cloud Catalyst is supported.
■ NetBackup clients must be on version 8.3.0.1 or later to support Client Direct
enabled backups to the Cloud Catalyst media server.
The technical specifications for the NetBackup 5240 Appliance are listed in the
appendix of the NetBackup 5240 Appliance Product Description. Table 6-2 lists a
subset of minimum requirements for the media server.
DDR4 RDIMM
10 Gb Ethernet ports 5
If you don’t use a NetBackup appliance, the following table describes the minimum
system requirements for a media server that is configured for Cloud Catalyst.
Cores 4
System memory 16 GB
10 Gb Ethernet ports 1
Note: Multiple networking interface cards (NIC) may be
desired for redundancy and connectivity reasons.
Cache size 1 TB
Optimal streams 16
Note: Cloud Catalyst uses cloud connectors during initial configuration. Certificate
validation against Certificate Revocation List (CRL) secures the calls that the cloud
connectors initiate. After configuration, nbrmms periodically polls Cloud Catalyst
storage server and makes a call to the cloud connector to check its state using the
cloud connector’s CRL.
For more information on Certificate Revocation List (CRL), Review the section
Certificate validation against Certificate Revocation List (CRL) in NetBackup Cloud
Administrator’s Guide.
After initial configuration, Cloud Catalyst uses Online Certificate Status Protocol
(OCSP) for securing calls during data transfer to and from the cloud. Review the
following section for more information on OCSP:
See “Certificate validation using Online Certificate Status Protocol (OCSP)”
on page 272.
See “Optimized duplication is used to copy data from an MSDP storage server to
a Cloud Catalyst storage server (preferred use case)” on page 257.
Step 1 Configure an MSDP One or more A NetBackup environment may contain one or more media
storage server (source) media servers servers that serve as source MSDP storage servers for one or
more Cloud Catalyst appliances.
Step 2 Create an account with a Cloud service Make sure that you have an account with a cloud service provider
cloud service provider provider and that you know the user name and password.
Step 3 Configure KMS (optional) Master server NetBackup Cloud Catalyst supports MSDP Key Management
Service (KMS) encryption.
Step 4 Configure an appliance Appliance A Cloud Catalyst Appliance is the target for the optimized
as a Cloud Catalyst duplication. It is a dedicated appliance for cloud storage.
storage server (target)
A NetBackup environment may contain one or more Cloud
Catalyst appliances. There should be one appliance for each
different cloud service provider that is used in the environment.
The default path for the Local cache directory: is /msdpc. You
can change this directory by clicking on Browse.
https://www.veritas.com/content/support/en_US/Appliances.html
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 267
Configuring a Cloud Catalyst storage server for deduplication to the cloud
Step 5 Configure the target to Appliance To configure load balancing for push optimized duplications, the
allow load balancing for Cloud Catalyst Appliance must allow access from the source
push optimized media server.
duplications (optional)
Note: The push configuration is recommended if using load
balancing for Cloud Catalyst.
Step 6 Configure a storage Master server See “Configuring a storage lifecycle policy for NetBackup Cloud
lifecycle policy Catalyst” on page 273.
Note: The backup and the duplication jobs can also be run
manually from the NetBackup Administration Console or by
using the bpduplicate command.
Step 7 Configure a backup Master server Create a backup policy that uses the SLP. In the Policy storage
policy field of the policy Attributes tab, select the SLP.
Step 8 Protect the MSDP data Media server ■ Manually create an MSDP catalog backup policy for every
and catalog MSDP storage server and Cloud Catalyst storage server in
the environment.
See “About protecting the MSDP catalog” on page 233.
See “Configuring an MSDP catalog backup” on page 239.
Note: After the MSDP catalog backup policy is created, it
is inactive. Assign a storage unit to it and then make it active
by setting the Go into effect at option on the Attributes tab.
■ Save a configuration file for every MSDP storage server and
Cloud Catalyst storage server in the environment.
See “About saving the MSDP storage server configuration”
on page 227.
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 268
Configuring a Cloud Catalyst storage server for deduplication to the cloud
Step 1 Configure an MSDP One or more A NetBackup environment may contain one or more media
storage server (source) media servers servers that serve as source MSDP storage servers for Cloud
Catalyst storage servers.
Step 2 Create an account with a Cloud service Make sure that you have an account with a cloud service provider
cloud service provider provider and that you know the user name and password.
Step 3 Deploy host name-based Master server This step is applicable and required for media server versions
(Conditional) certificates 8.0 to 8.1.2 only.
1 /usr/openv/netbackup/bin/admincmd/bpnbaz
–ProvisionCert host_name
Step 4 Configure a Cloud Media server A Cloud Catalyst storage server is the target for the optimized
Catalyst storage server duplication. It is a dedicated media server for cloud storage.
(target)
■ Run the Cloud Storage Server Configuration Wizard in the
Java GUI selecting the Enable NetBackup Cloud Catalyst
check box and entering a path for the local cache directory.
■ The administrator configures a local cache directory as part
of configuring a Cloud Catalyst storage server. The minimum
size for this directory is 4 TB, however, an environment may
see benefits from a larger local cache directory.
See “About the Cloud Catalyst cache” on page 278.
■ If desired, select the Encrypt data using MSDP KMS
encryption in the wizard to configure encryption.
If the Encrypt data using MSDP KMS encryption option is
selected, one of following operations is carried out:
■ If none of the KMS servers is configured, by default
NetBackup KMS is configured.
■ If NetBackup KMS server is configured, you need to
provide key name and key passphrase.
■ If external KMS server is configured on the master server,
you must enter key group name. You need to ensure that
a key with a custom attribute set with value of key group
name already exists in the external KMS server.
For more information about KMS configuration, see the
NetBackup Security and Encryption Guide.
■ A NetBackup environment may contain one or more media
servers that serve as Cloud Catalyst storage servers. There
should be one Cloud Catalyst storage server for each different
cloud service provider that is used in the environment.
Step 5 Ensure that a common Media server If the Cloud Catalyst media server was added as a load balancing
media server is media server during the MSDP storage server wizard (Step 1),
configured to allow pull optimized duplication is configured.
optimized duplication
See the following topic for configuration considerations:
Step 6 Protect the MSDP data Media server ■ Create an MSDP catalog backup policy for every MSDP
and catalog storage server and Cloud Catalyst storage server in the
environment.
See “About protecting the MSDP catalog” on page 233.
See “Configuring an MSDP catalog backup” on page 239.
Note: After the MSDP catalog backup policy is created, it
is inactive. Assign a storage unit to it and then make it active
by setting the Go into effect at option on the Attributes tab.
■ Save a configuration file for every MSDP storage server and
Cloud Catalyst storage server in the environment.
See “About saving the MSDP storage server configuration”
on page 227.
Step 7 Configure a storage Master server See “Configuring a storage lifecycle policy for NetBackup Cloud
lifecycle policy Catalyst” on page 273.
The backup and the duplication jobs can also be run manually
from the NetBackup Administration Console or by using the
bpduplicate command.
Step 8 Configure a backup Master server Create a backup policy that uses the SLP. In the Policy storage
policy field of the policy Attributes tab, select the SLP.
Configuring a Cloud Catalyst storage server as the target for the MSDP storage
servers
1 On the master server, in the NetBackup Administration Console, click
Configure Cloud Storage Servers to launch the Cloud Storage Server
Configuration Wizard.
2 In the wizard, enable the option: Enable NetBackup Cloud Catalyst.
3 Enter a path to use as the Local cache directory. Deduplicated data is written
to this directory on the cloud storage server before it is uploaded to the cloud.
Notes:
■ The minimum size for the local cache directory is 4 TB, however, an
environment may see benefits from a larger local cache directory.
■ This path should be to a file system which is dedicated for Cloud Catalyst
cache use. Inaccurate cache eviction occurs if the path shares any storage
with other data or applications.
■ NetBackup manages the files in the local cache directory. Users should not
manually delete files in this directory.
■ The NetBackup Cloud Administrator's Guide contains information about all
of the available options and switches which are common to non-Cloud
Catalyst and Cloud Catalyst environments.
http://www.veritas.com/docs/DOC5332
See “About the Cloud Catalyst cache” on page 278.
For more information about KMS configuration, see the NetBackup Security
and Encryption Guide.
5 As part of the storage server wizard, a disk pool and a storage unit are
configured.
When the storage lifecycle policy is created, this storage unit name is used in
the Storage unit field for the Duplication operation.
See “Configuring a storage lifecycle policy for NetBackup Cloud Catalyst”
on page 273.
Note: When a storage server has SSL enabled, the OCSP validation is enabled
by default when you upgrade to NetBackup Cloud Catalyst 8.2.1.
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 273
Configuring a Cloud Catalyst storage server for deduplication to the cloud
Note: Remember these credentials because the same credentials are required
during disaster recovery of the Cloud Catalyst storage server.
If esfs.json is changed while vxesfsd is stopped, the changes take effect the
next time vxesfsd starts.
4 Some items require that you restart vxesfsd before the new configuration can
take effect.
These items are indicated in Table 6-6.
■ Before vxesfsd is stopped, verify that no processes currently use vxesfsd,
including the current working directory of any user.
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 275
About the Cloud Catalyst esfs.json configuration file
Log
Path /usr/openv/ This setting indicates the directory where logs for Yes
the vxesfsd process are created.
netbackup/logs
The logs include the following:
■ esfs_database
■ esfs_filesystem
■ esfs_monitor
■ esfs_storage
■ esfs_storagemanager
Size 10485760 This value (in bytes) controls the maximum size No
that a single log file is allowed to grow. Once the
file reaches that approximate size, the log file is
closed and another log file is opened.
Level 1 This value determines the logging level and what No. Changes
information is included in the logs: may not take
effect until the
■ 1: Errors only
next restart.
■ 2: Warnings and errors
■ 3: All log entries
Monitor
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 276
About the Cloud Catalyst esfs.json configuration file
StorageManager
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 277
About the Cloud Catalyst esfs.json configuration file
FileSystem
Note: If you initiate a restore from Glacier or Glacier Deep Archive, NetBackup
initiates a warming step. NetBackup does not proceed with the restore until all the
data is available in S3 storage to be read.
The warming step is always done if using Amazon, even if the data is in the Cloud
Catalyst cache. For storage classes other than Glacier and Glacier Deep Archive,
the warming step is almost immediate with no meaningful delay. For Glacier and
Glacier Deep Archive, the warming step may be immediate if files were previously
warmed and are still in S3 Standard storage. However, it may take several minutes,
hours, or days depending on settings being used.
6 Set the Sampling interval to one or more to enable throttling (zero means that
throttling is disabled).
7 Click Apply and OK to close the dialog.
See “About the Cloud Catalyst esfs.json configuration file” on page 274.
■ If the Cloud Catalyst media server manages the duplication operation, it is target
controlled duplication. The MSDP media server must allow access from the
Cloud Catalyst media server. (See Example target controlled configuration:
Source media storage server is not RHEL or is pre-8.1.)
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 281
Configuring source control or target control optimized duplication for Cloud Catalyst
The MSDP media server can be of any supported platform or NetBackup version.
See the considerations that are listed in Table 6-7.
Source Advantage: The NetBackup Disk Manager (bpdm) runs on the source media server, which prevents
controlled overloading the Cloud Catalyst storage server during duplication jobs.
Disadvantage: Requires that the MSDP storage server be of a specific platform and NetBackup
version.
Target Advantage: The MSDP media server can be of any supported platform or NetBackup version.
controlled
Disadvantage: The NetBackup tape manager (bptm) may run on the Cloud Catalyst media server
during backups. NetBackup may inadvertently use the Cloud Catalyst storage server for load balancing
during backups.
Note: In MSDP optimized duplication, the spad process on the source pool reads
the source side data and sends the data to the target pool.
Note: Do not uncheck the box for the Cloud Catalyst host credentials. Only uncheck
the box when directed by technical support staff.
Example: In Figure 6-4, do not uncheck the box next to CCat_st_server when
configuring the storage unit properties on target.
Note: Do not uncheck the box for the Cloud Catalyst host credentials. Only uncheck
the box when directed by technical support staff.
Example: In Figure 6-5, do not uncheck the box next to CCat_st_server when
configuring the storage unit properties on target.
Media Server Cloud RHEL 7.3 or RHEL 7.3 or None RHEL 7.3 or
Deduplication Catalyst later and later and later and
Pool storage NetBackup NetBackup NetBackup
server 8.1 or later. 8.1 or later. 8.1 or later.
Cloud Media Server RHEL 7.3 or None RHEL 7.3 or RHEL 7.3 or
Catalyst Deduplication later and later and later and
storage Pool NetBackup NetBackup NetBackup
server 8.2 or later. 8.2 or later. 8.2 or later.
Cloud Cloud RHEL 7.3 or RHEL 7.3 or RHEL 7.3 or RHEL 7.3 or
Catalyst Catalyst later and later and later and later and
storage storage NetBackup NetBackup NetBackup NetBackup
server server 8.1 or later. 8.1 or later. 8.1 or later. 8.1 or later.
5 Any remaining data on the cloud storage may now be manually removed.
FSDB
(ESFS database)
Open Cloud
Storage Storage
manager Daemon
Monitor (ocsd) Cloud
vxesfsd
MSDP provider
File system
ESFS
Job is done only when data is
MSDP writes uploaded to the cloud
User space
data to FUSE
Kernel space
MSDP cache files
(local cache)
FUSE Linux or VxFS
Note that a single Cloud Catalyst storage server can only write to one cloud provider
and to one bucket within that cloud provider.
vxesfsd includes three subcomponents that perform various tasks as part of ESFS:
esfs_monitor Logs the job status updates and periodic tasks such as cache
eviction and FSDB backups.
esfs_database Logs the FSDB actions that are specific to the ESFS
database.
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 289
NetBackup Cloud Catalyst workflow processes
Backup workflow
Figure 6-7 Backup data workflow for a Cloud Catalyst storage server
bptm
MSDP server
spad, spoold
OST
File system interface
libstspigateway
libstspimsdpcld ESFS
libstspipd vxesfsd
ocsd
bpdm
Read side handle Write side handle
OST OST
MSDP server
spad libstspipd libstspigateway
spoold libstspimsdpcld
libstspipd
Data movement
optimized dup
copy_extent, copy_image MSDP server
spad
spoold
ESFS
vxesfsd
ocsd
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 292
Disaster recovery for Cloud Catalyst
If you have configured IAM role in the EC2 instance, use the following command:
-python /usr/openv/pdde/pdag/scripts/ims_system_config.py
-k dummy -s dummy -b <name_S3_bucket>
-sp <s3_http_port>: Specify HCP storage server HTTP port (Default is 80)
-sps <s3_https_port>: Specify HCP storage server HTTP port (Default is 443)
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 295
About image sharing in cloud using Cloud Catalyst
-ssl <ssl_usage>: Specify whether to use SSL. (0- Disable SSL. 1- Enable SSL.
Default is 1.) If SSL is disabled, it uses <s3_http_port> to make connection to
<s3_hostname>. Otherwise, it uses <s3_https_port>.
Table 6-9 Steps for image sharing and the command options
Step Command
List all the backup images that are in the cloud nbimageshare --listimage
Table 6-9 Steps for image sharing and the command options (continued)
Step Command
nbimageshare --singleimport
<client> <policy> <backupID>
nbimageshare --batchimport
<image_list_file_path>
"ec2:CreateTags"
"ec2:DescribeImportImageTasks"
"ec2:ImportImage"
"iam:ListRolePolicies"
"iam:ListRoles"
"iam:GetRole"
"iam:GetRolePolicy"
"iam:CreateRole"
"iam:PutRolePolicy"
/usr/openv/netbackup/bin/nbkms -createemptydb
/usr/openv/netbackup/bin/nbkms
/usr/openv/netbackup/bin/nbkmscmd -discovernbkms -autodiscover
If Cloud Catalyst is not set for optimized deduplication or is not an AIR target,
you cannot use image sharing. If Amazon Glacier is enabled in Cloud Catalyst,
you cannot use image sharing.
In these scenarios to disable image sharing:
Modify the <install_directory>/etc/puredisk/spa.cfg file and add the
following configuration item:
EnableIMandTIR=false
Workaround:
■ You can change the maximum policy size limit for the vmimport role.
■ You can list and delete the existing policies using the following commands:
■ The recover operation includes AWS import process. Therefore, a vmdk image
cannot be recovered concurrently in two restore jobs at the same time.
■ The image sharing feature can recover the virtual machines that satisfy the
Amazon Web Services VM import prerequisites.
For more information about the prerequisites, refer to the following article:
https://docs.aws.amazon.com/vm-import/latest/userguide/vmie_prereqs.html
■ If you cannot obtain the administrator password to use an AWS EC2 instance
that has a Windows OS, the following error is displayed:
Password is not available. This instance was launched from a custom
AMI, or the default password has changed. A password cannot be
retrieved for this instance. If you have forgotten your password,
you can reset it using the Amazon EC2 configuration service. For
more information, see Passwords for a Windows Server Instance.
This error occurs after the instance is launched from an AMI that is converted
using image sharing.
For more information, refer to the following articles:
■ Amazon Elastic Compute Cloud Common Messages
Configuring deduplication to the cloud with NetBackup Cloud Catalyst 300
About image sharing in cloud using Cloud Catalyst
■ One MSDP storage server can be configured to support multiple storage targets,
including one local storage target and zero or more cloud storage targets. You
can move data to local and multiple cloud targets simultaneously.
■ The cloud targets can be from the same or different providers, either public, or
private. For example, AWS, Azure, HCP, etc.
■ The cloud targets can be added on demand after the MSDP server is configured
and active.
■ Multiple cloud targets can coexist in a single cloud bucket or multiple buckets
that are distributed in a single or different cloud providers.
■ The data and metadata for local storage and multiple cloud targets are isolated
to support Multiple Tenant usage.
■ Optimized deduplication is supported within one MSDP server scope so that
data can be stored to local storage first and then duplicated to cloud targets in
the same media server.
■ Disaster recovery from the cloud targets is enhanced and more straightforward.
■ Feature is well integrated with the MSDP cluster solution.
Based on the OpenStorage Technology (OST), the new architecture uses multiple
Logical storage unit (LSU) to manage and move data. These LSUs can be
customized independently to meet different customer requirements. For example,
as pure local target (same as MSDP in NetBackup 8.2 or earlier), or local target
plus one or more cloud targets.
Starting with NetBackup 8.3, you can configure MSDP from the NetBackup Web
UI. You can refer to the NetBackup Web UI documentation for more details.
This chapter focuses on how to use the command line interface to configure MSDP.
Limitations
■ Instant Access for cloud LSU is not supported.
■ Universal share for cloud LSU is not supported.
■ Accelerator for cloud LSU of AWS Glacier and Deep Archive is not supported.
■ Cloud DR for cloud LSU of AWS Glacier and Deep Archive is not supported if
the storage server name changes.
■ AIR for cloud LSU of AWS Glacier and Deep Archive is not supported.
■ Classic AIR for cloud LSU as replication target is not supported.
■ Synthetic backup for cloud LSU of AWS Glacier and Deep Archive is not
supported.
■ Snowball is not supported.
■ FIPS is not supported.
■ Multi-threaded Agent must be disabled when a Client-Direct backup is used by
NetBackup clients that have NetBackup version earlier than 8.3.
■ If you select a load balancing media server that has NetBackup version earlier
than 8.3, then the cloud LSUs are not listed. Even if you select cloud LSUs with
media servers that have NetBackup version earlier than 8.3, the backups can
fail.
The following steps describe the method to create a cloud storage unit using
the command line:
1 Create an MSDP storage server.
See “Configuring MSDP server-side deduplication” on page 56.
2 Create a cloud instance alias.
For example:
Example 1: Creating an Amazon S3 cloud instance alias
# /usr/openv/netbackup/bin/admincmd/csconfig cldinstance -as -in
amazon.com -sts <storage server> -lsu_name <lsu name>
The cloud alias name is <storage server>_<lsu name>, and is used to create
a bucket.
3 Create a new bucket (Optional)
For example:
# /usr/openv/netbackup/bin/nbcldutil -createbucket -storage_server
<storage server>_<lsu name> -username <cloud user> -bucket_name
<bucket name>
MSDP cloud support 305
Creating a cloud storage unit
V7.5 "operation" "add-lsu-cloud" string Specifies the value “add-lsu-cloud” for adding a new
cloud LSU.
V7.5 "lsuCloudUser" " " string Specifies the cloud user name.
V7.5 "lsuCloudBucketName" " " string Specifies the cloud bucket name.
V7.5 "lsuCloudBucketSubName" " " string Multiple cloud LSUs can use the same cloud bucket, this
value distinguishes different cloud LSUs.
Key group name must have valid characters: A-Z, a-z, 0-9,
_ (underscore), - (hyphen), : (colon), . (period), and space.
Note: All encrypted LSUs in one storage server must use the same
keygroupname and kmsservername. If you use the nbdevconfig command to
add a new encrypted cloud Logical storage unit (LSU) and an encrypted LSU
exists in this MSDP, the keygroupname must be the same as the keygroupname
in the previous encrypted LSU.
For more information, See “About MSDP Encryption using NetBackup KMS
service” on page 76.
Create a configuration file and then run the following nbdevconfig command:
# /usr/openv/netbackup/bin/admincmd/nbdevconfig -setconfig
-storage_server <storage server> -stype PureDisk -configlist
<configuration file path>
Note: The parameter <storage server> must be the same as the parameter
<storage server> in Step 2.
MSDP cloud support 307
Updating cloud credentials for a cloud LSU
Note: You can also create the disk pool can from the NetBackup Web UI or
NetBackup Administration Console.
Note: You can also create the storage unit from the NetBackup Web UI or
NetBackup Administration Console.
V7.5 "operation" "update-lsu-cloud" string Use the value “update-lsu-cloud” to update some
cloud LSU parameters.
V7.5 "lsuCloudUser" " " string Specifies the cloud user name.
For example:
MSDP cloud support 308
Updating encryption configurations for a cloud LSU
V7.5 "operation" "update-lsu-cloud" string You can only update the KMS status from disabled to
enabled.
V7.5 "lsuKmsEnable" "YES" string Specifies the KMS status for the cloud LSU.
Key group name must have valid characters: A-Z, a-z, 0-9,
_ (underscore), - (hyphen), : (colon), . (period), and space.
Example to enable KMS status from disabled status to enabled status for cloud
LSU “s3amazon”:
Note: All encrypted LSUs in one storage server must use the same keygroupname
and kmsservername. If you use the nbdevconfig command to add a new encrypted
cloud Logical storage unit (LSU) and an encrypted LSU exists in this MSDP, the
keygroupname must be the same as the keygroupname in the previous encrypted
LSU.
For more information, See “About MSDP Encryption using NetBackup KMS service”
on page 76.
/usr/openv/pdde/pdcr/bin/pddecfg -a listcloudlsu
dsid, lsuname, storageId, CachePath
3, S3Volume, server1_ S3Volume/cloud-bucket1/sub1, /msdp/data/ds_3
4, S3Volume2, server1_ S3Volume2/cloud-bucket1/sub2,
/msdp/data/ds_4
V7.5 "operation" "delete-lsu-cloud" string The value “delete-lsu-cloud” for deleting the MSDP
cloud LSU configurations in spad.
For example:
# /usr/openv/pdde/pdconfigure/pdde stop
# /usr/openv/netbackup/bin/nbsvcmon
V7.5 "operation" " " string The value must be “set-replication” for adding a new
replication target.
V7.5 " rephostname" " " string Specifies the replication target's host name.
V7.5 "replogin" " " string Specifies the replication target storage server's user name.
V7.5 "reppasswd" " " string Specifies the replication target storage server's password.
V7.5 "repsourcevolume" " " string Specifies the replication source volume name.
V7.5 "reptargetvolume" " " string Specifies the replication target volume name.
Example:
See “About the storage lifecycle policies required for Auto Image Replication”
on page 192.
See “Creating a storage lifecycle policy” on page 194.
V7.5 "operation" " " string The value must be “delete-replication” for deleting a new
replication target.
V7.5 " rephostname" " " string Specifies the replication target's host name.
V7.5 "repsourcevolume" " " string Specifies the replication source volume name.
V7.5 "reptargetvolume" " " string Specifies the replication target volume name.
For example:
For example, there is a target storage server and user name of the server is userA
and there is a cloud LSU s3cloud1 in the target storage server. To replicate image
from an old storage server to the cloud LSU of the target server, you can use the
following user name while adding the A.I.R target:
userA?LSU=s3cloud1
You must also create an import SLP to local volume of target storage server in the
target master server. Then select the imported SLP while creating the target A.I.R
SLP on the source side. When A.I.R runs, the import job in target side shows the
policy name as SLP_No_Target_SLP in the Activity Monitor, but the data is sent
to cloud.
If the NetBackup client version is 8.2 or earlier, the client direct backup from the
old client to cloud LSU of one storage server might fail. During the backup if mtstrmd
is used on the client side, the job fails with a media write error. To disable mtstrmd
at the client side, open the configuration file pd.conf on the client and change the
following:
MTSTRM_BACKUP_ENABLED = 1 to MTSTRM_BACKUP_ENABLED = 0.
■ Windows
install_path\Veritas\NetBackup\bin\ost-plugins
When doing a client direct backup with a cloud LSU and an old client, the client
does only client-side deduplication.
To use cloud LSU, the load balance server of the storage server must not be an
earlier version (NetBackup 8.2 or earlier). If there are new and old load balancers,
new load balance server is selected automatically to make sure that the job can be
done successfully. When you are restoring a backup image on cloud LSU and you
select the media server explicitly, the media server selected must not be an earlier
version of NetBackup.
MSDP cloud support 315
About the configuration items in cloud.json, contentrouter.cfg and spa.cfg
UseMemForUpload If it is set to true, the upload cache directory is mounted in memory as true
tmpfs. It is especially useful for high speed cloud that disk speed is
bottleneck. It can also reduce the disk competition with local LSU. The
value is set to true if the system memory is enough.
CachePath The path of the cache. It is created under an MSDP volume according to NA
the space usage of MSDP volumes. It will reserve some space that local
LSU cannot write beyond. Usually you do not need to change this path,
unless in some case that some volumes are much freer than others,
multiple cloud LSUs may be distributed to the same disk volume. For
performance consideration, you may need to change this option to make
them distributed to different volumes. This path can be changed to reside
in a non-MSDP volume.
When you add a new cloud LSU, the value of UploadCacheGB is equal
to CloudUploadCacheSize. You can later change this value in the
cloud.json file.
MSDP cloud support 316
About the configuration items in cloud.json, contentrouter.cfg and spa.cfg
DownloadDataCacheGB It is the maximum space usage of data file, mainly the SO BIN file. The 500
larger this cache, the more data files can reside in the cache. Then there
is no need to download these files from cloud when doing restore.
Note: The initial value of DataCacheGB in the cloud.json file is the
value of CloudDataCacheSize in the contentrouter.cfg file.
When you add a new cloud LSU, the value of DataCacheGB is equal to
CloudDataCacheSize. You can later change this value in the
cloud.json file.
DownloadMetaCacheGB It is the maximum space usage of metadata file, mainly the DO file and 500
SO BHD file. The larger this cache, the more meta files can reside in the
cache. Then there is no need to download these files from cloud when
doing restore.
Note: The initial value of MetaCacheGB in the cloud.json file is the
value of CloudMetaCacheSize in the contentrouter.cfg file.
When you add a new cloud LSU, the value of MetaCacheGB is equal to
CloudMetaCacheSize. You can later change this value in the
cloud.json file.
MapCacheGB It is the max space usage of map file that is used for compatibility of MD5 5
type fingerprint. The larger this cache, the more map files can reside in
the cache.
Note: The initial value of MapCacheGB in the cloud.json file is the
value of CloudMapCacheSize in the contentrouter.cfg file.
When you add a new cloud LSU, the value of MapCacheGB is equal to
CloudMapCacheSize. You can later change this value in the
cloud.json file.
DeleteConnNum Maximum number of concurrent connections to the cloud provider for 100
deleting. Increasing this value is helpful especially for high latency network.
KeepData Keep uploaded data to data cache. The value always false if UseMem is false
true.
KeepMeta Keep uploaded meta to meta cache, always false if UseMem is true. false
ReadOnly LSU is read only, cannot write and delete on this LSU. false
WriteThreadNum The number of threads for writing data to the data container in parallel 2
that can improve the performance of IO.
RebaseThresholdMB Rebasing threshold (MB), when image data in container less than the 4
threshold, all of the image data in this container will not be used for
deduplication to achieve good locality. Allowed values: 0 to half of
MaxFileSizeMB, 0 = disabled
Adding a new cloud LSU fails if no partition has free space more than the
following:
CloudDataCacheSize + CloudMapCacheSize +
CloudMetaCacheSize + CloudUploadCacheSize +
WarningSpaceThreshold * partition size
Note: Each Cloud LSU has a cache directory. The directory is created under an
MSDP volume that is selected according to the disk space usage of all the MSDP
volumes. Cloud LSU reserves some disk space for cache from that volume, and
the local LSU cannot utilize more disk space.
The initial reserved disk space for each of the cloud LSU is the sum of values of
UploadCacheGB, DownloadDataCacheGB, DownloadMetaCacheGB, and MapCacheGB
in the <STORAGE>/etc/puredisk/cloud.json file. The disk space decreases when
the caches are used.
There is a Cache options in crcontrol --dsstat 2 output:
# crcontrol --dsstat 2
Path = /msdp/data/dp1/1pdvol
Data storage
The Cache option is the currently reserved disk space by cloud for this volume. The
disk space is the sum of the reserved space for all cloud LSUs that have cache
directories on this volume. The actually available space for Local LSU on this volume
is Avail – Cache.
CloudDataCacheSize Default data cache size when adding Cloud LSU. 500 GiB
CloudMapCacheSize Default map cache size when adding Cloud LSU. 5 GiB
CloudMetaCacheSize Default meta cache size when adding Cloud LSU. 500 GiB
CloudUploadCacheSize Default upload cache size when adding Cloud LSU. 12 GiB
MaxCloudCacheSize Specify the maximum cloud cache size in percentage. It is based on total 20 %
system memory, swap space excluded.
CloudBits The number of top-level entries in the cloud cache. This number is Auto-sized
(2^CloudBits). Increasing this value improves cache performance, at according to
the expense of extra memory usage. Minimum value = 16, maximum MaxCloudCacheSize
value = 48.
DCSCANDownloadTmpPath While using the dcscan to check cloud LSU, data gets downloaded to disabled
this folder. For details, see the dcscan tool in cloud support section.
Dcscan downloads data container from the cloud. The default download path is
<STORAGE>/tmp/DSID_#dsid, where #dsid is dependent on the cloud LSU DSID
value. Different cloud storage providers have different DSID values. You do not
need to know the DSID value, dcscan gets the DSID value automatically. The
default download path can be modified in the contentrouter.cfg file using the
DCSCANDownloadTmpPath field.
While using the dcscan tool to look at cloud data, -a option is disabled, because it
downloads all data containers from cloud, it is an expensive operation. The -fixdo
option is disabled as well, because dcscan only downloads data container from the
cloud. Others operations are same as the local LSU.
MSDP cloud support 320
About the tool updates for cloud support
SEEDUTIL:
Seedutil can be used for seeding a backup for a better deduplication rate. It creates
links in the <destination client name> directory to all the backup files found in
the path <client name>/<policy name> that have <backup ID> in their names.
The user needs to know which DSID value the cloud LSU has used. That DSID value
needs to be given to the seedutil, to let seedutil know which cloud LSU will seed
a client. If you do a seeding for a local LSU, the default DSID is 2, you do not need
to give the DSID value. Seedutil cannot seed across different DSIDs.
For example,/usr/openv/pdde/pdag/bin/seedutil -seed -sclient
<source_client_name> -spolicy <source_policy_name> -dclient
<destination_client_name> -dsid <dsid_value>.
CRCONTROL:
Using crcontrol –clouddsstat option to show cloud LSU datastore usage. DSID
value needs to be given. As cloud storage has unlimited space, the size is
hard-coded to 8 PB.
For example:
CRSTATS:
Using crstats -cloud -dsid option to show the cloud LSU statistics. DSID value
needs to be given. As cloud storage has unlimited space, the size is hard-coded
to 8 PB.
For example:
PDDECFG:
/usr/openv/pdde/pdcr/bin/pddecfg -a listcloudlsu
dsid, lsuname, storageId, CachePath
3, S3Volume, amazon_1/cloud-bucket1/sub1, /msdp/data/ds_3
4, S3Volume2, amazon_1/cloud-bucket1/sub2, /msdp/data/ds_4
0 Reinstall and configure storage See the NetBackup Administrator's Guide, Volume I.
server
1 Create an empty local LSU Refer to the "Configure/Reconfigure MSDP local storage" topic.
1 Expire the backup images Expire all backup images that reside on the deduplication disk storage.
Warning: Do not delete the images. They are imported back into
NetBackup later in this process. If you use the bpexpdate command
to expire the backup images, use the -nodelete parameter.
2 Delete old storage server-related Refer to the "Recovering from an MSDP storage server failure" topic.
configurations
Delete the storage units that use the disk pool.
4 Create an empty local LSU. Refer to the "Configure/Reconfigure MSDP local storage" topic.
0 Reinstall and configure storage See the NetBackup Administrator's Guide, Volume I.
server
1 Reuse existing local storage path Refer to the "Configure/Reconfigure MSDP local storage" topic.
/usr/openv/netbackup/bin/bp.start_all
MSDP cloud support 323
About the disaster recovery for cloud LSU
1 Expire the backup images. Expire all backup images that reside on the deduplication disk storage.
Warning: Do not delete the images. They are imported back into
NetBackup later in this process. If you use the bpexpdate command
to expire the backup images, use the -nodelete parameter.
2 Delete old storage server-related Refer to the "Recovering from an MSDP storage server failure" topic.
configurations.
Delete the storage units that use the disk pool.
4 Reuse existing local storage path Refer to the "Configure/Reconfigure MSDP local storage" topic.
/usr/openv/netbackup/bin/bp.start_all
If alias does not exist, you can use the csconfig command to add
them.
/usr/openv/netbackup/bin/admincmd/csconfig
cldinstance -as -in amazon.com -sts <storageserver>
-lsu_name <lsuname>
/usr/openv/netbackup/bin/bp.start_all
Warning: The time taken can be high if the siz eof the containers is
large. Use the following command to get the status:
/usr/openv/pdde/pdcr/bin/cacontrol --catalog
clouddrstatus lsu
MSDP cloud support 326
About Image Sharing using MSDP cloud
/usr/openv/pdde/pdcr/bin/pddecfg -a
startdatafullcheck
/usr/openv/pdde/pdcr/bin/crcontrol --processqueue
Note: Do not use nbdevconfig to modify cloud LSU or add new cloud LSU in image
sharing server, that might cause an issue in the image sharing server.
If Amazon Glacier or deep Archive is enabled in MSDP cloud, you cannot use Image
sharing.
As there is a requirement of free disk spaces by cloud LSU, ensure that you have
enough disk space in the default mount point /storage, or you can use -p parameter
of ims_system_config.py to specify a different mount point to meet the requirement
of free disk spaces.
Chapter 8
Monitoring deduplication
activity
This chapter includes the following topics:
For the method to show the MSDP compression rate, See “Viewing MSDP job
details” on page 330.
On UNIX and Linux, you can use the NetBackup bpdbjobs command to display
the deduplication rate. However, you must configure it to do so.
See “To configure the bpdbjobs command to display the MSDP deduplication rate”
on page 330.
To view the global MSDP deduplication ratio
1 In the NetBackup Administration Console, expand Media and Device
Management > Credentials > Storage Server
2 Select the deduplication storage server.
3 On the Edit menu, select Change.
4 In the Change Storage Server dialog box, select the Properties tab. The
Deduplication Ratio field displays the ratio.
Monitoring deduplication activity 330
Viewing MSDP job details
To view the MSDP deduplication rate for a backup job in the Activity Monitor
1 In the NetBackup Administration Console, click Activity Monitor.
2 Click the Jobs tab.
The Deduplication Rate column shows the rate for each job.
Not all columns appear by default. Click View > Column Layout to show or
hide columns.
To configure the bpdbjobs command to display the MSDP deduplication rate
◆ Add a DEDUPRATIO BPDBJOBS_COLDEFS entry in the bp.conf file on the media
server on which you run the command.
The bpdbjobs command then shows the deduplication rate in its output.
3 To view the details for a specific job, double-click on the job that is displayed
in the Jobs tab pane.
4 In the Job Details dialog box, click the Detailed Status tab.
The deduplication job details are described in a different topic.
See “MSDP job details” on page 331.
Field descriptions
Table 8-1 describes the deduplication activity fields.
Field Description
Dedup space saving The percentage of space that is saved by data deduplication (data is not written again).
Compression space The percentage of space that is saved because the deduplication engine compressed some
saving data before writing it to storage.
cache hits The percentage of data segments in the backup that is represented in the local fingerprint
cache. The deduplication plug-in did not have to query the database about those segments
If the pd.conf file FP_CACHE_LOCAL parameter is set to 0 on the storage, the cache hits
output is not included for the jobs that run on the storage server.
CR sent The amount of data that is sent from the deduplication plug-in to the component that stores
the data. In NetBackup, the NetBackup Deduplication Engine stores the data.
If the storage server deduplicates the data, it does not travel over the network. The
deduplicated data travels over the network when the deduplication plug-in runs on a computer
other than the storage server, as follows:
Field Description
CR sent over FC The amount of data that is sent from the deduplication plug-in over Fibre Channel to the
component that stores the data. In NetBackup, the NetBackup Deduplication Engine stores
the data.
dedup The percentage of data that was stored already. That data is not stored again.
multi-threaded stream Indicates that the Deduplication Multi-Threaded Agent processed the backup.
used
See “About the MSDP Deduplication Multi-Threaded Agent” on page 59.
PDDO stats Indicates that the job details are for storage on the following destinations:
rebased The percentage of segments that were rebased (that is, defragmented) during the backup.
Those segments had poor data locality.
NetBackup reports backup job completion only after backup rebasing is completed.
Using OpenStorage Indicates that the restore travels over the client-direct data path and does not use NetBackup
client direct to media server components to process the data.
restore...
■ If other applications use the storage, NetBackup cannot report usage accurately.
NetBackup requires exclusive use of the storage.
Table 8-2 describes the options for monitoring capacity and usage.
Option Description
Change Storage Server The Change Storage Server dialog box Properties tab displays
dialog box storage capacity and usage. It also displays the global
deduplication ratio.
This dialog box displays the most current capacity usage that is
available in the NetBackup Administration Console.
Disk Pools window The Disk Pools window of the Administration Console displays
the values that were stored when NetBackup polled the disk
pools. NetBackup polls every 5 minutes; therefore, the value may
not be as current as the value that is displayed in the Change
Storage Server dialog box.
View container command A command that is installed with NetBackup provides a view of
storage capacity and usage within the deduplication container
files.
Disk Pool Status report The Disk Pool Status report displays the state of the disk pool
and usage information.
Option Description
Disk Logs report The Disk Logs report displays event and message information.
A useful event for monitoring capacity is event 1044; the following
is the description of the event in the Disk Logs report:The usage
of one or more system resources has exceeded a
warning level.
The nbdevquery The nbdevquery command shows the state of the disk volume
command and its properties and attributes. It also shows capacity, usage,
and percent used.
http://www.veritas.com/docs/DOC5332
For systems that host a Media Server Deduplication Pool, you can use the
following crcontrol command to show information about each partition:
/usr/openv/pdde/pdcr/bin/crcontrol --dsstat 3
Size The size of the storage that NetBackup can use: the Raw size of the
storage minus the file system Reserved space.
If the file system has a concept of root reserved space (such as EXT3
or VxFS), that space cannot be used for storage. The crcontrol
command does not include reserved space in the available space.
Unlike the crcontrol command, some operating system tools report
root reserved space as usable space.
Used The amount of deduplicated data that is stored on the file system.
NetBackup obtains the file system used space from the operating
system.
Report Description
Images on Disk The Images on Disk report generates the image list present on the
disk storage units that are connected to the media server. The
report is a subset of the Images on Media report; it shows only
disk-specific columns.
Disk Logs The Disk Logs report displays the media errors or the informational
messages that are recorded in the NetBackup error catalog. The
report is a subset of the Media Logs report; it shows only
disk-specific columns.
Disk Storage Unit The Disk Storage Unit Status report displays the state of disk
Status storage units in the current NetBackup configuration.
For disk pool capacity, see the disk pools window in Media and
Device Management > Devices > Disk Pools.
Multiple storage units can point to the same disk pool. When the
report query is by storage unit, the report counts the capacity of
disk pool storage multiple times.
Disk Pool Status The Disk Pool Status report displays the state of disk pool and
usage information. This report displays only when a license that
enables NetBackup disk features is installed.
Monitoring deduplication activity 338
About monitoring MSDP processes
Replication The job that replicates a backup image to a target master displays in the Activity Monitor as a
Replication job. The Target Master label displays in the Storage Unit column for this type of job.
Similar to other Replication jobs, the job that replicates images to a target master can work on
multiple backup images in one instance.
The detailed status for this job contains a list of the backup IDs that were replicated.
Import The job that imports a backup copy into the target master domain displays in the Activity Monitor as
an Import job. An Import job can import multiple copies in one instance. The detailed status for an
Import job contains a list of processed backup IDs and a list of failed backup IDs.
Note that a successful replication does not confirm that the image was imported at the target master.
If the data classifications are not the same in both domains, the Import job fails and NetBackup does
not attempt to import the image again.
Failed Import jobs fail with a status 191 and appear in the Problems report when run on the target
master server.
The image is expired and deleted during an Image Cleanup job. Note that the originating domain
(Domain 1) does not track failed imports.
Chapter 9
Managing deduplication
This chapter includes the following topics:
This example output is shortened; more flags may appear in actual output.
This example output is shortened; more flags may appear in actual output.
The following describes the options that require the arguments that are specific
to your domain:
Managing deduplication 344
Managing MSDP servers
-setattribute The attribute is the name of the argument that represents the
attribute new functionality.
Warning: If you have load balancing servers, do not select any or all of them
in the Media Servers pane of the NetBackup Administration Console. If
you do, the change operation fails.
4 In the Change Storage Server dialog box, select the Properties tab.
5 For the property to change, select the value in the Value column.
6 Change the value.
7 Click OK.
-setattribute The attribute is the name of the argument that represents the
attribute functionality.
Managing deduplication 346
Managing MSDP servers
See “About changing the MSDP storage server name or storage path” on page 346.
Managing deduplication 347
Managing MSDP servers
Step 1 Ensure that no deduplication Deactivate all backup policies that use deduplication storage.
activity occurs
See the NetBackup Administrator's Guide, Volume I:
http://www.veritas.com/docs/DOC5332
Step 2 Expire the backup images Expire all backup images that reside on the deduplication disk storage.
Warning: Do not delete the images. They are imported back into NetBackup
later in this process.
If you use the bpexpdate command to expire the backup images, use the
-nodelete parameter.
http://www.veritas.com/docs/DOC5332
Step 3 Delete the storage units that See the NetBackup Administrator's Guide, Volume I:
use the disk pool
http://www.veritas.com/docs/DOC5332
Step 4 Delete the disk pool See “Deleting a Media Server Deduplication Pool” on page 364.
Step 5 Delete the deduplication See “Deleting an MSDP storage server” on page 349.
storage server
Step 7 Delete the deduplication host Each load balancing server contains a deduplication host configuration file.
configuration file If you use load balancing servers, delete the deduplication host configuration
file from those servers.
Step 8 Delete the identity file and the Delete the following files from the MSDP storage server, depending on
file system table file operating system:
UNIX:
/storage_path/data/.identity
/storage_path/etc/puredisk/fstab.cfg
Windows:
storage_path\data\.identity
storage_path\etc\puredisk\fstab.cfg
Managing deduplication 348
Managing MSDP servers
Table 9-1 Changing the storage server name or storage path (continued)
Step 9 Change the storage server See the computer or the storage vendor's documentation.
name or the storage location
See “Use fully qualified domain names” on page 41.
Step 10 Reconfigure the storage When you configure deduplication, select the host by the new name and
server enter the new storage path (if you changed the path). You can also use a
new network interface.
Step 11 Import the backup images See the NetBackup Administrator's Guide, Volume I:
http://www.veritas.com/docs/DOC5332
3 Select the deduplication storage server, and then select Edit > Change.
4 In the Change Storage Server dialog box, select the Media Servers tab.
5 Clear the check box of the media server you want to remove.
6 Click OK.
Windows: install_path\Program
Files\Veritas\pdde\PDDE_deleteConfig.bat
On UNIX/Linux:
/usr/openv/volmgr/bin/tpconfig -add -storage_server sshostname
-stype PureDisk -sts_user_id UserID -password PassWord
On UNIX/Linux:
/usr/openv/volmgr/bin/tpconfig -delete -storage_server sshostname
-stype PureDisk -sts_user_id UserID
This example output is shortened; more flags may appear in actual output.
The following describes the options that require the arguments that are specific
to your domain:
-setattribute The attribute is the name of the argument that represents the
attribute new functionality.
4 In the Change Disk Pool dialog box, click Refresh to update the disk pool
replication properties.
If NetBackup discovers changes, your actions depend on the changes
discovered.
See “How to resolve volume changes for Auto Image Replication” on page 358.
5 Change the other properties as necessary.
See “Media Server Deduplication Pool properties” on page 116.
6 Click OK.
7 If you clicked Refresh and the Replication value for the PureDiskVolume
changed, refresh the view in the Administration Console.
Managing deduplication 358
Managing Media Server Deduplication Pools
Outcome Description
NetBackup discovers the new The new volumes appear in the Change Disk Pool dialog box. Text in the dialog box
volumes that you can add to the changes to indicate that you can add the new volumes to the disk pool.
disk pool.
Managing deduplication 359
Managing Media Server Deduplication Pools
Outcome Description
The replication properties of all of A Disk Pool Configuration Alert pop-up box notifies you that the properties of all of the
the volumes changed, but they volumes in the disk pool changed, but they are all the same (homogeneous).
are still consistent.
You must click OK in the alert box, after which the disk pool properties in the Change
Disk Pool dialog box are updated to match the new volume properties
If new volumes are available that match the new properties, NetBackup displays those
volumes in the Change Disk Pool dialog box. You can add those new volumes to the
disk pool.
In the Change Disk Pool dialog box, select one of the following two choices:
■ OK. To accept the disk pool changes, click OK in the Change Disk Pool dialog box.
NetBackup saves the new properties of the disk pool.
NetBackup can use the disk pool, but it may no longer match the intended purpose
of the storage unit or storage lifecycle policy. Change the storage lifecycle policy
definitions to ensure that the replication operations use the correct source and target
disk pools, storage units, and storage unit groups. Alternatively, work with your storage
administrator to change the volume properties back to their original values.
■ Cancel. To discard the changes, click Cancel in the Change Disk Pool dialog box.
NetBackup does not save the new disk pool properties. NetBackup can use the disk
pool, but it may no longer match the intended use of the storage unit or storage
lifecycle policy.
Managing deduplication 360
Managing Media Server Deduplication Pools
Outcome Description
The replication properties of the A Disk Pool Configuration Error pop-up box notifies you that the replication properties
volumes changed, and they are of some of the volumes in the disk pool changed. The properties of the volumes in the
now inconsistent. disk pool are not homogeneous.
In the Change Disk Pool dialog box, the properties of the disk pool are unchanged, and
you cannot select them (that is, they are dimmed). However, the properties of the individual
volumes are updated.
Because the volume properties are not homogeneous, NetBackup cannot use the disk
pool until the storage configuration is fixed.
NetBackup does not display new volumes (if available) because the volumes already in
the disk pool are not homogeneous.
To determine what has changed, compare the disk pool properties to the volume
properties.
See “Viewing the replication topology for Auto Image Replication” on page 165.
Work with your storage administrator to understand the changes and why they were
made. The replication relationships may or may not have to be re-established. If the
relationship was removed in error, re-establishing the relationships seem justified. If you
are retiring or replacing the target replication device, you probably do not want to
re-establish the relationships.
The disk pool remains unusable until the properties of the volumes in the disk pool are
homogenous.
In the Change Disk Pool dialog box, click OK or Cancel to exit the Change Disk Pool
dialog box.
Managing deduplication 361
Managing Media Server Deduplication Pools
Outcome Description
NetBackup cannot find a volume A Disk Pool Configuration Alert pop-up box notifies you that an existing volume or
or volumes that were in the disk volumes was deleted from the storage device:
pool.
NetBackup can use the disk pool, but data may be lost.
To protect against accidental data loss, NetBackup does not allow volumes to be deleted
from a disk pool.
To continue to use the disk pool, do the following:
■ Use the bpimmedia command or the Images on Disk report to display the images
on the specific volume.
■ Expire the images on the volume.
■ Use the nbdevconfig command to set the volume state to DOWN so NetBackup
does not try to use it.
-setattribute The attribute is the name of the argument that represents the
attribute new functionality.
■ UNIX: /usr/openv/netbackup/bin/admincmd
■ Windows: install_path\NetBackup\bin\admincmd
To change the state to DOWN, the disk pool in which the volume resides must not
be busy. If backup jobs are assigned to the disk pool, the state change fails. Cancel
the backup jobs or wait until the jobs complete.
To change the MSDP disk volume state
◆ Change the disk volume state; the following is the command syntax:
UNIX: /usr/openv/netbackup/bin/admincmd/nbdevconfig -changestate
-stype PureDisk -dp disk_pool_name –dv PureDiskVolume -state state
Windows: install_path\NetBackup\bin\admincmd\nbdevconfig
-changestate -stype PureDisk -dp disk_pool_name –dv PureDiskVolume
-state state
Warning: Veritas recommends that you do not disable the data integrity checking.
If you do so, NetBackup cannot find and repair or report data corruption.
Windows: install_path\Veritas\pdde\pddecfg –a
enabledataintegritycheck
Windows: install_path\Veritas\pdde\pddecfg –a
disabledataintegritycheck
Windows: install_path\Veritas\pdde\pddecfg –a
getdataintegritycheck
Enable CRC does not run if queue processing is active or during disk read
or write operations.
UNIX: /usr/openv/pdde/pdcr/bin/crcontrol
--crccheckon
Windows: install_path\Veritas\pdde\Crcontrol.exe
--crccheckon
Windows: install_path\Veritas\pdde\Crcontrol.exe
--crccheckoff
Enable fast Fast check CRC mode begins the check from container 64 and
checking does not sleep between checking containers.
When the fast CRC ends, CRC behavior reverts to the behavior
before fast checking was invoked.
UNIX: /usr/openv/pdde/pdcr/bin/crcontrol
--crccheckrestart
Windows: install_path\Veritas\pdde\Crcontrol.exe
--crccheckrestart
Managing deduplication 369
Configuring MSDP data integrity checking behavior
Windows: install_path\Veritas\pdde\Crcontrol.exe
--crccheckstate
Warning: Veritas recommends that you do not disable the data integrity checking.
If you do so, NetBackup cannot find and repair or report data corruption.
Managing deduplication 370
Configuring MSDP data integrity checking behavior
Table 9-3 The contentrouter.cfg file parameters for data integrity checking
EnableCRCCheck true Enable or disable cyclic redundancy checking (CRC) of the data
container files.
The longer the sleep interval, the more time it takes to check containers.
The greater the number of containers, the less time it takes to check
all containers, but the more system resources it takes.
ShutdownCRWhenError false Stops the NetBackup Deduplication Manager when a data loss is
discovered.
GarbageCheckRemainDCCount 100 The number of containers from failed jobs not to check for garbage. A
failed backup or replication job still produces data containers. Because
failed jobs are retried, retaining those containers means NetBackup
does not have to send the fingerprint information again. As a result,
retried jobs consume less time and fewer system resources than when
first run.
The greater the number of days, the fewer the objects that are checked
each day. The greater the number of days equals fewer storage server
resources consumed each day.
Managing deduplication 371
About managing MSDP storage read performance
Table 9-4 spa.cfg file parameters for data integrity checking (continued)
However, the data segments in a backup may become scattered across the disk
storage each time the client is backed up. Such scattering is a normal consequence
of deduplication.
NetBackup includes a process, called rebasing, that helps to maintain the data
segments in as few container files as possible. Rebasing improves performance
for the operations that read from the storage, such as restores and duplications.
NetBackup writes all of the data segments from a backup into new container files
even though the segments exist on storage already. Future backups then refer to
the new copies of those segments rather than the old copies until any changes
because of future rebasing. Deduplication rates for the backup jobs that perform
rebasing are lower than for the jobs that do not rebase the data.
After the rebasing, NetBackup reclaims the storage space that the rebased data
segments used.
Table 9-5 describes the rebasing operations.
Type Description
Normal backup The rebasing that occurs during a backup if the normal rebasing
rebasing criteria are met, as follows:
Backup rebasing occurs only for the full backups that pass through
the normal MSDP backup process. For example, the NetBackup
Accelerator backups do not pass through the MSDP backup process.
Type Description
Periodic backup The rebasing that occurs during a backup if the periodic rebasing
rebasing criteria are met, as follows:
■ The container has not been rebased within the last 3 months.
■ For that backup, the data segments in the container consume less
space than the FP_CACHE_REBASING_THRESHOLD value. The
FP_CACHE_REBASING_THRESHOLD parameter is in the pd.conf
file.
See “MSDP pd.conf file parameters” on page 210.
Backup rebasing occurs only for the full backups that pass through
the normal MSDP backup process. For example, the NetBackup
Accelerator backups do not pass through the MSDP backup process.
Server-side rebasing The storage rebasing that occurs on the server if the rebasing criteria
are met. Server-side rebasing includes the deduplicated data that
does not pass through the normal MSDP backup process. For
example, the NetBackup Accelerator backups do not pass through
the MSDP backup process.
Parameter Description
RebaseMaxPercentage The maximum percentage of the data segments to be rebased in a file. For any
file, if the percentage of the data segments reaches this threshold, the remainder
of the data segments are not rebased.
RebaseMaxTime The maximum time span in seconds of data segments to be rebased in a file. If
this threshold is reached, NetBackup does not rebase the remainder of the data
segments.
RebaseMinContainers The minimum number of containers in which a file’s data segments are stored for
the file to be eligible for rebasing. If the number of containers in which a file’s data
segments are stored is less than RebaseMinContainers, NetBackup does not
rebase the data segments.
RebaseScatterThreshold The data locality threshold for a container. If the total size of a file’s data segments
in a container is less than RebaseScatterThreshold, NetBackup rebases all
of the file’s data segments.
During the queue processing, the Deduplication Engine reclaims some of the
storage space on which the data segments reside. Some is reclaimed during
data compaction. If a different backup image requires a data segment, the
segment is not removed.
Various internal parameters control whether a container file is compacted.
See “About MSDP container files” on page 335.
If you manually delete an image that has expired within the previous 24 hours, the
data becomes garbage. It remains on disk until removed by the next garbage
collection process. Garbage collection occurs during data integrity checking.
See “About MSDP data integrity checking” on page 366.
See “Deleting backup images” on page 364.
Type Description
Normal restore The MSDP storage server first rehydrates (that is, reassembles) the data. NetBackup then chooses
the least busy media server to move the data to the client. (NetBackup chooses the least busy
media server from those that have credentials for the NetBackup Deduplication Engine.) The
media server bptm process moves the data to the client.
The following media servers have credentials for the NetBackup Deduplication Engine:
Restore directly to The storage server can bypass the media server and move the data directly to the client.
the client NetBackup does not choose a media server for the restore, and the restore does not use the
media server bptm process.
You must configure NetBackup to bypass a media server and receive the restore data directly
from the storage server.
By default, NetBackup rehydrates the data on the storage server except for client-side
deduplication clients. Those clients rehydrate the data. You can configure NetBackup so that the
data is rehydrated on the storage server rather than the client. See the RESTORE_DECRYPT_LOCAL
parameter in the MSDP pd.conf file.
client. In this case, you can specify that the media server at the remote site as the
restore server.
Scenario 1 Yes Configure the client in another domain and restore directly to the client.
Scenario 2 No Create the client in the recovery domain and restore directly to the
client. This is the most likely scenario.
The steps to recover the client are the same as any other client recovery. The actual
steps depend on the client type, the storage type, and whether the recovery is an
alternate client restore.
For restores that use Granular Recovery Technology (GRT), an application instance
must exist in the recovery domain. The application instance is required so that
NetBackup has something to recover to.
media server you specify. Specify the same server for the Restore server
as for the Original backup server.
See “Forcing restores to use a specific server” in the NetBackup
Administrator's Guide, Volume I:
http://www.veritas.com/docs/DOC5332
This procedure sets the FORCE_RESTORE_MEDIA_SERVER option. Configuration
options are stored in the bp.conf file on UNIX systems and the registry on
Windows systems.
■ Create the touch file USE_BACKUP_MEDIA_SERVER_FOR_RESTORE on the
NetBackup master server in the following directory:
UNIX: usr/openv/netbackup/db/config
Windows: install_path\Veritas\Netbackup\db\config
This global setting always forces restores to the server that did the backup.
It applies to all NetBackup restore jobs, not just deduplication restore jobs.
If this touch file exists, NetBackup ignores the FORCE_RESTORE_MEDIA_SERVER
and FAILOVER_RESTORE_MEDIA_SERVER settings.
Restore from a shadow If NetBackup detects corruption in the MSDP catalog, the Deduplication Manager restores
copy the catalog automatically from the most recent shadow copy. The automatic restore process
also plays a transaction log so that the recovered MSDP catalog is current.
Although the shadow copy restore process is automatic, a restore procedure is available if
you need to recover from a shadow copy manually.
See “Restoring the MSDP catalog from a shadow copy” on page 381.
Recovering MSDP 381
Restoring the MSDP catalog from a shadow copy
Recover from a backup If you configured an MSDP catalog backup policy and a valid backup exists, you can recover
the catalog from a backup. As a general rule, you should only attempt to recover the MSDP
catalog from a backup if you have no alternatives. As an example: A hardware problem or
a software problem results in the complete loss of the MSDP catalog and the shadow copies.
The greatest chance for a successful outcome when you recover the MSDP catalog from
a backup is when the recovery is guided. An unsuccessful outcome may cause data loss.
For the customers who need to recover the MSDP catalog, Veritas wants to guide them
through the process. Therefore, to recover the MSDP catalog from a backup, contact your
Veritas support representative. You can refer the support representative to Knowledge Base
Article 000047346, which contains the recovery instructions.
Caution: You must determine if your situation is severe enough to recover the
catalog. Veritas recommends that you contact your Veritas Support representative
before you restore or recover the MSDP catalog. The Support representative can
help you determine if you need to recover the catalog or if other solutions are
available.
Restore the entire MSDP In this scenario, you want to restore the entire catalog from
catalog from a shadow copy one of the shadow copies.
Restore a specific MSDP The MSDP catalog is composed of multiple small database
database file files. Those files are organized in the file system by the client
name and policy name, as follows:
UNIX:
/database_path/databases/catalogshadow/2/ClientName/PolicyName
Windows:
database_path\databases\catalogshadow\2\ClientName\PolicyName
You can restore the database files for a client and a policy
combination. The restore of a specific client’s and policy’s
database files is always from the most recent shadow copy.
4 Enable all policies and storage lifecycle policies that back up to the Media
Server Deduplication Pool.
5 Restart the jobs that were canceled before the recovery.
To restore a specific MSDP database file from a shadow copy
1 If any MSDP jobs are active for the client and the backup policy combination,
either cancel them or wait until they complete.
2 Disable the policies and storage lifecycle policies for the client and the backup
policy combination that back up to the Media Server Deduplication Pool.
Recovering MSDP 383
Recovering from an MSDP storage server disk failure
3 Change to the shadow directory for the client and policy from which you want
to recover that database file. That directory contains the database files from
which to recover. The following are the pathname formats:
UNIX:
/database_path/databases/catalogshadow/2/ClientName/PolicyName
Windows:
database_path\databases\catalogshadow\2\ClientName\PolicyName
5 Enable all policies and storage lifecycle policies that back up to the Media
Server Deduplication Pool.
6 If you canceled jobs before you recovered the database files, restart them.
Note: This procedure describes recovery of the disk on which the NetBackup media
server software resides not the disk on which the deduplicated data resides. The
disk may or may not be the system boot disk.
Step 1 Replace the disk. If the disk is a system boot disk, also install the operating system.
See the hardware vendor and operating system documentation.
Step 2 Mount the storage. Ensure that the storage and database are mounted at the same locations.
Step 3 Install and license the See NetBackup Installation Guide for UNIX and Windows:
NetBackup media server
http://www.veritas.com/docs/DOC5332
software.
See “About the MSDP license” on page 51.
Step 4 Delete the deduplication host Each load balancing server contains a deduplication host configuration file.
configuration file If you use load balancing servers, delete the deduplication host configuration
file from those servers.
Step 5 Delete the credentials on If you have load balancing servers, delete the NetBackup Deduplication
deduplication servers Engine credentials on those media servers.
Step 6 Add the credentials to the Add the NetBackup Deduplication Engine credentials to the storage server.
storage server
See “Adding NetBackup Deduplication Engine credentials” on page 351.
Step 7 Get a configuration file If you did not save a storage server configuration file before the disk failure,
template get a template configuration file.
Step 8 Edit the configuration file See “Editing an MSDP storage server configuration file” on page 229.
Step 9 Configure the storage server Configure the storage server by uploading the configuration from the file you
edited.
Step 10 Add load balancing servers If you use load balancing servers in your environment, add them to your
configuration.
NetBackup recommends that you consider the following items before you recover:
■ The new computer must use the same byte order as the old computer.
Warning: If the new computer does not use the same byte order as the old
computer, you cannot access the deduplicated data. In computing, endianness
describes the byte order that represents data: big endian and little endian. For
example, SPARC processors and Intel processors use different byte orders.
Therefore, you cannot replace an Oracle Solaris SPARC host with an Oracle
Solaris host that has an Intel processor.
■ Veritas recommends that the new computer use the same operating system as
the old computer.
■ Veritas recommends that the new computer use the same version of NetBackup
as the old computer.
If you use a newer version of NetBackup on the new computer, ensure that you
perform any data conversions that may be required for the newer release.
If you want to use an older version of NetBackup on the replacement host,
contact your Veritas support representative.
Step 1 Expire the backup images Expire all backup images that reside on the deduplication disk storage.
Warning: Do not delete the images. They are imported back into NetBackup
later in this process.
If you use the bpexpdate command to expire the backup images, use the
-nodelete parameter.
http://www.veritas.com/docs/DOC5332
Step 2 Delete the storage units that See the NetBackup Administrator's Guide, Volume I:
use the disk pool
http://www.veritas.com/docs/DOC5332
Step 3 Delete the disk pool See “Deleting a Media Server Deduplication Pool” on page 364.
Step 4 Delete the deduplication See “Deleting an MSDP storage server” on page 349.
storage server
Recovering MSDP 386
Recovering from an MSDP storage server failure
Step 5 Delete the deduplication host Each load balancing server contains a deduplication host configuration file.
configuration file If you use load balancing servers, delete the deduplication host configuration
file from those servers.
Step 6 Delete the credentials on If you have load balancing servers, delete the NetBackup Deduplication
deduplication servers Engine credentials on those media servers.
Step 7 Configure the new host so it When you configure the new host, consider the following:
meets deduplication
■ You can use the same host name or a different name.
requirements
■ You can use the same Storage Path or a different Storage Path. If you
use a different Storage Path, you must move the deduplication storage
to that new location.
■ If the Database Path on the original host is different that the Storage
Path, you can do one of the following:
■ You can use the same Database Path.
■ You can use a different Database Path. If you do, you must move
the deduplication database to the new location.
■ You do not have to continue to use a different Database Path. You
can move the databases directory into the Storage Path and then
specify only the Storage Path when you configure the storage server.
■ You can use the host’s default network interface or specify a network
interface.
If the original host used a specific network interface, you do not have to
use the same interface name.
■ If you had configured the previous MSDP storage server to use MSDP
Encryption using KMS service, you must use the same configuration for
the new MSDP storage server.
Step 8 Connect the storage to the Use the storage path that you configured for this replacement host.
host
See the computer or the storage vendor's documentation.
Step 9 Install the NetBackup media See the NetBackup Installation Guide for UNIX and Windows:
server software on the new
http://www.veritas.com/docs/DOC5332
host
Recovering MSDP 387
Recovering the MSDP storage server after NetBackup catalog recovery
Step 10 Reconfigure deduplication You must use the same credentials for the NetBackup Deduplication Engine.
Step 11 Import the backup images See the NetBackup Administrator's Guide, Volume I:
http://www.veritas.com/docs/DOC5332
Warning: If the new computer does not use the same byte order as the old
computer, you cannot access the deduplicated data. In computing, endianness
describes the byte order that represents data: Big endian and little endian. For
example, SPARC processors and Intel processors use different byte orders.
Therefore, you cannot replace an Oracle Solaris SPARC host with an Oracle
Solaris host that has an Intel processor.
■ Veritas recommends that the new computer use the same operating system as
the old computer.
■ Veritas recommends that the new computer use the same version of NetBackup
as the old computer.
If you use a newer version of NetBackup on the new computer, ensure that you
perform any data conversions that may be required for the newer release.
Replacing MSDP hosts 389
Replacing the MSDP storage server host computer
Step 1 Expire the backup images Expire all backup images that reside on the deduplication disk storage.
Warning: Do not delete the images. They are imported back into NetBackup
later in this process.
If you use the bpexpdate command to expire the backup images, use the
-nodelete parameter.
http://www.veritas.com/docs/DOC5332
Step 2 Delete the storage units that See the NetBackup Administrator's Guide, Volume I:
use the disk pool
http://www.veritas.com/docs/DOC5332
Step 3 Delete the disk pool See “Deleting a Media Server Deduplication Pool” on page 364.
Step 4 Delete the deduplication See “Deleting an MSDP storage server” on page 349.
storage server
Step 5 Delete the deduplication host Each load balancing server contains a deduplication host configuration file.
configuration file If you use load balancing servers, delete the deduplication host configuration
file from those servers.
Step 6 Delete the credentials on If you have load balancing servers, delete the NetBackup Deduplication
deduplication servers Engine credentials on those media servers.
Step 7 Configure the new host so it When you configure the new host, consider the following:
meets deduplication
■ You can use the same host name or a different name.
requirements
■ You can use the same Storage Path or a different Storage Path. If you
use a different Storage Path, you must move the deduplication storage
to that new location.
■ If the Database Path on the original host is different that the Storage
Path, you can do one of the following:
■ You can use the same Database Path.
■ You can use a different Database Path. If you do, you must move
the deduplication database to the new location.
■ You do not have to continue to use a different Database Path. You
can move the databases directory into the Storage Path and then
specify only the Storage Path when you configure the storage server.
■ You can use the host’s default network interface or specify a network
interface.
If the original host used a specific network interface, you do not have to
use the same interface name.
■ If you had configured the previous MSDP storage server to use MSDP
Encryption using KMS service, you must use the same configuration for
the new MSDP storage server.
Step 8 Connect the storage to the Use the storage path that you configured for this replacement host.
host
See the computer or the storage vendor's documentation.
Step 9 Install the NetBackup media See the NetBackup Installation Guide for UNIX and Windows:
server software on the new
http://www.veritas.com/docs/DOC5332
host
Step 10 Reconfigure deduplication See “Configuring MSDP server-side deduplication” on page 56.
Step 11 Import the backup images See the NetBackup Administrator's Guide, Volume I:
http://www.veritas.com/docs/DOC5332
Chapter 12
Uninstalling MSDP
This chapter includes the following topics:
■ Deactivating MSDP
Deactivating MSDP
You cannot remove the deduplication components from a NetBackup media server.
You can disable the components and remove the deduplication storage files and
the catalog files. The host remains a NetBackup media server.
This process assumes that all backup images that reside on the deduplication disk
storage have expired.
Warning: If you remove deduplication and valid NetBackup images reside on the
deduplication storage, data loss may occur.
Uninstalling MSDP 392
Deactivating MSDP
Step 1 Remove client deduplication Remove the clients that deduplicate their own data from the client
deduplication list.
Step 2 Delete the storage units that See the NetBackup Administrator's Guide, Volume I:
use the disk pool
http://www.veritas.com/docs/DOC5332
Step 3 Delete the disk pool See “Deleting a Media Server Deduplication Pool” on page 364.
Step 4 Delete the deduplication See “Deleting an MSDP storage server” on page 349.
storage server
Deleting the deduplication storage server does not alter the contents of the
storage on physical disk. To protect against inadvertent data loss, NetBackup
does not automatically delete the storage when you delete the storage server.
Step 6 Delete the deduplication host Each load balancing server contains a deduplication host configuration file.
configuration file If you use load balancing servers, delete the deduplication host configuration
file from those servers.
Step 7 Delete the storage directory Delete the storage directory and database directory. (Using a separate
and the database directory database directory was an option when you configured deduplication.)
Warning: If you delete the storage directory and valid NetBackup images
reside on the deduplication storage, data loss may occur.
Multi- NetBackup
Threaded Deduplication
Deduplication
Agent Engine (spoold)
plug-in
Proxy NetBackup
plug-in Deduplication
Manager (spad)
Component Description
The plug-in runs on the deduplication storage server and on load balancing servers.
Multi-Threaded Agent The NetBackup Deduplication Multi-Threaded Agent uses multiple threads for
asynchronous network I/O and CPU core calculations. The agent runs on the storage
server, load balancing servers, and clients that deduplication their own data.
NetBackup Deduplication The NetBackup Deduplication Engine is one of the storage server core components.
Engine It provides many of the deduplication functions, which are described in Table 13-2.
The binary file name is spoold, which is short for storage pool daemon; do not confuse
it with a print spooler daemon. The spoold process appears as the NetBackup
Deduplication Engine in the NetBackup Administration Console.
NetBackup Deduplication The deduplication manager is one of the storage server core components. The
Manager deduplication manager maintains the configuration and controls internal processes,
optimized duplication, security, and event escalation.
The deduplication manager binary file name is spad. The spad process appears as
the NetBackup Deduplication Manager in the NetBackup Administration Console.
Proxy plug-in The proxy plug-in manages control communication with the clients that back up their
own data. It communicates with the OpenStorage proxy server (nbostpxy) on the
client.
Reference database The reference database stores the references that point to every data segment of which
a file is composed. Unique fingerprints identify data segments. The reference database
is partitioned into multiple small reference database files to improve scalability and
performance.
The reference database is separate from the NetBackup catalog. The NetBackup
catalog maintains the usual NetBackup backup image information.
Table 13-2 describes the components and functions within the NetBackup
Deduplication Engine.
Deduplication architecture 395
MSDP server components
Component Description
Connection and Task The Connection and Task Manager manages all of the
Manager connections from the load balancing servers and the clients
that deduplicate their own data. The Connection and Task
Manager is a set of functions and threads that does the
following:
Data integrity checking The NetBackup Deduplication Engine checks the integrity of
the data and resolves integrity problems.
Data Store Manager The Data Store Manager manages all of the data container
files. The datastore Manager is a set of functions and threads
that provides the following:
Index Cache Manager The Index Cache Manager manages the fingerprint cache.
The cache improves fingerprint lookup speed.
Reference Database Engine The Reference Database Engine stores the references that
point to the data segments, such as read-from or write-to
references. It manipulates a single database file at a time.
Deduplication architecture 396
Media server deduplication backup process
Component Description
bpbkar
Control path
Data path Client
Media Server Deduplication Pool
The following list describes the backup process when a media server deduplicates
the backups and the destination is a Media Server Deduplication Pool:
■ The NetBackup Job Manager (nbjm) starts the Backup/Restore Manager (bpbrm)
on a media server.
■ The Backup/Restore Manager starts the bptm process on the media server and
the bpbkar process on the client.
■ The Backup/Archive Manager (bpbkar) on the client generates the backup
images and moves them to the media server bptm process.
Deduplication architecture 397
Media server deduplication backup process
The Backup/Archive Manager also sends the information about files within the
image to the Backup/Restore Manager (bpbrm). The Backup/Restore Manager
sends the file information to the bpdbm process on the master server for the
NetBackup database.
■ The bptm process moves the data to the deduplication plug-in.
■ The deduplication plug-in retrieves a list of IDs of the container files from the
NetBackup Deduplication Engine. Those container files contain the fingerprints
from the last full backup for the client. The list is used as a cache so the plug-in
does not have to request each fingerprint from the engine.
■ The deduplication plug-in separates the files in the backup image into segments.
■ The deduplication plug-in buffers the segments and then sends batches of them
to the Deduplication Multi-Threaded Agent. Multiple threads and shared memory
are used for the data transfer.
■ The NetBackup Deduplication Multi-Threaded Agent processes the data
segments in parallel using multiple threads to improve throughput performance.
The agent then sends only the unique data segments to the NetBackup
Deduplication Engine.
If the host is a load balancing server, the Deduplication Engine is on a different
host, the storage server.
■ The NetBackup Deduplication Engine writes the data to the Media Server
Deduplication Pool.
The first backup may have a 0% deduplication rate, although a 0% rate is
unlikely. Zero percent means that all file segments in the backup data are unique.
The Figure 13-3 diagram shows the backup process when a media server
deduplicates the backups. The destination is a PureDisk Deduplication Pool. A
description follows.
bpdbm
bptm plug-in
Control path
Data path
bpbkar
Client
PureDisk storage pool
The following list describes the backup process when a media server deduplicates
the backups and the destination is a PureDisk Deduplication Pool:
■ The NetBackup Job Manager (nbjm) starts the Backup/Restore Manager (bpbrm)
on a media server.
■ The Backup/Restore Manager starts the bptm process on the media server and
the bpbkar process on the client).
■ The Backup/Archive Manager (bpbkar) generates the backup images and moves
them to the media server bptm process.
The Backup/Archive Manager also sends the information about files within the
image to the Backup/Restore Manager (bpbrm). The Backup/Restore Manager
sends the file information to the bpdbm process on the master server for the
NetBackup database.
■ The bptm process moves the data to the deduplication plug-in.
■ The deduplication plug-in retrieves a list of IDs of the container files from the
NetBackup Deduplication Engine. Those container files contain the fingerprints
from the last full backup for the client. The list is used as a cache so the plug-in
does not have to request each fingerprint from the engine.
■ The deduplication plug-in compares the file fingerprints and the segment
fingerprints against the fingerprint list in its cache.
■ The deduplication plug-in performs file fingerprinting calculations.
■ The deduplication plug-in sends only unique data segments to the PureDisk
Deduplication Pool.
Deduplication architecture 399
MSDP client components
Component Description
The following list describes the backup process for an MSDP client to a Media
Server Deduplication Pool:
■ The NetBackup Job Manager (nbjm) starts the Backup/Restore Manager (bpbrm)
on a media server.
■ The Backup/Restore Manager probes the client to determine if it is configured
and ready for deduplication.
■ If the client is ready, the Backup/Restore Manager starts the following processes:
The OpenStorage proxy server (nbostpxy) on the client and the data moving
processes (bpbkar) on the client and bptm on the media server).
NetBackup uses the proxy plug-in on the media server to route control information
from bptm to nbostpxy.
■ The Backup/Archive Manager (bpbkar) generates the backup images and moves
them to the client nbostpxy process by shared memory.
The Backup/Archive Manager also sends the information about files within the
image to the Backup/Restore Manager (bpbrm). The Backup/Restore Manager
sends the file information to the bpdbm process on the master server for the
NetBackup database.
■ The client nbostpxy process moves the data to the deduplication plug-in.
Deduplication architecture 401
MSDP client–side deduplication backup process
bpdbm
bpbkar Deduplication
Proxy server
(nbostpxy) plug-in
bpbrm Proxy
plug-in
bptm
PureDisk storage pool
Media server
The following list describes the backup process for an MSDP client to a PureDisk
storage pool:
■ The NetBackup Job Manager (nbjm) starts the Backup/Restore Manager (bpbrm)
on a media server.
■ The Backup / Restore Manager probes the client to determine if it is configured
and ready for deduplication.
■ If the client is ready, the Backup/Restore Manager starts the following processes:
The OpenStorage proxy server (nbostpxy) on the client and the data moving
processes (bpbkar on the client and bptm on the media server).
NetBackup uses the proxy plug-in on the media server to route control information
from bptm to nbostpxy.
■ The Backup/Archive Manager (bpbkar) generates the backup images and moves
them to the client nbostpxy process by shared memory.
The Backup/Archive Manager also sends the information about files within the
image to the Backup/Restore Manager (bpbrm). The Backup/Restore Manager
sends the file information to the bpdbm process on the master server for the
NetBackup database.
■ The client nbostpxy process moves the data to the deduplication plug-in.
Deduplication architecture 403
MSDP client–side deduplication backup process
■ The deduplication plug-in retrieves a list of fingerprints from the last full backup
of the client. The list is used as a cache so the plug-in does not have to request
each fingerprint from the engine.
■ The deduplication plug-in performs file fingerprint calculations.
■ The deduplication plug-in sends only unique data segments to the PureDisk
storage pool.
Chapter 14
Troubleshooting
This chapter includes the following topics:
■ Unable to obtain the administrator password to use an AWS EC2 instance that
has a Windows OS
Windows install_path\NetBackup\logs
UNIX /usr/openv/logs
You can access logging controls in Logging host properties. You can also manage
unified logging with the following commands:
vxlogmgr Manages the log files that the products that support unified logging
generate.
UNIX /usr/openv/logs
Windows install_path\NetBackup\logs
STDATE Long Integer or string Provide the start date in seconds or STDATE = 98736352
in the locale-specific short date and
STDATE = '4/26/11 11:01:00
time format. For example, a locale
AM'
can have the format 'mm/dd/yy
hh:mm:ss AM/PM'
ENDATE Long Integer or string Provide the end date in seconds or ENDATE = 99736352
in the locale-specific short date and
ENDATE = '04/27/11 10:01:00
time format. For example, a locale
AM'
can have the format 'mm/dd/yy
hh:mm:ss AM/PM'
1 = WARNING
2 = ERR
3 = CRIT
4 = EMERG
Example Description
(PRODID == 51216) && ((PID == 178964)|| ((STDATE == Retrieves the log file message for the
'2/5/15 09:00:00 AM') && (ENDATE == '2/5/15 NetBackup product ID 51216 between
12:00:00 PM')) 9AM and 12PM on 2015-05-02.
((prodid = 'NBU') && ((stdate >= ‘11/18/14 Retrieves the log messages for the
00:00:00 AM’) && (endate <= ‘12/13/14 12:00:00 PM’))) || NetBackup product NBU between
((prodid = 'BENT') && ((stdate >= ‘12/12/14 00:00:00 2014-18-11 and 2014-13-12 and the log
AM’) && (endate <= ‘12/25/14 12:00:00 PM’))) messages for the NetBackup product
BENT between 2014-12-12 and
2014-25-12.
(STDATE <= ‘04/05/15 0:0:0 AM’) Retrieves the log messages that were
logged on or before 2015-05-04 for all
of the installed Veritas products.
Item Example
Item Example
Display specific Display the log messages for NetBackup (51216) that show only the date, time, message
attributes of the log type, and message text:
messages
vxlogview --prodid 51216 --display D,T,m,x
Display the latest log Display the log messages for originator 116 (nbpem) that were issued during the last 20
messages minutes. Note that you can specify -o nbpem instead of -o 116:
Display the log Display the log messages for nbpem that were issued during the specified time period:
messages from a
specific time period # vxlogview -o nbpem -b "05/03/15 06:51:48 AM"
-e "05/03/15 06:52:48 AM"
Display results faster You can use the -i option to specify an originator for a process:
# vxlogview -i nbpem
The vxlogview -i option searches only the log files that the specified process (nbpem)
creates. By limiting the log files that it has to search, vxlogview returns a result faster. By
comparison, the vxlogview -o option searches all unified log files for the messages that
the specified process has logged.
Note: If you use the -i option with a process that is not a service, vxlogview returns the
message "No log files found." A process that is not a service has no originator ID in the file
name. In this case, use the -o option instead of the -i option.
The -i option displays entries for all OIDs that are part of that process including libraries (137,
156, 309, etc.).
Search for a job ID You can search the logs for a particular job ID:
The jobid= search key should contain no spaces and must be lowercase.
When searching for a job ID, you can use any vxlogview command option. This example
uses the -i option with the name of the process (nbpem). The command returns only the
log entries that contain the job ID. It misses related entries for the job that do not explicitly
contain the jobid=job_ID.
Troubleshooting 409
About legacy logging
Windows install_path\NetBackup\logs
install_path\Volmgr\debug
UNIX /usr/openv/netbackup/logs
/usr/openv/volmgr/debug
It is recommended that you do not use symbolic links or hard links inside legacy
log folders.
If any process runs for a non-root or non-admin user and there is no logging
happening under legacy log folders, you can create a folder using the mklogdir
command for the required user.
To run a command line for a non-root or non-admin user (troubleshooting when the
NetBackup services are not running), it is recommended that you create user folders
for the specific command line. You can create the folders either using the mklogdir
command or manually with the non-root or non-admin user privileges.
To use legacy logging, a log file directory must exist for a process. If the directory
is not created by default, you can use the Logging Assistant or the mklogdir batch
files to create the directories. Or, you can manually create the directories. When
logging is enabled for a process, a log file is created when the process begins.
Each log file grows to a certain size before the NetBackup process closes it and
creates a new log file.
You can use the following batch files to create all of the log directories:
■ Windows: install_path\NetBackup\Logs\mklogdir.bat
■ UNIX: /usr/openv/netbackup/logs/mklogdir
More information
See the NetBackup Commands Reference Guide for a complete description about
the mklogdir command.
and on each media server that you use for your feature. The log files reside in the
following directories:
■ UNIX: /usr/openv/netbackup/logs/
■ Windows: install_path\NetBackup\logs\
More information about NetBackup logging is available in the NetBackup Logging
Reference Guide, available through the following URL:
http://www.veritas.com/docs/DOC5332
To create log directories for NetBackup commands
◆ Depending on the operating system, run one of the following scripts:
UNIX: /usr/openv/netbackup/logs/mklogdir
Windows: install_path\NetBackup\logs\mklogdir.bat
To create the tpconfig command log directory
◆ Depending on the operating system, create the debug directory and the
tpcommand directory (by default, the debug directory and the tpcommand directory
do not exist). The pathnames of the directories are as follows:
UNIX: /usr/openv/volmgr/debug/tpcommand
Windows: install_path\Veritas\Volmgr\debug\tpcommand
Most interaction occurs on the NetBackup media servers. Therefore, the log files
on the media servers that you use for disk operations are of most interest.
Warning: The higher the log level, the greater the affect on NetBackup performance.
Use a log level of 5 (the highest) only when directed to do so by a Veritas
representative. A log level of 5 is for troubleshooting only.
Specify the NetBackup log levels in the Logging host properties on the NetBackup
master server. The log levels for some processes specific to certain options are set
in configuration files as described in Table 14-4.
Backups and restores N/A Messages appear in the log files for the following processes:
■ The bpbrm backup and restore manager. The following is the path to the log
files:
UNIX: /usr/openv/netbackup/logs/bpbrm
Windows: install_path\Veritas\NetBackup\logs\bpbrm
■ The bpdbm database manager. The following is the path to the log files:
UNIX: /usr/openv/netbackup/logs/bpdbm
Windows: install_path\Veritas\NetBackup\logs\bpdbm
■ The bptm tape manager for I/O operations. The following is the path to the
log files:
UNIX: /usr/openv/netbackup/logs/bptm
Windows: install_path\Veritas\NetBackup\logs\bptm
Troubleshooting 412
NetBackup MSDP log files
Catalog shadow copies N/A The MSDP catalog shadow copy process writes messages to the following log
files and directories:
UNIX:
/storage_path/log/spad/spad.log
/storage_path/log/spad/sched_CatalogBackup.log
/storage_path/log/spad/client_name/
Windows:
storage_path\log\spad\spad.log
storage_path\log\spad\sched_CatalogBackup.log
storage_path\log\spad\client_name\
Client deduplication N/A The client deduplication proxy plug-in on the media server runs under bptm,
proxy plug-in bpstsinfo, and bpbrm processes. Examine the log files for those processes
for proxy plug-in activity. The strings proxy or ProxyServer embedded in the
log messages identify proxy server activity.
They write log files to the following directories:
■ For bptm:
UNIX: /usr/openv/netbackup/logs/bptm
Windows: install_path\Veritas\NetBackup\logs\bptm
■ For bpstsinfo:
Windows: /usr/openv/netbackup/logs/admin
UNIX: /usr/openv/netbackup/logs/bpstsinfo
Windows: install_path\Veritas\NetBackup\logs\admin
Windows: install_path\Veritas\NetBackup\logs\stsinfo
■ For bpbrm:
UNIX: /usr/openv/netbackup/logs/bpbrm
Windows: install_path\Veritas\NetBackup\logs\bpbrm
Client deduplication N/A The deduplication proxy server nbostpxy on the client writes messages to files
proxy server in the following directory, as follows:
UNIX: /usr/openv/netbackup/logs/nbostpxy
Windows: install_path\Veritas\NetBackup\logs\nbostpxy.
Troubleshooting 413
NetBackup MSDP log files
Deduplication N/A The following is the path name of the log file for the deduplication configuration
configuration script script:
■ UNIX: storage_path/log/pdde-config.log
■ Windows: storage_path\log\pdde-config.log
NetBackup creates this log file during the configuration process. If your
configuration succeeded, you do not need to examine the log file. The only reason
to look at the log file is if the configuration failed. If the configuration process fails
after it creates and populates the storage directory, this log file identifies when
the configuration failed.
Deduplication plug-in N/A The DEBUGLOG entry and the LOGLEVEL in the pd.conf file determine the log
location and level for the deduplication plug-in. The following are the default
locations for log files:
■ UNIX: /var/log/puredisk/pdplugin.log
■ Windows: C:\pdplugin.log
You can configure the location and name of the log file and the logging level. To
do so, edit the DEBUGLOG entry and the LOGLEVEL entry in the pd.conf file.
Device configuration 178 The Disk Service Manager process that runs in the Enterprise Media Manager
and monitoring (EMM) process.
Device configuration 202 The storage server interface process that runs in the Remote Manager and
and monitoring Monitor Service. RMMS runs on media servers.
Device configuration 230 The Remote Disk Service Manager interface (RDSM) that runs in the Remote
and monitoring Manager and Monitor Service. RMMS runs on media servers.
Troubleshooting 414
NetBackup MSDP log files
drcontrol utility N/A You must run the drcontrol utility on the MSDP storage server host. The
command requires administrator privileges.
The utility creates a log file and displays its pathname in the command output.
The utility writes log files to the following directory, depending on the operating
system:
UNIX:
/[storage_path]/log/drcontrol/policy_admin
/storage_path/log/drcontrol/dedupe_catalog_DR
Windows:
storage_path\log\drcontrol\policy_admin
storage_path\log\drcontrol\dedupe_catalog_DR
Installation N/A The NetBackup installation process writes information about the installation of
the deduplication components to a log file in the following directory:
■ UNIX: /var/log/puredisk
■ Windows: %ALLUSERSPROFILE%\Symantec\NetBackup\InstallLogs
NetBackup N/A The NetBackup Deduplication Engine writes several log files, as follows:
Deduplication Engine
■ Log files in the storage_path/log/spoold directory, as follows:
■ The spoold.log file is the main log file
■ The storaged.log file is for queue processing.
■ A log file for each connection to the engine is stored in a directory in the
storage path spoold directory. The following describes the pathname to
a log file for a connection:
hostname/application/TaskName/MMDDYY.log
For example, the following is an example of a crcontrol connection log
pathname on a Linux system:
/storage_path/log/spoold/server.example.com/crcontrol/Control/010112.log
Usually, the only reason to examine these connection log files is if a Veritas
support representative asks you to.
■ A VxUL log file for the events and errors that NetBackup receives from polling.
The originator ID for the deduplication engine is 364.
Troubleshooting 415
NetBackup MSDP log files
NetBackup 364 The NetBackup Deduplication Engine that runs on the deduplication storage
Deduplication Engine server.
NetBackup N/A The log files are in the /storage_path/log/spad directory, as follows:
Deduplication Manager
■ spad.log
■ sched_QueueProcess.log
■ SchedClass.log
■ A log file for each connection to the manager is stored in a directory in the
storage path spad directory. The following describes the pathname to a log
file for a connection:
hostname/application/TaskName/MMDDYY.log
For example, the following is an example of a bpstsinfo connection log
pathname on a Linux system:
/storage_path/log/spoold/server.example.com/bpstsinfo/spad/010112.log
Usually, the only reason to examine these connection log files is if a Veritas
support representative asks you to.
You can set the log level and retention period in the Change Storage Server
dialog box Properties tab.
Optimized duplication N/A For optimized duplication and Auto Image Replication, The following are the log
and replication files that provide information:
■ The NetBackup bptm tape manager for I/O operations. The following is the
path to the log files:
UNIX: /usr/openv/netbackup/logs/bptm
Windows: install_path\Veritas\NetBackup\logs\bptm
■ The following is the path name of MSDP replication log file:
/storage_path/log/spad/replication.log
Troubleshooting 416
Troubleshooting MSDP installation issues
Resilient network 387 The Remote Network Transport Service (nbrntd) manages resilient network
connections connection sockets. It runs on the master server, on media servers, and on
clients. Use the VxUL originator ID 387 to view information about the socket
connections that NetBackup uses.
Note: If multiple backup streams run concurrently, the Remote Network
Transport Service writes a large amount of information to the log files. In such
a scenario, Veritas recommends that you set the logging level for OID 387 to 2
or less. To configure unified logs, see the following guide:
http://www.veritas.com/docs/DOC5332
Resilient network N/A The deduplication plug-in logs information about keeping the connection alive.
connections
For more information about the deduplication plug-in log file, see “Deduplication
plug-in” in this table.
/net/nbstore/vol/test_data/PDDE_packages/suse/NB_FID2740_LinuxS_x86_
20090713_6.6.0.27209/linuxS_x86/catalog/anb/NB.file_trans: symbol
Troubleshooting 417
Troubleshooting MSDP configuration issues
cat /etc/SuSE-release
SUSE Linux Enterprise Server 10 (x86_64)
VERSION = 10
PATCHLEVEL = 2
Diagnosis The PDDE_initConfig script was invoked, but errors occurred during
the storage initialization.
Second, examine the tpconfig command log file errors about creating
the credentials for the server name. The tpconfig command writes
to the standard NetBackup administrative commands log directory.
Second, restart the NetBackup Administration Console. This step clears cached
information from the failed attempt to display the disk volume.
Examine the disk error logs to determine why the volume was marked DOWN.
If the storage server is busy with jobs, it may not respond to master server disk
polling requests in a timely manner. A busy load balancing server also may cause
this error. Consequently, the query times out and the master server marks the
volume DOWN.
If the error occurs for an optimized duplication job: verify that source storage server
is configured as a load balancing server for the target storage server. Also verify
that the target storage server is configured as a load balancing server for the source
storage server.
See “Viewing MSDP disk errors and events” on page 433.
Then, restart both components. Do not change the values of the other two shared
memory parameters.
The SharedMemoryEnabled parameter is stored in the following file:
storage_path\etc\puredisk\agent.cfg
If the job details also include errors similar to the following, it indicates that an image
clean-up job failed:
This error occurs if a deduplication backup job fails after the job writes part of the
backup to the Media Server Deduplication Pool. NetBackup starts an image
cleanup job, but that job fails because the data necessary to complete the image
clean-up was not written to the Media Server Deduplication Pool.
Deduplication queue processing cleans up the image objects, so you do not need
to take corrective action. However, examine the job logs and the deduplication logs
to determine why the backup job failed.
See “About MSDP queue processing” on page 365.
See “NetBackup MSDP log files” on page 410.
Note: RefDBEngine and refdb do not refer to nor are they related to the open
source RefDB reference database and bibliography tool.
Troubleshooting 423
Troubleshooting MSDP operational issues
To delete the disk pool, you must first delete the image fragments. The nbdelete
command deletes expired image fragments from disk volumes.
To delete the fragments of expired images
◆ Run the following command on the master server:
UNIX: /usr/openv/netbackup/bin/admincmd/nbdelete -allvolumes -force
Windows: install_path\NetBackup\bin\admincmd\nbdelete -allvolumes
-force
The -allvolumes option deletes expired image fragments from all volumes that
contain them.
Troubleshooting 426
Troubleshooting MSDP operational issues
The -force option removes the database entries of the image fragments even if
fragment deletion fails.
2 Cancel the incomplete jobs by running the following command for each backup
ID returned by the previous command (xxxxx represents the backup ID):
UNIX: /usr/openv/netbackup/bin/admincmd/nbstlutil cancel -backupid
xxxxx
■ The Both IPv4 and IPv6 option is enabled for the master server, the media
server that hosts the NetBackup Deduplication Engine, and the client. The Both
IPv4 and IPv6 option is configured in the Network Settings host properties.
■ The IPv6 network is configured as a preferred network for the master server,
the media server that hosts the NetBackup Deduplication Engine, and the client.
The preferred network Match (Above network will be preferred for
communication) property also is enabled. Preferred networks are configured
in the Preferred Networks host properties.
■ The IPv6 network is chosen for the backup.
Examine the bpbrm log file for an error similar to the following:
probe_ost_plugin: sts_get_server_prop_byname failed: error 2060057
If the error message appears, the NetBackup host name cache may not contain
the correct host name mapping information. The cache may be out of sync if DNS
changes in your network environment were not fully propagated throughout your
environment. It takes some amount of time for DNS changes to propagate throughout
a network environment.
To resolve the problem, do the following on the NetBackup master server and on
the MSDP storage server:
1. Stop the NetBackup services.
2. Run the following command:
UNIX: /usr/openv/netbackup/bin/bpclntcmd -clearhostcache
Windows: install_path\NetBackup\bin\bpclntcmd.exe -clearhostcache
3. Start the NetBackup services.
For more information about client deduplication logging, see the description of
“Client deduplication proxy plug-in” in the “MSDP log files” topic.
See “NetBackup MSDP log files” on page 410.
The messages may indicate a client name case sensitivity issue in your MSDP
environment. For a description of the problem and the procedures to resolve it, see
the following Veritas tech note:
http://www.veritas.com/docs/TECH207194
The NetBackup Deduplication Engine Examine the Disk Logs report for errors that include
(spoold) was too busy to respond. the name PureDisk. Examine the disk monitoring
services log files for details from the deduplication
plug-in.
A user tampered with the storage. Users must not add files to, change files on, delete
files from, or change file permissions on the
storage. If a file was added, remove it.
Storage capacity was increased. If you grew the storage, you must restart the
NetBackup services on the storage server so the
new capacity is recognized.
Firewall ports are not open. Ensure that ports 10082 and 10102 are open in
any firewalls between the deduplication hosts.
A duplication job to PureDisk storage may fail for a large number of small files. This
failure may occur if the storage unit maximum fragment is at the default setting.
Messages similar to the following may appear in the NetBackup job details:
Messages similar to the following also may appear in a log file on the PureDisk
content router that processed the duplication operation. The log file resides in a
directory that corresponds to the connection to the content router for the duplication
operation. The name of the directory is the NetBackup storage server IP address,
and it resides in the /Storage/log/spoold directory on the content router.
To resolve this problem, reduce the fragment size of the NetBackup storage unit
that represents the PureDisk storage. Depending on the source storage content,
you may have to reduce the fragment size significantly.
Operation Activity Monitor job details Status in bpdm and bptm log
files
The message may indicate a client name case sensitivity issue in your MSDP
environment. For a description of the problem and the procedures to resolve it, see
the following Veritas tech note:
http://www.veritas.com/docs/TECH207194
See “About MSDP storage capacity and usage reporting” on page 333.
See “About MSDP container files” on page 335.
See “Viewing storage usage within MSDP container files” on page 335.
Troubleshooting 432
Troubleshooting MSDP operational issues
Examining the disk log reports for threshold warnings can give you an idea of when
a storage full condition may occur.
How NetBackup performs maintenance can affect when storage is freed up for use.
See “About MSDP queue processing” on page 365.
See “About the MSDP data removal process” on page 374.
Although not advised, you can reclaim free space manually.
See “Processing the MSDP transaction queue manually” on page 365.
Catalog backup
Table 14-8 describes error messages that may occur when you create or update
a catalog backup policy. The messages are displayed in the shell window in which
you ran the drcontrol utility. The utility also writes the messages to its log file.
Code or Description
message
1 Fatal error in an operating system or deduplication command that the drcontrol utility calls.
110 The command cannot find the necessary NetBackup configuration information.
140 The user who invoked the command does not have administrator privileges.
227 This error code is passed from the NetBackup bplist command. The MSDP catalog backup policy
you specified does not exist or no backups exist for the given policy name.
For more information about status codes and error messages, see the following:
■ The Troubleshooter in the NetBackup Administration Console.
■ The NetBackup Status Codes Reference Guide available through the following
webpage:
http://www.veritas.com/docs/DOC5332
Troubleshooting 433
Viewing MSDP disk errors and events
1009 8 Authorization Authorization request from <IP> for user <USER> denied
(<REASON>).
1015 1 Critical Task creation failed, could not initialize task class on
server PureDisk:server1.example.com on host
server1.example.com.
1044 multiple multiple The usage of one or more system resources has exceeded a
warning level. Operations will or could be suspended.
Please take action immediately to remedy this situation.
1057 A data corruption has been detected. The data consistency check
detected a data loss or data corruption in the Media Server Deduplication Pool
(MSDP) and reported the affected backups.
The backup ID and policy name appear in the NetBackup Disk Logs report
and the storage_path/log/spoold/storaged.log file on the storage
server.
■ /usr/openv/netbackup/logs/esfs_filesystem
■ /usr/openv/netbackup/logs/esfs_monitor
■ /usr/openv/netbackup/logs/esfs_storage
■ /usr/openv/netbackup/logs/esfs_storagemanager
Diagnosis One possible cause is that cache eviction cannot clean up any
data from the local cache directory for non-MSDP system file writes.
Since the upload speed to the cloud is slower than the hard disk
drive write speed, this situation can arise when the partition for the
local cache directory is small. Increase the space that is allowed
for the local cache directory.
Make sure the SSL (Secure Sockets Layer) is configured properly on the private
cloud server.
NetBackup supports only Certificate Authority (CA)-signed certificates while it
communicates with cloud storage in the SSL mode. Ensure that the cloud server
has CA-signed certificate. If it does not have the CA-signed certificate, data transfer
between NetBackup and the cloud provider fails in the SSL mode.
The NetBackup Cloud Administrator's Guide contains information about SSL mode
in private clouds:
http://www.veritas.com/docs/DOC5332
Error Reason
/usr/openv/esfs/log/ops/esfs-config.log.*
An error occurred in ESFS initialization This error may occur if the esfs_init.sh script is run without all of
the required parameters.
/usr/openv/esfs/log/ops/esfs-config.log.*
FUSE not installed NetBackup installs the Filesystem in Userspace (FUSE) upon installation.
If the FUSE version is too old, this message appears.
Unsupported ESFS operating system A Cloud Catalyst storage server can be configured on a media server
only if the host is running Red Hat Enterprise Linux 7.3 or later.
http://www.netbackup.com/compatibility
The Cloud Catalyst service could not be Check that the vxesfsd daemon has sufficient memory to run.
started
Check the following log for more specific information:
/usr/openv/esfs/log/ops/esfs-config.log. *
Troubleshooting 442
Troubleshooting Cloud Catalyst issues
Error Reason
An ESFS mount could not be found This error indicates that something is preventing the NetBackup
Extendable Storage File System (ESFS) from starting properly. There
may also be a problem with the vxesfsd process.
/usr/openv/esfs/log/ops/esfs-config.log. *
See “Configuring source control or target control optimized duplication for Cloud
Catalyst” on page 280.
This error is more likely to occur when a relatively small disk is used for
demonstration or proof-of-concept testing. To prevent this error in a production
environment, make sure that there is at least 4 TB of disk space available for the
cache. Make sure that the cache is located on a dedicated file system that is used
only for the Cloud Catalyst local cache.
After receiving a media write error (84) and addressing the cause, restart all
NetBackup services to avoid a media open error (83).
The UNIX df command can be used to determine the disk space in the local cache
directory. For example:
df /esfs
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/loop0 1007896 1007896 0 100% /esfs
See the following topic for more information about Cloud Catalyst local cache
considerations:
See “About the Cloud Catalyst cache” on page 278.
/usr/openv/netbackup/bin/nbsvcmon -terminate
/usr/openv/pdde/pdconfigure/pdde stop
/usr/openv/esfs/bin/vxesfsd stop
/usr/openv/esfs/bin/vxesfsd start
/usr/openv/pdde/pdconfigure/pdde start
/usr/openv/netbackup/bin/nbsvcmon &
Troubleshooting 446
Troubleshooting Cloud Catalyst issues
/etc/init.d/NetBackup stop
/etc/init.d/NetBackup start
Note that after the hard kill of vxesfsd, the target remains busy.
To verify if the ESFS mount point is used by any processes
1 Unmount the mount directory:
umount mount_dir
Or
lsof mount_dir
Or
lsof | grep mount_dir
3 Restart vxesfsd :
/usr/openv/esfs/bin/vxesfsd start
/usr/openv/pdde/pdconfigure/pdde start
This command adds a new legacy cloud service or connector to Tomcat server's
server.xml file. Restart the nbwmc service.
esfs_cleanup.sh script
In certain conditions, use the esfs_cleanup.sh script on the Cloud Catalyst storage
server to remove all Cloud Catalyst-related information from the host.
For example, when Cloud Catalyst cloud storage is decommissioned. (See
“Decommissioning Cloud Catalyst cloud storage” on page 286.)
The script is located in the following directory:
/usr/openv/esfs/scripts/esfs_cleanup.sh
Note: Use this script with care as it deletes all data, data loss occurs if used
incorrectly. Contact Veritas Technical Support for assistance.
Note: Since the tool reads data from the cloud, it is likely that the NetBackup
customer might incur a financial cost by the cloud vendor. Consider contacting
Veritas Support before running the tool.
Where:
■ path_to_local_cache is the path to the local cache directory.
Troubleshooting 448
Troubleshooting Cloud Catalyst issues
■ The -strict option provides an additional check to compare the file size in the
local cache with the file size in the ESFS database.
■ The -verbose option enables logging to the following directory:
/usr/openv/esfs/log/ops/esfs_check
Output line 1 means that file /data/2/97.bin appears in the ESFS database and
should also be in the cloud, but it is not.
Output line 2 means that file /data/2/97.bin appears in the ESFS database, but
is neither in the cloud nor the local cache.
If the esfs_check tool finds an inconsistency, contact Veritas Support.
setlsu_ioctl tool
This tool is used in a disaster recovery scenario to set the cloud bucket name.
The setlsu_ioctl tool is located in the following directory: /usr/openv/esfs/bin
It uses the following syntax:
./setlsu_ioctl /cache_directory/storage/proc/cloud.lsu name_of bucket
Troubleshooting 449
Unable to obtain the administrator password to use an AWS EC2 instance that has a Windows OS
cred_ioctl tool
This tool is used in a disaster recovery scenario to set the cloud credentials.
The cred_ioctl tool is located in the following directory: /usr/openv/esfs/bin
The cred_ioctl uses this syntax:./cred_ioctl [-update] <mountPath>
<cloud_username> <cloud_password>
■ Check if the NetBackup certificate is deployed to the media server that is used
to configure the OpenStorage server. When certificate is not correctly deployed,
the following error logs can be found in pdplugin log:
If there is an error log, the issue is that different NetBackup domains use the
same MSDP user to access one MSDP storage server that is not supported by
multi-domain.
For Windows:
<install_path>\Veritas\pdde\crcontrol.exe --taskstat
2. Check the client column for the list of clients that belong to the NetBackup
domain, identify the work load of the clients from one domain, and then work
load of one domain.
3. Run the bpplclients command on one NetBackup domain to list all clients
of that domain.
Appendix A
Migrating to MSDP
storage
This appendix includes the following topics:
NetBackup cannot use the storage hardware while PureDisk uses it for storage.
The structure of the PureDisk storage is different than the structure of the storage
for integrated NetBackup deduplication. The disk systems cannot be used
simultaneously by both NetBackup and PureDisk. The PureDisk images on the
storage cannot be transferred to the deduplication storage server storage.
Therefore, to replace a NetBackup 5000 series appliance with the NetBackup Media
Server Deduplication Option, migrate the storage to a NetBackup Media Server
Deduplication Pool.
Step 1 Install and configure See the NetBackup Installation Guide for UNIX and
NetBackup Windows:
http://www.veritas.com/docs/DOC5332
Migrating to MSDP storage 453
Migrating from another storage type to MSDP
Step 3 Redirect your backup jobs Redirect your backup jobs to the NetBackup Media
Server Deduplication Pool.
http://www.veritas.com/docs/DOC5332
Step 4 Decommission your appliance Optionally, after the backup images expire on your
NetBackup 5000 series appliance, decommission
your NetBackup 5000 series appliance.
Step 2 Redirect your backup Redirect your backup jobs to the media server
jobs deduplication pool storage unit. To do so, change the
backup policy storage destination to the storage unit for
the deduplication pool.
http://www.veritas.com/docs/DOC5332
Step 3 Repurpose the storage After all of the backup images that are associated with
the storage expire, repurpose that storage.
N
M nbstserv process 194
maintenance processing. See queue processing NDMP
master servers storage units 199
inter-node authentication for clustering 181 NetBackup
Maximum concurrent jobs 124 naming conventions 18
Maximum fragment size 123 NetBackup 5200 series appliance
Media open error (83) 444 as a storage destination 20
media server deduplication NetBackup 5300 series appliance
process 396 as a storage destination 20
Index 460
T
Targeted A.I.R. 181
throttling, configuring for Cloud Catalyst 438
topology of storage 164–165
troubleshooting
database system error 417
deduplication backup jobs fail 420
deduplication processes do not start 420
general operational problems 424
host name lookup 419
installation fails on Linux 416
no volume appears in disk pool wizard 419
server not found error 418
trusted master servers
adding 173
for Auto Image Replication 167
removing 179
U
unified logging 404
format of files 405
uninstalling media server deduplication 391
UploadThreads Cloud Catalyst setting 277
UTC (Coordinated Universal Time) 160
V
Validate Across Backup Policies button in SLP 199