Recommendation for sizing the catalog

Article ID: HOWTO56093 | Created: 2011-07-25 | Updated: 2012-01-24 How To for NetBackup Enterprise Server 7.0 7.1

Recommendation for sizing the catalog

A little background information is in order. The term catalog refers to all of the following components:

The image database, which contains information about what has been backed up. It is by far the largest part of the catalog (more than 90%). NetBackup data in relational databases. NetBackup configuration files.

See About the NetBackup catalog. The size of the NetBackup catalog depends on the number of files in the backups and the number of copies of the backups that are retained. As a result, the catalog has the potential to grow quite large. You should consider two additional factors, however, when estimating the ultimate size of the catalog: can it be backed up in an acceptable time window, and can the general housekeeping activities complete within their execution windows. The time that is required to complete a catalog backup depends on the amount of space it occupies. The time that is required for the housekeeping activities depends on the number of entries it contains. Note that NetBackup's catalog archiving feature can be used to move older catalog data to other disk or tape storage. Archiving can reduce the size of the catalog on disk and thus reduce the backup time. Archiving, however, does not decrease the amount of time that is required for housekeeping activities. Symantec recommends that you plan your environment to meet the following criteria for the catalog:

The amount of data that is held in the online catalog should not exceed 750 GB. Archiving can be used to keep the online portion of the catalog below this value. The total number of catalog entries should not exceed 2,000,000. This number equals the total of all retained copies of all backups from all clients held both online and in the catalog archive.

Table: Guidelines for catalog size (these guidelines are not hard limits)

Catalog guideline

Notes on the guidelines

Potential consequences of exceeding the guidelines

Catalog guideline

Notes on the guidelines

Potential consequences of exceeding the guidelines

Online catalog should not contain more than 750 GB of data

Use archiving to keep the online portion of the catalog below 750 GB. The reasons for the 750 GB guideline relate to catalog backup and recovery times:

The potential results of exceeding 750 GB are the following:

A catalog backup is not a simple flatfile backup:

o

The time that is required to back up the catalog could become excessive. Using incremental backups can reduce backup time but increase recovery time. Consider creating a service level agreement (SLA) for recovery. Replication can help in most cases but a catalog backup is still required for insurance or fallback purposes.

A catalog backup is based on a synchronization check point and requires extra time to process the work list. Catalog backup uses locking to allow other NetBackup jobs to be active concurrently (extra processing time is required).

No snapshot mechanism exists for catalog backup. Catalog backups typically run at around 30 MB/second. It takes about 7 hours to back up 750 GB. (Restore rates are faster because of less processing overhead).

The total number of catalog entries should not exceed 2,000,000

Reasons for the 2,000,000-entry guideline:

Cleanup jobs run every 12 hours.

o

The potential results of too many catalog entries are the following:

Cleanup jobs check the information in each image metadata file. Processing 30 to 50 records a second, cleanup jobs need between 5 and 9 hours to complete the run if the catalog contains 1,000,000 entries

If the image count grows to such an extent that the cleanup job takes more than 24 hours to complete, the next cleanup job is skipped. If the next cleanup

Catalog guideline

Notes on the guidelines

Potential consequences of exceeding the guidelines

(longer if 2,000,000).
o

If the jobs fail to complete in 12 hours, the next run of each job is cancelled and the run interval extends to 24 hours. A cleanup job frequency of once in 24 hour usually has a negligible effect on overall operation.

job is skipped, the recycling of expired media is delayed. As a result, more scratch media are required. Note: In general, it is safe to miss a few cleanup jobs; image counts of two to three million may not cause serious problems.

The image cleanup does the following:

o

Removes expired images from the catalog. Reclaims the disk space on disk storage devices. Prunes TIR information, and optionally compresses catalog files.

The media cleanup deassigns empty tapes and returns them to the scratch pool.

Note that the actual limits of acceptable catalog performance are influenced by the speed of the storage and the power of the server. Your performance may vary significantly from the guideline figures provided in this section.

Note:

If you expect your catalog to exceed these limits, consider deploying multiple NetBackup domains in your environment.

The following Symantec document describes best practices for the catalog layout: http://www.symantec.com/docs/TECH144969 More information on catalog archiving is available in the NetBackup Administrator's Guide, Volume I. See Designing your backup system

See How to calculate the size of your NetBackup image database See How to calculate the size of the NetBackup relational database (NBDB)

Legacy ID