DB2 9.7 For LUW New Feature - Index Compression

DB2 9.
7 f or LUW New Feat ur e:

I ndex c ompr essi on
Applies to:
All SAP releases running on DB2 Version 9.7for Linux, UNIX and Windows Version.
For more information, visit the Landscape Design and Architecture homepage.
Summary
This article describes the new feature of index compression available with DB2 9.7 and how to use it an SAP
environment. This new feature provides additional storage savings by compressing table indexes. A new tool
provided by SAP to evaluate and compress the indexes is also explained in this article.

Author: Marianne Nesiem
Company: IBM Canada
Created on: 28 J uly 2009
Author Bio
Marianne Nesiem is a member of the IBM SAP Integration and Support Centre at the IBM
Toronto Lab. Her current activities include testing and certification of SAP R/3 with every new
release of DB2 UDB and providing assistance to customers running SAP in a DB2 UDB
environment with problem analysis and troubleshooting. She is also a customer advocate,
providing custom-tailored support for large customer accounts running SAP and DB2. She has
been the sole author of multiple White papers on the topic of SAP and DB2 UDB and has been involved in
many customer briefings and presentations and has chaired multiple workshops on the same topic. She is a
DB2 Certified Solutions Expert and a UNIX systems administrator, and has experience in application
development on J ava for databases and other applications.
SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BOC - boc.sap.com
2009 SAP AG 1
DB2 9.7 for LUW New Feature: Index compression
Table of Contents
Introduction ......................................................................................................................................................... 3
How Does Index Compression Work .................................................................................................................. 3
Compression Algorithms ................................................................................................................................. 3
Variable Slot Directory ................................................................................................................................................. 3
RID List Compression .................................................................................................................................................. 4
Prefix Compression ...................................................................................................................................................... 6
How to Enable Index Compression .................................................................................................................... 9
Verify the State of the Indexes ........................................................................................................................ 9
ADMIN_GET_INDEX_INFO ......................................................................................................................................... 9
ADMIN_GET_INDEX_COMPRESS_INFO ................................................................................................................ 11
How to Activate Index Compression ............................................................................................................. 13
Activating Index Compression for New Indexes: ........................................................................................................ 13
Activating Index Compression for Existing Indexes: .................................................................................................. 13
Compressing Indexes ................................................................................................................................... 15
Index Compression Tool ................................................................................................................................... 17
Enabling Tables for Data and Index Compression ....................................................................................... 17
Enabling Index Compression: ....................................................................................................................... 21
Restrictions ....................................................................................................................................................... 24
Case Study ....................................................................................................................................................... 24
Related Content ................................................................................................................................................ 25
Disclaimer and Liability Notice .......................................................................................................................... 26
2009 SAP AG 2
Introduction
More and more organizations are facing increasing storage cost as more data is being generated and more
existing data is being retained for longer periods of time. Since more disk space is consumed, the need for
innovations to reduce storage and total cost is becoming crucial for business success.
Data compression was introduced in DB2 V9.1 to allow table data to be compressed and reduce storage
space. Customers have seen great success using data compression and many have achieved very high
compression rates for some of their tables.
In many database environments, it is common to have many very large indexes that are defined on a large
table. Similar to data objects, index objects can now be compressed to further reduce disk consumption and
storage cost with minimal effect on the system performance for most cases.
How Does Index Compression Work
To compress index objects, multiple compression algorithms can be used. The database manager chooses
the best compression algorithms based on the data layout to minimize the storage space.
Contrary to data compression, no dictionary is required to compress indexes as the database manager uses
fixed algorithms.
Only data in leaf pages is compressed to reduce CPU and I/O cost to compress, uncompress and fetch the
data. A major benefit of compressing leaf pages is that there are less frequent page splits and smaller index
levels. Because the number of index levels is reduced, an index search starting from the root page requires
fewer I/O corresponding to the number of index levels. Similarly an index scan requires fewer I/O
corresponding to the total number of leaf pages. Furthermore, more index keys can be cached in the buffer
pool. The buffer pool hit ratios increase and the index page physical reads and writes decrease. On the other
hand, since the index keys are now compressed, extra CPU is needed to uncompress the index keys back to
the uncompressed format before returning the result back to the users, or to compress the index key to the
compressed format before updating an index page.
The degree of compression achieved will vary based on the type of index, as well as the data the index
contains. For example, the database manager can compress an index with a large number of duplicate keys
by storing an abbreviated format of the record identifier (RID) for the duplicate keys. In an index where there
is a high degree of commonality in the prefixes of the index keys, the database manager can apply
compression based on the similarities in prefixes of index keys.
The following section describes the different compression algorithms being used.
Compression Algorithms
If index compression is enabled, DB2 considers multiple compression algorithms for index pages:
Variable Slot Directory
Each index page begins with a page header, followed by a slot directory. Each two byte entry in the slot
directory corresponds to a different index key on the page. The entry itself is the byte-offset into the index
page where the index key begins.

2009 SAP AG 3
In previous releases, a slot directory was pre-allocated and fixed based on the minimum index key size.
Normally the slot directory occupies only a little space. However, if an index contains many duplicates or a
variable length key part where the actual size of the values are longer than its minimum size, much of the
pre-allocated slot directory will not be used. Some slots may have been unused and the unused slot directory
space may prevent the insertion of new records on the index page. In DB2 V97, a slot directory is
dynamically adjusted in order to fit as many index keys into an index page as possible. A slot directory will
not be fixed or pre-allocated in a compressed index; as a result, every byte on an index page is used wisely.

RID List Compression
An index key contains a key value and a key data list. A key data contains a Record ID (RID) and its flags.
DB2 keeps only the RIDs for each duplicate entry in an index page. The RIDs in a RID list are sorted. For
indexes on tables in LARGE and TEMPORARY table spaces, the size of a RID is 8 bytes for partitioned
tables and 6 bytes for other tables. For indexes on tables in REGULAR tablespaces, the size of a RID is 6
bytes for partitioned tables and 4 bytes for other tables. Instead of saving the full version of a RID, we can
save some space by storing the delta between two RIDs. The delta is further compressed if some encoding
algorithms are applied. The encoded delta can be as small as 1 byte. If the indexes have low cardinality and
a high cluster ratio, the savings will be significant.

Example:
If we have 6 records stored on page 4 for a regular table in a LARGE tablespace.
In an uncompressed index, DB2 saves the following RIDs:

2009 SAP AG 4

<00 00 00 04, 00 00>,
<00 00 00 04, 00 01>,
<00 00 00 04, 00 02>,
<00 00 00 04, 00 03>,
<00 00 00 04, 00 04>,
<00 00 00 04, 00 05>
<00 00 00 04, 00 06>

In a compressed index, DB2 compresses the RID list:

<00 00 00 04, 00 00>,
<1>,
<1>,
<1>,
<1>,
<1>,
<1>

2009 SAP AG 5
The following image shows a comparison of the uncompressed and compressed index page:

Prefix Compression
Because the key values are sorted on index pages, very often the key values of two adjacent keys are very
similar on an index with many different entries. It is common for two adjacent keys to share the same
prefixes. During index creation or insertion, DB2 compares the new key with adjacent index keys and finds
the longest common prefixes between them. If the page is full, DB2 tries to find the optimal prefixes to use
for each page by constructing a representation of the prefixes using some Meta data structure. Each prefix is
then compared with neighboring keys to find an optimal prefix that uses less space and update the Meta data
accordingly.
If the cardinality of the initial portion of the index keys is low relative to the number of index records, the
prefix compression savings will be significant.

2009 SAP AG 6
Example:
If we have the following uncompressed index:

(a, a, a),
(a, a, b),
(a, a, c),
(b, b, b),
(b, b, c)

DB2 keep (a, a) and (b, b) as the common prefixes in a compressed index. In addition, DB2 knows the
first three keys use prefix (a, a) and will keep ( , , a), ( , , b), ( , , c). Similarly, DB2 knows the fourth and
fifth keys use prefix (b, b) and only keeps ( , , b) and ( , , c) for the fourth and fifth keys.
After compression, the index will look similar to the following:

(a, a, a),
( , , b),
( , , c),
(b, b, b),
( , , c)

2009 SAP AG 7
The following image shows how the page looks after prefix compression:

DB2 uses common prefix compression and RID list compression for index leaf pages, which represent the
majority of the index space. In some cases, the database manager might choose to use more than one
algorithm at the same time to achieve the highest compression rate.
2009 SAP AG 8
How to Enable Index Compression
By default, new indexes that were created on row-based compressed tables are compressed. If a table is
created or altered with the COMPRESS attribute set to Y, both data and indexes on that table are set for
compression and any new indexes created on that table are compressed by default. At that point, it is not
required to explicitly compress the indexes upon creation.
As of DB2 9.7, you can now also compress temporary tables, hence indexes on temporary tables can also
be compressed.
Verify the State of the Indexes
There are two new table functions that you can use to retrieve more detailed information on indexes as
described in the following sections.
ADMIN_GET_INDEX_INFO
This new table function displays detailed information about the indexes, such as the compression information
as well as the logical and physical size of the indexes.

Syntax
>>-ADMIN_GET_INDEX_INFO--(--objecttype--, --objectschema--, --objectname--)-><

Where:
Objecttype is T for table and I for index.
Objectschema is the name of the schema where the object belongs
Objectname is the name of the object
ADMIN_GET_INDEX_INFO returns, for example, the following important values:

Value Description
COMPRESS_ATTR The following values are returned:
Y
If index compression is enabled
N
If index compression is not enabled
INDEX_COMPRESSED The following values are returned:
Y
If the index is compressed
N
If the index is not compressed
INDEX_OBJ ECT_L_SIZE Returns the logical size of the index object in KB. Note that
the same size (which is equivalent to the total size) will be
returned for all indexes
2009 SAP AG 9

INDEX_OBJ ECT_P_SIZE Returns the physical size of the index object in KB. Note that
the same size (which is equivalent to the total size) will be
returned for all indexes
Example:

% db2 " select indname , compress_attr, index_compressed, index_object_p_size from table
(admin_get_index_info ('T', 'SAPLRP', 'MARA')) as t"
INDNAME COMPRESS_ATTR INDEX_COMPRESSED INDEX_OBJ ECT_P_SIZE
-----------------------------------------------------------------------------------------------------------------
MARA~0 Y N 5344
MARA~A Y N 5344
MARA~L Y N 5344
MARA~O Y N 5344
MARA~PMA Y N 5344
MARA~T Y N 5344
6 record(s) selected.

The previous example shows that all the indexes belonging to table MARA are set for compression but have
not yet been compressed. Note that INDEX_OBJ ECT_P_SIZE is identical for all indexes as it represents the
total size of all the indexes on that table.
2009 SAP AG 10
ADMIN_GET_INDEX_COMPRESS_INFO
This new function returns information that is specific to index compression, such as the compression savings
estimate (before the index is compressed) and the index compression statistics (after the index is
compressed) from the system catalog table.

Syntax
--ADMIN_GET_INDEX_COMPRESS_INFO--(--objecttype--,-- objectschema--,--objectname--,--
dbpartitionnum--,--datapartitionid--)

Where:
Objecttype is T for table and I for index.
Objectschema is the name of the schema where the object belongs
Objectname is the name of the object
Dbpartitionnum is the database partition number. To specify that the data is requested for all partition
use the value -2. For a non-partitioned system, use the value 0.
Datapartitionid is the database partition ID. To specify that the data is requested for all partition, use
the value -2. For a non-partitioned system, use the value 0.

ADMIN_GET_INDEX_COMPRESS_INFO returns, for example, the following important values:
Value Description
COMPRESS_ATTR The following values are returned:
Y
If index compression is enabled
N
If index compression is not enabled

INDEX_COMPRESSED The following values are returned:
Y
If the index is compressed
N
If the index is not compressed
PCT_PAGES_SAVED Before an index is compressed, it provides an
estimated percentage of leaf pages saved based on
the existing index keys. After index compression, this
parameter provides the percentage of savings after
compression.
To get an accurate value for PC_PAGES_SAVED,
you should perform an RUNSTATS on that table. If
the information is unavailable or if the statistics on the
table need to be refreshed, this parameter returns -1.

2009 SAP AG 11
If this function is executed in a newly upgraded database the value for COMPRESS_ATTR as well as
INDEX_COMPRESSED will be N by default and the value for PCT_PAGES_SAVED will be -1 until the
runstats command is executed on the table.

Example:

% db2 select INDNAME, COMPRESS_ATTR, INDEX_COMPRESSED, PCT_PAGES_SAVED from table (
admin_get_index_compress_info ('T', 'SAPLRP', 'MARA', 0, 0)) as t

INDNAME COMPRESS_ATTR INDEX_COMPRESSED PCT_PAGES_SAVED
---------------------------------------------------------------------------------------------------------------
MARA~0 N N 31
MARA~A N N 71
MARA~L N N 57
MARA~O N N 71
MARA~PMA N N 71
MARA~T N N 57

Note that in this example, all existing indexes do not have index compression enabled. The value of
PCT_PAGES_SAVED, which shows the estimated total savings after compression, is displayed per index in
this case.

2009 SAP AG 12
How to Activate Index Compression
You can explicitly activate or deactivate index compression on a table using the COMPRESS clause of the
CREATE INDEX and ALTER INDEX statements
Activating Index Compression for New Indexes:
To activate compression on a new index, use the CREATE INDEX SQL statement:

Syntax

>>-CREATE INDEX--index-name -----------------------------------
...
>--+----------------------------------+------------------------->
'--------COMPRESS------+-NO -+-----'
'-YES-'
...

Example:

% db2 create index MARA~6 compress yes

The value of the COMPRESS attribute specifies whether index compression is enabled. To activate index
compression on a new index, the value is set to Yes- To disable index compression for a new index, the
value is set to No. Note that if the index is created in a table where data compression is enable there is no
need to set the compression attribute, as it is set to YES by default as explained earlier.
Activating Index Compression for Existing Indexes:
For existing indexes, you can use the ALTER INDEX SQL statement as follows:
Syntax
>>-ALTER INDEX--index-name ------------------------------------ >>

>--COMPRESS--+-NO -+------------------------------------------| >
'-YES-'

Example:

% db2 alter index MARA~0 compress yes

To activate or deactivate index compression on an existing index , the value of the COMPRESS attribute
again is set to YES or NO.

2009 SAP AG 13
Another method to modify existing indexes other than using the ALTER INDEX SQL statement is to drop the
index and re-create it. In addition, based on the state of the table, the index is created as compressed or
uncompressed. Both methods are as effective and use approximately the same amount of time to complete.

To activate compression on multiple indexes, you can use a script similar to the following example:

Example:
To activate index compression for all indexes belonging to table MARA, issue the following command:
% db2 -x "select 'alter index ' || CHR(34) || RTRIM(tabschema) || CHR(34) || '.' || CHR(34) || INDNAME ||
CHR(34) || ' compress yes;' from syscat.indexes where tabname=MARA" > alter_indexes_mara.clp

A CLP script is generated that you can execute as follows:
% db2 tvf alter_indexes_mara.clp

To verify that index compression has been activated, you can also use the admin function
ADMIN_GET_INDEX_COMPRESS_INFO.

Example:

db2 => select INDNAME, COMPRESS_ATTR, INDEX_COMPRESSED, PCT_PAGES_SAVED from table (
admin_get_index_compress_info ('T', 'SAPLR1', 'MARA', 0, 0)) as t
---------------------------------------------------------------------------------------------------------------
MARA~0 Y N 31
MARA~A Y N 71
MARA~L Y N 57
MARA~O Y N 71
MARA~PMA Y N 71
MARA~T Y N 57


Note that in this example the COMPRESS_ATTR value has been changed to Y. The value of
INDEX_COMPRESSED remains N until a table or index reorganization is performed.
2009 SAP AG 14
Compressing Indexes
Once you activated compression using one of the previous methods, the indexes are ready to be
compressed. To compress the indexes, the REORG command is used. The reorganization can performed
online of offline.

Syntax

>>-REORG
>----TABLE --- table-name --| INDEX--index-name ----------------------------------- >>
>----INDEXES ALL FOR TABLE table-name ----------------------------- >>

Examples:

% db2 reorg table MARA index MARA~T

In the following example, only index MARA~T is compressed:
% db2 reorg indexes all for table MARA

In the following example, all the indexes belonging to the table MARA are compressed:
% db2 reorg indexes all for table MARA inplace

In this example, all indexes in table MARA are compressed while allowing user access.
If you want to compress both data and indexes at the same time, you can run the REORG TABLE command
(after setting the compression attribute to Y on both indexes and data). Note that in this case, the REORG
operation must be performed offline.
If you want to drop and re-create the indexes as compressed, you do not have to perform the REORG
operation.
Example:

% db2 reorg table MARA

To run the REORG command on multiple tables, you can use a script similar to the example used in the
previous section.
Example:
To compress all the indexes in your database, issue the following command:
% db2 -x "select 'reorg indexes all for table ' || CHR(34) || RTRIM(tabschema) || CHR(34) || '.' || CHR(34) ||
TABNAME || CHR(34) || ';' from syscat.tables" > reorg_indexes.clp

2009 SAP AG 15
A CLP script is generated that you can be execute as follows:
% db2 tvf reorg_indexes.clp

To verify that the indexes have been compressed, use the same admin function,
ADMIN_GET_INDEX_COMPRESS_INFO. To receive an accurate value for PCT_PAGES_SAVED, you
have to perform an RUNSTATS.
Example:

db2 => select INDNAME, COMPRESS_ATTR, INDEX_COMPRESSED, PCT_PAGES_SAVED from table (
admin_get_index_compress_info ('T', 'SAPLR1', 'MARA', 0, 0)) as t
---------------------------------------------------------------------------------------------------------------
MARA~0 Y Y 31
MARA~A Y Y 71
MARA~L Y Y 57
MARA~O Y Y 71
MARA~PMA Y Y 71
MARA~T Y Y 57

Note: To deactivate index compression, you can either re-create the indexes without compression or alter the indexes
using the following SQL statement:
ALTER INDEX --- index-name COMPRESS NO
Afterwards, you should perform an index REORG to disable index compression.

2009 SAP AG 16
Index Compression Tool
The ABAP-based row compression tool has been extended by SAP to enable index compression. Using
these additions, you can identify the indexes that are suitable for compression and have these indexes
compressed online. The row compression tool is available as a transport and you can install it using
transaction STMS or using the tp command for Unicode systems.
For the purpose of this article, we shall only discuss the new additions to the tool. For more information about
the row compression tool, see SAP Note 980067.
This addition is available with DB2 LUW 9.7 only and will not be visible if you run the tool on a previous DB2
version. Full index compression support is also planned for the next update of the DBA Cockpit.
Enabling Tables for Data and Index Compression
1. In your SAP system, call transaction SE38 and enter /ISIS/ZCOMP for the program name.
The Enabling Tables for Row Compression screen appears:

The following tab pages are available::
2009 SAP AG 17
Tab Page Description.
Candidates Lists all the tables that are suitable for data and index compression
Indexes Lists all the tables that have their data compressed (or enabled for
compression) but not the indexes
Compressed Lists all the tables that have both data and indexes compressed
Exception Lists all the tables that you do not want to be compressed. To
remove a table from that list, select the respective row and choose
Delete from Exception.
Export Includes a list of tables that have been compressed and a sample
has been generated to be used for migration and system copy. For
more information, please review the SAP system copy guide for the
required release.
By default, the candidates that are suitable for data and index compression are displayed on the
Candidates tab page.
2. To identify tables that are candidate for compression, choose Identify Candidates for Compression.
The dialog box Schedule Job to Identify Table Candidates appears.

3. Enter the number of tables that you would like to retrieve and start the job by choosing OK.
A new list of candidates is displayed on the Candidates tab page after that job is completed.
2009 SAP AG 18
4. To estimate the storage saving for one or more tables, select the table(s) from the list and choose
the Estimate pushbutton.
The dialog box Schedule Job to Estimate Storage Savings appears:

5. Specify the time when you want the job to run and choose OK.
6. On the Enabling Tables for Row Compression screen, choose Refresh View. The latest estimated
savings as well as the timestamp of the latest compression estimate are displayed.
2009 SAP AG 19
7. To enable tables for compression, select the respective table(s) and choose Compress.
The dialog box Schedule Row Compression J ob appears:

You can specify one of the following options:
Enable Table and Index Compression
Enables compression and an offline REORG is performed at a later point in time. In this
case, the table is enabled for compression and the catalog information is updated but the
actual data and indexes are not compressed. The table longer appears on the Candidates
tab page but will appear the Compressed tab page.
Enable Table and Index Compression, and Offline REORG
Enables compression and an offline REORG is performed at the same time. In this case, the
table is enabled for compression and is taken offline to compress data and indexes. This
action is followed by a RUNSTATS to update the statistical information of that table. If you
want to use a temporary tablespace, select Use Temporary Tablespace.

2009 SAP AG 20
Enabling Index Compression:
1. On the screen Enabling Tables for Row Compression, choose the Indexes tab page.
A list of all the tables is displayed that have their data compressed but not the indexes.

2009 SAP AG 21
2. To estimate the savings after index compression, select one or more tables and choose the
Estimate pushbutton.

3. Specify the time when you want the job to run and choose OK.
4. On the Enabling Tables for Row Compression screen, choose Refresh View. the latest
estimated savings as well as the timestamp of the latest compression estimate are displayed.
5. To enable index compression, select the respective table(s) and choose the Compress
pushbutton.
2009 SAP AG 22
The dialog box Schedule Index Compression Job appears:

You can specify one of the following options:
Enable Index Compression and Online Index REORG
Enables index compression and a REORG of all the indexes of that table is performed
online. This option enables index compression on all the indexes of the selected table(s)
and updates the system catalog data accordingly. The indexes are compressed while
the table is still online. This job is also followed by a RUNSTATS to update the table
statistics.
Offline Table and Index REORG
Enables data and index REORG. This option is used for tables that had the compression
attribute set but that have yet not been compressed. Choose this option to start an
offline REORG of the table (data and indexes). If you want to use a temporary
tablespace, select Use Temporary Tablespace
Enable Index Compression and Offline Index REORG
Enable index compression and perform an offline index REORG. This option is identical
to the first option except that the table is taken offline to perform this action.

2009 SAP AG 23
Restrictions
Index compression is not supported for the following:
Catalog indexes
MDC block indexes
XML path indexes and meta indexes
Index specifications
Case Study
A case study was performed in the lab on an SAP ERP customer database used for testing. The following
table outlines the results:

Original Size After Data and Index
Compression
Savings
DATA_P_SIZE 567187840 158071552
-72%
INDEX_P_SIZE 130968320 66381600
-49%
Total Size 698156160 224453152
-68%

From the table above, we notice that the total data size has decreased by 72% after data compression and
the total index size has decreased by 49% after index compression. The total database size has decreased
by 68% after compressing the data and indexes

2009 SAP AG 24
Related Content
Enabling DB2 for LUW 97's LOB Inlining in an SAP Environment
SSL Support in DB2 9.7 Running SAP ECC6 SR3 ABAP+J AVA system with SSL connections on DB2 Database
Temporary Table Compression
Enable Automatic Storage for Your SAP Database and Table Spaces
Reclaiming Space from Multidimensional Clustering (MDC) Tables
Reducing the High Water Mark
Rotating Diagnostic and Administration Notification Logs
2009 SAP AG 25
2009 SAP AG 26
Disclaimer and Liability Notice
This document may discuss sample coding or other information that does not include SAP official interfaces and therefore is not
supported by SAP. Changes made based on this information are not supported and can be overwritten during an upgrade.
SAP will not be held liable for any damages caused by using or misusing the information, code or methods suggested in this document,
and anyone using these methods does so at his/her own risk.
SAP offers no guarantees and assumes no responsibility or liability of any type with respect to the content of this technical article or
code sample, including any liability resulting from incompatibility between the content within this document and the materials and
services offered by SAP. You agree that you will not hold, or seek to hold, SAP responsible or liable with respect to the content of this
document.

DB2 9.7 For LUW New Feature - Index Compression

Uploaded by

Copyright:

Available Formats

You might also like

DB2 9.7 For LUW New Feature - Index Compression

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DB2 9.7 For LUW New Feature - Index Compression

Uploaded by

Copyright:

Available Formats

DB2 9.

7 f or LUW New Feat ur e:

You might also like