The Objective:

The goal is to create a reliable predictive model that will suggest tables and indexes
which will measurably benefit from reorganization, predict the reduction in I/O
(logical I/O - consistent gets and physical I/O - physical reads) after the
reorganization, and suggest changes that will prevent a reoccurrence of the
fragmentation (i.e. new pctfree, new blocksize, etc.):

Oracle tables/indexes can be reorganized, sometimes resulting in faster SQL

execution speeds

AWR (STATSPACK) has a history of how the tables were accessed by historical SQL
(dba_hist_sql_plan, stats$sql_plan, etc.), including the I/O and CPU costs
associated with each step of SQL execution.

We can see the current internal structure of every table/index

(e.g. chain_cnt, clustering_factor, etc.)

Some possible Rules for your AI engine:

To get you started, here is a list of possible conditions that may contribute to
table/index fragmentation. Remember, you don't need to pre-justify your rule set.
The only thing that counts is that your rules make accurate predictions.

Index fast full scans will run faster after index reorganization whenever the
density of the index entries becomes greater. In other words, it takes less
time to read 100,000 entries from a 100 block index than reading the entries
from a 500 block index.

Multi-block Index range scans will run faster when the data blocks are arranged
in index-key order and when the data blocks have a high number of row entries
(as evidenced by clustering_factor in dba_indexes).

Large-table full-table scans will run faster after reorganization when the table
has excessive chained or relocated rows, or low block density after massive DML
(updates and deletes).

Why Rebuilt Indexes are Problems

A tightly packed index is more efficient for the database as long as no changes are
made in the underlying table. Once you start making changes, the index blocks start to
split and reintroduce the fluff. Not only is fluff reintroduced, but there is redo
created as blocks are added to the index. The higher the rate of change, the faster
your nicely paced index will return to a steady state of fluff.

When to Rebuild Indexes

The only time you need to rebuild indexes is when the cost to rebuild is less that the
performance gained. The first one is obvious. Pack your tables and indexes (rebuild
into the soon to be read-only tablespace) tightly and they will stay that way. The
second is much more difficult. First, it is a continuing process since the index will
move toward fluff with use. Second, there is the cost of rebuilding the index AND the
cost of the additional redo as the index changes. There is only one method to
determine is rebuilding an indexes benefits your database, testing.

Oracle Index rebuilding

There is a lot of great information about Oracle index rebuilding, and current
research reveals how Oracle rebuilds indexes. Index rebuilding is risk free when done
properly, and on OTN, Chen Shapira reports that an index rebuild reduced index range
scan I/O:
"We noticed no change of the execution plan after rebuild. The plan was exactly the
same, but an index scan that required lots of block reads and was very costly, now
takes significantly less. Which is exactly what we needed!"

Oracle author claims to have perfected script to detect bad indexes

In this article, author Boris Milrud says that Oracle indexes are not self-balancing
and proposed a method for detecting when an index needs rebuilding. Milrud notes that
Oracle b-tree indexes become out of balance via high DML and he claims that they have
lower performance:
"Oracle indexes are not self-balancing. They become fragmented after a large number of
INSERTs and DELETEs, which may lead to significant performance degradation. In this
10-Minute Solution I show you how to detect these "out-of-shape" indexes and cure
them.
The author claims that these statistics are relevant to the index rebuilding decision:
"The following INDEX_STATS columns are especially useful:

height refers to the maximum number of levels encountered within the

index. An index could have 90 percent of the nodes at three levels, but
excessive splitting and spawning in one area of the index with heavy DML
operations could make nodes in that area to have more than three levels.

lf_rows refers to the total number of leafs nodes in the index.

del_lf_rows refers to the number of leaf rows that have been marked
deleted as a result of table DELETEs."

Milrud also claims that an index rebuild made a batch job run 1000x faster:
"In other words, massive DELETEs seemed to damage certain index areas, but leave other
areas intact.
Anyway, it was clear to me that it was high time to fix that index! After I added code
to rebuild the index at the end of the procedure and recompiled it, the next run took
only 14 seconds."

How rare are "bad" indexes?

You cannot generalize to say that index rebuilding for performance is rare, or even
medium rare, it depends on many factors, most importantly the characteristics of the
application.

In scientific applications (clinical, laboratory) where large datasets are added

and removed, the need to rebuild indexes is "common".
Conversely, in system that never update or delete rows, index rebuilding rarely
improves performance.
In systems that do batch DML jobs, index rebuilding "often" improves SQL
performance.

Oracle MOSC note 122008.1 has the officially authorized script to detect indexes that
benefit from rebuilding. This script detects indexes for rebuilding using these
rules: Rebuild the index when these conditions are true:
- deleted entries represent 20% or more of the current entries.
- the index depth is more then 4 levels

Oracle's index rebuilding guidelines appear in MOSC note 77574.1 (dated April 2007)
recommends that indexes be periodically examined to see if they are candidates for an
index rebuild:
?When an index is skewed, parts of an index are accessed more frequently than others.
As a result, disk contention may occur, creating a bottleneck in performance.
It is important to periodically examine your indexes to determine if they have become
skewed and might need to be rebuilt.?
Oracle index nodes are not physically deleted when table rows are deleted, nor are the
entries removed from the index. Rather, Oracle "logically" deletes the index entry and
leaves "dead" nodes in the index tree where that may be re-used if another adjacent
entry is required.
However, when large numbers of adjacent rows are deleted, it is highly unlikely that
Oracle will have an opportunity to re-use the deleted leaf rows, and these represent
wasted space in the index. In addition to wasting space, large volumes of deleted leaf
nodes will make index fast-full scans run for longer periods.
These deleted leaf nodes can be easily identified by running the IDL.SQL script.

Oracle index nodes are not physically deleted when table rows are deleted, nor are the
entries removed from the index. Rather, Oracle "logically" deletes the index entry and
leaves "dead" nodes in the index tree where that may be re-used if another adjacent
entry is required.
However, when large numbers of adjacent rows are deleted, it is highly unlikely that
Oracle will have an opportunity to re-use the deleted leaf rows, and these represent
wasted space in the index. In addition to wasting space, large volumes of deleted leaf
nodes will make index fast-full scans run for longer periods.
These deleted leaf nodes can be easily identified by running the IDL.SQL script.
The number of deleted leaf rows
The term "deleted leaf node" refers to the number of index inodes that have been
logically deleted as a result of row deletes. Remember that Oracle leaves "dead" index
nodes in the index when rows are deleted. This is done to speed up SQL deletes, since
Oracle does not have to allocate resources to rebalance the index tree when rows are
deleted.
Index height
The height of the index refers to the number of levels that are spawned by the index
as a result in row inserts. When a large amount of rows are added to a table, Oracle
may spawn additional levels of an index to accommodate the new rows.
Oracle indexes can support many millions of entries in three levels. Any Oracle index
that has spawned to a 4th level followed by a large delete job might benefit from
rebuilding to restore the index to it's pristine state.
Gets per index access
The number of "gets" per access refers to the amount of logical I/O that is required
to fetch a row with the index. As you may know, a logical "get" is not necessarily a
physical I/O since much of the index may reside in the Oracle buffer cache.
Unfortunately, Oracle does not make it easy to capture this information. In Oracle we
must issue these commands to populate the statistics in dba_indexes and related
dictionary tables:
ANALYZE INDEX index_name COMPUTE STATISTICS
ANALYZE INDEX index_name VALIDATE STRUCTURE
We might want to rebuild an index if the ?block gets? per access is excessive. This
happens when an index becomes "sparse" after high delete activity, making full-index
scans requires unnecessary I/O. Another rebuild condition would be cases wheredeleted
leaf nodes comprise more than 20% of the index nodes.
As you may know, you can easily rebuild an Oracle index with the command:
ALTER INDEX index_name REBUILD tablespace FLOP;
Done properly during scheduled downtime, rebuilding an index is 100% safe. Note the
use of the tablespace option. When rebuilding multi-gigabyte indexes, many DBA's will
rebuild partitioned indexes into a fresh, empty tablespace for greater manageability.
( I use the convention ts_ndexname_flip, and ts_indexname_flop)
The ALTER INDEX index_name REBUILD command is very safe way to rebuild indexes. Here
is the syntax of the command:
alter index index_name
rebuild

tablespace tablespace_name
storage (initial new_initial next new_next freelists new_freelist_number )
Unlike the traditional method where we drop the index and recreate it, the REBUILD
command does not require a full table scan of the table, and the subsequent sorting of
the keys and rowids. Rather, the REBUILD command will perform the following steps:
1. Walk the existing index to get the index keys.
2. Populate temporary segments with the new tree structure.
3. Once the operation has completed successfully, drop the old tree, and rename the
temporary segments to the new index.
As you can see from the steps, you can rebuild indexes without worrying that you will
accidentally lose the index. If the index cannot be rebuilt for any reason, Oracle
will abort the operation and leave the existing index intact. Only after the entire
index has been rebuilt does Oracle transfer the index to the new b-tree.
Most Oracle administrators run this script, and then select the index that they would
like to rebuild. Note that the TABLESPACE clause should always be used with the ALTER
INDEX REBUILD command to ensure that the index is not rebuilt within the default
tablespace (usually SYS).
Be aware that it's always a good idea to move an index into another tablespace and you
must have enough room in that tablespace to hold all of the temporary segments
required for the index rebuild, so most Oracle administrators will double-size index
tablespaces with enough space for two full index trees.
there are some that are especially important:

CLUSTERING_FACTOR ? This is one of the most important index statistics because

it indicates how well sequenced the index columns are to the table rows.
Ifclustering_factor is low (about the same as the number
of dba_segments.blocks in the table segment) then the index key is in the same
order as the table rows and index range scans will be very efficient, with
minimal disk I/O. As clustering_factorincreases (up to dba_tables.num_rows),
the index key is increasingly out of sequence with the table rows. Oracle?s
cost-based SQL optimizer relies heavily upon clustering_factor to decide whether
to use the index to access the table.

HEIGHT - As an index accepts new rows, the index blocks split. Once the index
nodes have split to a predetermined maximum level the index will ?spawn? into a
new level.

BLOCKS ? This is the number of blocks consumed by the index. This is dependent
on the db_block_size. In Oracle9i and beyond, many DBAs create b-tree indexes
in very large blocksizes (db_32k_cache_size) because the index will spawn less.
Robin Schumacher has noted in his book Oracle Performance
Troubleshooting notes ?As you can see, the amount of logical reads has been
reduced in half simply by using the new 16K tablespace and accompanying 16K data
cache. Clearly, the benefits of properly using the new data caches and multiblock tablespace feature of Oracle9i and above are worth your investigation and
trials in your own database.?

PCT_USED ? This metric is very misleading because it looks identical to

thedba_tables pct_used column, but has a different meaning. Normally,
the pct_usedthreshold is the freelist unlink threshold, while in index_stats
pct_used is the percentage of space allocated in the b-tree that is being used.

his gives us a high-level idea of Oracle threshold for spawning an index onto
new levels. We can take this same approach and attempt to answer the following
questions:

1 - At what point does an index spawn to another level (height)? It should be a

function of blocksize, key length and the number of keys.
2 - The number of deleted leaf nodes may not be enough to trigger an index
rebuild. This is because if clustering_factor is low
(dba_indexes.clustering_factor ~= dba_segments.blocks), then the rows are added
in order, and the index is likely to reuse the deleted leaf nodes. On the other
hand, if clustering_factor is high
(dba_indexes.clustering_factor ~=dba_tables.num_rows), and the majority of
queries use the index with fast-full scans or index range scans, then a rebuild
of the underlying table (to resequence the rows) may be beneficial.
To illustrate, assume I have an index on the last_name column of a 1,000,000 row
table and the clustering_factor is close to the number of blocks, indicating
that the rows are in the same sequence as the index. In this case, a bulk delete
of all people whose last_name begins with the letter ?K? would leave a dense
cluster of deleted leaf nodes on adjacent data blocks within the index
tablespace. This large section of space is more likely to be reused than many
tiny chunks.

This Kim Floss article shows the Oracle 10g segment advisor recommending a rebuild of
an index:
?The page lists all the segments (table, index, and so on) that constitute the object
under review. The default view ("View Segments Recommended to Shrink") lists any
segments that have free space you can reclaim.?

"In other words, massive DELETEs seemed to damage certain index areas, but leave other
areas intact.

Anyway, it was clear to me that it was high time to fix that index! After I added code
to rebuild the index at the end of the procedure and recompiled it, the next run took
only 14 seconds."
Index behavior and Oracle blocksize
Because the blocksize affects the number of keys within each index block, it follows
that the blocksize will have an effect on the structure of the index tree. All else
being equal, large 32k blocksizes will have more keys per block, resulting in a
flatter index than the same index created in a 2k tablespace.
Today, most Oracle tuning experts utilize the multiple blocksize feature of Oracle
because it provides buffer segregation and the ability to place objects with the most
appropriate blocksize to reduce buffer wast
"A bigger block size means more space for key storage in the branch nodes of B-tree
indexes, which reduces index height and improves the performance of indexed queries."

Here is a great script to show SQL access patterns, grouped by full-table scans, index range scans
and index unique scans.

Full table scans and counts

Note that "K" indicates in the table is in the KEEP pool.

OWNER
-------------SYS
SYSTEM
DONALD
DONALD
DONALD
DONALD
DONALD

NAME
-----------------------DUAL
SQLPLUS_PRODUCT_PROFILE
PAGE
RWU_PAGE
PAGE_IMAGE
SUBSCRIPTION
PRINT_PAGE_RANGE

NUM_ROWS C K
BLOCKS NBR_FTS
--------- - - -------- -------N
2
97,237
N K
2
16,178
3,450,209 N
932,120
9,999
434 N
8
7,355
18,067 N
1,104
5,368
476 N K
192
2,087
10 N K
32
874

--**************************************************************
-- Object Access script report
--- 2012 by Donald K. Burleson
--No part of this SQL script may be copied. Sold or distributed
-without the express consent of Donald K. Burleson
--**************************************************************
-- ********************************************************
-- Report section
-- ********************************************************
spool plan.lst
set echo off

set feedback on
set pages 999;
column nbr_FTS
column num_rows
column blocks
column owner
column name
column ch

format
format
format
format
format
format

9,999,999
999,999,999
999,999
a14;
a24;
a1;

column object_owner heading "Owner"

format a12;
column ct
heading "# of SQL selects" format 999,999;
select
object_owner,
count(*)
ct
from
v$sql_plan
where
object_owner is not null
group by
object_owner
order by
ct desc
;
--spool access.lst;
set heading on;
set feedback on;
ttitle 'full table scans and counts| |The "K" indicates that the table is in the KEEP Pool (Oracle8).'
select
p.owner,
p.name,
t.num_rows,
-ltrim(t.cache) ch,
decode(t.buffer_pool,'KEEP','Y','DEFAULT','N') K,
s.blocks blocks,
sum(a.executions) nbr_FTS
from
dba_tables
t,
dba_segments s,
v$sqlarea
a,
(select distinct
address,
object_owner owner,
object_name name
from
v$sql_plan
where
operation = 'TABLE ACCESS'
and
options = 'FULL') p
where
a.address = p.address
and
t.owner = s.owner
and
t.table_name = s.segment_name
and
t.table_name = p.name
and
t.owner = p.owner
and
t.owner not in ('SYS','SYSTEM')
having
sum(a.executions) > 1
group by
p.owner, p.name, t.num_rows, t.cache, t.buffer_pool, s.blocks
order by
sum(a.executions) desc;
column nbr_RID

format 999,999,999

column num_rows format 999,999,999

column owner
format a15;
column name
format a25;
ttitle 'Table access by ROWID and counts'
select
p.owner,
p.name,
t.num_rows,
sum(s.executions) nbr_RID
from
dba_tables t,
v$sqlarea s,
(select distinct
address,
object_owner owner,
object_name name
from
v$sql_plan
where
operation = 'TABLE ACCESS'
and
options = 'BY ROWID') p
where
s.address = p.address
and
t.table_name = p.name
and
t.owner = p.owner
having
sum(s.executions) > 9
group by
p.owner, p.name, t.num_rows
order by
sum(s.executions) desc;
--*************************************************
-- Index Report Section
--*************************************************
column
column
column
column
column
column

nbr_scans
num_rows
tbl_blocks
owner
table_name
index_name

format
format
format
format
format
format

999,999,999
999,999,999
999,999,999
a9;
a20;
a20;

ttitle 'Index full scans and counts'

select
p.owner,
d.table_name,
p.name index_name,
seg.blocks tbl_blocks,
sum(s.executions) nbr_scans
from
dba_segments seg,
v$sqlarea s,
dba_indexes d,
(select distinct
address,
object_owner owner,
object_name name
from
v$sql_plan
where
operation = 'INDEX'
and
options = 'FULL SCAN') p
where
d.index_name = p.name
and
s.address = p.address
and
d.table_name = seg.segment_name
and
seg.owner = p.owner

having
sum(s.executions) > 9
group by
p.owner, d.table_name, p.name, seg.blocks
order by
sum(s.executions) desc;
ttitle 'Index range scans and counts'
select
p.owner,
d.table_name,
p.name index_name,
seg.blocks tbl_blocks,
sum(s.executions) nbr_scans
from
dba_segments seg,
v$sqlarea s,
dba_indexes d,
(select distinct
address,
object_owner owner,
object_name name
from
v$sql_plan
where
operation = 'INDEX'
and
options = 'RANGE SCAN') p
where
d.index_name = p.name
and
s.address = p.address
and
d.table_name = seg.segment_name
and
seg.owner = p.owner
having
sum(s.executions) > 9
group by
p.owner, d.table_name, p.name, seg.blocks
order by
sum(s.executions) desc;
ttitle 'Index unique scans and counts'
select
p.owner,
d.table_name,
p.name index_name,
sum(s.executions) nbr_scans
from
v$sqlarea s,
dba_indexes d,
(select distinct
address,
object_owner owner,
object_name name
from
v$sql_plan
where
operation = 'INDEX'
and
options = 'UNIQUE SCAN') p
where
d.index_name = p.name
and
s.address = p.address
having
sum(s.executions) > 9
group by
p.owner, d.table_name, p.name
order by
sum(s.executions) desc;
spool off

So When Does An Oracle B-Tree Index Increase

In Height ? (Almost Grown) April 3, 2008
Posted by Richard Foote in Index Height, Index statistics, Oracle General,Oracle Indexes, Oracle
Myths.
trackback

So when does an Oracle B-Tree index actually increase in height ?

Ive basically been asked this same question a number of times over the past few days with regard to
the discussions on indexes and different block sized tablespaces, so I thought it might be worth quickly
sharing the answer to a wider audience.
Imagine a new, empty table and a corresponding new, empty index. At this stage, the index structure
basically consists of one, empty block. The index has a BLEVEL of 0 (from DBA_INDEXES) and a HEIGHT
of 1 (from INDEX_STATS), yes it can be confusing ;) This block is basically the Root block of the index as
its the first (and currently only) block to be accessed during an index scan, but at this stage is used to
also store the actual index entries as well (and so can kinda be viewed as being a Leaf block as well).
We now start to insert rows into the table and thus row entries into the index. These index entries
basically consist of the indexed column(s) and its corresponding ROWID, and are sorted based on the
indexed column values.
Eventually, this single index block will fill; Oracle simply cant add any more index entries into it. Now
comes the fun bit.
When Oracle wants to insert a new index entry but it cant as this Root index block is full, Oracle will
allocate two new index blocks. If the new index entry is the maximum value currently to be indexed,

Oracle will move all the index entries from the full block and put it into one of the new index blocks and
place the new index entry into the other block. This is known as a 90-10 index block split.
If the new index entry isnt the maximum value, Oracle will place the lower 1/2 valued index entries
into one new block and the other 1/2 into the other new block. This is known as a 50-50 index block
split.
These two new blocks are now the new leaf blocks in the index structure.
The contents of the previously single filled block is now totally replaced with pointers to the two new
blocks. This block therefore remains the Root block in the index structure. These pointers basically
consist of the Relative Block Address (RBA) to the new index blocks and a value which represents the
lowest indexed value found in the specific referenced leaf block. These indexed values in the Root
block are now used by Oracle as the method by which it can navigate the index structure to find the
specific index leaf block containing a required indexed entry.
The index has just increased in height and now has a BLEVEL of 1 and a HEIGHT of 2.
As we continue to add more rows into the table, we add more index entries into our 2 leaf blocks.
Eventually they will fill again and will again perform either a 90-10 or 50-50 block split depending on
the new index value to be inserted. With a non Root block split, only one additional index block is
allocated and the index entries are distributed between the full and new index block. Each time a leaf
block splits in a BLEVEL 1 index, a new entry is also added into the Root block to point to the new Leaf
block.
Once we have enough Leaf blocks, the Root block will again eventually fill. At this point, Oracle will
again allocate two new blocks and distribute the contents of the Root block into these two new blocks,
again 90-10 or 50-50 depending on the new indexed value to be inserted. The contents of the Root
block is now totally replaced with pointers to these 2 new Branch blocks which of course in turn now
contain the pointers to the Leaf blocks.
The index has again increased in height and we now have an index with a BLEVEL of 2 and a HEIGHT of
3.
As the leaf blocks continue fill and split, a new entry is added to the corresponding Branch block each
time. When these Branch blocks fill and split, a new entry is added to the Root block. When the Root
block eventually fills, it will again allocate 2 new blocks and so the index grows in height again.

So basically, an index increases in height whenever the index Root block splits and the two new
allocated blocks result in a new level within the index structure. Note the index Root block remains the
same throughout the entire life of the index, no matter the index height.
Note also a Root block split is the only time an index increases in height. Therefore, the number of
levels between the Root block and any/all of the Leaf blocks is always and must always be the same.
Hence, an Oracle B-Tree index is always structurally height balanced, always.
About these ads