Professional Documents
Culture Documents
LOBInternals
LOBInternals
LOBInternals
Julian Dyke
Independent Consultant
Basic Files
Secure Files
Can be stored
within database (internal)
outside database (external)
For each row with an internal LOB column, the LOB can:
Be NULL
Point to an empty BLOB, CLOB or NCLOB
Point to a BLOB, CLOB or NCLOB value
LONGs LOBs
Single column per table Multiple columns per table
For example:
T1
LOB
TABLE
CREATE TABLE t1
(
c1 NUMBER, SYS_IL0000055604C00002$$
LOB
c2 CLOB, LOB INDEX
);
SYS_LOB0000055604C00002$$
LOB
LOB DATA
T2
LOB
TABLE
SYS_IL0000055607C00002$$ SYS_IL0000055607C00003$$
LOB LOB
LOB INDEX LOB INDEX
SYS_LOB0000055607C00002$$ SYS_LOB0000055607C00003$$
LOB LOB
LOB DATA LOB DATA
LOB and and LOB index segments are equi-partitioned with base table
CREATE TABLE t3
(
c1 NUMBER,
c2 CLOB
)
PARTITION BY RANGE (c1)
(
PARTITION p1 VALUES LESS THAN (100),
PARTITION p2 VALUES LESS THAN (200)
);
T3
LOB
TABLE
P1 P2
TABLE PARTITION TABLE PARTITION
SYS_IL0000055612C00002$$
LOB
LOB INDEX
SYS_LOB_P23 SYS_LOB_P24
INDEX PARTITION INDEX PARTITION
SYS_LOB0000055612C00002$$
LOB
LOB
SYS_LOB_P21 SYS_LOB_P22
LOB PARTITION LOB PARTITION
P1 P2
TABLE PARTITION TABLE PARTITION
SYS_IL0000055633C00003$$
LOB
LOB INDEX
SYS_LOB_P41 SYS_LOB_P44
SYS_IL_P23
INDEX PARTITION SYS_IL_P24
INDEX PARTITION
SYS_LOB0000055633C00003$$
LOB
LOB
SYS_LOB_P31 SYS_LOB_P34
LOB PARTITION LOB PARTITION
CREATE TABLE t1
(
c1 NUMBER,
c2 CLOB,
);
C2 occupies 0 bytes
Header LOBID
00 54 0 01 0 0c 0 00 0 01 0 00 0 01 0 00 00 0 89 3d
0 2 0 0 0 0 0 0
00 10 0 00 0 00 0 00 0 00 0 00 0 00 0 00
9 0 0 0 0 0 0
Body Length Flags LOB Length Version
ab 0, row 0, @0x1f62
tl: 54 fb: --H-FL-- lb: 0x1 cc: 2
col 0: [ 1] 80
col 1: [48]
00 54 00 01 02 0c 00 00 00 01 00 00 00 01 00 00 00 00 8a 7b 00 1c 09 00 00
00 00 00 00 0c 00 00 00 00 00 01 58 58 58 58 58 58 58 58 58 58 58 58
0 54 00 0 02 0 00 0 00 0 00 0 00 0 00 0 00 0 8a 7b
0 1 c 0 1 0 1 0 0
0 1c 09 0 00 0 00 0 00 0 00 0 00 0 00 0 58 5 58 58
0 0 0 0 c 0 0 1 8
5 58 58 5 58 5 58 5
8 0x1C
8 = 28 8 8 0x0C = 12
00 54 0 01 0 0c 0 00 0 01 0 00 0 01 0 00 00 0 8a ad
0 2 0 0 0 0 0 0
01 a0 0 00 0 00 0 00 0 90 0 00 0 00 0 01 58 5 58 58
9 0 0 1 0 0 0 8
58 58 5 58 5 58 5 58 5 58 5 58 5 58 5 58 58 5 58 58
0x01A0
8 = 416
8 8 8 0x0190
8 = 8400 8 8
2 ©58 58Julian
2008 5 58 5
Dyke 58 5 58 5 58 5 58 5 58 5 juliandyke.com
5858 5 58 58
Basic Files
LOB Inline Storage
Example 3 - 3964 byte LOB - ENABLE STORAGE IN ROW
0 54 00 0 02 0 00 0 00 0 00 0 00 0 00 0 00 0 8c a
0 1 c 0 1 0 1 0 0 1
0f 8c 09 0 00 0 00 0 0f 7 00 0 00 0 00 0 58 5 58 5
0 0 0 c 0 0 1 8 8
5 58 58
5 58 5 58 5 58 5 58 5 58 5 58 5 58 5 58 5
8 0x0F8C8= 3980 8 8 8 8 = 3964
0x0F7C 8 8 8 8
5 58 58 5 58 5 58 5 58 5 58 5 58 5 58 5 58 5 58 5
2 8
© 2008 8
Julian Dyke 8 8 8 8 8 juliandyke.com
8 8 8
Basic Files
LOB Out-of-Line Storage
Example 4 - 3965 byte LOB - ENABLE STORAGE IN ROW
00 54 00 01 0 0c 0 00 0 01 00 00 0 01 0 00 00 0 8d 05
2 0 0 0 0 0
00 14 05 00 0 00 0 00 0f 7d 00 00 0 00 0 02 01 0 01 b5
0 0 0 0 0
0x0F7D = 3965
2 © 2008 Julian Dyke juliandyke.com
Basic Files
LOB Out-of-Line Storage
Out-of-line LOB data is stored in a PAGETABLE MANAGED LOB BLOCK
Each block has
a 56 byte header
a 4 byte trailer
For example for a 8192 byte block
Header
56 bytes
LOB Data
8132 bytes
Trailer
4 bytes
00 54 00 01 0 0c 0 00 0 01 00 00 0 01 0 00 00 0 8d ff
2 0 0 0 0 0
00 34 05 00 0 00 0 08 1 55 00 00 0 00 0 02 01 0 02 5f
0 0 0 0 0 0
01 00 02 60 0 00 0 5c 0 00 02 5d 0 00 0 5e 010 02 6e
1 2 1 1 2 0 of LOB
DBA
0x34 =0152 00 02 6f 0 8 chunks
00 0 70 0 0x1055
00 02bytes
6a chunks
1 2 1
8 x 8132 = 65056 0x1055 = 4181 65056 + 4181+ 69237
00 54 0 01 02 0 00 00 0 01 00 00 0 01 0 00 00 0 90 25
0 c 0 0 0 0
00 40 0 00 00 0 00 0c 0 00 00 00 0 00 0 02 01 0 02 3d
5 0 0 0 0 0
01 00 0 3e 01 0 02 3f 0 00 02 40 0 00 0 3c 01 0 02 37
2 0 1 1 2 0
01 00 0 38 01 0 02 32 0 00 02 33 0 00 0 LOB
34 Index
01 0exists
02 35
0x40 = 264 12 0chunks 1 0 bytes 1 2 but is empty
0
3 01 00 Julian
© 2008 0 36 Dyke 12 x 8132 = 97584 juliandyke.com
Basic Files
LOB Out-of-Line Storage
Example 7- 97585 byte LOB - ENABLE STORAGE IN ROW
00 54 00 01 0 0c 0 00 0 01 0 00 00 0 00 00 0 00 90 89
2 0 0 0 1 0
00 40 05 00 0 00 0 0c 0 01 0 00 00 0 00 02 0 00 03 2d
0 0 0 0 0 1
01 00 03 2e 0 00 0 2f 0 00 0 30 01 0 03 2c 0 00 03 27
1 3 1 3 0 1
01 00 03 28 0 00 0 22 0 00 0 23 01
LOB Index is used 03 25
0 03 th 24 0 00
0x40 = 64 1
12 chunks
3 1
1 byte
3 0for 13 chunk
1
3 ©012008
00 Julian
03 26Dyke juliandyke.com
Basic Files
LOB Out-of-Line Storage
Example 7 continued - 97585 byte LOB - ENABLE STORAGE IN ROW
LOB Index
Key 00 00 00 0 00 0 00 00 9 89 0 00 00 0
1 0 0 0 c
Data 01 00 03 3f 0 00 0 00 0 00 0 00 00 0 00 00
0 0 0 0 0
00 00 00 00 0 00 0 00 0 00 0 00 00 0 00 00
0 0 0 0 0
LOB Index is always unique
Data is always 32 bytes
00 54 0 01 02 0 00 00 0 01 00 00 0 01 0 00 00 0 90 ed
0 c 0 0 0 0
00 40 0 00 00 0 00 17 0 e4 00 00 0 00 0 02 01 0 03 35
5 0 0 0 0 0
01 00 0 36 01 0 03 37 0 00 03 38 0 00 0 34 01 0 03 5f
3 0 1 1 3 0
12 chunks in row
01 00 0 60 01 0 03 5a 0 00 03 5b 0 00 0 11 5c 01 0in index
chunks 03 5d
3 0
0x17 = 231 0xe4 = 2281 3 0
chunks bytes
3 01 00 Julian
© 2008 0 5e Dyke juliandyke.com
Basic Files
LOB Out-of-Line Storage
Example 8 continued - 187264 byte LOB - ENABLE STORAGE IN ROW
Lob Index
row#0[7982] flag: ------, lock: 2, len=50, data:(32):
01 00 03 6f 01 00 03 70 01 00 03 6a 01 00 03 6b 01 00 03 6c 01 00 03 6d 01
00 03 6e 01 00 03 57
col 0; len 10; (10): 00 00 00 01 00 00 00 00 90 ed
col 1; len 4; (4): 00 00 00 0c
row#1[7932] flag: ------, lock: 2, len=50, data:(32):
01 00 03 58 01 00 03 52 01 00 03 53 01 00 03 54 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00
col 0; len 10; (10): 00 00 00 01 00 00 00 00 90 ed
col 1; len 4; (4): 00 00 00 14
Key 00 00 00 0 00 0 00 00 9 ed 0 00 00 0
1 0 0 0 c
01 00 03 6f 0 00 0 70 01 00 03 6a 0 00 0 6b
Data
1 3 1 3
01 00 03 6c 0 00 0 6d 01 00 03 6e 0 00 0 57
Key 00 00 00 0 00
1 0 00
3 00 9 ed 0 00 00
1 1 3
1 0 0 0 4
Data 01 00 03 58 0 00 0 52 01 00 03 53 0 00 0 54
1 3 1 3
3 © 200800 00 Dyke
Julian 00 00 0 00 0 00 00 00 00 00 0 00 juliandyke.com
0 00
Basic Files
Chunk Size
Each internal LOB has a chunk size
Chunks can be manipulated individually
Default chunk size is block size
Maximum chunk size is 32768 for all tablespace block sizes
Chunk size is:
specified in bytes
rounded up to nearest block size
00 54 00 01 0 0c 0 00 00 01 0 00 00 0 00 0 00 0 94 d5
2 0 0 1 0 0
00 1c 05 00 0 00 0 08 10 55 0 00 00 0 00 0 01 0 04 45
0 0 0 0 2 0
4 01
x 8132
00 =04
32528
55 0 2 x00
32528
0 =5d65056 0x1055 = 4181 65056 + 4181+ 69237
3 © 2008 Julian Dyke1 4 juliandyke.com
Basic Files
Chunk Size
In Oracle 11.1 maximum LOB size is determined by chunk size:
CREATE TABLE t1 (
c1 NUMBER,
c2 CLOB,
c3 CLOB,
c4 CLOB
)
LOB (c2) STORE AS (CHUNK 8192),
LOB (c3) STORE AS (CHUNK 16384),
LOB (c4) STORE AS (CHUNK 32768);
SELECT
dbms_lob.getchunksize (c2), dbms_lob.get_storage_limit (c2),
dbms_lob.getchunksize (c3), dbms_lob.get_storage_limit (c3),
dbms_lob.getchunksize (c4), dbms_lob.get_storage_limit (c4)
FROM t1;
Reported in DBA_LOB.PCTVERSION
Can be set to 0 for read-only LOBs
PCTVERSION should be set to a low value if:
updates and reads are not concurrent
the LOB is written once and then read-only
PCTVERSION should be set to a high value if:
there are large numbers of queries
there is heavy read and write activity
DECLARE
l_clob CLOB;
l_str VARCHAR2(4000);
BEGIN
l_str := LPAD ('Y',4000,'Y');
00 54 00 01 0 0c 0 00 00 01 00 00 0 01 0 00 00 00 93 45
2 0 0 0
00 24 05 00 0 00 0 04 00 f0 00 00 0 00 0 02 01 00 03 fd
0 0 0 0
01After
00 03 fe 0
Update 00 0 ff 01 00 04 00 0 00 0 fc
1 3 1 3
tl: 62 fb: --H-FL-- lb: 0x1 cc: 2
col 0: [ 1] 80
col 1: [56]
00 54 00 01 02 0c 00 00 00 01 00 00 00 01 00 00 00 00 93 45 00 24 05 00 00
00 00 04 00 f0 00 00 00 00 00 03 01 00 03 fd 01 00 04 0f 01 00 03 ff 01 00
04 00 01 00 03 fc
00 54 00 01 0 0c 0 00 00 01 00 00 0 01 0 00 00 00 93 45
2 0 0 0
00 24 05 00 0 00 0 04 00 f0 00 00 0 00 0 02 01 00 03 fd
0 0 0 0
01 00 04 0f 0 00 0 ff 01 00 04 00 0 00 0 fc
1 Updated
3 Chunk 1 3
4 © 2008 Julian Dyke juliandyke.com
Basic Files
Read Consistency
Example 9 (continued)
Key 00 01 4 14 fe c0 0 00 00 00 01 00 0 0f
9 0 3
Data 01 00 03 fe 0 00 0 00 00 00 0 00 00 0 00 00
0 0 0 0
00 00 00 00 0 00 0 00 00 00 0 00 00 0 00 00
0 0 0 0
LOB Index Free List
3FD 3FE
40F 3FF 400 3FC 3FE
LOB Data
DECLARE
l_clob CLOB;
l_str VARCHAR2(4000);
BEGIN
l_str := LPAD ('Y',4000,'Y');
00 54 00 01 0 0c 0 00 00 01 00 00 0 01 0 00 00 00 93 45
2 0 0 0
00 24 05 00 0 00 0 04 05 bf 00 00 0 00 0 04 01 00 03 fd
0 0 0 0
01 00 04 0f 0 00 0 ff 01 00 03 fe 0 00 0 10
1
Updated
3
Chunk Updated 1Chunk 4
4 © 2008 Julian Dyke juliandyke.com
Basic Files
Read Consistency
Example 10 (continued) - Lob Index
Key 00 01 49 1 fe c 00 00 0 00 0 00 03 0f
4 0 0 1
Data 01 00 03 fc 0 00 0 00 00 00 0 00 00 0 00 00
1 4 0 0
00 00 00 00 0 00 0 00 00 00 0 00 00 0 00 00
0 0 0 0
LOB Index Free List
3FD 3FE
40F 3FF 400
3FE 3FC
410 3FC
3FE 400
LOB Data
DECLARE
l_clob CLOB;
l_amount NUMBER := 18000;
BEGIN
SELECT c2 INTO l_clob
FROM t22
WHERE c1 = 0
FOR UPDATE;
DBMS_LOB.ERASE (l_clob,l_amount,12000);
END;
00 54 00 01 0 0c 0 00 0 01 00 00 0 01 0 00 0 00 95 6b
2 0 0 0 0 0
00 34 05 00 0 00 0 08 1 55 00 00 0 00 0 03 0 00 04 6d
0 0 0 0 0 1
4
01 00 04
© 2008 Julian Dyke
0
7d 0 00 0
0
00 0
1
00 04 7e 0
1
00 0
4 juliandyke.com
6c 0
1
00 04 7f
Basic Files
Partial Deletion
Example 11 continued - 69237 byte LOB - Erase 18000 bytes from offset 12000
After ERASE - Lob Index
01 00 04 6e 0 00 0 6f 01 00 0 70 00 0 00 00
Data 1 4 4 0
00 00 00 00 0 00 0 00 00 00 0 00 00 0 00 00
0 LOB Index
0 0 0 Free List
46D 46E
47D 46F 470
47E 46C 47F 480 47A 47B 46E 46F 470
LOB Data
Reported in DBA_LOB.CACHE
Can be NO, CACHEREADS or YES
NOCACHE NOLOGGING
STORE AS BASICFILE
Default
Behaves as Oracle 10.2 and below
STORE AS SECUREFILE
Oracle 11.1 and above
Supports several new features including
LOB compression
LOB encryption
LOB deduplication
Metadata only logging
Configurable retention time
LOB Data Segments are still stored in blocks of type trans_data (block type 6)
Note
CHUNK is advisory - only maintained for backward compatibility
PCTVERSION not supported - use RETENTION instead
00 54 0 01 0 0c 0 00 0 01 0 00 0 01 0 00 00 0 89 3d
0 2 0 0 0 0 0 0
00 10 0 00 0 00 0 00 0 00 0 00 0 00 0 00
9 0 0 0 0 0 0
Secure File
tl: 36 fb: --H-FL-- lb: 0x1 cc: 2
col 0: [ 1] 80
col 1: [30]
00 54 00 01 02 0c 00 80 00 01 00 00 00 01 00 00 00 00 99 53 00 0a 48 90 00
04 00 00 00 00
00 54 00 01 0 0c 0 80 0 01 00 00 0 01 0 00 00 0 99 53
2 0 0 0 0 0
00 0a 48 90 0 04 0 00 0 00
0 0 0
00 54 0 01 02 0c 0 80 0 01 00 00 0 01 0 00 00 0 98 8b
0 0 0 0 0 0
00 1f 4 90 00 19 2 00 0 0e 75 01 0
0 2 1 20x10E75 = 69237
01 01 0 05 6d 04 # Blocks in extent
0
5 © 2008
01 01Julian
0 Dyke
05 80 01DBA of first block in extent juliandyke.com
Secure Files
Lob Blocks
Out-of-line LOB data is stored in a trans data block (block type 6)
Each block has
a 128 byte header
a 4 byte trailer
For example for a 8192 byte block
Header
128 bytes
LOB Data
8060 bytes
Trailer
4 bytes
COMPRESS MEDIUM
Faster (less resource)
COMPRESS HIGH
Better compression (less storage)
Defined in $ORACLE_HOME/rdbms/admin/dbmslobu.sql
Subroutines include
GETINODE
GETLOBMAP
GETEXTENTS
GETINODE
Reports LOB header
GETLOBMAP
Reports LOB extent header
Numeric parameter is extent number (0..N-1)
GETEXTENTS
Reports LOB extents in more detail
Pipelined function - takes REF CURSOR as parameter
info@juliandyke.com