1 Data Storage Structures-Ch13

I Eid 064 0 S it I Gt w 45
so
Chapter 13: Data Storage Structures
I 81 41 im W.tl F
Textbook Database System Concepts, 7

th Ed.
©Silberschatz, Korth and Sudarshan
See www.db-book.com for conditions on re-use
fixedlength
V lab I gth
T
File Organization ‫يتم تخزين قاعدة البيانات‬
‫ كل‬.‫كمجموعة من امللفات‬
▪ The database is stored as a collection of files. Each file is a sequence

ii a
of records. A record is a sequence of fields.
▪ One approach
• Assume record size is fixed
• Each file has records of one particular type only
• Different files are used for different relations
This case is easiest to implement; will consider variable length records

later
▪ We assume that records are smaller than a disk block

.
block o g
records L
Database System Concepts - 7th Edition 13.2 ©Silberschatz, Korth and Sudarshan
IT dit al s
b i Fixed-Length Records m size is a 4
:‫نهج بسيط‬
▪ Simple approach: gal d i ‫• تخزين السجل‬
to
• Store record i starting from byteyn  (i – 1), where n is the size of
n ‫بد ًءا من البايت‬
each record.
e ‫ هو‬n ‫ حيث‬، (i - 1)
‫حجم‬
• Record access is simple but records may cross blocks 3
isaiah
4 I am
▪ Modification: do not allow records to cross block boundaries was
o Wii memory
it u A
j A 1
41 m m is
I cantstonge a2e
it is ai pi i s
i it
size
or fields
real is Kim
wi
f
att
AH semi g's
105 am It is
0 d sit Jot El
40 o 1
o w
W 911 w I 21
12 11 40
yo
4041 is Yo n
Yo 3 1 80
Fixed-Length Records
3 cross block wa y't j t
▪ Deletion of record i: alternatives:
• move records i + 1, . . ., n to i, . . . , n – 1 to sat onin sp I
• move record n to i
• do not move records, but link all free records on a free list
Record 3 deleted
3 a m
Y
at I s
80 as 80

• move records i + 1, . . ., n to i, . . . , n – 1
• move record n to i
am
• do not move records, but link all free records on a free list
Record 3 deleted and replaced by record 11
o 411 It

• move records i + 1, . . ., n to i, . . . , n – 1
• move record n to i Ipointer s.is seas
soawww81
stackorlinklist
• do not move records, but link all free records on a free list uh
46 im
ID wi't 3 It I do
Ii z Je 4 freespace of
6.66a86 N pm
a 6064,151
66 0 4 4356
116s.mil 5
visit gW
record
WE
wi ios out record12 4356
too ms a
‫تنشأ السجالت ذات الطول‬
ti i size Variable-Length Records ‫املتغير في أنظمة قواعد‬
length
:‫البيانات بعدة طرق‬
s
▪ Variable-length records arise in database systems in several ways:

• Storage of multiple record types in a file.
• Record types that allow variable lengths for one or more fields such
as strings (varchar) namevonchar so ma von
a
j se s sooofields (used in some older data
• Record types that allow repeating
models).
▪ Attributes are stored in order I again
w mis n
▪ Variable length attributes represented by fixed size (offset, length), with
lo i 50 I
actual data stored after all fixed length attributes
▪ Null values represented by null-value bitmap
88 If IAEA
Stl 55 1Bytes
Variable length
attributes(offset, length),
ex for variable length
array JI Si is 0 518 03 151 p at 6,1 6
É W
M
m e at IU in 1 I on
9 fixed variable lenghth I I m
b
i i m
a 9581
t a wot gé asi wi web a ski in t d e was variable2 n I
w si mi y www.t a it it a
U ok a I di et i saw a en mi si
i am a low
Variable-Length Records: Slotted Page Structure
‫▪ يحتوي رأس الصفحة‬
:‫املشقوقة على‬
‫• عدد إدخاالت السجل‬
‫• نهاية املساحة الخالية‬
4,11 Gum w
▪1 Slotted page ya
Header a
header contains:
• number of record entries W D B ET a
• end of free space in the block It'swww.a.us
• location and size of each record Go e si owl owl 413 is
Headers I I 211Datapiled 6
it
▪ Records can be moved around within a page to keep them contiguous
with no empty space between them; entry in the header must be
updated.
▪ Pointers should not point directly to record — instead they should point
to the entry for the record in header.
aww it 8
sit to
ah M FB Header
video
Storing Large Objects
picha
re
▪ E.g., blob/clob types Binary Large Object / Character Large Object
▪ Records must be smaller than pages ti sa o ous

m
i
▪ Alternatives:
• Store as files in file systems
• Store as files managed by database

wa sa y ti som e k b omno.tw i
• Break into pieces and store in multiple tuples in separate relation
▪ PostgreSQL TOAST Ei la ma
Examples:
create table pictures(name varchar(32) not null primary key, pic blob(16M));
bdf a
CREATE TABLE documents (id INT, text CLOB)
I D B 20 01 b mi 911 É i d
I ok Organization ‫ يمكن‬- ‫▪ كومة‬
of Records in Files ‫وضع سجل في أي‬
‫مكان في امللف‬
I Ini se on spy in E e 1ms i

▪ Heap – record can be placed anywhere in the file where there is space
a 66 so gl
oforder,
a swim
▪ Sequential – store records in sequential based on the ta
value of the
search key of each record
ox I Job I d I 0.5 si 8
▪ In a multitable clustering file organization records of several different
relations can be stored in the same file do oh l Ii spin
• Motivation: store related records on the same block to minimize I/O
▪ B+-tree file organization
Im bi mi Ia si a
• Ordered storage even with inserts/deletes is on um
• More on this in Chapter 14
is Im y se search
jam key ma
▪ Hashing – a hash function computed on search key; the result specifies in
which block of the file the record should be placed
ate a s i wi Ew
• More on this in Chapter 14
theadvancedmethodofan indexedsequential accessmethod
technique to directlysearchthelocation of desireda

data on the disk without use index
th
Database System Concepts - 7 Edition 13.10 ©Silberschatz, Korth and Sudarshan
unorderedfile
‫يمكن وضع‬
Heap File Organization ‫السجالت في أي‬
it stows lot
▪ Records can be placed anywhere in the file where there is free space
▪ Records usually do not move once allocated 8061,18 alone
▪ Important to be able to efficiently find free space within file is winless so
file soratan
is's ▪ Free-space map
stats • data structure to track which blocks have free space to store records.
byte obits
• Array with 1 entry per block. Each entry is a few bits to a byte, and
records fraction of block that is free
bytes a 61st bit I
• Example free-space map for a file with 16 blocks. 3 bits per block, value
divided by 8 indicates fraction of block that is free.
• For example, a value of 7 indicates that at least 7∕8th of the space in the
block is free.
Can have second-level free-space map
• In example below, each entry stores maximum from 4 entries of first-level
free-space map
▪ Free space map written to disk periodically, OK to have wrong (old) values
for some entries (will be detected and fixed)
Sequential File Organization
‫▪ مناسب للتطبيقات التي‬
▪ Suitable for applications that require sequential processing of ‫تتطلب معالجة متسلسلة لـ‬
the entire file
‫امللف بأكمله ▪ يتم ترتيب‬
▪ The records in the file are ordered by a search-key ‫السجالت املوجودة في امللف‬
iw.sk i
Sequential File Organization (Cont.)
skins it onis i ni ‫ استخدم‬- ‫▪ الحذف‬
wi g i sat in e is z si
▪ Deletion – use pointer chains
▪ Insertion –locate the position where the record is to be inserted
g wht • if there is free space insert there
an alsees go ok
insent • if no free space, insert the record in an overflow block
• In either case, pointer chain must be updated
▪ Need to reorganize the file
from time to time to restore
sequential order
i am sit in I 28
WE Eat is up Il Il N
Multitable Clustering File Organization
‫قم بتخزين عدة عالقات في ملف واحد باستخدام منظمة ملف تجميع متعددة‬
Store several relations in one file using a multitable clustering
file organization
department
instructor
C
multitable clustering create cluster command
of department and cluster key is dept name.
instructor
Multitable Clustering File Organization (cont.)
as a s
▪ good for queries involving department ⨝ instructor, and for queries
involving one single department and its instructors
▪ bad for queries involving only department
▪ results in variable size records
▪ Can add pointer chains to link records of a particular relation
- ‫▪ مناسب لالستفسارات املتعلقة بمدرس القسم ولالستفسارات إشراك قسم واحد ومعلميه‬
- ‫قسما فقط‬
ً ‫سيئ لالستعالمات التي تتضمن‬
- ‫ينتج عنه سجالت متغيرة الحجم‬
‫يمكن إضافة سالسل مؤشرات لربط سجالت عالقة معينة‬
‫ يمكن تقسيم السجالت‬:‫تجزئة الجدول‬
Partitioning ‫ذات العالقة إلى أصغر‬
an 851 I im sitI t l
▪ Table partitioning: Records in a relation can be partitioned into smaller
relations that are stored separately
▪ E.g., transaction relation may be partitioned into
It
transaction_2018, transaction_2019, etc.
▪ Queries written on transaction must access records in all partitions

is a a ie wi so do I
• Unless querysa
i
has a selection such as year=2019, in which case only
one partition in needed
N
▪ Partitioning
I
l et
•
I wi o txt i wa i g
Reduces costs of some operations such as free space management
in if I É im F I 061,5
• Allows different partitions to be stored on different storage devices
▪ E.g., transaction partition for current year on SSD, for older years
on magnetic disk
solidstatedrive d
in Ms Wi axe
oket km data
ma am in s.iqData Dictionary Storage ‫يقوم قاموس البيانات‬
Data
t t w wt w Data st.ms so
The Data dictionary (also called system catalog) stores metadata; that is,
data about data, such as
10 ▪ Information about relations

• names of relations
• names, types and lengths of attributes of each relation
• names and definitions of views
• integrity constraints praimnykeynullvalue b i
▪ User and accounting information, including passwords am we
▪ Statistical and descriptive data Datasi m a sun
• number of tuples in each relation
▪ Physical file organization information
• How relation is stored (sequential/hash/…)
• Physical location of relation
▪ Information about indices (Chapter 14)
Relational Representation of System Metadata
44 w 81 if a 1 4 93 l d
▪ Relational
representation on my
disk
s
▪ Specialized data s
structures designed s
for efficient access,
in memory
iii i am i mis É
ownie ti s inou
it
me
admin
analyst
‫العالئقية التمثيل على القرص‬

‫ في الذاكرة‬، ‫▪ البيانات املتخصصةالهياكل املصممة للوصول الفعال‬
Data sina.im ‫▪ الكتل هي وحدات لكل من‬
O.nl I Sti a Storage Access .‫تخصيص التخزين ونقل البيانات‬
wife pata si b
most
▪ Blocks are units of both storage allocation and data transfer.
▪ Database system seeks to minimize the number of block transfers between

disks
misty
a
diversion the disk and memory. We can reduce the number of disk accesses by
law
after it keeping as many blocks as possible in main memory.
▪ Obiang's
Buffer – portion of
metastasis
main memory
soya ma oil se
available to store copies of disk blocks.
im
orgies mainmemory is Mihaly Buffer 13 um
g ▪ Buffer manager – subsystem responsible for allocating buffer space in main
memory. ran d
rimsie ewmdai
e eHands
wiersma
processing
am
‫▪ تستدعي‬
Buffer Manager ‫البرامج مدير املخزن‬
▪ Programs call on the buffer manager when they need a block from disk.
• If the block is already in the buffer, buffer manager returns the address of
1am IT
WE1 Data the block in main memory disk
18 I s i w w's is
I
• If the block is not in the buffer, the buffer manager
Bufferto so did as l
▪ Allocates space in the buffer for the block
a a web 41 Buffer is pal oh Is it
• Replacing (throwing out) some other block, if required, to make
space for the new block.
Buffer b so bi
a mix at I a e
• Replaced block written back to disk only if it was modified since
the most recent time that it was written to/fetched from the disk.
▪ Reads the block from the disk to the buffer, and returns the address of
the block in main memory to requester. isaim

Jew a
Buffer Manager -Buffer replacement strategy
too disk t.LI 1,512
▪ Pinned block: memory block that is not allowed to be written back to disk
• Pin done before reading/writing datadfrom a block :‫▪ كتلة مثبتة‬
‫كتلة ذاكرة ال‬
• Unpin done when read /write is complete wait t.is
a basin• Multiple concurrent pin/unpin operations possible
I Wide ▪ Keep a pin count, buffer block can be evicted only if pin count = 0
nasty
▪ Shared and exclusive locks on buffer tags looks's s i list
pinlunpinsiksbxo.ws.is
• Needed to prevent concurrent operations from reading page contents
twists she as they are moved/reorganized, and to ensure only one
move/reorganize at a time
was listen
E ist 4051 40 mn.gs iv girls
• Readers get shared lock, updates to a block require exclusive lock cow
Block
m.nw.wi• Locking rules: g us.ie
▪ Only one process can get exclusive lock at a time jam
a rites s buffer
i sie
▪ Shared lock cannot be concurrently with exclusive lock
▪ Multiple processes may be given shared lock concurrently
s www.wsis
i e.skobwsharedlock
Buffer w Jw I I l 431 051 2 it s t I ‫تستبدل معظم أنظمة‬
Buffer-Replacement Policies ‫التشغيل الكتلة األقل‬
▪ Most operating systems replace the block least recently used (LRU strategy)
• i te
Idea behind LRU – use pastipattern
sa vie in ya i am
of block references as a predictor of
future references
• t.I.tk
LRU can be bad for some queries
▪ Example of bad access pattern for LRU: when computing the join of 2 relations r
and s by a nested loops
tuple m e
for each tuple tr of r do
for each tuple ts of s do
if the tuples tr and ts match … iw.is
join
When there is no room left in the buffer, a block must be

evicted, that is, removed, from the buffer before a new
one can be read in. Most operating systems use a least
recently used (LRU) scheme, in which the block that was
referenced least recently is written back to disk and is
removed from the buffer
Buffer-Replacement Policies (Cont.)
one g processes wi d pis i win
sa is is
▪ Toss-immediate strategy – frees the space occupied by a block as soon
as the final tuple of that block has been processed
it s t
▪ Most recently used (MRU) strategy – system must pin the block
currently being processed. After the final tuple of that block has been
processed, the block is unpinned, and it becomes the most recently used
block. processed
aye wi sa wa e x i aim I
i pin a just
‫ تحرر املساحة التي يشغلها الكتل بأسرع ما‬- ‫▪ إستراتيجية القذف الفوري‬
‫يمكن‬
(MRU) ‫نظرا ألنه تمت معالجة املجموعة النهائية لتلك الكتلة ▪ إستراتيجية‬ً
‫ يجب على النظام تثبيت الكتلة‬- ‫مؤخرا‬
ً ‫املستخدمة‬
‫ بعد أن تم إجراء املجموعة النهائية لتلك الكتلة‬.‫قيد املعالجة حاليا‬
‫ وتصبح أحدث استخدامات‬، ‫ يتم إلغاء تثبيت الكتلة‬، ‫تمت معالجته‬
.‫الكتلة‬
9 im g s1st w w j
Optimization of Disk Block Access
‫يدعم مديرو املخزن‬
‫املخرجات القسرية للكتل‬
s p.jo qt II é mi I six o pas system w yo t o g p ie
bi ti ▪ Buffer managers support forced output of blocks for the purpose of recovery
5 is w son i
ask aswrite
Nonvolatile buffers speed up disk writes by writing blocks to a non-
volatile RAM or flash buffer immediately I E a w
rmmo so of wi u o Eso wi a t.si as an Buffywww.wl
• Writes can be reordered to minimize disk arm movement di
▪ Log disk – a disk devoted to writing a sequential log of block updates
• Used exactly like nonvolatile RAM
www.t.am al 0418 I o to e
▪ Write to log disk is very fast since no seeks are required
f
▪ Journaling file systems write data in-order to NV-RAM or log disk
• Reordering without journaling: risk of corruption of file system data
risk if u É 561 43 o ul s t
431 T I I b in WI Is 65 at
16 M at A
Column-Oriented Storage
ً ‫▪ يُعرف‬
‫أيضا باسم التمثيل العمودي‬
▪ Also known as columnar representation ‫▪ تخزين كل سمة من سمات العالقة‬
▪ Store each attribute of a relation separately ‫بشكل منفصل مثال‬
▪ Example new II 138
Columnar Representation :‫▪ الفوائد‬
‫ اإلخراج‬/ ‫• تقليل اإلدخال‬
‫في حالة الوصول إلى بعض‬
▪ Benefits:
He a aim I a lady
• Reduced IO if only some attributes are accessed
• Improved CPU cache performance si si i m
Into
• Improved compression imJiya
• Vector processing on modern CPU architectures
▪ Drawbacks
• Cost of tuple reconstruction from columnar representation
• Cost of tuple deletion and update
tax am
• Cost of decompression
www.t▪ Columnar representation found to be more efficient for decision support than
noisy row-oriented representation
▪ Traditional row-oriented representation preferable for transaction processing
▪ Some databases support both representations
• Called hybrid row/column stores
rowrepresentation it d
and
columnpresentation
ORC :‫والباركيه‬
Columnar File Representation ‫ملف‬
Optimized Row Columnar ( ORC) file format

format
imized ▪ ORC and Parquet: file Now on
wcolumnar
moon
formats with columnar me awie
representation
storage inside file Columnar
▪ Very popular for big-data

applications
▪ Orc file format shown on
right:
Parquet is a column-oriented
binary file format
data www
Storage Organization in Main-Memory Databases
‫▪ يمكن تخزين‬
‫السجالت مباشرة‬
▪ Can store records directly in
memory without a buffer manager
byuse
▪ Column-oriented storage can be 2
used in-memory for decision
support applications
• Compression reduces
wi I I1,51 memory requirement
Wity asm
Ii sway
End of Chapter 13

1 Data Storage Structures-Ch13

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

1 Data Storage Structures-Ch13

Uploaded by

Copyright:

Available Formats

I Eid 064 0 S it I Gt w 45

Textbook Database System Concepts, 7

▪ The database is stored as a collection of files. Each file is a sequence

• Each file has records of one particular type only

• Different files are used for different relations

This case is easiest to implement; will consider variable length records

▪ We assume that records are smaller than a disk block

▪ Deletion of record i: alternatives:

▪ Deletion of record i: alternatives:

▪ Variable-length records arise in database systems in several ways:

t a wot gé asi wi web a ski in t d e was variable2 n I

▪ E.g., blob/clob types Binary Large Object / Character Large Object

▪ Records must be smaller than pages ti sa o ous

• Store as files managed by database

I Ini se on spy in E e 1ms i

theadvancedmethodofan indexedsequential accessmethod

technique to directlysearchthelocation of desireda

▪ E.g., transaction relation may be partitioned into

▪ Queries written on transaction must access records in all partitions

10 ▪ Information about relations

‫العالئقية التمثيل على القرص‬

▪ Database system seeks to minimize the number of block transfers between

the block in main memory to requester. isaim

When there is no room left in the buffer, a block must be

Optimized Row Columnar ( ORC) file format

▪ Very popular for big-data

You might also like