Professional Documents
Culture Documents
Database Storage: Intro To Database Systems Andy Pavlo
Database Storage: Intro To Database Systems Andy Pavlo
03 Part I
ADMINISTRIVIA
OVERVIEW
COURSE OUTLINE
D I S K- O R I E N T E D A R C H I T E C T U R E
S TO R A G E H I E R A R C H Y
Faster
Smaller
CPU Registers
Expensive
Volatile CPU Caches
Random Access
Byte-Addressable DRAM
Non-Volatile SSD
Sequential Access
Block-Addressable
HDD
Slower
Network Storage Larger
Cheaper
CMU 15-445/645 (Fall 2019)
10
S TO R A G E H I E R A R C H Y
Faster
Smaller
CPU Registers
Expensive
CMU 15-721
CPU Caches
Memory DRAM
SSD
Disk HDD
Slower
Network Storage Larger
Cheaper
CMU 15-445/645 (Fall 2019)
10
S TO R A G E H I E R A R C H Y
Faster
Smaller
CPU Registers
Expensive
CMU 15-721
CPU Caches
Memory DRAM
Non-volatile Memory
SSD
Disk HDD
Slower
Network Storage Larger
Cheaper
CMU 15-445/645 (Fall 2019)
11
ACCESS TIMES
D I S K- O R I E N T E D D B M S
Database File
1 2 3 4 5 … Pages
Disk
CMU 15-445/645 (Fall 2019)
13
D I S K- O R I E N T E D D B M S
Buffer Pool
Memory
Database File
1 2 3 4 5 … Pages
Disk
CMU 15-445/645 (Fall 2019)
13
D I S K- O R I E N T E D D B M S
Get page #2 Execution
Engine
Buffer Pool
Memory
Database File
1 2 3 4 5 … Pages
Disk
CMU 15-445/645 (Fall 2019)
13
D I S K- O R I E N T E D D B M S
Get page #2 Execution
Engine
Buffer Pool
Directory
Memory
Database File
1 2 3 4 5 … Pages
Disk
CMU 15-445/645 (Fall 2019)
13
D I S K- O R I E N T E D D B M S
Get page #2 Execution
Engine
Buffer Pool
Directory Header
2
Memory
Database File
1 2 3 4 5 … Pages
Disk
CMU 15-445/645 (Fall 2019)
13
D I S K- O R I E N T E D D B M S
Get page #2 Execution
Engine
Buffer Pool
Directory Header
1 2 3 4 5 … Pages
Disk
CMU 15-445/645 (Fall 2019)
13
D I S K- O R I E N T E D D B M S
Get page #2 Execution
Lecture 5 Engine
Buffer Pool
Directory Header
1 2 3 4 5 … Pages
Disk
CMU 15-445/645 (Fall 2019)
15
W H Y N OT U S E T H E O S ?
On-Disk File
CMU 15-445/645 (Fall 2019)
15
W H Y N OT U S E T H E O S ?
On-Disk File
CMU 15-445/645 (Fall 2019)
15
W H Y N OT U S E T H E O S ?
On-Disk File
CMU 15-445/645 (Fall 2019)
15
W H Y N OT U S E T H E O S ?
On-Disk File
CMU 15-445/645 (Fall 2019)
15
W H Y N OT U S E T H E O S ?
On-Disk File
CMU 15-445/645 (Fall 2019)
15
W H Y N OT U S E T H E O S ?
On-Disk File
CMU 15-445/645 (Fall 2019)
16
W H Y N OT U S E T H E O S ?
W H Y N OT U S E T H E O S ?
Full Usage
There are some solutions to this
problem:
→ madvise: Tell the OS how you expect to
read certain pages.
→ mlock: Tell the OS that memory ranges
cannot be paged out.
→ msync: Tell the OS to flush memory
ranges out to disk. Partial Usage
W H Y N OT U S E T H E O S ?
D ATA B A S E S TO R A G E
T O D AY ' S A G E N D A
File Storage
Page Layout
Tuple Layout
F I L E S TO R A G E
S TO R A G E M A N A G E R
D ATA B A S E PA G E S
D ATA B A S E PA G E S
4KB
There are three different notions of
"pages" in a DBMS:
→ Hardware Page (usually 4KB)
→ OS Page (usually 4KB)
→ Database Page (512B-16KB)
PA G E S TO R A G E A R C H I T E C T U R E
D ATA B A S E H E A P
H E A P F I L E : PA G E D I R E C T O R Y Page
Data
The DBMS maintains special pages
that tracks the location of data pages Directory
Page
in the database files.
Data
The directory also records the number
of free slots per page.
…
Page
The DBMS has to make sure that the
directory pages are in sync with the Data
data pages.
CMU 15-445/645 (Fall 2019)
29
T O D AY ' S A G E N D A
File Storage
Page Layout
Tuple Layout
PA G E H E A D E R
Page
Every page contains a header of meta-
Header
data about the page's contents.
→ Page Size
→ Checksum
→ DBMS Version Data
→ Transaction Visibility
→ Compression Information
PA G E L AYO U T
Two approaches:
→ Tuple-oriented
→ Log-structured
T U P L E S TO R A G E
Page
How to store tuples in a page?
Num Tuples = 0
T U P L E S TO R A G E
Page
How to store tuples in a page?
Num Tuples = 30
Strawman Idea: Keep track of the Tuple #1
number of tuples in a page and then Tuple #2
just append a new tuple to the end.
Tuple #3
T U P L E S TO R A G E
Page
How to store tuples in a page?
Num Tuples = 230
Strawman Idea: Keep track of the Tuple #1
number of tuples in a page and then
just append a new tuple to the end.
→ What happens if we delete a tuple? Tuple #3
T U P L E S TO R A G E
Page
How to store tuples in a page?
Num Tuples = 30
Strawman Idea: Keep track of the Tuple #1
number of tuples in a page and then Tuple #4
just append a new tuple to the end.
→ What happens if we delete a tuple? Tuple #3
T U P L E S TO R A G E
Page
How to store tuples in a page?
Num Tuples = 30
Strawman Idea: Keep track of the Tuple #1
number of tuples in a page and then Tuple #4
just append a new tuple to the end.
→ What happens if we delete a tuple? Tuple #3
→ What happens if we have a variable-
length attribute?
S LOT T E D PA G E S
Slot Array
The most common layout scheme is
called slotted pages. Header
S LOT T E D PA G E S
Slot Array
The most common layout scheme is
called slotted pages. Header
S LOT T E D PA G E S
Slot Array
The most common layout scheme is
called slotted pages. Header
LO G -S T R U C T U R E D F I L E O R G A N I Z AT I O N
Page
Instead of storing tuples in pages, the
DBMS only stores log records. INSERT id=1,val=a
New Entries
INSERT id=2,val=b
…
attributes that were modified.
LO G -S T R U C T U R E D F I L E O R G A N I Z AT I O N
Page
To read a record, the DBMS scans the
log backwards and "recreates" the INSERT id=1,val=a
Reads
DELETE id=4
INSERT id=3,val=c
UPDATE val=X (id=3)
UPDATE val=Y (id=4)
…
CMU 15-445/645 (Fall 2019)
35
LO G -S T R U C T U R E D F I L E O R G A N I Z AT I O N
Page
To read a record, the DBMS scans the
log backwards and "recreates" the INSERT id=1,val=a
…
CMU 15-445/645 (Fall 2019)
35
LO G -S T R U C T U R E D F I L E O R G A N I Z AT I O N
Page
To read a record, the DBMS scans the
log backwards and "recreates" the id=1,val=a
id=2,val=b
tuple to find what it needs. id=3,val=X
id=4,val=Y
Build indexes to allow it to jump to
locations in the log.
T O D AY ' S A G E N D A
File Storage
Page Layout
Tuple Layout
T U P L E L AYO U T
TUPLE HEADER
Tuple
Each tuple is prefixed with a header Header Attribute Data
that contains meta-data about it.
→ Visibility info (concurrency control)
→ Bit Map for NULL values.
T U P L E D ATA
Tuple
Attributes are typically stored in the Header a b c d e
order that you specify them when you
create the table.
CREATE TABLE foo (
a INT PRIMARY KEY,
This is done for software engineering b INT NOT NULL,
reasons. c INT,
d DOUBLE,
We re-order attributes automatically e FLOAT
in CMU's new DBMS… );
D E N O R M A L I Z E D T U P L E D ATA
D E N O R M A L I Z E D T U P L E D ATA
foo
Can physically denormalize (e.g., "pre Header a b
join") related tuples and store them
together in the same page.
→ Potentially reduces the amount of I/O for
common workload patterns. bar
→ Can make updates more expensive. Header c a
Header c a
Header c a
D E N O R M A L I Z E D T U P L E D ATA
foo
Can physically denormalize (e.g., "pre Header a b c c c …
join") related tuples and store them
together in the same page. foo bar
→ Potentially reduces the amount of I/O for
common workload patterns.
→ Can make updates more expensive.
D E N O R M A L I Z E D T U P L E D ATA
foo
Can physically denormalize (e.g., "pre Header a b c c c …
join") related tuples and store them
together in the same page. foo bar
→ Potentially reduces the amount of I/O for
common workload patterns.
→ Can make updates more expensive.
RECORD IDS
C O N C LU S I O N
NEXT CLASS
Value Representation
Storage Models