Professional Documents
Culture Documents
ch04.2 - File System Impl
ch04.2 - File System Impl
Outline
File-System Structure
File-System Operations
Directory Implementation
Allocation Methods
Free-Space Management
Efficiency and Performance
Recovery
Example: WAFL File System
Operating System Concepts – 10th Edition 14.2 Silberschatz, Galvin and Gagne ©2018
1
Objectives
Operating System Concepts – 10th Edition 14.3 Silberschatz, Galvin and Gagne ©2018
File-System Structure
File structure
• Logical storage unit
• Collection of related information
File system resides on secondary storage (disks)
• Provided user interface to storage, mapping logical to physical
• Provides efficient and convenient access to disk by allowing data to
be stored, located retrieved easily
Disk provides in-place rewrite and random access
• I/O transfers performed in blocks of sectors (usually 512 bytes)
File control block (FCB) – storage structure consisting of information
about a file
Device driver controls the physical device
File system organized into layers
Operating System Concepts – 10th Edition 14.4 Silberschatz, Galvin and Gagne ©2018
2
Layered File System
Operating System Concepts – 10th Edition 14.5 Silberschatz, Galvin and Gagne ©2018
Operating System Concepts – 10th Edition 14.6 Silberschatz, Galvin and Gagne ©2018
3
File System Layers (Cont.)
Operating System Concepts – 10th Edition 14.7 Silberschatz, Galvin and Gagne ©2018
Operating System Concepts – 10th Edition 14.8 Silberschatz, Galvin and Gagne ©2018
4
File-System Operations
We have system calls at the API level, but how do we implement their
functions?
• On-disk and in-memory structures
Boot control block contains info needed by system to boot OS from that
volume
• Needed if volume contains OS, usually first block of volume
Volume control block (superblock, master file table) contains volume
details
• Total # of blocks, # of free blocks, block size, free block pointers or array
Directory structure organizes the files
• Names and inode numbers, master file table
Operating System Concepts – 10th Edition 14.9 Silberschatz, Galvin and Gagne ©2018
Per-file File Control Block (FCB) contains many details about the file
• Typically inode number, permissions, size, dates
• NFTS stores into in master file table using relational DB structures
Operating System Concepts – 10th Edition 14.10 Silberschatz, Galvin and Gagne ©2018
5
In-Memory File System Structures
Mount table storing file system mounts, mount points, file system types
System-wide open-file table contains a copy of the FCB of each file
and other info
Per-process open-file table contains pointers to appropriate entries in
system-wide open-file table as well as other info
The following figure illustrates the necessary file system structures
provided by the operating systems
Plus buffers hold data blocks from secondary storage
Open returns a file handle for subsequent use
Data from read eventually copied to specified user process memory
address
Operating System Concepts – 10th Edition 14.11 Silberschatz, Galvin and Gagne ©2018
Operating System Concepts – 10th Edition 14.12 Silberschatz, Galvin and Gagne ©2018
6
Directory Implementation
Operating System Concepts – 10th Edition 14.13 Silberschatz, Galvin and Gagne ©2018
An allocation method refers to how disk blocks are allocated for files:
Each file occupies set of contiguous blocks
• Best performance in most cases
• Simple – only starting location (block #) and length (number of blocks)
are required
• Problems include:
Finding space for file,
Knowing file size,
External fragmentation, need for compaction off-line (downtime)
or on-line
Operating System Concepts – 10th Edition 14.14 Silberschatz, Galvin and Gagne ©2018
7
Contiguous Allocation
LA/512
Operating System Concepts – 10th Edition 14.15 Silberschatz, Galvin and Gagne ©2018
Extent-Based Systems
Many newer file systems (i.e., Veritas File System) use a modified
contiguous allocation scheme
Extent-based file systems allocate disk blocks in extents
An extent is a contiguous block of disks
• Extents are allocated for file allocation
• A file consists of one or more extents
Operating System Concepts – 10th Edition 14.16 Silberschatz, Galvin and Gagne ©2018
8
Allocation Methods - Linked
Operating System Concepts – 10th Edition 14.17 Silberschatz, Galvin and Gagne ©2018
Linked Allocation
Each file is a linked list of disk blocks: blocks may be scattered anywhere
on the disk
Scheme. Ex: a file of 5 blocks: start at block 9, and continue at block 16,
then block 1, then block 10, and finally block 25
Q
Mapping LA/511
R
Operating System Concepts – 10th Edition 14.18 Silberschatz, Galvin and Gagne ©2018
9
Allocation Methods – FAT
FAT (File Allocation Table) - An important variation on linked allocation
• Beginning of volume has table, indexed by block number
• Much like a linked list, but faster on disk and cacheable
• New block allocation simple
Ex:
• a file consisting of disk blocks
217, 618, and 339
index table
Operating System Concepts – 10th Edition 14.19 Silberschatz, Galvin and Gagne ©2018
Operating System Concepts – 10th Edition 14.20 Silberschatz, Galvin and Gagne ©2018
10
Combined Scheme: UNIX UFS
4K bytes per block, 32-bit addresses
More index blocks than can be addressed with 32-bit file pointer
Operating System Concepts – 10th Edition 14.26 Silberschatz, Galvin and Gagne ©2018
Performance
Best method depends on file access type
• Contiguous great for sequential and random
Linked good for sequential, not random
Declare access type at creation -> select either contiguous or linked
Indexed more complex
• Single block access could require 2 index block reads then data
block read
• Clustering can help improve throughput, reduce CPU overhead
For NVM, no disk head so different algorithms and optimizations
needed
• Using old algorithm uses many CPU cycles trying to avoid non-
existent head movement
• With NVM goal is to reduce CPU cycles and overall path needed
for I/O
Operating System Concepts – 10th Edition 14.27 Silberschatz, Galvin and Gagne ©2018
11
Free-Space Management
File system maintains free-space list to track available
blocks/clusters (Using term “block” for simplicity)
• the free-space list is implemented as a bitmap or bit vector.
Bit vector or bit map (n blocks)
0 1 2 n-1 1 block[i] free
bit[i] =
… 0 block[i] occupied
Ex: 001111001111110001100000011100000 ...
Block number calculation:
(number of bits per word) * (number of 0-value words) + offset of first 1 bit
CPUs have instructions to return offset within word of first “1” bit
Ex: blocks 2, 3, 4, 5, 8, 9, 10, 11, 12, 13, 17, 18, 25, 26, and
27 were free and the rest of the blocks were allocated.
Operating System Concepts – 10th Edition 14.30 Silberschatz, Galvin and Gagne ©2018
12
Free-Space Management (Cont.)
Grouping: A modification of the free-list approach
• Modify linked list to store address of next n-1 free blocks in first
free block, plus a pointer to next block that contains free-block-
pointers (like this one)
Operating System Concepts – 10th Edition 14.31 Silberschatz, Galvin and Gagne ©2018
Operating System Concepts – 10th Edition 14.32 Silberschatz, Galvin and Gagne ©2018
13
Free-Space Management (Cont.)
Operating System Concepts – 10th Edition 14.33 Silberschatz, Galvin and Gagne ©2018
Operating System Concepts – 10th Edition 14.34 Silberschatz, Galvin and Gagne ©2018
14
Efficiency and Performance (Cont.)
Performance
• Keeping data and metadata close together
• Buffer cache – separate section of main memory for frequently used
blocks
• Synchronous writes sometimes requested by apps or needed by OS
No buffering / caching – writes must hit disk before acknowledgement
Asynchronous writes more common, buffer-able, faster
• Free-behind and read-ahead – techniques to optimize sequential access
• Reads frequently slower than writes
Operating System Concepts – 10th Edition 14.35 Silberschatz, Galvin and Gagne ©2018
Page Cache
A page cache caches pages rather than disk blocks using virtual memory
techniques and addresses
Memory-mapped I/O uses a page cache
Routine I/O through the file system uses the buffer (disk) cache
This leads to the following figure
I/O Without a Unified Buffer Cache
Operating System Concepts – 10th Edition 14.36 Silberschatz, Galvin and Gagne ©2018
15
Recovery
Operating System Concepts – 10th Edition 14.37 Silberschatz, Galvin and Gagne ©2018
Operating System Concepts – 10th Edition 14.38 Silberschatz, Galvin and Gagne ©2018
16
Example: WAFL File System
Used on Network Appliance “Filers” – distributed file system
appliances
“Write-anywhere file layout”
Serves up NFS, CIFS, http, ftp
Random I/O optimized, write optimized
• NVRAM for write caching
Similar to Berkeley Fast File System, with extensive modifications
The WAFL File Layout
Operating System Concepts – 10th Edition 14.39 Silberschatz, Galvin and Gagne ©2018
Snapshots in WAFL
Operating System Concepts – 10th Edition 14.40 Silberschatz, Galvin and Gagne ©2018
17
The Apple File System
Operating System Concepts – 10th Edition 14.41 Silberschatz, Galvin and Gagne ©2018
Operating System Concepts – 10th Edition Silberschatz, Galvin and Gagne ©2018
18
Linked Allocation
block = pointer
Mapping
Q
LA/511
R
Block to be accessed is the Qth block in the linked chain of blocks
representing the file.
Operating System Concepts – 10th Edition 14.43 Silberschatz, Galvin and Gagne ©2018
19