Section 5.

Trends In Distributed File Systems

Professor: Dr. Zhang
CSc 8320
Advanced Operating Systems

Presented by:

Tu Tran
Outline

 New Hardware
 Scalability
 Wide Area Networking
 Mobile Users
 Fault Tolerance
 Multimedia
 Interesting Research Papers/Projects
New Hardware

 Costs continue to drop at an amazing rate

 Rapidly dropping memory costs make it possible to

have every larger databases in main memory
 Servers could have entire file system in main
memory

 Advantage:
• Gain in performance
• Simplify file system structure
New Hardware (cont’d)

• Simpler to store each file contiguously in

memory (UNIX – tree, MS-DOS – link listed)

 Disadvantage:
• Power fails, all files stored are lost. However,
current technology allows to back up
continuously files on tapes or other storage
media.

 RAM disk and file system

New Hardware (cont’d)
 Optical Disk
» Write-Once Read Many (WORM)
» CD-ROM/DVD burners
» Excellent backup and archiving method
» Increasingly cheap, but slow.

 Huge capacity networks

» Server with a main memory & a fast fiber optic network.
May not need client’s cache and server’s disk. Files are
backed up by optical disk.
New Hardware (cont’d)

» Problem: memory cache in multiprocessor is similar to

the case in which two clients are caching the same file.

» Solution: design a simple network interface which has a

bit map, one bit per cached file.

 Specialized hardware for sophisticated systems

» Real-time support

» Distributed synchronization and control

Scalability

 Distributed system size strongly affects algorithm

choice

» Working well for 100 machines means nothing for 10,000.

 Centralized algorithms do not scale well

» Opening a file requires to contact the centralized server to

record the fact that the file is open. As the system grows,
the algorithms do not work well.
Scalability (cont’d)

» Solution: partitioning the system into smaller units often

helps

 Broadcasts are a problem

» Consider CPU broadcasting one message per second

» N of these generate N interrupts at N machines
» Not a problem for N=10
» VERY problematic for N=10,000
Scalability (cont’d)

 Data structures become important with scaling

» Linear search easiest and fastest for 10

» Self abuse for even 100

 Strict semantics (Unix) are harder to implement

as systems scale

» Design Principle: use weakest semantics that make

sense
Scalability (cont’d)

» Trade off ease of programming with scalability

 Name space

» A single Unix-like file tree. How long can/should path

names get?

» Partition the tree into smaller trees.

Wide Area Networking

 Virtually all distributed system research has been

done in the context of LANs

» Considerable changes with WAN context

» Latency
» Loss
» Cost
» Interaction
Wide Area Networking (cont’d)

 WAN access of major economic importance

» WWW commerce
» Video on demand
» Distributed Virtual Environments

 Economies of scale

» IP phones
Mobile Users

 Network addressing is a big challenge - Mobile IP

» May be transparent to distributed computing level

 Often seen as highly variable communication

bandwidth

» Isolated
» Wireless
» Wired
Mobile Users (cont’d)

 Interesting effects on caching

» CODA file system claims to support mobility and

intermittent connection

 Rapidly Deployable Radio Network – RDRN[2]

» Wireless end-user and network nodes

» Steerable communication beams
» Self-organizing network structure
Fault Tolerance
 Most systems are not fault tolerant

» But the general population expects things to work

» Phone system  IP phones?

 Requires considerable redundancy

» Hardware
» Communication infrastructure
» Software
» Data – File replication
Fault Tolerance (cont’d)

 Downtimes and periodic crashes will become

less and less acceptable as computers
spread to non-specialists

» ATM machines
» Microwaves
» Phone system (IP mode)
Multimedia

 Current data files are rarely more than a few

megabytes

» Multimedia files can exceed gigabytes

» Compression clearly popular because of this and
has a fundamental affect on network requirements
and economics
Multimedia (cont’d)

 Video-on-demand

» Significant affect on network traffic

» Perhaps also on file systems
» Real-time support is interesting as well
Interesting Research Papers/Projects

 xFS: Serverless Network File Service [3]

 Distributed file systems for grid computing [4]

 Mobile and Distributed file systems [5,6,7]

 A Scalable Distributed File System [8]

 References
[1] Andrew S. Tanenbaum. Distributed
Operating Systems. Prentice-Hall,
Englewood Cliffs, NJ 07632, USA, 1995

[2] Rapidly Deployable Radio Network,

http://www.ittc.ku.edu/RDRN/

[3] xFS: Serverless Network File Service,

http://now.cs.berkeley.edu/Xfs/xfs.html

[4] http://www.gridbus.org/papers/gridtech.pdf
 References (cont’d)
[5] MFS: an Adaptive Distributed File System
for Mobile Hosts
http://www.cs.cornell.edu/Info/Projects/Spinglass/public_pdfs/mfs.pdf

[6] Azzedine Boukerche, Raed Al-Shaikh, Bo Marleau, "Disconnection-

Resilient File System for Mobile Clients," lcn, pp. 608-614, The IEEE
Conference on Local Computer Networks 30th Anniversary (LCN'05)l,
2005.

http://ieeexplore.ieee.org/iel5/10397/33047/01550932.pdf?tp=&arnumb
er=1550932&isnumber=33047

[7] Haddock-FS, http://www.gsd.inesc-id.pt/~jpbarreto/Haddock-FS.html

[8] DiFFS: a Scalable Distributed File System

http://www.hpl.hp.com/techreports/2001/HPL-2001-19.pdf