Storage Systems

NPTEL Course
Jan 2012
(Lecture 01)
K. Gopinat
!n"ian !nstitute o# Science

!ntro"uction

$y Storage Systems%

Ear&ier' (processing ( storage) an" net)or*ing

No)' processing+ storage an" net)or*ing

,ast net)or*s ena-&e separation #rom .processing/

0e1ices+ Protoco&s+ Layers2Systems

3&" an" Ne) 0e1ices' Tape+ 0rum+ 0is*+ So&i" State

Protoco&s' N,S+ C&ou" storage 4P!

Layers2Systems' Goog&e ,S+ 5ai& storage

!ssues

3&"er' concurrency )it CP6+ an"&ing "e1ice "i1ersity

Ne)er' sca&e+ "istri-ution+ error mgmt+ security+ 7T+ 8oS+

managea-i&ity

$y is storage "i##erent%

Consi"er &ong term storage (mu&tip&e "eca"es)'

Store" "ata can -e accesse" "eca"es &ater9

,ormats+ "e1ices+ etc can cange

0ata not interpreta-&e un&ess au:i&iary

in#ormation a&so store" (7ecursion pro-&em9)

Consi"er security' $y storage security "i##erent

#rom+ say+ net)or* security%

Net)or* security is across .space/

Net)or* trans#ers appen )itin a sort time

(un&ess space pro-e net) pac*ets to P&uto9)

Storage security is across -ot .space an" time/

!# using *eys+ *eys may a1e to sur1i1e years9

Naming an" Storing

T)o important #unctions'

Gi1e a name to an o-;ect

Can in1o&1e some processing (eg. a"" name to an in"e:)

3r+ name itse&# may -e compute" -ase" on contents

Store an o-;ect

5ay in1o&1e oter rea"s or stores9

5ay in1o&1e signi#icant computation

Compression+ encryption+ co"ing+ or "e"up&ication' remo1e re"un"ancy

5ay nee" to *eep au:. in#ormation a-out o-;ect

5eta"ata 1s "ata< recursion% (meta"ata a-out meta"ata...)

5eta"ata &oss 1s "ata &oss

4ccess (r2))' "e1ice speci#ic aspects "etermine spee"

7ea"s se=uentia&+ non>se=uentia& or ran"om

$rites in>p&ace or out>o#>p&ace

Large persistent "ata structures

,or processing+ parts nee" to -e -rougt into mem.

T)o copies' .in memory/ an" .on>"is*/

4tomicity an" Consistency issues

4&goritmic aspects nee" to -e care#u&&y ta*en care o#

Num-er o# o-;ects can -e in -i&&ions

Si?e o# o-;ect can -e in giga-ytes

4s time progresses+ ne)er a&gs nee"e" as sca&e canges

5ai& "irectories can a1e 1000@s o# msgs+ eac a #i&e9

Creating a #i&e re=uires &oc*ing "irectory

Concurrent creates to same "irectory may -ecome &oc*>-oun"

Critica& )it )e->&e1e& storage systems

5any ne)er mo"e&s (eg. *ey>1a&ue stores) since c. 2000

E:t2 ,S
(#rom $i*ipe"ia)


Goog&e ,S
(#rom $i*ipe"ia)


0eep Storage Stac*

Aarious types o# a-stractions in stac*

0e1ice

B&oc*

,i&e

4pp&ication &e1e& (-u##ering in &i-c+ #or eg.)

,iner su-&ayers in eac &ayer

SCS! as upper ("e1ice>speci#ic)+ mi" (protoco& speci#ic)

an" &o)er (pysica& communication &ayer)

,or sca&a-i&ity+ net)or* stac* part o# storage stac*

4 goo" part o# stac* in *erne&

!ncreasing&y+ storage stac* migrating out o# *erne& )it

separation o# processing an" storage (eg. G,S)

Storage Caracteristics

Concurrency arises natura&&y

$i"e "isparity in spee"s o# memory an" storage

Ca1e to mas* s&o)ness o# storage

5a*e processing an" storage go at teir o)n rates an"

use interrupts (or sometimes po&&ing) to signa&
comp&etion o# s&o) storage operations

Cistoric reason )y operating systems "e1e&ope"

Storage as to -e typica&&y persistent o1er time

4mount increases typica&&y )it time

But not a&& imp o1er time< *eep imp part in #ast storage%

.Cacing/ an" .tiering/ arise natura&&y

Ca1e to coose 2 o# D' spee"+ capacity or cost

Storage Per#ormance

Storage o#ten s&o)est component

Cace9

$itin sing&e "e1ice+ e##iciency -y'

merging re=uests

sce"u&ing re=uests in an or"er tat is -est )rt "e1ice

(out>o#>or"er e:ecution commonp&ace)

Ciger &e1e& so#t)are as to )or* aroun" tis aspect

!# a particu&ar or"er re=uire"+ &e#t to .user/

Semantica&&y not muc guarantee"

4syncronous processing o#ten use"' aio

Para&&e&ism across mu&tip&e "e1ices2trea"s'

5u&tip&e Cea"s (0is*s)

5u&tip&e cips (SS0s)

3ptimi?ation ,rame)or*

0ue to s&o)ness o# "e1ices+ optimi?ation o#

accesses important

eg. )at to cace+ )at to pre#etc%

But usage patterns typica&&y not *no)n a priori

Big "i##erence in per#ormance )eter se=uentia&

access or ran"om

System s&o) i# too many on "eman" migrations #rom

s&o) to #ast tier o# storage (&atency "e&ays)

3#ten+ opts. critica& an" o1erri"e .semantics/

3ut>o#>or"er processing typica&

Comp&e: iger>&e1e& so#t)are

Learning on te ;o- important

Simp&e an" ro-ust meto"s use#u&

Storage Protoco&s

!nterrupt "ri1en rater tan )ait2po&&

3n comp&etion+ interrupt CP6 or CB4

To a1oi" interrupt o1erea"+ CB4 or simi&ar agents

Ce&ps Segmentation an" 7eassem-&y (S47)

Sp&it>pase transactions common

#or eg' on comp&etion o# (a &ong) see*+ s&a1e ta*es -us

Protoco& en"points pre#era-&y .1irtua&i?a-&e/

SCS! "e1ices can -e on an e&ectrica& -us+ net)or* or

!nternet i# pysica& &ayer an"&e" correct&y

Protoco&s sur1i1e muc &onger

0e1ices can a1e ar-itrary structure as &ong as tey spea*

SCS! protoco&

E1en -ig ser1ers9

Summary

Storage systems "esign as many rami#ications

#or te rest o# te system

Pro1i"e a-stractions -ase" on app&ication

nee"s an" "e1ices

0esign nee"s to -e sensiti1e to cost+ "e1ices+

managea-i&ity

!ntro"uce ne)er a-stractions )it time

eg. *ey 1a&ue stores

Storage systems nee" to sca&e to support &arge

sca&e computing systems