Memory Organization

We designed computers, with memory subsystem comprised solely of ROM and RAM
This is fine for simple computers that perform specific tasks, such as controlling a microwave oven
or comple! computers, a memory subsystem consisting only of such physical memory would be
relatively slow and somewhat limited
"omputer designers utili#e several methods to design a memory system hierarchy that maximizes
overall system performance at an acceptable cost
One important component of memory hierarchy is cache memory$ This is high speed memory that
reduces the amount of time the "%& needs in order to access data
Another component of memory hierarchy is virtual memory, which expands the amount of memory
the CPU can access while minimizing cost
6.1 Hierarchical Memory Systems:
A computer system usually is not constructed using a single type of memory
'everal types are used
They comprise the hierarchical memory system of the computer
The most well known element of the memory subsystem is the physical memory, which is constructed
using dynamic random access memory ()RAM* chips
As processor speeds increase, physical memory becomes a bottleneck in system performance
A computer system with only physical memory would spend most of its time waiting for data
This is the motivation behind using cache memory
"ache memory is constructed using 'RAM chips
"ache memory sits between "%& and main memory
A cache controller copies data from physical memory to cache memory before or when the "%& needs
it
"ache memory itself can be arranged hierarchically, usually with two levels
+$ ,+ cache
-$ ,- cache
The goal of cache memory is to minimi#e the processor.s memory access time at a reasonable cost
/n addition to the cache levels, each level may contain - separate caches
0 )ata cache
0 /nstruction cache
(Motivation: to improve performance, especially in pipelined systems)
%rinciple behind 1arvard architecture
On the otherhand of the hierarchy is virtual memory
"onsider a processor that supplies 2-3bit address
6.2 Cache Memory !ssociative Memory
!ssociative Memory:
"ache memory can be constructed using either 'RAM or associative memory, depending upon the
mapping scheme used
Associative memory is also called as Content Addressable Memory
Most of the types of memory are accessed by specifying the address (including 'RAM*
Associative memory is accessed differently
A memory unit accessed by content is called an Associative Memory or Content Addressable Memory
This memory is accessed simultaneously and in parallel on the basis of content rather than address
An associative memory is more expensive than random access memory, because each cell must have
storage capability as well as logic circuits for matching its content with e!ternal argument
or this reason, associative memories are used where the search time is very short and critical
Associative memory consists of set of words "alid #it
4ach word, in addition to data bits, contains additional bit 3 valid bit (labeled 5*
/t indicates whether the data is valid or not$ This is important in determining data matches $ata
re%ister
To read a value from associative memory, the "%& must specify the data value to be matched
This value is called argument or data$
Mas& 'e%ister
Mask register is used to in connection with checking portion of data bits
4ach bit that is to be checked is set to +
The other bits are set to 6
Mach 'e%ister
Match register contains one bit for each location in the associative memory$
/f a location generates a match, its bit in the match register is set to +, otherwise it is set to 6
Once match occurs, circuitry within associative memory would copy this data to its output register
/f more than one match occurs, the data can be read out se7uentially
(ritin% data to associative memory
/s straight forward
The "%& supplies data to data register and asserts write signal
The associative memory checks for a location whose valid bit is 6$ /f it finds, it stores the data in that
location and sets the valid bit to +
/f no such location is found, it clears a location in order to store data$$
Match )o%ic:
Associative memory checks each location in parallel
Match occurs if
(+* for every bit position that has a value of + in the mask register, the location.s bits are same as data
register
(-* the location.s valid bit is set to +
"onsider +8!9 associative memory whose content is given below$ What data should be stored in data
and mask registers to select the last location:
Cache memory Mappin% *echni+ues:
'eplacin% data in the cache:
)irect mapping offers the easiest solution to the replacement problem$
The associative cache allows any location in the physical memory to be mapped to any location in
cache$ Three typical replacement strategies are
0 /O
0 ,R&
0 Random
The set3associative cache also needs replacement strategy$ (igure ;$;*
Cache performance
"ache hits and cache misses
1it ratio
Average memory access time
Where Tc< cache access time, Tp<memory access time, h< hit ratio
VIRTUAL MEMORY
p C T h hT TM * + ( + =
Basic Idea: The basic idea of virtual memory is to create an illusion of memory that is as
large as a disk (in gigabytes) and as fast as memory (in nanoseconds).
Most advanced CPUs can address more memory locations than physically exist in
their computers. For examle! a comuter "hose CPU issues #$%bit addresses can
directly access &'( of memory! much more than most comuters have.
Terminology
Physical address: )n address in main memory is called physical address * the set of
such addresses is called memory space
Virt!al"Logical address : ) logical address is the address generated by CPU (or) an
address used by a rogrammer * the set of such addresses is called address space
Address s#ace o$ a Process:% +et of addresses available for a Process
Virt!al Memory: There are t"o rimary methods for imlementing virtual memory,
Paging
%egmentation
VIRTUAL MEMORY & PA'I('
-n Paging! the entire range of logical addresses (the addresses that can be outut by the
CPU)! is divided into contiguous blocks called! Pages.
.ach age is the same si/e! and each logical address resides exactly in one age.
Physical memory is divided into nonoverlaing $rames.
The si/e of each frame is the
same as the si/e of one age.
Consider 0elatively +imle CPU!
"hich has a logical address sace
"ith 1&2 locations. -n this
system! there is 312 of hysical
memory and the age si/e is &2.
The memory is sho"n in the
4gure given belo",
)ig: One #ossi*le memory con+g!ration $or the
Relati,ely %im#le -PU
Paging moves ages from the disk to frames of the hysical memory so data can be
accessed by the rocessor.
)ny age can occuy any frame.
This leads to several issues that must be resolved by a aging system,
5hen should a age be moved into
hysical memory6 (7emand Paging)
8o" does the CPU 4nd data in hysical
memory! esecially if its logical address
is not the same as its hysical address6
()ddress translation done by MMU)
5hat haens "hen all the frames have
ages and the CPU needs to access data from a age )ig: MMU
con+g!ration .ithin the memory hierarchy not currently
stored in hysical memory6 (0elacement )lgorithms)
)ll of these issues are handled by the memory management unit (MMU) as sho"n above.
The logical address is outut from the CPU to the MMU.
The MMU converts this address to a hysical address! "hich it sulies to the cache and
hysical memory.
-f the data is not located in hysical memory! it generates a Page )a!lt/ and moves that
age from the s"a disk to a frame (removing another age if necessary).
Throughout this rocess! the CPU has no information as to the actual hysical location it
is accessing.
The MMU uses a Page Ta*le to kee track of "hich ages are in "hich frames.
Consider a Page Table for the 0elatively +imle CPU described earlier! "ith 312 of hysical
memory and &2 age si/e. The Page Table has 31 entries (locations)! one entry for each
age in the logical address sace. .ach location contains )rame n!m*er! Valid *it (also
called Presence *it)! -o!nt and 0irty *it.
o )rame n!m*er: indicates in "hich frame (in main memory) the age from virtual
memory is loaded
o Valid *it: indicates "hether the virtual memory age is available in main memory
o -o!nt 1 0irty *its: are used in connection "ith age relacement. 7irty bit (also
called Modi$y *it) is needed to indicate if the age has been altered since it "as
last loaded into main memory. -f no change has been made! the age does not
have to be "ritten to the disk "hen it needs to be s"aed out
The MMU uses the age table to calculate the re9uired hysical addresses. To do so! the
MMU treats the logical address as t"o arts,
Logical address: 2Page (!m*er/ O3set4
)ig:Address Translation in Paging Mechanism )ig:
Address Translation E5am#le
This maing rocess "orks "hen the age resides in hysical memory. (ut "hat
haens "hen the age is not currently located in hysical memory6
Page )a!lt: 5hen the re9uired age is not currently located in hysical memory! age
fault (interrut) is generated * the MMU causes the age to be coied from the disk to a
frame in hysical memory. 5hile bringing re9uired age from disk to memory! if there are
unused frames! the re9uired age is coied there. :ther"ise! the MMU "ill use
0elacement )lgorithm.
0emand Paging: The rocess of bringing re9uired age from disk to main memory as
and when it is required (on demand) is called demand aging. The ractice of demand
aging is used by virtually all MMUs.
Im#ortant Points:
67 %!##ort (eeded $or Virt!al Memory:
o 8ard"are must suort aging and segmentation
o :erating system must be able to manage the movement of ages and;or
segments bet"een secondary memory and main memory
87 Each #rocess has its o.n #age ta*le
Translation Loo9aside B!3er:
-n Paging! each virtual memory reference can cause t"o hysical memory accesses,
:ne to fetch the age table
:ne to fetch the data
To overcome this roblem a high%seed cache is set u for age table entries. This is
called Translation Loo9aside B!3er :TLB;7 T<( contains age table entries that have
been most recently used. T<( can be constructed using )ssociative memory.
'iven a virtual address! rocessor examines the T<(.
-f age table entry is resent (T<( hit)! the frame number is retrieved and the real
address is formed.
-f age table entry is not found in the T<( (T<( miss)! the age number is used to index
the rocess age table.
)ig: Address Translation in Paging !sing TLB
VIRTUAL MEMORY:: %E'ME(TATIO(
+egmentation is another method of allocating memory.
-n +egmentation! the program is divided
into segments
) segment is a logical unit such as,
subroutine! or data structure
Unlike ages! segments can vary in size
The MMU uses %egment Ta*le to kee track
of the segments resident in
memory.
+ince a segment can start at one of many
addresses and can be of any si/e! each
segment table entry must include the start
address and segment size
%egmentation Architect!re
<ogical address consists of a t"o tule,
2segmentn!m*er/ o3set4 )ig!re:
User<s ,ie. o$ Program
Segment Table = mas logical address to hysical addresses
.ach table entry has,
*ase = contains the starting hysical address "here the segments reside in
memory.
limit = seci4es the length of the segment.
%egmentation & Address Translation
The logical address is artitioned into a segment number and o>set
The segment number is inut to the
segment table,
-f the segment is located in
memory! oututs the starting
address of the segment and the
segment si/e
-f the segment is not in hysical
memory! it generates %egment
)a!lt! "hich causes the MMU to
load the segment into memory
The o>set is comared to the segment
si/e.
)ig: Address
Translation in %egmentation
-f the o>set is greater than or e9ual to the segment si/e! it indicates that the location is
not art of the segment! an error is generated
-f the o>set is valid! then it is added to the start of the segment address to generate the
correct hysical address
)s "ith aging! a +egmented MMU can also have a T<( to seed u the translation
%egmentation :: 0ra.*ac9s
-n Paging! the age number is sent to the age table to roduce frame number. This value
is concatenated "ith the o>set to roduce hysical address
-n segmentation! the start address of the segment is added to the o>set to roduce
hysical address
The process of addition is much more time consuming than concatenation
-om#arison o$ Paging and %egmentation:
Paging su>ers from I(TER(AL )RA'ME(TATIO( (unused sace "ithin the block of
memory is called internal fragmentation)
+egmentation su>ers from E=TER(AL )RA'ME(TATIO( (unused sace bet"een blocks
in memory is called external fragmentation)
Paged %egmentation :Paging > %egmentation;
-t is ossible to combine segmentation and aging by constructing a segment $rom
#ages! rather than as one contiguous block of memory
-n this scenario! a logical address is broken into # arts )ig: Address
translation in Paged %egmentation
2%egment n!m*er/ Page n!m*er/
and O3set4
The +egment number is inut to the
%egment Ta*le
8o"ever! instead of oututting a start
address! the segment table oututs a
#ointer to one o$ se,eral #age ta*les
(one for each segment)
The age table then oututs the
corresonding frame number in main
memory! or issues a Page )a!lt if the
age is not resident in memory
The frame number is concatenated "ith
o>set to roduce the hysical address
Ad,antages
The allocation of segments to physical memory is simpler since it is no longer
necessary to 4nd one contiguous block large enough to hold entire segment. The ages
that comrise the segment can be located any"here in main memory
Tradeo3 :: )lthough external fragmentation is eliminated! internal fragmentation is
introduced
?o longer necessary to add o>set! quicker concatenation can be used to the frame
number
0isad,antages
T"o levels of table look us
Virt!al Memory :: Memory Protection
) Multitasking :+ may have several di>erent rograms (or arts of rograms ) resident in
memory simultaneously
These rogram comonents may belong to one user or to di>erent users
The :+ may also have comonents resident in memory
+ome mechanism is needed to ensure that one comonent does not over"rite another!
and that no comonent can read data from another comonent
-n addition! a comonent can be rotected from its o"ner by restricting its mode of
access
)ll of these fall under memory rotection
) referable otion to rotect memory is to #rotect the segment or #age! rather than
individual location
To accomlish this! rotection bits are added to
each entry in the segment or age table
These bits might signify that a segment or
age is read only! read;"rite! or execute! and
that it is a user segment or age or is o"ned by
the :+. This is referred to as %!#er,isor or
?ernal mode
)lso it is necessary to kee track of "hich
segments or ages are associated "ith "hich
users
A multi tasking OS may allow more than
one user or process to access a segment
%haring o$ segments
+hared resources! such as the dynamic link libraries (7<<s) in Microsoft 5indo"s! fall into
this category +o do! share data structures
)ig: %haring o$ %egments
Beyond the *asics o$ -ache and Virt!al Memory
+o far "e have seen fundamental rinciles and simli4ed imlementations of cache
and virtual memory
-n ractice! comlex structures are needed as the si/e of memory increases
Beyond the *asics o$ -ache Memory
-n ractice! $ levels of cache are used in the real "orld
They are,
o <3 cache
o <$ cache
<evel 3 cache is included "ithin the CPU! imroves the erformance of the system
-ncluding the <evel $ cache "ithin the CPU imroves erformance even more
Beyond the *asics o$ -ache Memory
)nother di>erence is the use of %PLIT -A-@E
-nstead of having a uni4ed le,el 6 cache that holds both instructions and data! an <3
cache is usually designed as t"o arallel caches
o Instr!ction cache ,, stores only rogram instructions
o 0ata cache :: stores only data
The comiler usually assigns instructions and data to di>erent ages or segments
This simli4es the assignment of values to caches
Moti,ation ,, behind dividing the cache into instruction and data caches is imroving
erformance! esecially in Pielined +ystems
Beyond the *asics o$ Virt!al Memory
For smaller comuter systems! the
aging and segmentation schemes for
managing @irtual Memory "ork "ell
8o"ever! their si/es become
rohibitively large as the si/e of the
virtual memory address sace increases
.xamle, CPU "ith #$%bit address si/e!
is caable of accessing & '( of memory
Page si/e = 312
Page Table contains $A12 (i.e.
$
3B
C$1$!3&&) entries in its age table
:ne "ay to resolve this roblem is to
imlement the age table as a
multilevel hierarchy
) Primary Table contains ointers to
secondary tables
The rimary table can be stored in
memory * secondary tables could be
moved in and out of hysical memory
as needed
The Al#ha micro#rocessor suorts
u to & levels of age tables
)ig: M!ltile,el
Page Ta*le @ierarchy
Real Aorld E5am#le: : Memory Management in a Penti!m"Aindo.s Personal
-om#!ter
Consider a Personal Comuter that has an -ntel Pentium Microrocessor and runs
Microsoft 5indo" ?T.
First "e look at the +ystemDs
Cache Memory! and then "e
examine its @irtual Memory
Memory hierarchy is sho"n in
the Figure.
The system has 312 of <3
cache "ithin the Pentium Microrocessor
The cache is divided evenly! "ith B2 for instructions (code) and B2 for data
-ode -ache
The Code Cache is $%"ay set%associative "ith a line si/e of #$ bytes
.ach line has $E%bit tag
The entries in this cache are "rite%rotected
The code cache also has its o"n #$%entry translation lookaside bu>er
The T<( is constructed using associative memory
0ata -ache
The data cache is also $%"ay set%associative "ith a line si/e of #$ bytes
-t also has a T<(! but its T<( is larger than the T<( of the code cache
The data cache T<( has 1& entries for age of si/e & 2(
Unlike code cache! the CPU can "rite data to the data cache
.ach individual line can be set for either "rite%back or "rite%through
The PC also has an external <$ cache
This <$ cache is constructed using +0)M * is !ni+ed cache
-t is &%"ay set%associative and has a si/e of $A12(
Virt!al memory
Unlike cache! "hich is managed exclusively by microrocessor! @irtual Memory is handled
by both the Pentium and 5indo"s ?T.
Together! they suort a &%'( address sace
The lo"%order $ '( address sace is allocated to individual rocesses
The uer $ '( are reserved for oerating system rocesses
The Pentium can use either segmentation or paging
(ut 5indo"s ?T virtual memory manager uses paging
The higher order bits in the #$ bits select an entry in the age table
The lo"er order 3E bits secify the o>set