3.

System Memory

Hierarchical Mem Org 1

 System Memory:

 Introduction
 semiconductor memory technology
 hierarchical memory organization

Hierarchical Mem Org 2

Hierarchical Memory Organization

Hierarchical Mem Org 3

 Outline

 Hierarchical organization: goals

 Justification
 Design Requirements
 Cached memory system
 Performance factors
 Cache organization
 Cache management policies
 Memory hierarchy Examples

Hierarchical Mem Org 4

 Hierarchical Memory Organization:Goals

 The performance related considerations for

system memory
 Speed: access time should be as low as
possible to maximize performance in terms of
processing throughput.
 Size (capacity): it is desirable that memory be
of as large a capacity as possible so as to
maximize the quantity of information that may
be stored.
 Cost: economic feasibility places an upper limit
on system memory cost.

Hierarchical Mem Org 5

 Hierarchical Memory Organization: Goals

 The speed & size vs. cost dilemma.

 Memory technologies offering faster
speeds are invariably more expensive.

Realization of large system memory
with fast access time and at acceptable
cost is a challenge.

Hierarchical Mem Org 6

 Hierarchical Memory Organization: Goals

Solution:
 The hierarchical organization of system

memory.

Hierarchical Mem Org 7

Hierarchical Memory Organization:

3.3.2 Memory Hierarchy

 System memory is constructed as a

Hierarchy of memory components with
differing speeds.

Hierarchical Mem Org 8

 Memory Hierarchy

 Top of the hierarchy

 fastest memory components to hold
frequently required or “needed soon”
information.
 only very limited capacities are
economical feasible

Hierarchical Mem Org 9

 Memory Hierarchy

 Towards the bottom

 progressively slower memory
components
 to hold less frequently required
information
 larger capacities are possible.

Hierarchical Mem Org 10

 Typical Memory Hierarchy

 level0  Processor Registers

 level1  Cache memory
 level2  Main memory
 level3  Secondary storage

 [levelN  offline storage]

Hierarchical Mem Org 11

 Memory Hierarchy Operation

To access an item:

 First look for the item at the highest

level
 If the item not found at any level i,
look for it at level i+1.

Hierarchical Mem Org 12

Hierarchical Memory Organization: Justification

 The mean access time, Tamean will

depend on the % of access requests
satisfied at say Level1

Hierarchical Mem Org 13

Hierarchical Memory Organization: Justification

 The hierarchical design can only be

justifiable if it is indeed the case that a
very high % of access requests will be
satisfied at level1!

Hierarchical Mem Org 14

 Hierarchical Memory Organization: Justification

The Principal of locality:

 Memory references tend to be clustered

in certain regions in
 time

 space and

 ordering

Hierarchical Mem Org 15

 Hierarchical Memory Organization: Justification

 Items recently referenced are likely to be

referenced again (temporal locality).
 There is a tendency for a process to access
items whose addresses are near one another
(spatial locality).
 References to instructions and to data tend to
follow sequential order (sequential locality).

Hierarchical Mem Org 16

Hierarchical Memory Organization: Justification

 Therefore:
 by keeping the current “regions” in the
higher memory, performance close to that
of Level1, as measured by Tamean, is
realizable.

 The memory management system must try

to achieve this

Hierarchical Mem Org 17

Hierarchical Memory Design Requirements

 The information stored in an n level memory

hierarchy, L1 to Ln, should satisfy the three
important properties of

 inclusion
 coherence
 locality

Hierarchical Mem Org 18

 Hierarchical Memory Design Requirements

Inclusion
 Level n, contains a copy of all information
items in the system.
 During processing, a subset of items at level i
may be copied into level i-1 .

 So: IM1  IM2  IM3    Imn = I

Hierarchical Mem Org 19

 Hierarchical Memory Design Requirements

Coherency
 Accesses to different copies of same
information item in different memory
levels must be consistent.

Hierarchical Mem Org 20

 Hierarchical Memory Design Requirements

Locality of Reference:

 For any leveli, requests for information

at that level should be successful with
high probability.

Hierarchical Mem Org 21

 Hierarchical Memory Design Requirements

Locality of Reference:

 The highest level in the hierarchy at which a

copy of a given item is located should be
determined by the probability of that item
being required in the next (few) accesses.

Hierarchical Mem Org 22

 Memory hierarchy on Contemporary Systems

 Cache memory
 a small high speed memory to cache subsets of
main memory contents
 is implemented by the machine architecture.

 Virtual memory
 a large external memory a subset of which is
cached by main memory
 is is implemented by the operating system with
support from the machine architecture.

Hierarchical Mem Org 23

Cached Memory

Hierarchical Mem Org 24

 Cached Memory

 The cache (C) is a small but very fast

memory component intended to store a
subset of the contents of main memory (MM).
 The items in the cache at any time
should be those frequently accessed by
the processor or likely to be requested in
the immediate future.
 The objective is to minimize the number
of references to the main memory,
which is much slower.

Hierarchical Mem Org 25

 Cached Memory

 Components
 CPU
 MMU
 Cache and cache controller
 MM and memory controller
How it works

 CPU generates a virtual memory address

 MMU translates to a physical memory address
 The physical address is presented to the cache
controller

Hierarchical Mem Org 26

 Cached Memory

 Performance factors
 a cache hit
 a copy of the target is in cache
 the access request is satisfied by the cache

 a cache miss
 there is no copy of the target in the cache
 a copy of the target has to be brought in
from memory

Hierarchical Mem Org 27

 Cached Memory: Performance Factors

Performance factors
 Hit ratio:

 Search speed:

 Miss penalty:

Hierarchical Mem Org 28

 Outline

 Hierarchical organization: goals

 Justification
 Design Requirements
 Cached memory system
 Performance factors
 Cache management policies
 Memory hierarchy Examples
 [virtual to physical memory address translation]

Hierarchical Mem Org 29

 Cache management policies

Hierarchical Mem Org 30

 3.3.7 Cache management policies

 policies
 fetch policy:

 placement policy:
 replacement policy:
 update policy:
 Exercise 1
 goal of fetch, placement, replacement policies?
 goal of update policy?

Hierarchical Mem Org 31

Placement policies

 Defines where in the cache a block

copied in from memory may be placed

Hierarchical Mem Org 32

Placement policies
 Direct mapping:
 for each block there is only one frame to
which it may be copied

 Fully Associative:
– a block may be copied to any frame in the
cache

Hierarchical Mem Org 33

Placement policies
 Set associative:
– the cache frames are grouped into sets
– for each memory block there is only one set
to which it may be copied
– a block may be copied to any frame within
the set

Hierarchical Mem Org 34

Placement policies

 Comment on the possible effect of

placement policy on the performance of
a cached memory system.
 Exercise 2

Hierarchical Mem Org 35

 Full associative mapping

Fully Associative

 A block may be placed in any cache

frame.

 advantages? disadvantages? Exercise 3

Hierarchical Mem Org 36

 Set associative mapping

Set Associative mapping

 It indicates the set in which the block would be
found if a copy were in cache.

 advantages? disadvantages? Exercise 4

Hierarchical Mem Org 37

 Replacement Policy

 If a block is required to be brought into the

cache as a result of a miss,
 but all the possible target frames are

occupied
 one of the blocks must be replaced.

 Which one?
  replacement policy

Hierarchical Mem Org 38

 Example replacement policies

 Random:

 FIFO:

 LRU:

 NRU:

 which one is best? Implementation?

 Exercise 5
Hierarchical Mem Org 39
 Update policy (coherency management)

 The coherency requirement implies that

copies of the same item must be
consistent with respect to all accesses.
 How to maintain this consistency?
 Write-through

 Write-back
 Research on the above – Exercise 6

Hierarchical Mem Org 40

 Split vs. Unified Cache

 A split cache implies a separate cache for

data and for instructions.
 An unified cache implies that data and
instructions share the same cache memory.

 Explain the advantage, if any, of having a

split cache – Exercise 8

Hierarchical Mem Org 41

 Non-blocking cache

 In conventional cache systems, if there is a

cache miss, the processor must wait for the
block to be brought from memory into the
cache before it can proceed.

 Non-blocking cache allows other accesses to

cache to take place as the block is being
fetched from memory.

Hierarchical Mem Org 42

 some question on performance

 Explain the effect each of the following might

have on Tamean
 hit ratio  replacement policy
 miss penalty  update policy
 main memory speed  fetch policy
 system bus speed
 split cache
 non-blocking cache
 burst access mode
 memory interleaving

Hierarchical Mem Org 43

 System Memory: Summary

 Introduction
 design considerations; types; functions
 Semiconductor memory technology
 Underlying technology; types; organization;
performance enhancement
 Hierarchical memory organization
 goals; justification; cached memory: organization,
performance factors, examples; virtual memory
basic organization

Hierarchical Mem Org 44