Computer Architecture Unit 8

Unit 8 Memory Hierarchy Technology

Structure:
8.1 Introduction
Objectives
8.2 Memory Hierarchy
Cache memory organisation
Basic operation of cache memory
Performance of cache memory
8.3 Cache Addressing Modes
Physical address mode
Virtual address mode
8.4 Mapping
Direct mapping
Associative mapping
8.5 Elements of Cache Design
8.6 Cache Performance
Improving cache performance
Techniques to reduce cache miss
Techniques to decrease cache miss penalty
Techniques to decrease cache hit time
8.7 Shared Memory organisation
8.8 Interleaved Memory Organisation
8.9 Bandwidth and Fault Tolerance
8.10 Consistency Models
Strong consistency models
Weak consistency models
8.11 Summary
8.12 Glossary
8.13 Terminal Questions
8.14 Answers

8.1 Introduction
You can say that Memory system is the important part of a computer
system. The input data, the instructions necessary to manipulate the input
data and the output data are all stored in the memory.

Manipal University of Jaipur B1648 Page No. 171

Computer Architecture Unit 8

Memory unit is an essential part of any digital computer because computer

processes data only if it is stored somewhere in its memory. For example, if
computer has to compute f(x) = sinx for a given value of x, then first of all x
is stored somewhere in memory, then a routine is called that contains
program that calculates sine value of a given x. It is an indispensable
component of a computer. We will cover all this in this unit.
In the previous unit, we explored the software approach of exploiting
Instruction-level parallelism in which you studied types of branches, branch
handling, delayed branching, branch processing, and static branch
prediction. Also, you studied the Intel IA-64 architecture and Itanium
processor, ILP in the embedded and mobile markets.
In this unit, we will study memory hierarchy technology. We will cover cache
memory organisation, cache addressing modes, direct mapping and
associative caches. We will also discuss the elements of cache design,
techniques to reduce cache misses via parallelism, techniques to reduce
cache penalties, and techniques to reduce cache hit time. Also, we will
study the shared memory organisation and interleaved memory
organisation.
Objectives:
After studying this unit, you should be able to:
 explain the concept of cache memory organisation
 label different cache addressing modes
 explain the concept of mapping
 identify the elements of cache design
 analyse the concept of cache performance
 describe various techniques to reduce cache misses
 explain the concept of shared and interleaved memory organisation
 discuss bandwidth and fault tolerance
 discuss strong and weak consistency models

8.2 Memory Hierarchy

Computer memory is utilized for storing and retrieving data and instruction.
The memory system includes the managing and controlling of storage
devices along with information or algorithms contained in it. Basically
computers are used to enhance the speed of computing. Similarly the main

Manipal University of Jaipur B1648 Page No. 172

Computer Architecture Unit 8

aim of memory system is to give speedy and continuous access on memory

by CPU. Small computers do not require additional storage because they
have limited applications that can be easily fulfilled.
The General Purpose computers perform very well with the additional
storage capacity including the capacity of main memory. Main memory
directly deals with the processor. Auxiliary memory is a high-speed memory
which provides backup storage and not directly accessible by CPU but it is
connected with main memory. The early forms of auxiliary memory are
punched paper tape, punched cards and magnetic drums. Since 1980’s the
devices employed in auxiliary memory are tapes, optical and magnetic
disks. Cache memory is an extremely high speed memory utilized to boost
up the speed of computation by providing the required information and data
to the processor at high speed.
Cache memory is introduced in the system for just for overcoming the
difference of speed between main memory and CPU Cache memory stores
the program segments which is being executed in processor as well as the
temporary data required in current computations. Computer performance
rate increases because cache memory provides the segments and data at
very high speed. As Input/output processor is concerned with data transfer
among main memory and auxiliary memory, similarly cache memory is
concerned for information transfer between processor and main memory.
The objective of using memory system is to get maximum access speed
and to minimize the entire cost of memory organization.
Memories vary in their design, in their capacity and speed of operation that
is why we have a hierarchical memory system. A typical computer can have
all types of memories. According to their nearness to the CPU, memories
form a hierarchy structure as shown in figure 8.1.

S e c o n d a ry I/ O P ro c e s s o r M a in
S t o ra g e M e m o ry
D e vic e

A u x ilia ry
M e m o ry
CP U Cac he
M e m o ry

Figure 8.1: Memory Hierarchy in a Computer System

Manipal University of Jaipur B1648 Page No. 173

Computer Architecture Unit 8

Now, we let us discuss cache memory and the cache memory organisation.
8.2.1 Cache memory organisation
A cache memory is an intermediate memory between two memories
having large difference between their speeds of operation. Cache memory
is located between main memory and CPU. It may also be inserted between
CPU and RAM to hold the most frequently used data and instructions.
Communicating with devices with a cache memory in between enhances
the performance of a system significantly. Locality of reference is a common
observation that at a particular time interval, references to memory acts
limited for some localised memory areas. Its illustration can be given by
making use of control structure like 'loop'. Cache memories exploit this
situation to enhance the overall performance.
Whenever a loop is executed in a program, CPU executes the loop
repeatedly. Hence for fetching instructions, subroutines and loops act as
locality of reference to memory. Memory references also act as localised.
Table look-up procedure continually refers to memory portion in which table
is stored. These are the properties of locality of reference. Cache memory’s
basic idea is to hold the often accessed data and instruction in quick cache
memory, the total accessed memory time will attain almost the access time
of cache. The fundamental idea of cache organisation is that by keeping the
most frequently accessed instructions and data in the fast cache memory,
the average memory access time will reach near to access time of cache.
8.2.2 Basic operation of cache memory
Whenever CPU needs to access the memory, cache is examined. If the file
is found in the cache, it is read from the fast memory. If the file is missing in
cache then main memory is accessed to read the file. A set of files just
accessed by CPU is then transferred from main memory to cache memory.
8.2.3 Performance of cache memory
Cache memory performance is measured in terms of Hit Ratio. If the
processor detects a word in cache, while referring that word in main
memory is known to produce a “hit”. If processor cannot detect that word in
cache is known as “miss”. Hit ratio is a ratio of hits to misses. High hit ratio
signifies validity of "locality of reference". When the hit ratio is high, then the
processor accesses the cache memory rather than main memory.

Manipal University of Jaipur B1648 Page No. 174

Computer Architecture Unit 8

The main feature of cache memory is its spontaneous response. Hence,

time is not wasted in finding files in cache. When data is transformed from
main memory to cache memory this process is known as mapping process.
Self Assessment Questions
1. _______________ directly deals with the processor.
2. ____________ is a high-speed memory which provides backup
storage.
3. A ___________ memory is an intermediate memory between two
memories having large difference between their speeds of operation.
4. If the processor detects a word in cache, while referring that word in
main memory is known to produce a _________________.

8.3 Cache Addressing Modes

The operation to be performed is specified by the operation field of the
instruction. The execution of the operation is performed on some data
stored in computer registers or memory words. In program execution the
selection of operands depends upon the addressing mode of the instruction.
Addressing modes has a rule that says “the address field in the instruction is
translated or changed before the operand is referenced”. While accessing a
cache, the CPU can address the cache in two ways, as following
 Physical Address Mode
 Virtual Address Mode
Now let’s go into the details of these addressing modes
8.3.1 Physical address mode
In physical address mode, a physical memory address is used to access a
cache.
Implementation on unified cache: Generally, both instructions and data
are stored into the same cache. This design is called the Unified (or
Integrated) cache design. When it is implemented on unified cache, the
cache is indexed and tagged with physical address. When the processor
issues an address, the address is translated in Translation Lookaside
Buffer (TLB) or in Memory Management Unit (MMU) before any cache
lookup as illustrated in figure 8.2. TLB is a cache where a set of recently
looked entries is maintained.

Manipal University of Jaipur B1648 Page No. 175