Cache Memory

Basic Philosophy
Temporal Locality Spatial Locality

Basic Terms
Cache Block Miss/Hit Miss Rate/Hit Rate Miss Penalty Hit Time 3-Cs of caches
Conflict Compulsory Capacity

Direct Mapped Cache

Assume 5-bit address bus and cache with 8 entries

D 4 D3
TAG Processor D2 - D0 Index

Valid

TAG

DATA

Index 000 001 010 011 100 101 110 111

Data Bus

HIT

Direct Mapped Cache

First Load Valid

D 4 D3

TAG

DATA

Index 000 001 010 011 100 101 110 111

TAG
= 01

Processor D2 - D0 = 010

0 0 0 0 0 0 0 0

Data Bus LD R1, (01010) ;remember 5-bit address bus, assume data is 8-bit and AA16 is stored at this location First time, cause a MISS, data loaded from memory and cache HIT bit is set to 1

Direct Mapped Cache

After first load
Valid

D 4 D3

TAG

DATA

Index 000 001 010 011 100 101 110 111

TAG
= 01

Processor D2 - D0 = 010

0 0 1 0 0 0 0 0

01

AA

Data Bus LD R1, (01010) ; AA16 is stored at this location, Cache HIT bit is set to 1

Direct Mapped Cache

Second Load TAG = 11 Processor D2 - D0 = 010
Valid

D 4 D3

TAG

DATA

Index 000 001 010 011 100 101 110 111

0 0 1 0 0 0 0 0

01

AA

Data Bus LD R1, (11010) ; assume 99 at address 11010 Same index but different TAG will cause a MISS, data loaded from memory

Direct Mapped Cache

After Second Load
Valid

D 4 D3

TAG

DATA

Index 000 001 010 011 100 101 110 111

TAG
= 11

Processor D2 - D0 = 010

0 0 1 0 0 0 0 0

11

99

Data Bus LD R1, (11010) ;remember 5-bit address bus, assume 99 First time, same index but different TAG will cause a MISS, data loaded from memory

Cache Size Example Direct Mapped

32K X 48-bit Memory Processor Address Bus (32-bit)

Valid

TAG (15 bit)

DATA (32 bit)

1 1111 1111 1111 1111 1111 1111

0 0

Address Bus (A17 A2)

0 0 0

32 K Entries

0
0
0 0000 0000 0000 0000 0000 0000

A31- A2=18

Processor Address bus = 32 bit (A) Number of blocks in cache (entries) = 32K Tag Size = A- N- 2 = 32 15 2 (Byte offset) = 15

(15-bit)

Cache Storage = 128KB = 32 K Words (2N) with N = 15

Data Out

Cache Size = 128KB (data) + 32K X 15-bit (tag) + 32K X 1-bit (Hit bit) = 192KB

Cache Size Example (1) Two-Way Set Associative

Assume same processor (A = 32, D= 32) Assume same total storage of data = 128KB Two sets means we will have two direct mapped caches with 64KB (128/2) each. 64KB = 16K words To address 16K X 32-bit memory we need 14-bit index. Hence Tag Size = 32-14-2 = 16

Cache Size Example (1) Two-Way Set Associative

Valid

SET 1

16K X 49-bit Memories

(1 bit)

TAG (16 bit)

DATA (32 bit)

1111 1111 1111 1111 1111 1111

Valid

SET 2

(1 bit)

TAG (16 bit)

DATA (32 bit)

0 0 0

0 0

16 K Entries
Address Bus
(A16 A2)

0 0 0 0

Address Bus (A16 A2)

0 0 0 0

0000 0000 0000 0000 0000 0000

A31- A17

(16-bit)

(16-bit)

Size = 2 (Sets) X 16K X (32-bit + 16-bit + 1-bit) = 196KB

Data Out 2:1 MUX

Data Out

A31- A17

Cache Size Example (1) 4-Way Set Associative

Assume same processor (A = 32, D= 32) Assume same total storage of data = 128MB Four sets means we will have four direct mapped caches with 32KB (128/4) each. 32KB = 8K words To address 8K X 32-bit memory we need 13-bit address. Hence Tag Size = 32-13-2 = 17

Cache Size Example (1) 4-Way Set Associative

V

SET 1
TAG

8K X 50-bit Memories

17

SET 2
TAG

17

SET 3
TAG

17

SET 4
TAG

17

0 0 0

8M Entries
Address Bus (A15 A2)

0 0 0 0 0 0

8M Entries
Address Bus (A15 A2)

0 0 0 0 0 0

8M Entries
Address Bus (A15 A2)

0 0 0 0 0 0

Address Bus (A15 A2)

0 0 0 0

A31- A16

Data Out

Data Out

(17-bit)

(17-bit)

(17-bit)

(17-bit)

4:1 MUX

Size = 4 (Sets) X 8K X (32-bit + 17-bit + 1-bit) = 200KB

Data Out to processor

Data Out

A31- A16

A31- A16

A31- A16

Alpha 21264 Processor44-Bit Virtual Address

Organization of the data cache Alpha 21264

Byte Offset (A5 A0)
Valid

SET 1

512 Entries Cache (2 Sets) (Block Size = 64 bit)

DATA (64 bit)

(1 bit)

TAG (29 bit)

Valid

SET 2

(1 bit)

TAG (29 bit)

DATA (64 bit)

0 0
Index 512 entries Address Bus (A14 A6)

0 0
Index 512 entries Address Bus
(A14 A6)

0 0 0 0 0

0 0 0 0 0

A44- A15 (29-bit Tag)

(29-bit)

(29-bit)

Size = 2 (Sets) X 16K X (32-bit + 16-bit + 1-bit) = 196KB

Data Out 2:1 MUX

Data Out

A44- A15 (29-bit Tag)

Four Memory Hierarchy Questions

Where can a block be placed

Direct Mapped to Fully Associative

How a block is found

Tag Comparison

Which block should be replaced on a cache miss (only for sets)

LRU, Random, FIFO

4 Qs (Contd..)
What Happens on a Write?
Write Back Main Memory only updated when data is replaced from cache Write Through The information is updated in upper as well as lower level.
Write Allocate: Allocate data in cache on write Write No-Allocate: Only write to next level.

 Classifying Misses: 3 Cs

3 Cs of Caches

Compulsory The first access to a block is not in the cache, so the block must be brought into the cache. Also called cold start misses or first reference misses. (Misses in even an Infinite Cache) Capacity If the cache cannot contain all the blocks needed during execution of a program, capacity misses will occur due to blocks being discarded and later retrieved. (Misses in Fully Associative Size Cache) Conflict If block-placement strategy is set associative or direct mapped, conflict misses (in addition to compulsory & capacity misses) will occur because a block can be discarded and later retrieved if too many blocks map to its set. Also called collision misses or interference misses.