William Stallings

Computer Organization
and Architecture
8th Edition
Chapter 4
Cache Memory

Book by : Computer, Architecture and Organizations, 8th Edition ,William Stalling

Original Slides by : Adrian J Pullin
Cache Memory
 Lecture Outcomes
Understanding of:
•Computer Memory System
•Cache Memory principles
•Elements of Cache Design
•Mapping Function
 Characteristics
• Location
• Capacity
• Unit of transfer
• Access method
• Performance
• Physical type
• Physical characteristics
• Organisation
 Location
• CPU
• Internal
• External
 Capacity
• Word size
– The natural unit of organisation
• Number of words
– or Bytes
 Unit of Transfer
• Internal
– Usually governed by data bus width
• External
– Usually a block which is much larger than a word
• Addressable unit
– Smallest location which can be uniquely addressed
– Word internally
 Access Methods (1)
• Sequential
– Start at the beginning and read through in order
– Access time depends on location of data and previous location
– e.g. tape
• Direct
– Individual blocks have unique address
– Access is by jumping to vicinity plus sequential search
– Access time depends on location and previous location
– e.g. disk
 Access Methods (2)
• Random
– Individual addresses identify locations exactly
– Access time is independent of location or previous access
– e.g. RAM
• Associative
– Data is located by a comparison with contents of a portion of the
store
– Access time is independent of location or previous access
– e.g. cache
 Memory Hierarchy
• Registers
– In CPU
• Internal or Main memory
– May include one or more levels of cache
– “RAM”
• External memory
– Backing store
Memory Hierarchy - Diagram
 Performance
• Access time
– Time between presenting the address and getting the valid data
• Memory Cycle time
– Time may be required for the memory to “recover” before next
access
– Cycle time is access + recovery
• Transfer Rate
– Rate at which data can be moved
 Physical Types
• Semiconductor
– RAM
• Magnetic
– Disk & Tape
• Optical
– CD & DVD
• Others
– Bubble
– Hologram
 Physical Characteristics
• Decay
• Volatility
• Erasable
• Power consumption
 Organization
• Physical arrangement of bits into words
• Not always obvious
• e.g. interleaved
 The Bottom Line
• How much?
– Capacity
• How fast?
– Time is money
• How expensive?
 Hierarchy List
• Registers
• L1 Cache
• L2 Cache
• Main memory
• Disk cache
• Disk
• Optical
• Tape
 So you want fast?
• It is possible to build a computer which uses only static
RAM (see later)
• This would be very fast
• This would need no cache
– How can you cache cache?
• This would cost a very large amount
 Cache
• Small amount of fast memory
• Sits between normal main memory and CPU
• May be located on CPU chip or module
Cache and Main Memory
Cache/Main Memory Structure
 Cache operation – overview
• CPU requests contents of memory location
• Check cache for this data
• If present, get from cache (fast)
• If not present, read required block from main memory to cache
• Then deliver from cache to CPU
• Cache includes tags to identify which block of main memory is in
each cache slot
Cache Read Operation - Flowchart
 Cache Design
• Addressing
• Size
• Mapping Function
• Replacement Algorithm
• Write Policy
• Block Size
• Number of Caches
 Cache Addressing
• Where does cache sit?
– Between processor and virtual memory management unit
– Between MMU and main memory
• Logical cache (virtual cache) stores data using virtual addresses
– Processor accesses cache directly, not thorough physical cache
– Cache access faster, before MMU address translation
– Virtual addresses use same address space for different applications
• Must flush cache on each context switch
• Physical cache stores data using main memory physical addresses
 Size does matter
• Cost
– More cache is expensive
• Speed
– More cache is faster (up to a point)
– Checking cache for data takes time
 Comparison of Cache Sizes
Year of
Processor Type Introduction L1 cache L2 cache L3 cache
IBM 360/85 Mainframe 1968 16 to 32 KB — —
PDP-11/70 Minicomputer 1975 1 KB — —
VAX 11/780 Minicomputer 1978 16 KB — —
IBM 3033 Mainframe 1978 64 KB — —
IBM 3090 Mainframe 1985 128 to 256 KB — —
Intel 80486 PC 1989 8 KB — —
Pentium PC 1993 8 KB/8 KB 256 to 512 KB —
PowerPC 601 PC 1993 32 KB — —
PowerPC 620 PC 1996 32 KB/32 KB — —
256 KB to 1
PowerPC G4 PC/server 1999 32 KB/32 KB 2 MB
MB
IBM S/390 G4 Mainframe 1997 32 KB 256 KB 2 MB
IBM S/390 G6 Mainframe 1999 256 KB 8 MB —
Pentium 4 PC/server
High-end 2000 8 KB/8 KB 256 KB —
IBM SP server/ 2000 64 KB/32 KB 8 MB —
CRAY MTAb supercomputer
Supercomputer 2000 8 KB 2 MB —
Itanium PC/server 2001 16 KB/16 KB 96 KB 4 MB
SGI Origin
High-end server 2001 32 KB/32 KB 4 MB —
2001
Itanium 2 PC/server 2002 32 KB 256 KB 6 MB
IBM POWER5 High-end server 2003 64 KB 1.9 MB 36 MB
CRAY XD-1 Supercomputer 2004 64 KB/64 KB 1MB —
 Mapping Function
• Cache of 64kByte
• Cache block of 4 bytes
– i.e. cache is 16k (214) lines of 4 bytes
• 16MBytes main memory
• 24 bit address
– (224=16M)
 Direct Mapping
• Each block of main memory maps to only one cache line
– i.e. if a block is in cache, it must be in one specific
place
• Address is in two parts
• Least Significant w bits identify unique word
• Most Significant s bits specify one memory block
• The MSBs are split into a cache line field r and a tag of
s-r (most significant)
 Direct Mapping Address Structure

Tag s-r Line or Slot r Word w

8 14 2

• 24 bit address
• 2 bit word identifier (4 byte block)
• 22 bit block identifier
– 8 bit tag (=22-14)
– 14 bit slot or line
• No two blocks in the same line have the same Tag field
• Check contents of cache by finding line and checking Tag
Direct Mapping Cache Organization
Direct Mapping
Example
 Direct Mapping Summary
• Address length = (s + w) bits
• Number of addressable units = 2s+w words or bytes
• Block size = line size = 2w words or bytes
• Number of blocks in main memory = 2s+ w/2w = 2s
• Number of lines in cache = m = 2r
• Size of tag = (s – r) bits
 Direct Mapping pros & cons
• Simple
• Inexpensive
• Fixed location for given block
– If a program accesses 2 blocks that map to the same
line repeatedly, cache misses are very high
 Associative
Associative Mapping
Mapping
Example Example
 Associative Mapping Address Structure

Tag 22 bit Word

2 bit

• 22 bit tag stored with each 32 bit block of data

• Compare tag field with tag entry in cache to check for hit
• Least significant 2 bits of address identify which 16 bit
word is required from 32 bit data block
• e.g.
– Address Tag Data Cache
line
– FFFFFC FFFFFC 24682468 3FFF
 Associative Mapping Summary
• Address length = (s + w) bits
• Number of addressable units = 2s+w words or bytes
• Block size = line size = 2w words or bytes
• Number of blocks in main memory = 2s+ w/2w = 2s
• Number of lines in cache = undetermined
• Size of tag = s bits
 Set Associative Mapping
• Cache is divided into a number of sets
• Each set contains a number of lines
• A given block maps to any line in a given set
– e.g. Block B can be in any line of set i
• e.g. 2 lines per set
– 2 way associative mapping
– A given block can be in one of 2 lines in only one set
K-Way Set Associative Cache Organization
 Set Associative Mapping Address Structure

Word
Tag 9 bit Set 13 bit 2 bit

• Use set field to determine cache set to look in

• Compare tag field to see if we have a hit
• e.g
– Address Tag Data Set number
– 1FF 7FFC 1FF 12345678 1FFF
– 001 7FFC 001 11223344 1FFF
Two Way Set Associative Mapping Example
 Set Associative Mapping Summary

• Address length = (s + w) bits

• Number of addressable units = 2s+w words or bytes
• Block size = line size = 2w words or bytes
• Number of blocks in main memory = 2d
• Number of lines in set = k
• Number of sets = v = 2d
• Number of lines in cache = kv = k * 2d
• Size of tag = (s – d) bits
 Review Questions

❑What are the differences among direct mapping, associative mapping, and
set- associative mapping?
❑ For a direct-mapped cache, a main memory address is viewed as consisting
of three fields. List and define the three fields.
❑For an associative cache , a main memory address is viewed as consisting of two
fields. List and define the two fields.
❑For a set-associative cache, a main memory address is viewed as consisting of
three fields. List and define the three fields.
Thank you