Memory Organization 1

Lecture 40
Overview

 Memory Hierarchy

 Main Memory

 Auxiliary Memory

 Associative Memory

 Cache Memory

 Virtual Memory
Memory Organization 2
Lecture 40
Memory Hierarchy
Memory Hierarchy is to obtain the highest possible access speed while
minimizing the total cost of the memory system

Magnetic
tapes I/O Main
processor memory
Magnetic
disks

CPU Cache
memory

Register

Cache

Main Memory

Magnetic Disk

Magnetic Tape
Memory Organization 3
Lecture 40
Main Memory
RAM and ROM Chips
Typical RAM chip
Chip select 1 CS1
Chip select 2 CS2
Read RD 128 x 8 8-bit data bus
RAM
Write WR
7-bit address AD 7

CS1 CS2 RD WR Memory function State of data bus

0 0 x x Inhibit High-impedence
0 1 x x Inhibit High-impedence
1 0 0 0 Inhibit High-impedence
1 0 0 1 Write Input data to RAM
1 0 1 x Read Output data from RAM
1 1 x x Inhibit High-impedence

Typical ROM chip

Chip select 1 CS1

Chip select 2 CS2
512 x 8 8-bit data bus
ROM
9-bit address AD 9
Memory Organization 4
Lecture 40
Memory Address Map
Address space assignment to each memory chip

Example: 512 bytes RAM and 512 bytes ROM

Hexa Address bus

Component address 10 9 8 7 6 5 4 3 2 1
RAM 1 0000 - 007F 0 0 0 x x x x x x x
RAM 2 0080 - 00FF 0 0 1 x x x x x x x
RAM 3 0100 - 017F 0 1 0 x x x x x x x
RAM 4 0180 - 01FF 0 1 1 x x x x x x x
ROM 0200 - 03FF 1 x x x x x x x x x

Memory Connection to CPU

-RAM and ROM chips are connected to a CPU through the data and
address buses
-- The low-order lines in the address bus select the byte within the
chips and other lines in the address bus select a particular chip
through its chip select inputs
Memory Organization 5
Lecture 40
Connection of Memory to CPU
Address bus CPU
16-11 10 9 8 7-1 RD WR Data bus

Decoder
3 2 1 0
CS1
CS2

Data
RD 128 x 8
RAM 1
WR
AD7

CS1
CS2

Data
RD 128 x 8
RAM 2
WR
AD7

CS1
CS2

Data
RD 128 x 8
RAM 3
WR
AD7

CS1
CS2

Data
RD 128 x 8
RAM 4
WR
AD7

CS1
CS2
Data

1- 7 512 x 8
8
9 } AD9 ROM
Memory Organization 6
Lecture 41
Auxiliary Memory
Information Organization on Magnetic Tapes
file i

block 1 block 2
EOF
R1
block 3
R2 R3 R4
R5
R6
block 3 IRG
R1
EOF R3 R2
R5 R4 block 1
block 2

Organization of Disk Hardware

Moving Head Disk Fixed Head Disk

Track
Memory Organization 7
Lecture 41
Associative Memory
- Accessed by the content of the data rather than by an address
- Also called Content Addressable Memory (CAM)
Hardware Organization
Argument register(A)

Key register (K)

Match
register

Input Associative memory

array and logic
M
Read m words
Write n bits per word
Memory Organization 8
Lecture 41
Organization of CAM
A1 Aj An

K1 Kj Kn

i= word Word 1 C11 C1j C1n M1

j-= bit in word
Word i Ci1 Cij Cin Mi

Word m Cm1 Cmj Cmn Mm

Bit 1 Bit j Bit n

Internal organization of a typical cell Cij Aj Kj

Input

Write

R S Match
F ij logic To Mi
Read

Output
Memory Organization 9
Lecture 41
Match Logic

K1 A1 K2 A2 Kn An

F'i1 F i1 F' i2 F i2 .... F' in F in

Mi
Memory Organization 10
Lecture 42
Cache Memory
Locality of Reference
- The references to memory at any given time interval tend to be confined
within a localized areas
- This area contains a set of information and the membership changes
gradually as time goes by
- Temporal Locality
The information which will be used in near future is likely to be in use
already( e.g. Reuse of information in loops)
- Spatial Locality
If a word is accessed, adjacent(near) words are likely accessed soon
(e.g. Related data items (arrays) are usually stored together;
instructions are executed sequentially)
Cache
- The property of Locality of Reference makes the cache memory systems work
- Cache is a fast small capacity memory that should hold those information
which are most likely to be accessed

Main memory CPU

Cache memory
Memory Organization 11
Lecture 42
Performance of Cache
Memory Access
All the memory accesses are directed first to Cache
If the word is in Cache; Access cache to provide it to CPU
If the word is not in Cache; Bring a block (or a line) including
that word to replace a block now in Cache

- How can we know if the word that is required is there ?

- If a new block is to replace one of the old blocks, which one should we
choose ?

Performance of Cache Memory System

Hit Ratio - % of memory accesses satisfied by Cache memory system

Te: Effective memory access time in Cache memory system
Tc: Cache access time
Tm: Main memory access time

Te = Tc + (1 - h) Tm

Example: Tc = 0.4 s, Tm = 1.2s, h = 0.85%

Te = 0.4 + (1 - 0.85) * 1.2 = 0.58s
Memory Organization 12
Lecture 42
Memory and Cache Mapping – (Associative
Mapping)
Mapping Function
Specification of correspondence between main memory blocks and cache
blocks
Associative mapping
Direct mapping
Associative Mapping Set-associative mapping
- Any block location in Cache can store any block in memory
-> Most flexible
- Mapping Table is implemented in an associative memory
-> Fast, very Expensive
- Mapping Table
Stores both address and the content of the memory word
address (15 bits)

Argument register

Address Data
01000 3450
CAM 02777 6710
22235 1234
Memory Organization 13
Lecture 42
Cache Mapping – direct mapping
- Each memory block has only one place to load in Cache
- Mapping Table is made of RAM instead of CAM
- n-bit memory address consists of 2 parts; k bits of Index field and n-k bits of Tag field
- n-bit addresses are used to access main memory and k-bit Index is used to access the
Cache
n -k k
Addressing Relationships Tag(6) Index(9)

00 000 32K x 12 000

512 x 12
Main memory Cache memory
Address = 15 bits Address = 9 bits
Data = 12 bits Data = 12 bits
77 777 777
Direct Mapping Cache Organization
Memory
address Memory data
Cache memory
00000 1220
Index
address Tag Data
00777 2340 000 00 1220
01000 3450

01777 4560
02000 5670

777 02 6710
02777 6710
Memory Organization 14
Lecture 42
Cache Mapping – direct mapping
Operation

- CPU generates a memory request with (TAG;INDEX)

- Access Cache using INDEX ; (tag; data)
Compare TAG and tag
- If matches -> Hit
Provide Cache(data) to CPU
- If not match -> Miss
Search main memory and replace the block from cache memory

Direct Mapping with block size of 8 words

000 01 3450 Tag Block Word

Block 0
007 01 6578
010 INDEX
Block 1
017

Block 63 770 02
777 02 6710
Memory Organization 15
Lecture 42
Cache Mapping – Set Associative Mapping
- Each memory block has a set of locations in the Cache to load

Set Associative Mapping Cache with set size of two

Index Tag Data Tag Data

000 01 3450 02 5670

777 02 6710 00 2340

Memory Organization 17
Lecture 42
Cache Write
Write Through
When writing into memory
If Hit, both Cache and memory is written in parallel
If Miss, Memory is written
For a read miss, missing block may be overloaded onto a cache
block
Memory is always updated
-> Important when CPU and DMA I/O are both executing
Slow, due to the memory access time
Write-Back (Copy-Back)
When writing into memory
If Hit, only Cache is written
If Miss, missing block is brought to Cache and write into Cache
For a read miss, candidate block must be written back to the
memory
Memory is not up-to-date, i.e., the same item in
Cache and memory may have different value
Memory Organization 18
Lecture 43
Virtual Memory
Give the programmer the illusion that the system has a very large memory, even
though the computer actually has a relatively small main memory

Address Space(Logical) and Memory Space(Physical)

address space memory space

virtual address Mapping

(logical address) physical address

address generated by programs actual main memory address

Address Mapping
Memory Mapping Table for Virtual Address -> Physical Address
Virtual address

Virtual Memory Main memory

address address Main
mapping memory
register table register

Physical
Address
Memory table Main memory
buffer register buffer register
Memory Organization 19
Lecture 43
Address Mapping
Address Space and Memory Space are each divided into fixed size group
of words called blocks or pages
Page 0
1K words group Page 1
Page 2
Address space Memory space Block 0
Page 3
N = 8K = 213 M = 4K = 212 Block 1
Page 4
Block 2
Page 5
Block 3
Page 6
Page 7
Organization of memory Mapping Table in a paged system
Page no. Line number
1 0 1 0 1 0 1 0 1 0 0 1 1 Virtual address

Table Presence
address bit
000 0 Main memory
001 11 1 Block 0
010 00 1 Block 1
011 0 01 0101010011 Block 2
100 0 Block 3
Main memory
101 01 1 address register
Memory page table 110 10 1
111 0 MBR

01 1
Memory Organization 20
Lecture 43
Associative Memory Page Table
Assume that
Number of Blocks in memory = m
Number of Pages in Virtual Address Space = n
Page Table
- Straight forward design -> n entry table in memory Inefficient storage space
utilization
<- n-m entries of the table is empty
- More efficient method is m-entry Page Table
Page Table made of an Associative Memory
m words; (Page Number: Block Number)
Virtual address
Page no.
1 0 1 Line number Argument register

1 0 1 0 0 Key register

0 0 1 1 1
0 1 0 0 0 Associative memory
1 0 1 0 1
1 1 0 1 0
Page no. Block no.
Page Fault
Page number cannot be found in the Page Table
Memory Organization 21
Lecture 43
Page Replacement
Decision on which page to displace to make room for
an incoming page when no free frame is available
Modified page fault service routine
1. Find the location of the desired page on the backing store
2. Find a free frame
- If there is a free frame, use it
- Otherwise, use a page-replacement algorithm to select a victim frame
- Write the victim page to the backing store
3. Read the desired page into the (newly) free frame
4. Restart the user process valid/
frame invalid bit swap
out
1 victim
page
2 change to
f 0 v i invalid victim
3
4 swap
f v reset page
table for desired
new page page in
backing store
page table

physical memory
Memory Organization 22
Lecture 43
Page Replacement Algorithms
FIFO Reference string
7 0 1 2 0 3 0 4 2 3 0 3 2 1 2 0 1 7 0 1
7 7 7 2 2 2 4 4 4 0 0 0 7 7 7
0 0 0 3 3 3 2 2 2 1 1 1 0 0
1 1 1 0 0 0 3 3 3 2 2 2 1
Page frames

FIFO algorithm selects the page that has been in memory the longest time
Using a queue - every time a page is loaded, its
- identification is inserted in the queue
Easy to implement
May result in a frequent page fault

0 1 2 0 3 0 4 2 3 0 3 2 1 2 0 1 7 0 1
7 7 7 2 2 2 2 2 7
0 0 0 0 4 0 0 0
1 1 3 3 3 1 1
Page frames
Memory Organization 23
Lecture 43
Page Replacement Algorithms
LRU

- LRU uses the recent past as an approximation of near future.

Replace that page which has not been

used for the longest period of time

Reference string
7 0 1 2 0 3 0 4 2 3 0 3 2 1 2 0 1 7 0 1
7 7 7 2 2 4 4 4 0 1 1 1
0 0 0 0 0 0 3 3 3 0 0
1 1 3 3 2 2 2 2 2 7
Page frames

- LRU may require substantial hardware assistance

- The problem is to determine an order for the frames
defined by the time of last use