Module 5

Memory System
Text book: Computer organization &
Architecture- William Stallins
 Memory access methods
1. Sequential Access:-
In this method, the memory is accessed in a specific
linear sequential manner. The access time depends
on the location of the data. Eg. Magnetic tape
2. Random Access:-
Random access allows direct access to any memory
location with uniform access time. In this method,
memory cells are addressed using unique identifiers,
such as memory addresses. Eg: RAM, ROM
 Memory access methods
3. Direct Access:-
Direct access allows accessing memory locations
in any order without the need to sequentially
traverse the preceding locations. It utilizes a
direct access table or index to quickly locate the
desired memory location. Hard disk drives (HDD)
and solid-state drives (SSD) are examples of
direct access storage devices.
• 4. Associate Access:-
Associative access, also known as content-
addressable memory (CAM), allows data
retrieval based on content rather than memory
addresses. It compares the search pattern
against all memory locations simultaneously to
find a match. Associative access is commonly
used in caches and specialized hardware for
high-speed look ups. Eg: Cache
 Organization
• The basic element of a semiconductor memory
is the memory cell.
• They exhibit two stable (or semistable) states,
which can be used to represent binary 1 and 0.
• They are capable of being written into (at least
once), to set the state.
• They are capable of being read to sense the
state.
 Memory cell

• The select terminal, selects a memory cell for a read or

write operation.
• The control terminal indicates read or write. For
writing, the other terminal provides an electrical signal
that sets the state of the cell to 1 or 0.
 Flash memory
• Flash memory is a type of non-volatile computer memory
that can be electrically erased and reprogrammed. It is
commonly used in various electronic devices, such as USB
flash drives, solid-state drives (SSDs), memory cards,
smartphones, and other portable devices.
• Flash memory is based on a technology called floating-gate
transistors. These transistors store data as electrical
charges within the floating gate, which is isolated from the
rest of the transistor. The presence or absence of electrical
charges in the floating gate determines the state of each
memory cell, representing binary values (0s and 1s).
Types of RAM
Cache memory principles
• The cache consists of m blocks, called lines
• Each line contains K words, plus a tag of a few bits
• Each line also includes control bits (not shown),
such as a bit to indicate whether the line has been
modified since being loaded into the cache.
• The length of a line, not including tag and control
bits, is the line size.
• The line size may be as small as 32 bits.
• The number of lines is considerably less than the
number of main memory blocks.
• Each line includes a tag that identifies which
particular block is currently being stored.
• At any time, some subset of the blocks of
memory resides in lines in the cache.
• If a word in a block of memory is read, that
block is transferred to one of the lines of the
cache.
Cache read operation
 Cache read operation
• The processor generates the read address (RA)
of a word to be read.
• If the word is contained in the cache, it is
delivered to the processor.
• Otherwise, the block containing that word is
loaded into the cache, and the word is
delivered to the processor
Typical cache organization
• In this organization, the cache connects to the
processor via data, control, and address lines.
• The data and address lines also attach to data and
address buffers from which main memory is
reached.
• Data and address buffers attach to a system bus
• When a cache hit occurs, the data and address
buffers are disabled and communication is only
between processor and cache
• When a cache miss occurs, the desired address is
loaded onto the system bus and the data are
returned through the data buffer to both the cache
and the processor.
• When the processor attempts to access data, it
first checks the cache for the data.
• If the cache hits, the data is available
immediately.
• If the cache misses, the processor fetches the
data from main memory and places it in the
cache for future use.
• To accommodate the new data, the cache must
replace old data.
 Elements of Cache design: Mapping
techniques
• Because there are fewer cache lines than main memory blocks,
an algorithm is needed for mapping main memory blocks into
cache lines.
• Further, a means is needed for determining which main memory
block currently occupies a cache line.
• The relationship between the address of data in main memory
and the location of that data in the cache is called the mapping.
Each memory address maps to exactly one set in the cache.
• The choice of the mapping function dictates how the cache is
organized.
• Three techniques can be used: direct, associative, and set
associative
Direct Mapping
• Each block of main memory maps into one unique line of the
cache.
• First m blocks map to the m lines of cache
• The next m blocks of main memory map into the cache in the
same fashion
• Because many addresses map to a single line, the cache must
also keep track of the address of the data actually contained in
each set.
• Each main memory address can be viewed as consisting of three
fields:tag, line, word
- The least significant w bits identify a unique word or byte within a
block of main memory;
– The remaining s bits specify one of the 2s blocks of main memory.
– The cache logic interprets these s bits as a tag of s - r bits (most
significant portion) and a line field of r bits.
– This latter field identifies one of the m = 2r lines of the cache.
• The line field is used to directly select a cache line. If
the cache line is already occupied, the tag stored in that
cache line is compared to the tag derived from the
memory address.
• Checking for a Cache Hit or Miss: If the tags match
and the valid bit is set, it indicates a cache hit, meaning
the data associated with that memory address is already
in the cache, and the operation can be completed using
the cache. If the tags do not match or the valid bit is not
set, it results in a cache miss, requiring the data to be
fetched from main memory.
• The direct mapping technique is simple and
inexpensive to implement.
• Its main disadvantage is that there is a fixed
cache location for any given block.
• Thus, if a program happens to reference words
repeatedly from two different blocks that map
into the same line, then the blocks will be
continually swapped in the cache, and the hit
ratio will be low
 Associative mapping
• Associative mapping overcomes the disadvantage of
direct mapping by permitting each main memory
block to be loaded into any line of the cache.
• In this case, the cache control logic interprets a
memory address simply as a Tag and a Word field.
• The Tag field uniquely identifies a block of main
memory.
• To determine whether a block is in the cache, the
cache control logic must simultaneously examine
every line’s tag for a match.
• With associative mapping, there is flexibility as
to which block to replace when a new block is
read into the cache.
• The principal disadvantage of associative
mapping is the complex circuitry required to
examine the tags of all cache lines in parallel.
 Set-Associative mapping
• Set-associative mapping is a compromise that
exhibits the strengths of both the direct and
associative approaches while reducing their
disadvantages.
• In this case, the cache consists of a number
sets, each of which consists of a number of
lines.
• k-way set-associative mapping: k-number of
lines in each set
• With set-associative mapping, block Bj can be
mapped into any of the lines of set j.
• Each word maps into all the cache lines in a
specific set, so that main memory block B0
maps into set 0, and so on.
• The first n lines of main memory are direct
mapped into the n lines of each way; the next
group of n lines of main memory are similarly
mapped, and so on.
• For set-associative mapping, the cache control
logic interprets a memory address as three
fields: Tag, Set, and Word.
 Replacement Algorithms
• Once the cache has been filled, when a new
block is brought into the cache, one of the
existing blocks must be replaced.
• For direct mapping, there is only one possible
line for any particular block, and no choice is
possible.
• For the associative and set-associative
techniques, a replacement algorithm is
needed.
• Three methods: LFU, FIFO, LRU
• Least recently used (LRU): Replace that block in
the set that has been in the cache longest with
no reference to it.
• First-in-first-out (FIFO): Replace that block in
the set that has been in the cache longest.
• Least frequently used (LFU): Replace that block
in the set that has experienced the fewest
references
 Write policy
• When a block that is resident in the cache is to
be replaced, there are two cases to consider:
– If the old block in the cache has not been altered,
then it may be overwritten with a new block
without first writing out the old block.
– Otherwise, main memory must be updated by
writing the line of cache out to the block of
memory before bringing in the new block.
• Two write policies-1) write through 2) write back
• Write through: all write operations are made to main
memory as well as to the cache, ensuring that main
memory is always valid. Disadvantage: it generates
substantial memory traffic and may create a bottleneck
• Write back: With write back, updates are made only in
the cache. When an update occurs, a dirty bit, or use bit,
associated with the line is set. Then, when a block is
replaced, it is written back to main memory if and only if
the dirty bit is set. The problem with write back is that
portions of main memory are invalid, and hence accesses
by I/O modules can be allowed only through the cache.
This makes for complex circuitry and a potential
bottleneck
Improving Cache performance
Virtual memory

Memory hierarchy
 Virtual memory
• Most modern computer systems use a hard disk
(also called a hard drive) as the lowest level in the
memory hierarchy .
• Hard disk is large and cheap but terribly slow.
• The disk provides a much larger capacity than is
possible with a cost-effective main memory (DRAM).
• However, if a significant fraction of memory
accesses involve the disk, performance is dismal.
You may have encountered this on a PC when
running too many programs at once.
• The hard disc memory is called virtual memory,
and the smaller main memory is called physical
memory.
• Virtual addresses that specify the location in
virtual memory.
• Virtual memory is divided into virtual pages,
typically 4 KB in size.
• Physical memory is likewise divided into
physical pages of the same size.
• The physical memory holds a subset of most
recently accessed virtual memory.
• In this way, physical memory acts as a cache
for virtual memory.
• Thus, most accesses hit in physical memory at
the speed of DRAM, yet the program enjoys
the capacity of the larger virtual memory.
• The process of determining the physical
address from the virtual address is called
address translation.
• If the processor attempts to access a virtual
address that is not in physical memory, a page
fault occurs, and the operating system loads
the page from the hard disk into physical
memory.
• Page table: lookup table used to translate
virtual addresses to physical addresses
• The page table access translates the virtual
address used by the program to a physical
address.
• The physical address is then used to actually
read or write the data.
 Translation lookaside buffer
• Every virtual memory reference can cause two
physical memory accesses: one to fetch the
appropriate page table entry, and one to fetch the
desired data.
• Thus, a straightforward virtual memory scheme
would have the effect of doubling the memory
access time.
• To overcome this problem, most virtual memory
schemes make use of a special cache for page table
entries, called a translation lookaside buffer (TLB)
Operation of Paging and Translation
Lookaside Buffer (TLB)
• A virtual address will generally be in the form of a page
number, offset.
• First, the memory system consults the TLB to see if the
matching page table entry is present.
• If it is, the real (physical) address is generated by combining
the frame number with the offset.
• If not, the entry is accessed from a page table.
• Once the real address is generated, which is in the form of a
tag and a remainder, the cache is consulted to see if the block
containing that word is present . If so, it is returned to the
processor. If not, the word is retrieved from main memory