Memory Organization

Morris Mano chapter 12 pdf page-447

 Memory Hierarchy
• The memory unit that communicates
directly with the CPU is called the main
memory.
• Programs and data currently needed by
the processor reside in the main memory.
• Devices that provide backup storage
are called auxiliary memory.
• The most common auxiliary memory
devices used in the computer systems are
magnetic disks and tapes.
• They are used for storing system
programs, large data files and other
backup information.
• Internal Memory or Primary Memory
• Comprising of Main Memory, Cache Memory & CPU registers.
• This is directly accessible by the processor.
• External Memory or Secondary Memory
• Comprising of Magnetic Disk, Optical Disk, Magnetic Tape i.e. peripheral storage
devices which are accessible by the processor via I/O Module.
The list of memories from closet to the
processor to the farthest is given as below:
• Registers
• L1 Cache
• L2 Cache
• Main memory
• Magnetic disk
• Optical
• Tape
Registers
• Registers are small, high-speed memory units located in the CPU.
• They are used to store the most frequently used data and
instructions.
• Registers have the fastest access time and the smallest storage
capacity, typically ranging from 16 to 64 bits.
Cache Memory
• Cache memory is a small, fast memory unit located close to the CPU.
• It stores frequently used data and instructions that have been
recently accessed from the main memory.
• Cache memory is designed to minimize the time it takes to access
data by providing the CPU with quick access to frequently used data.
Cache Memory
• Cache memory is a small, fast memory unit located close to the CPU.
• It stores frequently used data and instructions that have been
recently accessed from the main memory.
• Cache memory is designed to minimize the time it takes to access
data by providing the CPU with quick access to frequently used data.
Main Memory
• It is the central storage unit in the computer system
• Main memory is used to store data and instructions that are currently
in use by the CPU.
• The principal technology used for the main memory is based on
semiconductor integrated circuits.
Types of Main Memory
• Static RAM: (SRAM)
• Static RAM stores the binary information in flip flops and information remains
valid until power is supplied.
• Static RAM has a faster access time and is used in implementing cache
memory.
• Dynamic RAM: (DRAM)
• It stores the binary information as a charge on the capacitor.
• It requires refreshing circuitry to maintain the charge on the capacitors after a
few milliseconds.
• It contains more memory cells per unit area as compared to SRAM.
Secondary Storage
• Secondary storage, such as hard disk drives (HDD) and solid-state
drives (SSD) , is a non-volatile memory unit that has a larger storage
capacity than main memory.
• It is used to store data and instructions that are not currently in use
by the CPU.
• Secondary storage has the slowest access time and is typically the
least expensive type of memory in the memory hierarchy.
Magnetic Disk
• Magnetic Disks are simply circular plates that are fabricated with
either a metal or a plastic or a magnetized material.
• The Magnetic disks work at a high speed inside the computer and
these are frequently used.
Magnetic Tape
• Magnetic Tape is simply a magnetic recording device that is covered
with a plastic film.
• Magnetic Tape is generally used for the backup of data.
• In the case of a magnetic tape, the access time for a computer is a
little slower and therefore, it requires some amount of time for
accessing the strip.
Random Access Memory (RAM) and Read
Only Memory (ROM)
Random Access Memory (RAM)
• Random Access Memory (RAM) is a type of computer memory that is
used to temporarily store data that the computer is currently using or
processing.
• RAM is volatile memory, which means that the data stored in it is lost
when the power is turned off.
• RAM is typically used to store the operating system, application
programs, and data that the computer is currently using.
• It is also called read-write memory or the main memory or the
primary memory .
Types of Random Access Memory (RAM)
• Static RAM:
• SRAM stands for Static Random Access Memory.
• It is a type of semiconductor which is widely used in computing devices and
microprocessors.

• Dynamic RAM:
• DRAM stands for Dynamic Random Access Memory.
• It is made of Capacitors and has smaller data life span than Static RAM.
Read-Only Memory (ROM)
• Read Only Memory (ROM) is a type of computer memory that is used to
permanently store data that does not need to be modified.
• ROM is non-volatile memory, which means that the data stored in it is
retained even when the power is turned off.
• ROM is typically used to store the computer’s BIOS (basic input/output
system), which contains the instructions for booting the computer, as well
as firmware for other hardware devices.
• Stores crucial information essential to operate the system, like the program
essential to boot the computer. (bootstrap loader)
• Used in embedded systems or where the programming needs no change.
• Used in calculators and peripheral devices.
Types of Read-Only Memory (ROM)
1. PROM (Programmable Read-Only Memory)
2. EPROM (Erasable Programmable Read Only Memory)
3. EEPROM (Electrically Erasable Programmable Read Only Memory)
4. MROM (Mask Read Only Memory)
• PROM (Programmable read-only memory):
• It can be programmed by the user.
• Once programmed, the data and instructions in it cannot be changed.
• EPROM (Erasable Programmable read-only memory):
• It can be reprogrammed.
• To erase data from it, expose it to ultraviolet light.
• To reprogram it, erase all the previous data.
• EEPROM (Electrically erasable programmable read-only memory):
• The data can be erased by applying an electric field, with no need for
ultraviolet light.
• We can erase only portions of the chip.
• MROM(Mask ROM):
• Mask ROM is a kind of read-only memory, that is masked off at the time of
production.
• Like other types of ROM, mask ROM cannot enable the user to change the
data stored in it.
• If it can, the process would be difficult or slow.
Memory allocation policies
• In the operating system, the following are four common memory
management techniques.
• Single contiguous allocation:
• Simplest allocation method used by MS-DOS. All memory (except some reserved for OS)
is available to a process.
• Partitioned allocation:
• Memory is divided into different blocks or partitions.
• Each process is allocated according to the requirement.
• Paged memory management:
• Memory is divided into fixed-sized units called page frames, used in a virtual memory
environment.
• Segmented memory management:
• Memory is divided into different segments (a segment is a logical grouping of the
process’ data or code).
• In this management, allocated memory doesn’t have to be contiguous.
Partition Allocation
• In Partition Allocation, when there is more than one partition freely
available to accommodate a process’s request, a partition must be
selected.
• To choose a particular partition, a partition allocation method is
needed.
• A partition allocation method is considered better if it avoids internal
fragmentation.
Different Allocation Policies
• When it is time to load a process into the main memory and
if there is more than one free block of memory of sufficient
size then the OS decides which free block to allocate.
• First Fit
• Best Fit
• Worst Fit
• Next Fit
First Fit
• In the first fit, the partition is
allocated which is the first
sufficient block from the top of
Main Memory.
• It scans memory from the
beginning and chooses the first
available block that is large
enough.
• Thus, it allocates the first hole
that is large enough.
Best Fit
• Best Fit Allocate the process to
the partition which is the first
smallest sufficient partition
among the free available
partition.
• It searches the entire list of
holes to find the smallest hole
whose size is greater than or
equal to the size of the process.
Worst Fit
• Worst Fit Allocate the process
to the partition which is the
largest sufficient among the
freely available partitions
available in the main memory.
• It is opposite to the best-fit
algorithm. It searches the
entire list of holes to find the
largest hole and allocate it to
process.
Next Fit
• Next fit is similar to the first fit
but it will search for the first
sufficient partition from the last
allocation point.
Cache memory
• The cache is a small and very fast memory, interposed between the
processor and the main memory.
• Its purpose is to make the main memory appear to the processor to
be much faster than it actually is.
• The effectiveness of this approach is based on a property of
computer programs called locality of reference.
• The cache memory can store a reasonable number of blocks at any
given time, but this number is small compared to the total number of
blocks in the main memory.
• The correspondence between the main memory blocks and those in
the cache is specified by a mapping function.
• The basic operation of the cache is as follows:
• When the CPU needs to access memory , the cache is examined.
• If the word is found in the cache, it is read from the fast memory.
• If the word addressed by the CPU is not found in the cache , the main
memory is accessed to read the word.
• A block of words containing the one just accessed is then transferred from main memory
to the cache memory.
• When the cache is full and a memory word (instruction or data) that is not in
the cache is referenced, the cache control hardware must decide which block
should be removed to create space for the new block that contains the
referenced word.
• The collection of rules for making this decision constitutes the cache’s replacement
algorithm.
• Locality of Reference
• Analysis of programs shows that most of their execution time
is spent in routines in which many instructions are executed
repeatedly.
• These instructions may constitute a simple loop, nested
loops, or a few procedures that repeatedly call each other.
• Locality of reference refers to a phenomenon in which a
computer program tends to access same set of memory
locations for a particular time period.
• There are two ways with which data or instruction is fetched
from main memory and get stored in cache memory.
• Temporal Locality
• Spatial Locality
• Temporal Locality
• Temporal locality means current data or instruction that is being fetched may
be needed soon. So , we should store that data or instruction in the cache
memory so that we can avoid again searching in main memory for the same
data.
• When CPU accesses the current main memory location for reading required
data or instruction, it also gets stored in the cache memory which is based on
the fact that same data or instruction may be needed in near future.
• This is known as temporal locality (If some data is referenced, then there is a
high probability that it will be referenced again in the near future.)
• Spatial Locality
• Spatial locality means instruction or data near to the current memory location
that is being fetched, may be needed soon in the near future.
• This is slightly different from the temporal locality.
• (Here we are talking about nearly located memory locations while in temporal
locality we were talking about the actual memory location that was being
fetched.)
• Cache Hit:
• When the CPU finds the required data in the cache memory, allowing for
quick access. On searching in the cache if data is found, a cache hit has
occurred.
• Cache Miss:
• When the required data is not found in the cache, forcing the CPU to retrieve
it from the slower main memory. On searching in the cache if data is not
found, a cache miss has occurred
• Hit ratio:
• The performance of the cache memory is frequently measured in terms of a
quantity called hit ratio.
• Hit ratio=(Number of cache hits)/(Number of searches)
Mapping Techniques
• The correspondence between the main memory blocks and those in
the cache is specified by a mapping function.
• Thus, transformation of data from main memory to cache memory is
referred to as a mapping process.
• There are three types of cache mappings namely:
• Associative Mapping
• Direct Mapping
• Set Associative Mapping
 Associative Mapping
• In Associative mapping method a
main memory block can be placed
into any cache block position.
• In this case, 12 bits tag is required to
identify a memory block when it is
resident in the cache.
• The tag bits of an address received
from the processor are compared to
the tag bits of each block of the cache
to see if the desired block is present.
• This is called the associative-mapping
technique.
• It gives complete freedom in choosing the cache location in which to
place the memory block, resulting in a more efficient use of the space
in the cache.
• When a new block is brought into the cache, it replaces an existing
block only if the cache is full.
• In this case, we need an algorithm to select the block to be replaced.
• A simple procedure is to replace blocks of the cache in round-robin order
whenever a new word is requested from main memory.
• This constitutes a FIFO replacement policy.
 Direct mapping
• The simplest way to determine cache
locations in which to store memory blocks is
the direct mapping technique.
• In this technique, block j of the main memory
maps onto block j modulo 128 of the cache.
• Thus, whenever one of the main memory
blocks 0, 128, 256, . . . is loaded into the
cache, it is stored in cache block 0 ,blocks 1,
129, 257, . . . are stored in cache block 1, and
so on.
• Since more than one memory block is
mapped onto a given cache block position,
contention may arise for that position even
when the cache is not full.
• For example, instructions of a program may start in block 1 and continue
in block 129, possibly after a branch.
• As this program is executed, both of these blocks must be transferred to
the block-1 position in the cache.
• Contention is resolved by allowing the new block to overwrite the
currently resident block.
• With direct mapping, the replacement algorithm is trivial.
• Placement of a block in the cache is determined by its memory address.
The memory address can be divided into three fields, The low-order 4 bits
select one of 16 words in a block.
• When a new block enters the cache, the 7-bit cache block field
determines the cache position in which this block must be stored.
• If they match, then the desired word is in that block of the cache. If
there is no match, then the block containing the required word must
first be read from the main memory and loaded into the cache.
• The direct-mapping technique is easy to implement, but it is not very
flexible.
Set-Associative Mapping
• Another approach is to use a
combination of the direct- and
associative-mapping techniques.
• The blocks of the cache are grouped
into sets, and the mapping allows a
block of the main memory to reside
in any block of a specific set.
• Hence, the contention problem of the
direct method is eased by having a
few choices for block placement.
• At the same time, the hardware cost is reduced by decreasing the size of the
associative search.
• An example of this set-associative-mapping technique is shown in Figure 8.18 for
a cache with two blocks per set.
• In this case, memory blocks 0, 64, 128, . . . , 4032 map into cache set 0, and they
can occupy either of the two block positions within this set.
• Having 64 sets means that the 6-bit set field of the address determines which set
of the cache might contain the desired block.
• The tag field of the address must then be associatively compared to the tags of
the two blocks of the set to check if the desired block is present.
• This two-way associative search is simple to implement. The number of blocks
per set is a parameter that can be selected to suit the requirements of a
particular computer.
• For the main memory and cache sizes in Figure 8.18, four blocks per set can be
accommodated by a 5-bit set field, eight blocks per set by a 4-bit set field, and so
on.
• The extreme condition of 128 blocks per set requires no set bits and corresponds
to the fully-associative technique, with 12 tag bits. The other extreme of one
block per set is the direct-mapping.
Architecture (L1, L2, L3)
• CPU cache is organized into multiple levels (L1, L2, L3), each with
varying sizes, speeds, and proximity to the CPU cores.
• L1 cache is the smallest and fastest, located closest to the CPU cores,
followed by L2 and L3 caches, which are larger but slower.
• L1 Cache:
• The L1 cache is usually split into two sections: the instruction cache and the
data cache.
• The instruction cache deals with the information about the operation that the
CPU must perform, while the data cache holds the data on which the
operation is to be performed.
• L1 cache is extremely fast but limited in size, typically ranging from a few KBs
to a few MBs per core.
• L2 Cache:
• L2 cache serves as a secondary cache, larger than L1 but slower in access
speed.
• It is shared among CPU cores within a single CPU core complex or chip.
• L3 Cache:
• L3 cache is the last level of cache in the hierarchy, shared among multiple CPU
cores or across multiple CPU sockets.
• It is larger than L2 cache but slower in access speed.
Cache coherence
• Cache coherence is a concept in computer architecture that ensures
consistency of data stored in multiple caches within a multiprocessor
system.
• When multiple processors (or cores) have their own caches, they may
store copies of the same memory location.
• Cache coherence ensures that all processors see the most recent and
correct value of the data, preventing inconsistencies.
Key Cache Coherence Issues
1. Multiple copies of data:
• If different processors cache the same memory block, updates to one copy
must be reflected in others.
2. Inconsistent reads:
• A processor may read stale data if another processor has modified it but not
updated the shared memory.
• When the processor finds a word in cache during a read operation,
the main memory is not involved in the transfer.
• If the operation is to write, there are two commonly used procedures
to update memory.
• Write-through policy:
• In the write-through policy, both cache and main memory are updated with
every write operation.
• Write-back policy:
• In the write-back policy, only the cache is updated and the location is marked
so that it can be copied later into main memory.
Associative memory
• Associative memory, also known as content-addressable memory
(CAM), is a type of memory that enables data retrieval based on
content rather than specific memory addresses.
• This allows for parallel searches of stored data, significantly enhancing
search efficiency.
Key Characteristics
• Content-Based Access: Data is accessed by providing a portion of the
content or a specific pattern, eliminating the need for explicit
memory addresses.​
• Parallel Search Capability: The memory hardware can simultaneously
compare the input search data against all stored entries, enabling
rapid data retrieval.​
• Applications: Associative memory is particularly useful in applications
requiring fast lookups, such as cache memory systems, translation
lookaside buffers (TLBs) in virtual memory implementations, and
networking devices like routers for quick routing table searches.
Interleaved memory
• Interleaved memory is a technique that enhances memory access
speed by dividing memory into multiple modules (or banks) and
distributing consecutive memory addresses across these modules.
• This arrangement allows the processor to access different memory
modules simultaneously, thereby reducing wait times and increasing
overall system performance.
 Key Aspects of Interleaved Memory
• Memory Modules:
• The main memory is partitioned into several modules, each capable of independent
operation.
• ​Address Distribution:
• Consecutive memory addresses are assigned to different modules in a sequential
manner.
• For example:
Address 0 → Module 0​Address 1 → Module 1​Address 2 → Module 2​Address 3 → Module 0 (and so on)
• Parallel Access:
• This configuration allows the CPU to access multiple memory modules concurrently,
thereby reducing access time and improving throughput.