OPERATING SYSTEMS AND NETWORKS

Prof. Dr. Andreas Wölfl

Department of Applied Computer Science,

Deggendorf Institute of Technology

andreas.woelfl@th-deg.de
MEMORY MANAGEMENT
 MEMORY MANAGEMENT

640K ought to be enough for anybody.

(Bill Gates, 1981)

Prof. Dr. Andreas Wölfl Operating Systems and Networks 1

MEMORY MANAGEMENT

1. CACHING
 CACHING

Cache
Hardware or software used to store data temporarily in a computing context.

• Small amount of faster, more expensive memory used to

CPU
improve the performance of frequently accessed data.
• Local to a cache client.
• Commonly used by the CPU and operating systems. Cache
• Decreases data access times and reduces latency.
RAM

Prof. Dr. Andreas Wölfl Operating Systems and Networks 2

 CACHING

Data access using caches:

• When a cache client attempts to access data, it first checks the cache.
• If the data is found, that is referred to as a cache hit.
• If the data is not found, that is referred to as a cache miss
• In case of a hit, the data is taken directly from the cache.
• In case of a miss, the data is fetched the ordinary way and put into the cache.
• How data gets evicted from the cache depends on the replacement
algorithm1 .

1
We will learn some algorithms later in the lecture
Prof. Dr. Andreas Wölfl Operating Systems and Networks 3
 CACHING

Caches used in a modern CPU:

• L1-Cache: Built onto the CPU core itself. Used to speed up frequently and
recently used instructions and data.
• L2-Cache: Separate from the CPU cores, but built onto the CPU package.
Used to speed up references to main memory.
• LLC/L3-Cache: Built onto the the CPU package and shared among cores.
Mainly used to tackle the cache coherence problem, which aims to keep all
cached copies of the same memory identical.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 4

 CACHING

Performance difference between L1/L2/L3: CPU

• Regsiter: 1 ns Core 1 Core 2

• L1 cache access: 2 ns Registers Registers

• L2 cache access: 5 ns
L1 Cache L1 Cache

• L3 cache access: 10 ns
• RAM access: 100 ns L2 Cache L2 Cache

Example: Intel Core i9 13900K

L3 Cache
• L1: 80kB per core.
• L2: 2MB per core.
RAM
• LLC: 36MB shared.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 5

MEMORY MANAGEMENT

2. ADDRESS SPACES
 ADDRESS SPACES

Major drawbacks of exposing physical memory to processes:

• Processes may trash the operating system (i.e., write garbage to memory
locations in kernel space → compromise the kernel).
• It is difficult to have multiple programs running at the same time (as two
processes may reference the same physical memory location).

Challenge: How can multiple programs execute without interfering with each
other when referencing main memory?

Prof. Dr. Andreas Wölfl Operating Systems and Networks 6

 ADDRESS SPACES

Address Space
The set of (virtual) addresses that a process can use to address memory.

• An address space is a kind of abstract memory for programs to live in.

• Isolation: each process is assigned an individual address space.
• The operating system is responsible to map the addresses in an address
space to the physical memory location in RAM.

→ For example, address 0x0028 in one program maps to a different phyisical

location than address 0x0028 in another program.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 7

 ADDRESS SPACES

Relocation
Maps each process’s address space onto a different part of physical memory

• Static Relocation: Add an offset equal to the starting location in memory to

all memory addresses during loading. E.g., for a program starting at 0x4000,
the address 0x0028 is rewritten to 0x4028.
• Dynamic Relocation: Uses two hardware registers, base and limit, which
define consecutive memory locations wherever there is room. Every time a
process references memory, the base value is added to the address. If the
result exceeds the limit, a fault is generated and the access is aborted.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 8

 ADDRESS SPACES

Process A Process B Main Memory

Registers Registers 0x7FF8 NOP

Limit: 0x3FFC Limit: 0x7FF8

Base: 0x0000 Base: 0x4000 JE 0x0000
CMP
Program Program 0x4000 ADD
0x3FFC NOP 0x3FFC NOP 0x3FFC NOP

0x0008 ADD 0x0008 JE 0x0000 0x0008 ADD

0x0004 MOV 0x0004 CMP 0x0004 MOV
0x0000 JMP 0x0008 0x0000 ADD 0x0000 JMP 0x0008

Prof. Dr. Andreas Wölfl Operating Systems and Networks 9

 ADDRESS SPACES

Swapping
Temporarily writing a process’s address space in its entirety to secondary
memory so that main memory can be made available for other processes.

• Simplest strategy to deal with memory overload that occurs if the physical
memory is not large enough to hold all processes.
• Causes idle processes to be stored mostly on disk, so they do not take up
any memory when not running.
• Swap out: Writing the address space to disk and removing it from memory.
• Swap in: Reading the address space from disk and loading it to memory.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 10

 ADDRESS SPACE

Methods to select the process to swap out:

• FIFO (First-in First-Out): selects the oldest process.

• More replacement algorithms are discussed later in the lecture.

Methods to select the location where to swap in:

• First fit: Scan the memory bottom-up until a hole is found that is big enough.
• Next fit: Variant of next fit where the scan starts from the place where it left
off the last time (instead of always at the beginning).
• Best fit: Searches the entire memory for the smalles hole that is adequate.
• Worst fit: Searches the entire memory for the largest hole that is adequate.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 11

 ADDRESS SPACES

C
C C C

A A
B B

A A A D D D D

Operating Operating Operating Operating Operating Operating Operating Operating

System System System System System System System System

t0 t1 t2 t3 t4 t5 t6 t7

Prof. Dr. Andreas Wölfl Operating Systems and Networks 12

 ADDRESS SPACES

External Fragmentation
Situation where there is a sufficient quantity of free memory available to satisfy
a processes memory requirements, but the process cannot be loaded because
the memory offered is in a non-contiguous manner.

• Caused by swapping processes in and out.

• Leads to significantly reduced capacity and performance.
• Techniques such as memory compaction, which relocate all processes
downward as far as possible, are impractical2 .

2
For example, on a 16GB machine that can copy 8 bytes in 8 ns, it would take about 16 seconds to
compact all of memory.
Prof. Dr. Andreas Wölfl Operating Systems and Networks 13
 ADDRESS SPACES

Preliminary conclusion: Realizing the concept of individual address spaces in

multiprogramming systems constitutes a major challenge.

Open problems:

• External fragmentation wastes a considerable amount of memory.

• All active processes must fit into memory, limiting the degree of multipro-
gramming (swapping actively running processes is impractical).
• A program that is larger than main memory cannot execute.
• Dynamically allocating memory3 is not taken into consideration.

→ Nevertheless, the idea of address spaces is good!

3
Requesting a certain amount of memory from the operating system during program execution.
Prof. Dr. Andreas Wölfl Operating Systems and Networks 14
 TASK

Task
Solve the task ”Swapping” in iLearn.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 15

MEMORY MANAGEMENT

3. VIRTUAL MEMORY
 VIRTUAL MEMORY

Virtual Memory
A memory allocation scheme in which secondary memory can be addressed as
though it were part of main memory.

• Each process is assigned a full virtual address space4 .

• The virtual address space consists of fixed-sized units called pages.
• The corresponding units in physical memory are called page frames.
• Both pages and page frames are generally the same size5 .
• Pages are mapped on the fly to either page frames or to disk.

4
From 0 to a maximum, even if a lesser amount of RAM is installed.
5
4kB in most operating systems
Prof. Dr. Andreas Wölfl Operating Systems and Networks 16
 VIRTUAL MEMORY

Virtual Address Space Physical Memory

60K - 64K X Page 7 28K - 32K
56K - 60K X 6 24K - 28K
52K - 56K X 5 20K - 24K
48K - 52K X 4 16K - 20K
44K - 48K 11 3 12K - 16K
40K - 44K 10 2 8K - 12K
36K - 40K 9 1 4K - 8K
32K - 36K 8 0 0K - 4K
28K - 32K 7
24K - 28K 6
Disk Page
20K - 24K 5
16K - 20K 4 frame
12K - 16K 3
8K - 12K 2
4K - 8K 1
0K - 4K 0

Prof. Dr. Andreas Wölfl Operating Systems and Networks 17

 VIRTUAL MEMORY

The mapping is programmed by the operating system in software, but performed

by the Memory Management Unit (MMU) in hardware:

The CU presents the virtual

addresses to the MMU
CPU RAM Disk
0x0007
Control Unit
0x0006
0x0005
0x0004
0x0003
MMU TLB 0x0002
0x0001

The MMU sends physical

addresses to RAM

Prof. Dr. Andreas Wölfl Operating Systems and Networks 18

 VIRTUAL MEMORY

1 MOV EAX, 0x2004 # load memory location 0x2004 into register EAX

Example: The program tries to access address 819610 using the instruction:

• The CU passes the virtual address 819610 to the MMU.

• The MMU identifies page 2 for 819610 , which maps to page frame 6.
• It transforms the address 819610 and outputs address 2458010 onto the bus.
• The RAM knows nothing at all about the MMU and just sees a request for
reading address 2458010 , which it honors.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 19

 TASK

Task
Solve the task ”Address Translation” in iLearn.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 20

 VIRTUAL MEMORY

Page Tables
Data structure in RAM maintained by the operating system that holds the
mapping between pages and page frames.

• The page number is used as an index into the page table, yielding the
number of the corresponding page frame as well as a present/absent bit.
• If the present/absent bit is 0, a kernel trap called page fault is generated.
• If the present/absent bit is 1, the page frame number found in the page table
is copied to the most significant 3 bits of the output, along with the 12 bit
offset, which is copied unmodified from the incoming virtual address.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 21

 VIRTUAL MEMORY

RAM MMU
Outgoing physical address (2458010)
Page Table
1 1 0 0 0 0 0 0 0 0 0 0 1 0 0
Frame P/A
15 000 0
14 000 0
13 000 0
12 000 0 110
11 111 1
10 000 0 12-bit offset copied
9 101 1 1
directly from input to
8 000 0 output
7 000 0 Present 0
6 000 0 Page Fault
bit?
5 011 1
4 100 1
3 000 1
2 110 1
1 001 1
0 001 1

0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0
Virtual page = 2 is used
as an index to the table Incoming virtual address (819610)

Prof. Dr. Andreas Wölfl Operating Systems and Networks 22

 VIRTUAL MEMORY

Modified Present/Absent

Page Frame Number

Referenced Protection

Structure of a Page Table entry:

• Protection: 3 bits that indicate what kinds of access are permitted (rwx).
• Referenced (R bit): Set if a page is accessed by the current instruction.
• Modified (M bit): Set as soon as a page is written to (stays set until unset).

Prof. Dr. Andreas Wölfl Operating Systems and Networks 23

 VIRTUAL MEMORY

What happens in case of a page fault trap?

• The operating system picks a page frame to be replaced.

• If the page frame is not already stored on disk or the M bit is set (page dirty),
the contents of the page frame are written to disk.
• The operating system fetches from disk the absent page that was just
referenced into the page frame just freed.
• Finally, the operating system update updates the respective page table
entry6 , and restarts the trapped instruction.

6
Writes the page frame number, sets the P/A bit and unsets the M bit.
Prof. Dr. Andreas Wölfl Operating Systems and Networks 24
 VIRTUAL MEMORY

Question: How does the operating system pick the page to be replaced?

Prof. Dr. Andreas Wölfl Operating Systems and Networks 25

 VIRTUAL MEMORY

The optimal page frame replacement algorithm:

• Label each page with the number of instructions that will be executed before
that page is first referenced (e.g, one page will not be used for 8 million
instructions, another page will not be used for 6 million instructions).
• On page fault: Pick the page with the highest label (i.e., push the page fault
that will fetch it back as far into the future as possible)

Problem: This algorithm is unrealizable. At the time of the page fault, the
operating system has no way of knowing when each of the pages will be
referenced next.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 26

 VIRTUAL MEMORY

Observation:

• Pages that have been heavily in use in the last few instructions will probably
be heavily used again soon.
• Conversely, pages that have not been used for ages will probably remain
unused for a long time.

The Least Recently Used (LRU) algorithm:

• Selects the page that has been unused for the longest time.
• Good approximation to the optimal algorithm (the best that we know).

Prof. Dr. Andreas Wölfl Operating Systems and Networks 27

 VIRTUAL MEMORY

Hit = Page available

Miss = Page fault

incoming pages
1 2 3 2 1 5 2 3

3 2 1 5 2 3
page
2 2 3 2 1 5 2
frames
1 1 1 1 3 2 1 5

Miss Miss Miss Hit Hit Miss Hit Miss

→ Problem: Special hardware required for an efficient implementation.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 28

 VIRTUAL MEMORY

Counter

Page Frame Number

Referenced

The Not Frequently Used algorithm (attempt to simulate LRU in software):

• With each entry in the page table, a counter is associated (initially 0).
• During each instruction, the operating system scans all pages in memory and
adds the R bit to the counter (i.e., the counter roughly keeps track of how
often each page has been referenced).
• Select the page with the lowest counter.

Problem: It never forgets anything → may keep undesired pages.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 29
 VIRTUAL MEMORY

Counter

10000000 1 Page Frame Number t0

01000000 0 Page Frame Number t1

00100000 0 Page Frame Number t2

10100000 1 Page Frame Number t3

11010000 1 Page Frame Number t4

01101000 0 Page Frame Number t5

Referenced

The Aging algorithm (small modification to NFU, simulates LRU quite well):

• Shift the counter to the right before appending the R bit to the left.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 30

 EXKURSUS

Page replacement in the Linux kernel (roughly described):

• The kernel tracks pages using a pair of LRU lists (active/inactive).

• Pages that have been recently accessed are kept on the ”active” list.
• Pages taken off the ”active” list are added to the ” inactive” list.
• If an ” inactive” page is accessed, it is promoted to the ”active” list.
• Pages are evicted only from the ” inactive” list.
• Important: Newly allocated pages first join the ” inactive” list.

→ Efficient in situations when a program reads sequentially through a file (the

contents are loaded into memory, but will never be used again).

Prof. Dr. Andreas Wölfl Operating Systems and Networks 31

 EXKURSUS

The multi-generational LRU patch of Yu Zhao (Google, 2021)7 :

• Adds more LRU lists to cover a range of page ages (generations).

• Change the way page scanning is done to reduce its overhead.

This work results in:

• Spectrum of page ages (highest access frequency in the oldest generation).

• The generations are smaller in size (compared to the previous lists)
• Only the youngest generation is scanned → efficiency gain.

Note: Linux Torvalds merged it to kernel 6.1. Benchmarks8 look promising.

7
https://lore.kernel.org/lkml/20220104202247.2903702-1-yuzhao@google.com/T/
8
https://www.phoronix.com/news/MGLRU-Reaches-mm-stable
Prof. Dr. Andreas Wölfl Operating Systems and Networks 32
 TASK

Task
Solve the task ”Page Replacement” in iLearn.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 33

 VIRTUAL MEMORY

Translation Lookaside Buffers (TLB)

Small table inside the MMU where each entry contains the information about
one page, including the page number, the M bit, the protection bits, and the
page frame.

• Based on the observation that most programs tend to make a large number
of references to a small number of pages (and not the other way arround).
• Typically 256 entries, implemented in hardware.
• Can be searched in parallel.
• Huge speedup.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 34

 VIRTUAL MEMORY

Example:

Valid Page Number M Bit Protection Page Frame

1 140 1 RW 31
1 20 0 RX 38
1 130 1 RW 29
1 129 1 RW 62
1 19 0 RX 50
1 21 0 RX 45
1 860 1 RW 14
1 861 1 RW 75

Prof. Dr. Andreas Wölfl Operating Systems and Networks 35

 VIRTUAL MEMORY

Page frame lookup with TLB:

• When a virtual address is presented to the MMU for translation, the

hardware first checks to see if its page number is present in the TLB.
• If a valid item is found and the access does not violate the protection bits,
the page frame is taken directly from the TLB.
• If the page number is not in the TLB, the MMU detects the muss and does an
ordinary page table lookup.
• It then evicts one of the entries from the TLB and replaces it with the page
table entry just looked up.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 36

 TASK

Task
Solve the task ”Translation Lookaside Buffer” in iLearn.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 37

MEMORY MANAGEMENT

4. MEMORY ALLOCATION
 MEMORY ALLOCATION

0xFFFF
Segments of a process’ address space: Kernel Space
high address
Stack
• Text: The instructions of the program.
• Data: Values of global and static variables.
• Heap: Area used for dynamic memory allocation.
• Stack: Area used for static memory allocation.
• Kernel Space: The kernel (e.g., system call Heap

procedures) is located at the top of the address Data

space. Device addresses are at the bottom. Text

low address
Kernel Space
0x0000

Prof. Dr. Andreas Wölfl Operating Systems and Networks 38

 MEMORY ALLOCATION

Heap
Memory region from which a running program can request chunks.

Example (in the C programming language):

1 int* ptr = (int*) malloc(50); // requests 50 bytes from the OS
2 free(ptr) // releases the 50 bytes to the OS

Note: How heap memory is organized depends on the programming language in

which a program is written.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 39

 MEMORY ALLOCATION

Internal Fragmentation
Situation where a bigger amount of memory is allocated than demanded.

• Memory allocation is said to by n-byte aligned when memory is allocated in

blocks that are multiples of n-bytes.
• Object-oriented programming languages such as C++, Java, and Python use
an 8-byte alignment for object allocation.
• Drawback: Memory alignment wastes memory due to internal fragmentation.
• Aligned access is faster because the bus to memory is typically 64 bit wide.
This implies that misaligned access can require 2 reads from memory.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 40

 MEMORY ALLOCATION

Example:

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
half word
byte
(upper bytes)
address space

byte half word (lower bytes)

half word

half word word (upper bytes

word word (lower bytes)

aligned transfers unaligned transfers

Prof. Dr. Andreas Wölfl Operating Systems and Networks 41

 MEMORY ALLOCATION

Stack
Memory region where data is added or removed in last-in-first-out (LIFO)
manner.

• Each entry of the stack is a structure called frame.

• A frame stores information about an invocation of a function.
• Keeps track of the nested sequence of currently active procedures.
• The top of the stack is the frame of the running call, i.e., the procedure that
has currently control over the execution.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 42

 MEMORY ALLOCATION

Usually9 , a stack frame includes:

• Parameters (variables declared in a procedure signature).

• Local variables (variables declared inside a procedure).
• The return address, i.e., the memory location of the instruction that caused
the procedure invocation.

Note: In many programming languages, variables and parameters hold memory

locations referencing the actual value in heap memory.

9
Varies among programming languages
Prof. Dr. Andreas Wölfl Operating Systems and Networks 43
 MEMORY ALLOCATION

Example (The diagram represents the stack during execution of line 2):

1 int foo(int a, int b) { Locals of bar

2 return a + b;
0x2016 Return Address stack frame
3 } int a = 15
Parameters of bar of foo()
4 int b = 5
5 void bar(int x, int y) { Locals of foo
int n = 15
6 int n = x + 10;
0x2008 Return Address stack frame
7 int z = foo(n, y); int x = 5 of bar()
8 printf("z=%d",z); Parameters of foo
int y = 5
9 }
Locals of main
10 stack frame
0x2000 Return Address of main()
11 int main() {
12 bar(5, 5); Parameters of main
13 }

Prof. Dr. Andreas Wölfl Operating Systems and Networks 44

 TASK

Task
Solve the task ”Fragmentation” in iLearn.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 45

MEMORY MANAGEMENT

5. SUMMARY
 SUMMARY

Summary
You should have accquired the competencies to

• Differentiate between different types and levels of memory.

• Explain the notion of an address space and understand why we need it.
• Recite techniques to realize an address space in an operating system.
• Explain and understand the concept of virtual memory.
• Describe the main functions of a Memory Management Unit (MMU).
• Distinguish and assess page replacement algorithms.
• Know the difference between text, data, stack, and heap memory.
• Differentiate between internal and external fragmentation and describe
techniques how to tackle it.

Prof. Dr. Andreas Wölfl Operating Systems and Networks 46