Memory Virtualization

VMware, Inc.
 Outline
• Background
• Virtualization Techniques
– Emulated TLB
– Shadow Page Tables
• Hardware-supported Memory Virtualization
– Nested Page Tables
 Computer System Organization

CPU Memory

MMU Controller

Local Bus

Interface
High-Speed
I/O Bus
Frame
NIC Controller Bridge
Buffer

LAN

Low-Speed
I/O Bus
CD-ROM USB
 Traditional Address Spaces

0 4GB
RAM
Frame
Devices ROM Physical
Buffer
Address Space
 Traditional Address Spaces

0 4GB
Current Process Operating System Virtual
Address Space

0 4GB
RAM
Frame
Devices ROM Physical
Buffer
Address Space
 Traditional Address Spaces

0 4GB
Process Virtual Address Space
Background Process Operating System
Background Process Operating System

0 4GB
Current Process Operating System Virtual
Address Space

0 4GB
RAM
Frame
Devices ROM Physical
Buffer
Address Space
 Memory Management Unit (MMU)
• Virtual Address to Physical Address Translation
– Works in fixed-sized pages
– Page Protection
• Translation Look-aside Buffer
– TLB caches recently used Virtual to Physical mappings
• Control registers
– Page Table location
– Current ASID
– Alignment checking
 Traditional Address Translation w/
Architected Page Tables

Virtual Address Physical Address

1 4 2 5

Operating System’s
Page Fault Handler

Process
Page Table
2
 Virtualized Address Spaces

0 4GB
Current Guest Process Guest OS Virtual
Address Spaces

0 4GB
Virtual RAM
Virtual
Frame
Virtual Virtual Physical
Buffer Devices ROM Address Spaces
 Virtualized Address Spaces

0 4GB
Current Guest Process Guest OS Virtual
Address Spaces

0 4GB
Virtual RAM
Virtual
Frame
Virtual Virtual Physical
Buffer Devices ROM Address Spaces

0 4GB
RAM Devices
Frame
ROM
Machine
Buffer
Address Space
 Virtualized Address Spaces
w/ Emulated TLB
0 4GB
Virtual Address Space

Guest Page Table

VMM PhysMap
0 4GB
Machine Address Space
 Virtualized Address Translation
w/ Emulated TLB

Virtual Address Machine Address

Emulated TLB Guest

0 4GB
Current Guest Process Guest OS Virtual
Address Spaces

0 4GB
Virtual RAM
Virtual
Frame
Virtual Virtual Physical
Buffer Devices ROM Address Spaces

0 4GB
RAM Devices
Frame
ROM
Machine
Buffer
Address Space
 Issues with Shadow Page Tables
• Positives
– Handle page faults in same way as Emulated TLBs
– Fast guest context switching
• Page Table Consistency
– Guest may not need invalidate TLB on writes to off-line
page tables
– Need to trace writes to shadow page tables to invalidate
entries
• Memory Bloat
– Caching guest page tables takes memory
– Need to determine when guest has reused page tables
Virtualized Address Translation
w/ Nested Page Tables

Virtual Address Machine Address

1 3

Guest 2 PhysMap 3
Page Table By VMM
 Interposition with Memory Virtualization
Page Sharing

Virtual Virtual

Physical Physical

VM1 VM2

Machine

Read-Only
Copy-on-wrte