research-article

Accelerating two-dimensional page walks for virtualized systems

Authors:

Benjamin Serebrin,

Francesco Spadini,

Srilatha ManneAuthors Info & Claims

ASPLOS XIII: Proceedings of the 13th international conference on Architectural support for programming languages and operating systems

Pages 26 - 35

https://doi.org/10.1145/1346281.1346286

Published: 01 March 2008 Publication History

Abstract

Nested paging is a hardware solution for alleviating the software memory management overhead imposed by system virtualization. Nested paging complements existing page walk hardware to form a two-dimensional (2D) page walk, which reduces the need for hypervisor intervention in guest page table management. However, the extra dimension also increases the maximum number of architecturally-required page table references.

This paper presents an in-depth examination of the 2D page table walk overhead and options for decreasing it. These options include using the AMD Opteron processor's page walk cache to exploit the strong reuse of page entry references. For a mix of server and SPEC benchmarks, the presented results show a 15%-38% improvement in guest performance by extending the existing page walk cache to also store the nested dimension of the 2D page walk. Caching nested page table translations and skipping multiple page entry references produce an additional 3%-7% improvement.

Much of the remaining 2D page walk overhead is due to low-locality nested page entry references, which result in additional memory hierarchy misses. By using large pages, the hypervisor can eliminate many of these long-latency accesses and further improve the guest performance by 3%-22%.

Supplementary Material

JPG File (1346286.jpg)

Download
16.25 KB

index.html (index.html)

Slides from the presentation

Download
.98 KB

ZIP File (p26-ravi--slides.zip)

Supplemental material for Accelerating two-dimensional page walks for virtualized systems

Download
17.29 MB

Audio only (1346286.mp3)

Download
10.17 MB

Video (1346286.mp4)

Download
140.42 MB

References

[1]

A. Saulsbury et al. Recency based TLB preloading. In Proceedings of the 27th Annual International Symposium on Computer Architecture (ISCA), 2000.

Digital Library

[2]

K. Adams and O. Agesen. A comparison of software and hardware techniques for x86 virtualization. In Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2006.

Digital Library

[3]

AMD I/O Virtualization Technology (IOMMU) Specification, February 2007.

[4]

AMD Programmer's Manual, Volume 2, September 2007.

[5]

D. Chang et al. Microarchitecture of HAL's memory management unit. In COMPCON '95: Technologies for the Information Superhighway, Digest of Papers, pages 272--279, 1995.

Digital Library

[6]

E. Perelman et al. Using SimPoint for accurate and efficient simulation. In Proceedings of the International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), 2003.

Digital Library

[7]

G. Neiger et al. Intel virtualization technology: Hardware support for efficient processor virtualization. Intel Technology Journal, 10(3):167--782, 2006.

[8]

P. H. Gum. System/370 extended architecture: facilities for virtual machines. IBM Journal of Research and Development, Nov. 1983.

Digital Library

[9]

TLBs, paging structure caches, and their invalidation. Intel Application Note, 317080-001, April 2007.

[10]

B.L. Jacob and T.N. Mudge. A look at several memory management units, TLB refill mechanisms, and page table organizations. Proceedings of the 8th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 1998.

Digital Library

[11]

G.B. Kandiraju and A. Sivasubramaniam. Going the distance for TLB prefetching: An application-driven study. Proceedings of the 29th Annual International Symposium on Computer Architecture (ISCA), pages 195--206, 2002.

Digital Library

[12]

J. Liedtke. Address space sparsity and fine granularity. 6th ACM SIGOPS European Workshop: Matching Operating Systems to Application Needs, pages 78--81, 1994.

Digital Library

[13]

M. Talluri et al. A new page table for 64-bit address spaces. In Proceedings of the 15th ACM Symposium on Operating Systems Principles (SIGOPS), 1996.

Digital Library

[14]

G.J. Popek and R.P. Goldberg. Formal requirements for virtualizable third generation architectures. Commun. ACM, 17(7):412--421, 1974.

Digital Library

[15]

B. Sander. Processor optimizations for system-level performance. Presentation at Microprocessor Fall Forum, 2006.

[16]

J. Smith and R. Nair. Virtual Machines: Versatile Platforms for Systems and Processes. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2005.

Digital Library

[17]

Software Optimization Guide for AMD Family 10h Processors, September 2007.

[18]

V. Makhija et al. VMmark: A scalable benchmark for virtualized systems. Technical report, VMWare, 2006.

[19]

T.-Y. Yeh. Low-power, high-performance architecture of the PWRficient processor family. Hot Chips 18, 2006.

Cited By

Jang SPark JKwon OLee YHong S(2024)Rethinking Page Table Structure for Fast Address Translation in GPUs: A Fixed-Size Hashed Page TableProceedings of the 2024 International Conference on Parallel Architectures and Compilation Techniques10.1145/3656019.3676900(325-337)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3656019.3676900
Zhou YWilkening MMickens JYu M(2024)SmartNIC Security Isolation in the Cloud with S-NICProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3650071(851-869)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3650071
Kwon OLee YPark JJang STak BHong S(2024)Distributed Page Table: Harnessing Physical Memory as an Unbounded Hashed Page Table2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00013(36-49)Online publication date: 2-Nov-2024
https://doi.org/10.1109/MICRO61859.2024.00013
Show More Cited By

Index Terms

Accelerating two-dimensional page walks for virtualized systems

Recommendations

Accelerating two-dimensional page walks for virtualized systems
ASPLOS '08

Nested paging is a hardware solution for alleviating the software memory management overhead imposed by system virtualization. Nested paging complements existing page walk hardware to form a two-dimensional (2D) page walk, which reduces the need for ...
Accelerating two-dimensional page walks for virtualized systems
ASPLOS '08

Nested paging is a hardware solution for alleviating the software memory management overhead imposed by system virtualization. Nested paging complements existing page walk hardware to form a two-dimensional (2D) page walk, which reduces the need for ...
Accelerating two-dimensional page walks for virtualized systems
ASPLOS '08

Nested paging is a hardware solution for alleviating the software memory management overhead imposed by system virtualization. Nested paging complements existing page walk hardware to form a two-dimensional (2D) page walk, which reduces the need for ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASPLOS XIII: Proceedings of the 13th international conference on Architectural support for programming languages and operating systems

March 2008

352 pages

ISBN:9781595939586

DOI:10.1145/1346281

General Chair:
Susan Eggers
University of Washington, USA
,
Program Chair:
James Larus
Microsoft Research, USA

ACM SIGPLAN Notices Volume 43, Issue 3
ASPLOS '08
March 2008
339 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/1353536
Issue’s Table of Contents
ACM SIGARCH Computer Architecture News Volume 36, Issue 1
ASPLOS '08
March 2008
339 pages
ISSN:0163-5964
DOI:10.1145/1353534
Issue’s Table of Contents
ACM SIGOPS Operating Systems Review Volume 42, Issue 2
ASPLOS '08
March 2008
339 pages
ISSN:0163-5980
DOI:10.1145/1353535
Issue’s Table of Contents

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 March 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ASPLOS08

Sponsor:

ASPLOS08: Architectural Support for Programming Languages and Operating Systems

March 1 - 5, 2008

WA, Seattle, USA

Acceptance Rates

ASPLOS XIII Paper Acceptance Rate 31 of 127 submissions, 24%;

Overall Acceptance Rate 535 of 2,713 submissions, 20%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

277
Total Citations
View Citations
4,096
Total Downloads

Downloads (Last 12 months)152
Downloads (Last 6 weeks)15

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jang SPark JKwon OLee YHong S(2024)Rethinking Page Table Structure for Fast Address Translation in GPUs: A Fixed-Size Hashed Page TableProceedings of the 2024 International Conference on Parallel Architectures and Compilation Techniques10.1145/3656019.3676900(325-337)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3656019.3676900
Zhou YWilkening MMickens JYu M(2024)SmartNIC Security Isolation in the Cloud with S-NICProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3650071(851-869)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3650071
Kwon OLee YPark JJang STak BHong S(2024)Distributed Page Table: Harnessing Physical Memory as an Unbounded Hashed Page Table2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00013(36-49)Online publication date: 2-Nov-2024
https://doi.org/10.1109/MICRO61859.2024.00013
Psomadakis SAlverti CKarakostas VKatsakioris CSiakavaras DNikas KGoumas GKoziris N(2024)Elastic Translations: Fast Virtual Memory with Multiple Translation Sizes2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00012(17-35)Online publication date: 2-Nov-2024
https://doi.org/10.1109/MICRO61859.2024.00012
Xue YLiu YNai LHuang J(2024)Hardware-Assisted Virtualization of Neural Processing Units for Cloud Platforms2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00011(1-16)Online publication date: 2-Nov-2024
https://doi.org/10.1109/MICRO61859.2024.00011
Zhao LWang ZLiu FJiang L(2024)Ninja: A Hardware Assisted System for Accelerating Nested Address Translation2024 IEEE 42nd International Conference on Computer Design (ICCD)10.1109/ICCD63220.2024.00071(426-433)Online publication date: 18-Nov-2024
https://doi.org/10.1109/ICCD63220.2024.00071
Du DYang BXia YChen H(2023)Accelerating Extra Dimensional Page Walks for Confidential ComputingProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3613424.3614293(654-669)Online publication date: 28-Oct-2023
https://dl.acm.org/doi/10.1145/3613424.3614293
Parra PDa Silva ALosa BGarcía JPolo ÓMartínez ASánchez S(2023)Tailor-made Virtualization Monitor Design for CPU Virtualization on LEON ProcessorsACM Transactions on Embedded Computing Systems10.1145/358470222:4(1-32)Online publication date: 17-Feb-2023
https://dl.acm.org/doi/10.1145/3584702
Gosakan KHan JKuszmaul WMubarek IMukherjee NSriram KTagliavini GWest EBender MBhattacharjee AConway AFarach-Colton MGandhi JJohnson RKannan SPorter DAamodt TJerger NSwift M(2023)Mosaic Pages: Big TLB Reach with Small PagesProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3582016.3582021(433-448)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3582016.3582021
Zhao KXue KWang ZSchatzberg DYang LManousis AWeiner JVan Riel RSharma BTang CSkarlatos DSolihin YHeinrich M(2023)Contiguitas: The Pursuit of Physical Memory Contiguity in DatacentersProceedings of the 50th Annual International Symposium on Computer Architecture10.1145/3579371.3589079(1-15)Online publication date: 17-Jun-2023
https://dl.acm.org/doi/10.1145/3579371.3589079
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten