Cited By
View all- Sha SZhang YLuo YWang XWang Z(2022)Accelerating Address Translation for Virtualization by Leveraging Hardware ModeIEEE Transactions on Computers10.1109/TC.2022.314567171:11(3047-3060)Online publication date: 1-Nov-2022
(No)Compromis: paging virtualization is not a fatality
Authors: 
Boris Teabe, Peterson Yuhala, Alain Tchana, Fabien Hermenier, Daniel Hagimont, and 
Gilles MullerAuthors Info & Claims
Boris Teabe, Peterson Yuhala, Alain Tchana, Fabien Hermenier, Daniel Hagimont, and Gilles MullerAuthors Info & ClaimsApril 2021
Pages 43 - 56
Abstract
Nested/Extended Page Table (EPT) is the current hardware solution for virtualizing memory in virtualized systems. It induces a significant performance overhead due to the 2D page walk it requires, thus 24 memory accesses on a TLB miss (instead of 4 memory accesses in a native system). This 2D page walk constraint comes from the utilization of paging for managing virtual machine (VM) memory. This paper shows that paging is not necessary in the hypervisor. Our solution Compromis, a novel Memory Management Unit, uses direct segments for VM memory management combined with paging for VM's processes. This is the first time that a direct segment based solution is shown to be applicable to the entire VM memory while keeping applications unchanged. Relying on the 310 studied datacenter traces, the paper shows that it is possible to provision up to 99.99% of the VMs using a single memory segment. The paper presents a systematic methodology for implementing Compromis in the hardware, the hypervisor and the datacenter scheduler. Evaluation results show that Compromis outperforms the two popular memory virtualization solutions: shadow paging and EPT by up to 30% and 370% respectively.
References
[1]
[n.d.]. Benefits of Virtualization. https://www.thrivenetworks.com/ blog/benefits-of-virtualization/.
[2]
[n.d.]. Inside Microsoft Azure datacenter hardware and software architecture with Mark Russinovich. https://www.youtube.com/watch? v= Lv8fDiTNHjk.
[3]
[n.d.]. Top 5 Business Benefits of Server Virtualization. https://blog.nhlearningsolutions.com/blog/top-5-ways-businessesbenefit-from-server-virtualization.
[4]
[n.d.]. Transparent Hugepages. https://lwn.net/Articles/359158/.
[5]
[n.d.]. X86 Paravirtualised Memory Management. https://wiki.xen. org/wiki/X86_Paravirtualised_Memory_Management.
[6]
Keith Adams and Ole Agesen. 2006. A Comparison of Software and Hardware Techniques for x86 Virtualization. In Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems (San Jose, California, USA) (ASPLOS XII). ACM, New York, NY, USA, 2-13. https://doi.org/10.1145/1168857. 1168860
[7]
Ole Agesen, Jim Mattson, Radu Rugina, and Jefrey Sheldon. 2012. Software Techniques for Avoiding Hardware Virtualization Exits. In Proceedings of the 2012 USENIX Conference on Annual Technical Conference (Boston, MA) ( USENIX ATC'12). USENIX Association, Berkeley, CA, USA, 35-35. http://dl.acm.org/citation.cfm?id= 2342821. 2342856
[8]
Jeongseob Ahn, Seongwook Jin, and Jaehyuk Huh. 2012. Revisiting Hardware-assisted Page Walks for Virtualized Systems. In Proceedings of the 39th Annual International Symposium on Computer Architecture (Portland, Oregon) (ISCA '12). IEEE Computer Society, Washington, DC, USA, 476-487. http://dl.acm.org/citation.cfm?id= 2337159. 2337214
[9]
Hanna Alam, Tianhao Zhang, Mattan Erez, and Yoav Etsion. 2017. Do-It-Yourself Virtual Memory Translation. In Proceedings of the 44th Annual International Symposium on Computer Architecture (Toronto, ON, Canada) ( ISCA '17). ACM, New York, NY, USA, 457-468. https://doi.org/10.1145/3079856.3080209
[10]
Michael Armbrust, Armando Fox, Rean Grifith, Anthony D. Joseph, Randy Katz, Andy Konwinski, Gunho Lee, David Patterson, Ariel Rabkin, Ion Stoica, and Matei Zaharia. 2010. A View of Cloud Computing. Commun. ACM 53, 4 (April 2010 ), 50-58. https://doi.org/10. 1145/1721654.1721672
[11]
Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, and Andrew Warfield. 2003. Xen and the art of virtualization. In IN SOSP. 164-177.
[12]
Thomas W. Barr, Alan L. Cox, and Scott Rixner. 2010. Translation Caching: Skip, Don'T Walk ( the Page Table). In Proceedings of the 37th Annual International Symposium on Computer Architecture (SaintMalo, France) (ISCA '10). ACM, New York, NY, USA, 48-59. https://doi.org/10.1145/1815961.1815970
[13]
Arkaprava Basu, Jayneel Gandhi, Jichuan Chang, Mark D. Hill, and Michael M. Swift. 2013. Eficient Virtual Memory for Big Memory Servers. In Proceedings of the 40th Annual International Symposium on Computer Architecture ( Tel-Aviv, Israel) (ISCA '13). ACM, New York, NY, USA, 237-248. https://doi.org/10.1145/2485922.2485943
[14]
Arkaprava Basu, Mark D. Hill, and Michael M. Swift. 2012. Reducing Memory Reference Energy with Opportunistic Virtual Caching. In Proceedings of the 39th Annual International Symposium on Computer Architecture (Portland, Oregon) (ISCA '12). IEEE Computer Society, Washington, DC, USA, 297-308. http://dl.acm.org/citation.cfm?id= 2337159. 2337194
[15]
Ravi Bhargava, Benjamin Serebrin, Francesco Spadini, and Srilatha Manne. 2008. Accelerating Two-dimensional Page Walks for Virtualized Systems. In Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems (Seattle, WA, USA) (ASPLOS XIII). ACM, New York, NY, USA, 26-35. https://doi.org/10.1145/1346281.1346286
[16]
Abhishek Bhattacharjee. 2013. Large-reach Memory Management Unit Caches. In Proceedings of the 46th Annual IEEE/ACM International Symposium on Microarchitecture (Davis, California) (MICRO-46). ACM, New York, NY, USA, 383-394. https://doi.org/10.1145/2540708.2540741
[17]
Abhishek Bhattacharjee. 2017. Translation-Triggered Prefetching. SIGARCH Comput. Archit. News 45, 1 (April 2017 ), 63-76. https://doi. org/10.1145/3093337.3037705
[18]
Abhishek Bhattacharjee, Daniel Lustig, and Margaret Martonosi. 2011. Shared Last-level TLBs for Chip Multiprocessors. In Proceedings of the 2011 IEEE 17th International Symposium on High Performance Computer Architecture (HPCA '11). IEEE Computer Society, Washington, DC, USA, 62-63. http://dl.acm.org/citation.cfm?id= 2014698. 2014896
[19]
Xiaotao Chang, Hubertus Franke, Yi Ge, Tao Liu, Kun Wang, Jimi Xenidis, Fei Chen, and Yu Zhang. 2013. Improving Virtualization in the Presence of Software Managed Translation Lookaside Bufers. In Proceedings of the 40th Annual International Symposium on Computer Architecture ( Tel-Aviv, Israel) (ISCA '13). ACM, New York, NY, USA, 120-129. https://doi.org/10.1145/2485922.2485933
[20]
cloudstack [n.d.]. Apache CloudStack-Open Source Cloud Computing. http://cloudstack.apache.org/.
[21]
Eli Cortez, Anand Bonde, Alexandre Muzio, Mark Russinovich, Marcus Fontoura, and Ricardo Bianchini. 2017. Resource Central: Understanding and Predicting Workloads for Improved Resource Management in Large Cloud Platforms. In Proceedings of the 26th Symposium on Operating Systems Principles (Shanghai, China) (SOSP '17). ACM, New York, NY, USA, 153-167. https://doi.org/10.1145/3132747.3132772
[22]
Guilherme Cox and Abhishek Bhattacharjee. 2017. Eficient Address Translation for Architectures with Multiple Page Sizes. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems (Xi'an, China) (ASPLOS '17). ACM, New York, NY, USA, 435-448. https://doi.org/10. 1145/3037697.3037704
[23]
filter [n.d.]. Nova filter scheduler. http://docs.openstack.org/developer/ nova/filter_scheduler.html.
[24]
Jayneel Gandhi, Arkaprava Basu, Mark D. Hill, and Michael M. Swift. 2014. Eficient Memory Virtualization: Reducing Dimensionality of Nested Page Walks. In Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture (Cambridge, United Kingdom) (MICRO-47). IEEE Computer Society, Washington, DC, USA, 178-189. https://doi.org/10.1109/MICRO. 2014.37
[25]
Jayneel Gandhi, Mark D. Hill, and Michael M. Swift. 2016. Agile Paging: Exceeding the Best of Nested and Shadow Paging. In Proceedings of the 43rd International Symposium on Computer Architecture (Seoul, Republic of Korea) (ISCA '16). IEEE Press, Piscataway, NJ, USA, 707-718. https://doi.org/10.1109/ISCA. 2016.67
[26]
Swapnil Haria, Mark D. Hill, and Michael M. Swift. 2018. Devirtualizing Memory in Heterogeneous Systems. In Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems (Williamsburg, VA, USA) ( ASPLOS '18). ACM, New York, NY, USA, 637-650. https://doi.org/10.1145/ 3173162.3173194
[27]
Nikhita Kunati and Michael M. Swift. 2018. Implementation of Direct Segments on a RISC-V Processor. In IN Second Workshop on Computer Architecture Research with RISC-V (CARRV), Co-located with ISCA.
[28]
Youngjin Kwon, Hangchen Yu, Simon Peter, Christopher J. Rossbach, and Emmett Witchel. 2016. Coordinated and Eficient Huge Page Management with Ingens. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (Savannah, GA, USA) ( OSDI'16). USENIX Association, Berkeley, CA, USA, 705-721. http://dl.acm.org/citation.cfm?id= 3026877. 3026931
[29]
Youngjin Kwon, Hangchen Yu, Simon Peter, Christopher J. Rossbach, and Emmett Witchel. 2017. Ingens: Huge Page Support for the OS and Hypervisor. SIGOPS Oper. Syst. Rev. 51, 1 (Sept. 2017 ), 83-93. https://doi.org/10.1145/3139645.3139659
[30]
Yashwant Marathe, Nagendra Gulur, Jee Ho Ryoo, Shuang Song, and Lizy K. John. 2017. CSALT: Context Switch Aware Large TLB. In Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture (Cambridge, Massachusetts) (MICRO-50 ' 17 ). ACM, New York, NY, USA, 449-462. https://doi.org/10.1145/3123939.3124549
[31]
Artemiy Margaritov, Dmitrii Ustiugov, Edouard Bugnion, and Boris Grot. 2019. Prefetched Address Translation. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture (Columbus, OH, USA) ( MICRO '52). Association for Computing Machinery, New York, NY, USA, 1023-1036. https://doi.org/10.1145/ 3352460.3358294
[32]
Vlad Nitu, Aram Kocharyan, Hannas Yaya, Alain Tchana, Daniel Hagimont, and Hrachya Astsatryan. 2018. Working Set Size Estimation Techniques in Virtualized Environments: One Size Does Not Fit All. Proc. ACM Meas. Anal. Comput. Syst. 2, 1, Article 19 ( April 2018 ), 22 pages. https://doi.org/10.1145/3179422
[33]
Vlad Nitu, Pierre Olivier, Alain Tchana, Daniel Chiba, Antonio Barbalace, Daniel Hagimont, and Binoy Ravindran. 2017. Swift Birth and Quick Death: Enabling Fast Parallel Guest Boot and Destruction in the Xen Hypervisor. In Proceedings of the 13th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (Xi'an, China) (VEE '17). ACM, New York, NY, USA, 1-14. https://doi.org/10.1145/3050748.3050758
[34]
Ashish Panwar, Aravinda Prasad, and K. Gopinath. 2018. Making Huge Pages Actually Useful. In Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems (Williamsburg, VA, USA) ( ASPLOS '18). ACM, New York, NY, USA, 679-692. https://doi.org/10.1145/3173162.3173203
[35]
Chang Hyun Park, Taekyung Heo, Jungi Jeong, and Jaehyuk Huh. 2017. Hybrid TLB Coalescing: Improving TLB Translation Coverage Under Diverse Fragmented Memory Allocations. In Proceedings of the 44th Annual International Symposium on Computer Architecture (Toronto, ON, Canada) ( ISCA '17). ACM, New York, NY, USA, 444-456. https://doi.org/10.1145/3079856.3080217
[36]
Binh Pham, Ján Vesel?, Gabriel H. Loh, and Abhishek Bhattacharjee. 2015. Large Pages and Lightweight Memory Management in Virtualized Environments: Can You Have It Both Ways?. In Proceedings of the 48th International Symposium on Microarchitecture (Waikiki, Hawaii) (MICRO-48). ACM, New York, NY, USA, 1-12. https://doi.org/10.1145/2830772.2830773
[37]
R. Raman, M. Livny, and M. Solomon. 1998. Matchmaking: Distributed Resource Management for High Throughput Computing. In Proceedings of the 7th IEEE International Symposium on High Performance Distributed Computing (HPDC '98). IEEE Computer Society, Washington, DC, USA, 140-. http://dl.acm.org/citation.cfm?id= 822083. 823222
[38]
Jee Ho Ryoo, Nagendra Gulur, Shuang Song, and Lizy K. John. 2017. Rethinking TLB Designs in Virtualized Environments: A Very Large Part-of-Memory TLB. In Proceedings of the 44th Annual International Symposium on Computer Architecture (Toronto, ON, Canada) ( ISCA '17). ACM, New York, NY, USA, 469-480. https://doi.org/10.1145/3079856. 3080210
[39]
Tom Shanley. 1996. Pentium Pro Processor System Architecture (1st ed.). Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA.
[40]
Siqi Shen, Vincent van Beek, and Alexandru Iosup. 2015. Statistical Characterization of Business-Critical Workloads Hosted in Cloud Datacenters. In 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2015, Shenzhen, China, May 4-7, 2015. 465-474.
[41]
Cristan Szmajda and Gernot Heiser. 2003. Variable Radix Page Table: A Page Table for Modern Architectures. In Advances in Computer Systems Architecture, Amos Omondi and Stanislav Sedukhin (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 290-304.
[42]
M. Talluri, M. D. Hill, and Y. A. Khalidi. 1995. A New Page Table for 64-bit Address Spaces. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, USA) ( SOSP '95). ACM, New York, NY, USA, 184-200. https://doi.org/10. 1145/224056.224071
[43]
Rich Uhlig, Gil Neiger, Dion Rodgers, Amy L. Santoni, Fernando C. M. Martins, Andrew V. Anderson, Steven M. Bennett, Alain Kagi, Felix H. Leung, and Larry Smith. 2005. Intel Virtualization Technology. Computer 38, 5 (May 2005 ), 48-56. https://doi.org/10.1109/ MC. 2005.163
[44]
Carl A. Waldspurger. 2002. Memory Resource Management in VMware ESX Server. SIGOPS Oper. Syst. Rev. 36, SI (Dec. 2002 ), 181-194. https://doi.org/10.1145/844128.844146
[45]
Xiaolin Wang, Jiarui Zang, Zhenlin Wang, Yingwei Luo, and Xiaoming Li. 2011. Selective Hardware/Software Memory Virtualization. In Proceedings of the 7th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (Newport Beach, California, USA) ( VEE '11). ACM, New York, NY, USA, 217-226. https://doi.org/10.1145/ 1952682.1952710
[46]
Idan Yaniv and Dan Tsafrir. 2016. Hash, Don'T Cache ( the Page Table). In Proceedings of the 2016 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Science (Antibes Juan-lesPins, France) (SIGMETRICS '16). ACM, New York, NY, USA, 337-350. https://doi.org/10.1145/2896377.2901456
Index Terms
- (No)Compromis: paging virtualization is not a fatality
Recommendations
MemX: Virtualization of Cluster-Wide Memory
ICPP '10: Proceedings of the 2010 39th International Conference on Parallel ProcessingWe present MemX -- a distributed system that virtualizes cluster-wide memory to support data-intensive and large memory workloads in virtual machines (VMs). MemX provides a number of benefits in virtualized settings: (1) VM workloads that access large ...
Reviewing the World of Virtualization
ISMS '12: Proceedings of the 2012 Third International Conference on Intelligent Systems Modelling and SimulationThe latest talk in IT industry is about server virtualization. Virtualization increase server utilization rates. Along with this, it lets you consolidate multiple operating systems and applications as per physical server and deploy new applications in ...
Agile paging: exceeding the best of nested and shadow paging
ISCA '16: Proceedings of the 43rd International Symposium on Computer ArchitectureVirtualization provides benefits for many workloads, but the overheads of virtualizing memory are not universally low. The cost comes from managing two levels of address translation---one in the guest virtual machine (VM) and the other in the host ...
Comments
Information & Contributors
Information
Published In
April 2021
200 pages
ISBN:9781450383943
DOI:10.1145/3453933
- General Chair:
- Ben L. Titzer,
- Program Chairs:
- Harry Xu,
- Irene Zhang
Copyright © 2021 ACM.
© 2021 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 April 2021
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
VEE '21: 17th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments
April 16, 2021
Virtual, USA
Acceptance Rates
Overall Acceptance Rate 80 of 235 submissions, 34%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 279Total Downloads
- Downloads (Last 12 months)23
- Downloads (Last 6 weeks)2
Other Metrics
Citations
Cited By
View all- Sha SZhang YLuo YWang XWang Z(2022)Accelerating Address Translation for Virtualization by Leveraging Hardware ModeIEEE Transactions on Computers10.1109/TC.2022.314567171:11(3047-3060)Online publication date: 1-Nov-2022
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in