research-article

Cache or Direct Access? Revitalizing Cache in Heterogeneous Memory File System

Authors:

Xinwei HuAuthors Info & Claims

DIMES '23: Proceedings of the 1st Workshop on Disruptive Memory Systems

Pages 38 - 44

https://doi.org/10.1145/3609308.3625272

Published: 23 October 2023 Publication History

Abstract

This paper revisits the value of cache in DRAM-PM heterogeneous memory file systems. The first contribution is a comprehensive analysis of the existing file systems on heterogeneous memory, including cache-based and DAX-based (direct access). We find that the DRAM cache still plays an important role in heterogeneous memory. The second contribution is a cache framework for heterogeneous memory, called FLAC. FLAC integrates the cache with the virtual memory management and proposes two technologies of zero-copy caching and parallel-optimized cache management, which deliver the benefits of fast application-storage data transfer and efficient DRAM-PM data synchronization/migration. We further implement a library file system upon FLAC. Microbenchmarks show that FLAC provides a performance increase of up to two orders of magnitude over existing file systems in file read/write. With a real-world application, FLAC achieves up to 77.4% and 89.3% better performance than NOVA and EXT4, respectively.

References

[1]

Daniel Bittman, Peter Alvaro, Pankaj Mehra, Darrell D. E. Long, and Ethan L. Miller. 2020. Twizzler: A Data-Centric OS for Non-Volatile Memory. In Proceedings of the USENIX Annual Technical Conference (ATC'20).

[2]

Youmin Chen, Youyou Lu, Bohong Zhu, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau, and Jiwu Shu. 2021. Scalable Persistent Memory File System with Kernel-Userspace Collaboration. In Proceedings of USENIX Conference on File and Storage Technologies (FAST'21).

[3]

Jungsik Choi, Jaewan Hong, Youngjin Kwon, and Hwansoo Han. 2020. Libnvmmio: Reconstructing Software IO Path with Failure-Atomic Memory-Mapped Interface. In Proceedings of the USENIX Annual Technical Conference (ATC'20).

[4]

CXL Consortium. 2023. Compute Express Link Specification Revision 3.0. https://www.computeexpresslink.org/download-the-specification

[5]

Gen-Z Consortium. 2022. Gen-Z Final Specifications. https://genzconsortium.org/specifications/

[6]

Johnathan Corbet. 2020. EXT4-DAX. https://lwn.net/Articles/717953

[7]

Peter Druschel and Larry L. Peterson. 1993. Fbufs: A High-Bandwidth Cross-Domain Transfer Facility. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP'93).

[8]

Subramanya R. Dulloor, Sanjay Kumar, Anil Keshavamurthy, Philip Lantz, Dheeraj Reddy, Rajesh Sankaran, and Jeff Jackson. 2014. System Software for Persistent Memory. In Proceedings of the Eleventh European Conference on Computer Systems (EuroSys'14).

Digital Library

[9]

Subramanya R. Dulloor, Amitabha Roy, Zheguang Zhao, Narayanan Sundaram, Nadathur Satish, Rajesh Sankaran, Jeff Jackson, and Karsten Schwan. 2016. Data Tiering in Heterogeneous Memory Systems. In Proceedings of the European Conference on Computer Systems (EuroSys'16).

Digital Library

[10]

Sangjin Han, Scott Marshall, Byung-Gon Chun, and Sylvia Ratnasamy. 2012. MegaPipe: A New Programming Interface for Scalable Network I/O. In Proceedings of the USENIX Conference on Operating Systems Design and Implementation (OSDI'12).

[11]

Mohammad Hedayati, Spyridoula Gravani, Ethan Johnson, John Criswell, Michael L. Scott, Kai Shen, and Mike Marty. 2019. Hodor: Intra-Process Isolation for High-Throughput Data Plane Libraries. In Proceedings of the USENIX Annual Technical Conference (ATC'19).

[12]

Intel. 2022. 3D XPoint Breakthrough Non-Volatile Memory. https://www.intel.com/content/www/us/en/architecture-and-technology/intel-micron-3d-xpoint-webcast.html

[13]

Intel. 2022. Persistent Memory Development Kit. https://pmem.io/pmdk

[14]

Joseph Izraelevitz, Jian Yang, Lu Zhang, Juno Kim, Xiao Liu, Amirsaman Memaripour, Yun Joon Soh, Zixuan Wang, Yi Xu, Subramanya R. Dulloor, Jishen Zhao, and Steven Swanson. 2019. Basic Performance Measurements of the Intel Optane DC Persistent Memory Module. CoRR (2019). https://doi.org/arXiv:1903.05714

[15]

Myoungsoo Jung. 2023. Hello bytes, bye blocks: PCIe storage meets compute express link for memory expansion (CXL-SSD). In Proceedings of Conference on Hot Topics in Storage and File Systems (HotStorage'23).

[16]

Rohan Kadekodi, Saurabh Kadekodi, Soujanya Ponnapalli, Harshad Shirwadkar, Gregory R. Ganger, Aasheesh Kolli, and Vijay Chidambaram. 2021. WineFS: A Hugepage-Aware File System for Persistent Memory That Ages Gracefully. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP'21).

Digital Library

[17]

Rohan Kadekodi, Se Kwon Lee, Sanidhya Kashyap, Taesoo Kim, Aasheesh Kolli, and Vijay Chidambaram. 2019. SplitFS: Reducing Software Overhead in File Systems for Persistent Memory. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP'19).

Digital Library

[18]

Juno Kim, Yun Joon Soh, Joseph Izraelevitz, Jishen Zhao, and Steven Swanson. 2020. SubZero: Zero-copy IO for Persistent Main Memory File Systems. In Proceedings of Asia-Pacific Workshop on Systems (APSys'20).

Digital Library

[19]

Youngjin Kwon, Henrique Fingler, Tyler Hunt, Simon Peter, Emmett Witchel, and Thomas Anderson. 2017. Strata: A cross media file system. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP'17).

Digital Library

[20]

Benjamin C. Lee, Ping Zhou, Jun Yang, Youtao Zhang, Bo Zhao, Engin Ipek, Onur Mutlu, and Doug Burger. 2010. Phase-Change Technology and the Future of Main Memory. IEEE Micro 30, 1 (2010), 143--143.

Digital Library

[21]

Ruibin Li, Xiang Ren, Xu Zhao, Siwei He, Michael Stumm, and Ding Yuan. 2022. ctFS: Replacing File Indexing with Hardware Memory Translation through Contiguous File Allocation for Persistent Memory. In Proceedings of USENIX Conference on File and Storage Technologies (FAST'22).

[22]

Yubo Liu, Hongbo Li, Yutong Lu, Zhiguang Chen, Nong Xiao, and Ming Zhao. 2020. HasFS: optimizing file system consistency mechanism on NVM-based hybrid storage architecture. Cluster Computing 23 (2020), 2510--2515.

Digital Library

[23]

Jiaxin Ou, Jiwu Shu, and Youyou Lu. 2016. A High Performance File System for Non-Volatile Main Memory. In Proceedings of the Eleventh European Conference on Computer Systems (EuroSys'16).

Digital Library

[24]

Vivek S. Pai, Peter Druschel, and Willy Zwaenepoel. 1999. IO-Lite: A Unified I/O Buffering and Caching System. In Proceedings of the USENIX Conference on Operating Systems Design and Implementation (OSDI'99).

[25]

Yuxin Ren, Gabriel Parmer, Teo Georgiev, and Gedare Bloom. 2016. CBufs: Efficient, System-Wide Memory Management and Sharing. In Proceedings of the ACM SIGPLAN International Symposium on Memory Management (ISMM'16).

Digital Library

[26]

Vasily A. Sartakov, Lluís Vilanova, and Peter Pietzuch. 2021. CubicleOS: A Library OS with Software Componentisation for Practical Isolation. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'21).

Digital Library

[27]

Timothy Stamler, Deukyeon Hwang, Amanda Raybuck, Wei Zhang, and Simon Peter. 2022. zIO: Accelerating IO-Intensive Applications with Transparent Zero-Copy IO. In Proceedings of the USENIX Conference on Operating Systems Design and Implementation (OSDI'22).

[28]

Haris Volos, Andres Jaan Tack, and Michael M. Swift. 2011. Mnemosyne: Lightweight Persistent Memory. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'11).

[29]

Yongfeng Wang, Yinjin Fu, Yubo Liu, Zhiguang Chen, and Nong Xiao. 2022. Characterizing and Optimizing Hybrid DRAM-PM Main Memory System with Application Awareness. In Proceedings of Design, Automation & Test in Europe Conference & Exhibition (DATE'22).

[30]

Hobin Woo, Daegyu Han, Seungjoon Ha, Sam H. Noh, and Beomseok Nam. 2023. On Stacking a Persistent Memory File System on Legacy File Systems. In Proceedings of USENIX Conference on File and Storage Technologies (FAST'23).

[31]

Jian Xu and Steven Swanson. 2016. NOVA: A Log-Structured File System for Hybrid Volatile/Non-Volatile Main Memories. In Proceedings of USENIX Conference on File and Storage Technologies (FAST'16).

[32]

Jifei Yi, Mingkai Dong, Fangnuo Wu, and Haibo Chen. 2022. HTMFS: Strong Consistency Comes for Free with Hardware Transactional Memory in Persistent Memory File Systems. In Proceedings of USENIX Conference on File and Storage Technologies (FAST'22).

Cited By

Liu YRen YLiu MLi HGuo HMiao XHu XChen HMa XWon Y(2024)Optimizing file systems on heterogeneous memory by integrating DRAM cache with virtual memory managementProceedings of the 22nd USENIX Conference on File and Storage Technologies10.5555/3650697.3650702(71-88)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.5555/3650697.3650702

Index Terms

Cache or Direct Access? Revitalizing Cache in Heterogeneous Memory File System
1. Information systems
  1. Information storage systems
    1. Information storage technologies
      1. Storage class memory
2. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        File systems management

Recommendations

Cache-aware block allocation for memory-technology storage targeted file systems
SAC '19: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing

In storage using Non-Volatile Memory (NVM), file placements affect the performance of processor cache. However, the impact of file system policies on processor cache has not been well studied. In particular, sequential block allocation of file systems ...
Advising cache for lower cache
IMCOM '17: Proceedings of the 11th International Conference on Ubiquitous Information Management and Communication

In a virtualized environment, such as a cloud computing environment, guest and host operating systems run simultaneously. Both of the operating systems have page caches for disk accesses. In such an environment, the host operating system cache does not ...
Heterogeneous way-size cache
ICS '06: Proceedings of the 20th annual international conference on Supercomputing

Set-associative cache architectures are commonly used. These caches consist of a number of ways, each of the same size. We have observed that the different ways have very different utilization, which motivates the design of caches with heterogeneous way ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DIMES '23: Proceedings of the 1st Workshop on Disruptive Memory Systems

October 2023

64 pages

ISBN:9798400703003

DOI:10.1145/3609308

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

In-Cooperation

USENIX

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

DIMES '23

Sponsor:

SIGOPS

DIMES '23: 1st Workshop on Disruptive Memory Systems

October 23, 2023

Koblenz, Germany

Acceptance Rates

DIMES '23 Paper Acceptance Rate 8 of 17 submissions, 47%;

Overall Acceptance Rate 8 of 17 submissions, 47%

Upcoming Conference

SOSP '24

Sponsor:
sigops

ACM SIGOPS 30th Symposium on Operating Systems Principles

November 4 - 6, 2024

Austin , TX , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
198
Total Downloads

Downloads (Last 12 months)198
Downloads (Last 6 weeks)6

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu YRen YLiu MLi HGuo HMiao XHu XChen HMa XWon Y(2024)Optimizing file systems on heterogeneous memory by integrating DRAM cache with virtual memory managementProceedings of the 22nd USENIX Conference on File and Storage Technologies10.5555/3650697.3650702(71-88)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.5555/3650697.3650702

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents