General-purpose Asynchronous Periodic Checkpointing in Hybrid Memory
Pages 675 - 684
Abstract
Non-volatile memory (NVM) is attractive because it enables us to make in-memory data structures persistent without serialization overhead. To implement persistent data structures durable against crashes, periodic checkpointing in NVM has been well studied. A remarkable technique that takes advantage of both DRAM and NVM (i.e., hybrid memory) for periodic checkpointing is mirroring with epoch-based write-address tracking. Its straightforward adoption, however, results in user thread blocking to checkpoint data structures mirrored in DRAM into NVM, from which applications suffer in throughput and responsiveness. To resolve this problem, we incorporate epoch-based versioning into this mirroring technique. The proposed method enables us to delegate checkpointing of data structures mirrored in DRAM into NVM to dedicated background threads that do not block user threads. We develop a system based on our method and evaluate it through experiments with memcached. Our system achieved +13% better throughput than an existing synchronous counterpart and such responsiveness that more than 50% of the performance of the original memcached kept for any time window of 0.5 ms in 99.83% of the entire execution.
References
[1]
David Aksun and James Larus. 2021. Durability Through NVM Checkpointing. 12th Non-Volatile Memories Workshop (Poster). http://nvmw.ucsd.edu/nvmw2021-program/nvmw2021-data/nvmw2021-paper17-final_version_your_extended_abstract.pdf
[2]
David Teksen Aksun. 2021. Software Support for Non-Volatile Memory (NVM) Programming. Ph. D. Dissertation. IINFCOM, EPFL, Lausanne. https://doi.org/10.5075/epfl-thesis-7187
[3]
Alexandro Baldassin, Jo ao Barreto, Daniel Castro, and Paolo Romano. 2021. Persistent Memory: A Survey of Programming Support and Implementations. ACM Comput. Surv. 54, 7 (2021), 152:1–152:37. https://doi.org/10.1145/3465402
[4]
Wentao Cai, Haosen Wen, Vladimir Maksimovski, Mingzhe Du, Rafaello Sanna, Shreif Abdallah, and Michael L. Scott. 2021. Fast Nonblocking Persistence for Concurrent Data Structures. In 35th International Symposium on Distributed Computing (DISC 2021)(Leibniz International Proceedings in Informatics (LIPIcs), Vol. 209). Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 14:1–14:20. https://doi.org/10.4230/LIPIcs.DISC.2021.14
[5]
Nachshon Cohen, David T. Aksun, Hillel Avni, and James R. Larus. 2019. Fine-Grain Checkpointing with In-Cache-Line Logging. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems(ASPLOS ’19). ACM, 441–454. https://doi.org/10.1145/3297858.3304046
[6]
Compute Express Link Consortium, Inc.2022. Compute Express Link (CXL) Specification: Revision 3.0, Version 1.0. https://www.computeexpresslink.org/download-the-specification.
[7]
Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. 2010. Benchmarking Cloud Serving Systems with YCSB. In Proceedings of the 1st ACM Symposium on Cloud Computing(SoCC ’10). ACM, 143–154. https://doi.org/10.1145/1807128.1807152
[8]
Michal Friedman, Erez Petrank, and Pedro Ramalhete. 2021. Mirror: Making Lock-Free Data Structures Persistent. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation(PLDI ’21). ACM, 1218–1232. https://doi.org/10.1145/3453483.3454105
[9]
Joseph Izraelevitz, Hammurabi Mendes, and Michael Scott. 2016. Linearizability of Persistent Memory Objects Under a Full-System-Crash Failure Model. In Distributed Computing, 30th International Symposium, DISC 2016(Lecture Notes in Computer Science, Vol. 9888). Springer, 313–327. https://doi.org/10.1007/978-3-662-53426-7_23
[10]
Ana Khorguani, Thomas Ropars, and Noel De Palma. 2022. ResPCT: Fast Checkpointing in Non-Volatile Memory for Multi-Threaded Applications. In Proceedings of the Seventeenth European Conference on Computer Systems(EuroSys ’22). ACM, 525–540. https://doi.org/10.1145/3492321.3519590
[11]
Wonbae Kim, Chanyeol Park, Dongui Kim, Hyeongjun Park, Young ri Choi, Alan Sussman, and Beomseok Nam. 2022. ListDB: Union of Write-Ahead Logs and Persistent SkipLists for Incremental Checkpointing on Persistent Memory. In 16th USENIX Symposium on Operating Systems Design and Implementation(OSDI ’22). USENIX, 161–177. https://www.usenix.org/conference/osdi22/presentation/kim
[12]
Wook-Hee Kim, R. Madhava Krishnan, Xinwei Fu, Sanidhya Kashyap, and Changwoo Min. 2021. PACTree: A High Performance Persistent Range Index Using PAC Guidelines. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles(SOSP ’21). ACM, 424–439. https://doi.org/10.1145/3477132.3483589
[13]
Leon Lee, Siphrey Xie, Yunus Ma, and Shimin Chen. 2022. Index Checkpoints for Instant Recovery in In-Memory Database Systems. Proc. VLDB Endow. 15, 8 (2022), 1671–1683. https://doi.org/10.14778/3529337.3529350
[14]
Se Kwon Lee, Jayashree Mohan, Sanidhya Kashyap, Taesoo Kim, and Vijay Chidambaram. 2019. Recipe: Converting Concurrent DRAM Indexes to Persistent-Memory Indexes. In Proceedings of the 27th ACM Symposium on Operating Systems Principles(SOSP ’19). ACM, 462–477. https://doi.org/10.1145/3341301.3359635
[15]
Mengxing Liu, Mingxing Zhang, Kang Chen, Xuehai Qian, Yongwei Wu, Weimin Zheng, and Jinglei Ren. 2017. DudeTM: Building Durable Transactions with Decoupling for Persistent Memory. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems(ASPLOS ’17). ACM, 329–343. https://doi.org/10.1145/3037697.3037714
[16]
Arlino Magalhaes, Jose Maria Monteiro, and Angelo Brayner. 2021. Main Memory Database Recovery: A Survey. ACM Comput. Surv. 54, 2 (2021), 46:1–46:36. https://doi.org/10.1145/3442197
[17]
Amirsaman Memaripour, Anirudh Badam, Amar Phanishayee, Yanqi Zhou, Ramnatthan Alagappan, Karin Strauss, and Steven Swanson. 2017. Atomic In-Place Updates for Non-Volatile Main Memories with Kamino-Tx. In Proceedings of the Twelfth European Conference on Computer Systems(EuroSys ’17). ACM, 499–512. https://doi.org/10.1145/3064176.3064215
[18]
Faisal Nawab, Joseph Izraelevitz, Terence Kelly, Charles B. Morrey III, Dhruva R. Chakrabarti, and Michael L. Scott. 2017. Dalí: A Periodically Persistent Hash Map. In Proceedings of the 31st International Symposium on Distributed Computing (DISC 2017)(Leibniz International Proceedings in Informatics (LIPIcs), Vol. 91). Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, 37:1–37:16. https://doi.org/10.4230/LIPIcs.DISC.2017.37
[19]
Haosen Wen, Wentao Cai, Mingzhe Du, Louis Jenkins, Benjamin Valpey, and Michael L. Scott. 2021. A Fast, General System for Buffered Persistent Data Structures. In Proceedings of the 50th International Conference on Parallel Processing(ICPP ’21). ACM, 73:1–73:11. https://doi.org/10.1145/3472456.3472458
[20]
Zhenwei Wu, Kai Lu, Andrew Nisbet, Wenzhe Zhang, and Mikel Luján. 2020. PMThreads: Persistent Memory Threads Harnessing Versioned Shadow Copies. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation(PLDI ’20). ACM, 623–637. https://doi.org/10.1145/3385412.3386000
[21]
Jian Yang, Juno Kim, Morteza Hoseinzadeh, Joseph Izraelevitz, and Steve Swanson. 2020. An Empirical Guide to the Behavior and Use of Scalable Persistent Memory. In 18th USENIX Conference on File and Storage Technologies(FAST ’20). USENIX, 169–182. https://www.usenix.org/conference/fast20/presentation/yang
[22]
Diyu Zhou, Yuchen Qian, Vishal Gupta, Zhifei Yang, Changwoo Min, and Sanidhya Kashyap. 2022. ODINFS: Scaling PM Performance with Opportunistic Delegation. In 16th USENIX Symposium on Operating Systems Design and Implementation(OSDI ’22). USENIX, 179–193. https://www.usenix.org/conference/osdi22/presentation/zhou-diyu
Index Terms
- General-purpose Asynchronous Periodic Checkpointing in Hybrid Memory
Recommendations
Multi-level queue NVM/DRAM hybrid memory management with language runtime support
RACS '15: Proceedings of the 2015 Conference on research in adaptive and convergent systemsNon-volatile memory devices (NVM) devices, such as PCM, STT-MRAM, and ReRAM, enable the integration of secondary storage into main memory. This integration reduces I/O access to slow block devices; however, it is currently unrealistic to construct a ...
NVM duet: unified working memory and persistent store architecture
ASPLOS '14: Proceedings of the 19th international conference on Architectural support for programming languages and operating systemsEmerging non-volatile memory (NVM) technologies have gained a lot of attention recently. The byte-addressability and high density of NVM enable computer architects to build large-scale main memory systems. NVM has also been shown to be a promising ...
Enabling Hybrid PCM Memory System with Inherent Memory Management
RACS '16: Proceedings of the International Conference on Research in Adaptive and Convergent SystemsReplacing the traditional volatile main memory, e.g., DRAM, with a non-volatile phase change memory (PCM) has become a possible solution to reduce the energy consumption of computing systems. To further reduce the bit cost of PCM, the development trend ...
Comments
Information & Contributors
Information
Published In
August 2023
858 pages
ISBN:9798400708435
DOI:10.1145/3605573
Copyright © 2023 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 September 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICPP 2023
ICPP 2023: 52nd International Conference on Parallel Processing
August 7 - 10, 2023
UT, Salt Lake City, USA
Acceptance Rates
Overall Acceptance Rate 91 of 313 submissions, 29%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 291Total Downloads
- Downloads (Last 12 months)245
- Downloads (Last 6 weeks)17
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in