Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3380446.3430624acmconferencesArticle/Chapter ViewAbstractPublication PagesmlcadConference Proceedingsconference-collections
research-article
Open access

Can Wear-Aware Memory Allocation be Intelligent?

Published: 16 November 2020 Publication History

Abstract

Many non-volatile memories (NVM) suffer from a severe reducedcell endurance and therefore require wear-leveling. Heap memory,as one segment, which potentially is mapped to a NVM, faces astrong application dependent characteristic regarding the amountof memory accesses and allocations. A simple deterministic strategyfor wear leveling of the heap may suffer when the available actionspace becomes too large. Therefore, we investigate the employmentof a reinforcement learning agent as a substitute for such a strategyin this paper. The agent's objective is to learn a strategy, which isoptimal with respect to the total memory wear out. We concludethis work with an evaluation, where we compare the deterministicstrategy with the proposed agent. We report that our proposedagent outperforms the simple deterministic strategy in several cases.However, we also report further optimization potential in the agentdesign and deployment.

Supplementary Material

MP4 File (3380446.3430624.mp4)
Presentation Video.

References

[1]
M. C. Carlisle and A. Rogers. Software caching and computation migration in olden. Technical Report TR-483--95, Princeton University.
[2]
Zhaoxia Deng, Lunkai Zhang, Nikita Mishra, Henry Hoffmann, and Frederic T Chong. Memory cocktail therapy: a general learning-based framework to optimize dynamic tradeoffs in nvms. In Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture, pages 232--244, 2017.
[3]
Christian Hakert, Kuan-Hsun Chen, Pual R Genssler, Georg von der Brüggen, Lars Bauer, Hussam Amrouch, Jian-Jia Chen, and Jörg Henkel. Softwear: Software-only in-memory wear-leveling for non-volatile main memory. arXiv preprint arXiv:2004.03244, 2020.
[4]
Christian Hakert, Kuan-Hsun Chen, Mikail Yayla, Georg von der Brüggen, Sebastian Blömeke, and Jian-Jia Chen. Software-based memory analysis environments for in-memory wear-leveling. In 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC), pages 651--658. IEEE, 2020.
[5]
Geoffrey E Hinton, Terrence Joseph Sejnowski, Tomaso A Poggio, et al. Unsupervised learning: foundations of neural computation. MIT press, 1999.
[6]
Kaixin Huang, Yijie Mei, and Linpeng Huang. Quail: Using nvm write monitor to enable transparent wear-leveling. Journal of Systems Architecture, 2020.
[7]
Kenneth C Knowlton. A fast storage allocator. Communications of the ACM, 1965.
[8]
Erik G Learned-Miller. Introduction to supervised learning. I: Department of Computer Science, University of Massachusetts, 2014.
[9]
W. Li, Z. Shuai, C. J. Xue, M. Yuan, and Q. Li. A wear leveling aware memory allocator for both stack and heap management in pcm-based main memory systems. In 2019 Design, Automation Test in Europe Conference Exhibition (DATE).
[10]
Long-Ji Lin. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine learning, 8(3--4):293--321, 1992.
[11]
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement learning. Nature, 2015.
[12]
A. Rogers, M. Carlisle, J. Reppy, and L. Hendren. Supporting dynamic data structures on distributed memory machines. ACM Transactions on Programming Languages and Systems, 17(2), March 1995.
[13]
Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction.
[14]
C. Wang, S. S. Vazhkudai, X. Ma, F. Meng, Y. Kim, and C. Engelmann. Nvmalloc: Exposing an aggregate ssd store as a memory partition in extreme-scale machines. In 2012 IEEE 26th International Parallel and Distributed Processing Symposium.
[15]
Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning.
[16]
Christopher John Cornish Hellaby Watkins. Learning from delayed rewards.
[17]
Michael Wunder, Michael L Littman, and Monica Babes. Classes of multiagent q-learning dynamics with epsilon-greedy exploration. In Proceedings of the 27th International Conference on Machine Learning (ICML-10). Citeseer, 2010.
[18]
S. Yu, N. Xiao, M. Deng, Y. Xing, F. Liu, Z. Cai, and W. Chen. Walloc: An efficient wear-aware allocator for non-volatile main memory. In 2015 IEEE 34th International Performance Computing and Communications Conference (IPCCC).

Cited By

View all
  • (2021)OCTO+: Optimized Checkpointing of B+ Trees for Non-Volatile Main Memory Wear-Leveling2021 IEEE 10th Non-Volatile Memory Systems and Applications Symposium (NVMSA)10.1109/NVMSA53655.2021.9628460(1-6)Online publication date: 18-Aug-2021

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MLCAD '20: Proceedings of the 2020 ACM/IEEE Workshop on Machine Learning for CAD
November 2020
183 pages
ISBN:9781450375191
DOI:10.1145/3380446
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 November 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. heap memory
  2. machine learning
  3. non-volatile memory
  4. reinforcement learning
  5. self learning
  6. wear-leveling

Qualifiers

  • Research-article

Conference

MLCAD '20
Sponsor:
MLCAD '20: 2020 ACM/IEEE Workshop on Machine Learning for CAD
November 16 - 20, 2020
Virtual Event, Iceland

Acceptance Rates

Overall Acceptance Rate 35 of 83 submissions, 42%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)48
  • Downloads (Last 6 weeks)9
Reflects downloads up to 12 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2021)OCTO+: Optimized Checkpointing of B+ Trees for Non-Volatile Main Memory Wear-Leveling2021 IEEE 10th Non-Volatile Memory Systems and Applications Symposium (NVMSA)10.1109/NVMSA53655.2021.9628460(1-6)Online publication date: 18-Aug-2021

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media