research-article

Open access

ARI: Adaptive LLC-memory traffic management

Authors:

Viacheslav V. Fedorov,

A. L. Narasimha Reddy,

Paul V. GratzAuthors Info & Claims

ACM Transactions on Architecture and Code Optimization (TACO), Volume 10, Issue 4

Article No.: 46, Pages 1 - 19

https://doi.org/10.1145/2543697

Published: 01 December 2013 Publication History

Abstract

Decreasing the traffic from the CPU LLC to main memory is a very important issue in modern systems. Recent work focuses on cache misses, overlooking the impact of writebacks on the total memory traffic, energy consumption, IPC, and so forth. Policies that foster a balanced approach, between reducing write traffic to memory and improving miss rates, can increase overall performance and improve energy efficiency and memory system lifetime for NVM memory technology, such as phase-change memory (PCM). We propose Adaptive Replacement and Insertion (ARI), an adaptive approach to last-level CPU cache management, optimizing the two parameters (miss rate and writeback rate) simultaneously. Our specific focus is to reduce writebacks as much as possible while maintaining or improving the miss rate relative to conventional LRU replacement policy. ARI reduces LLC writebacks by 33%, on average, while also decreasing misses by 4.7%, on average. In a typical system, this boosts IPC by 4.9%, on average, while decreasing energy consumption by 8.9%. These results are achieved with minimal hardware overheads.

References

[1]

Binkert, N., Beckmann, B., Black, G., Reinhardt, S. K., Saidi, A., Basu, A., Hestness, J., Hower, D. R., Krishna, T., Sardashti, S., Sen, R., Sewell, K., Shoaib, M., Vaish, N., Hill, M. D., and Wood, D. A. 2011. The gem5 simulator. SIGARCH Comput. Archit. News 39, 2, 1--7.

Digital Library

[2]

Corbató, F. J. 1968. A paging experiment with the multics system. Tech. rep., DTIC Document.

[3]

Dhiman, G., Ayoub, R., and Rosing, T. 2009. Pdram: A hybrid pram and dram main memory system. In Proceedings of the 46th Annual Design Automation Conference (DAC’09). ACM, New York, NY, 664--469.

Digital Library

[4]

Ferreira, A. P., Zhou, M., Bock, S., Childers, B., Melhem, R., and Mossé, D. 2010. Increasing pcm main memory lifetime. In Proceedings of the Conference on Design, Automation and Test in Europe (DATE’10). European Design and Automation Association, Belgium, 914--919.

Digital Library

[5]

Gao, H. and Wilkerson, C. 2010. A dueling segmented LRU replacement algorithm with adaptive bypassing. In Proceedings of the 1st JILP Workshop on Computer Architecture Competitions: Cache Replacement Championship (JWAC’10).

[6]

Gaur, J., Chaudhuri, M., and Subramoney, S. 2011. Bypass and insertion algorithms for exclusive last-level caches. In Proceedings of the 38th Annual International Symposium on Computer Architecture (ISCA’11). ACM, New York, NY, 81--92.

Digital Library

[7]

Goodman, J. R. 1983. Using cache memory to reduce processor-memory traffic. In Proceedings of the 10th Annual International Symposium on Computer Architecture (ISCA’83). ACM, New York, NY, 124--131.

Digital Library

[8]

International Technology Roadmap for Semiconductors (ITRS) Working Group. 2009. International Technology Roadmap for Semiconductors (ITRS), 2009 Edition.

[9]

Ishii, Y., Inaba, M., Hiraki, K., et al. 2010. Cache replacement policy using map-based adaptive insertion. In Proceedings of the 1st JILP Worskhop on Computer Architecture Competitions: Cache Replacement Championship (JWAC’10).

[10]

Jaleel, A., Borch, E., Bhandaru, M., Steely Jr., S. C., and Emer, J. 2010a. Achieving non-inclusive cache performance with inclusive caches: Temporal locality aware (TLA) cache management policies. In Proceedings of the 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’43). IEEE Computer Society, Washington, DC, 151--162.

Digital Library

[11]

Jaleel, A., Theobald, K. B., Steely, Jr., S. C., and Emer, J. 2010b. High performance cache replacement using re-reference interval prediction (RRIP). In Proceedings of the 37th Annual International Symposium on Computer Architecture (ISCA’10). ACM, New York, NY, 60--71.

Digital Library

[12]

Janapsatya, A., Ignjatović, A., Peddersen, J., and Parameswaran, S. 2010. Dueling clock: Adaptive cache replacement policy based on the clock algorithm. In Proceedings of the Conference on Design, Automation and Test in Europe (DATE’10). European Design and Automation Association, Belgium, 920--925.

Digital Library

[13]

Jiang, X., Mishra, A., Zhao, L., Iyer, R., Fang, Z., Srinivasan, S., Makineni, S., Brett, P., and Das, C. R. 2011. Access: Smart scheduling for asymmetric cache CMPs. In Proceedings of the 2011 IEEE International Symposium on High Performance Computer Architecture (HPCA’11). 527--538.

Digital Library

[14]

Jouppi, N. P. 1993. Cache write policies and performance. In Proceedings of the 20th Annual International Symposium on Computer Architecture (ISCA’93). ACM, New York, NY, 191--201.

Digital Library

[15]

Khan, S. and Jiménez, D. A. 2010. Insertion policy selection using decision tree analysis. In Proceedings of the 2010 IEEE International Conference on Computer Design (ICCD’10). 106--111.

[16]

Khan, S., Tian, Y., and Jiménez, D. A. 2010. Sampling dead block prediction for last-level caches. In Proceedings of the 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’10). 175--186.

Digital Library

[17]

Lee, B. C., Ipek, E., Mutlu, O., and Burger, D. 2009. Architecting phase change memory as a scalable dram alternative. In Proceedings of the 36th Annual International Symposium on Computer Architecture (ISCA’09). ACM, New York, NY, 2--13.

Digital Library

[18]

Lee, B. C., Zhou, P., Yang, J., Zhang, Y., Zhao, B., Ipek, E., Mutlu, O., and Burger, D. 2010. Phase-change technology and the future of main memory. IEEE Micro30, 1, 143--143.

Digital Library

[19]

Lee, C. J., Mutlu, O., Ebrahimi, E., Narasiman, V., and Patt, Y. N. 2010a. Dram-aware last-level cache replacement. Technical Report TR-HPS-2010-007. High Performance Systems Group, Department of Electrical and Computer Engineering, University of Texas at Austin.

[20]

Lee, C. J., Mutlu, O., Narasiman, V., Ebrahimi, E., and Patt, Y. N. 2010b. Dram-aware last-level cache writeback: Reducing write-caused interference in memory systems. Technical Report TR-HPS-2010-002. High Performance Systems Group, Department of Electrical and Computer Engineering, University of Texas at Austin.

[21]

Lee, D., Choi, J., Kim, J.-H., Noh, S. H., Min, S. L., Cho, Y., and Kim, C. S. 1999. On the existence of a spectrum of policies that subsumes the least recently used (LRU) and least frequently used (LFU) policies. In Proceedings of the 1999 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS’99). ACM, New York, NY, 134--143.

Digital Library

[22]

Lee, H.-H. S., Tyson, G. S., and Farrens, M. K. 2000. Eager writeback—a technique for improving bandwidth utilization. In Proceedings of the 33rd Annual ACM/IEEE International Symposium on Microarchitecture (MICRO’00). ACM, New York, NY, 11--21.

Digital Library

[23]

Loh, G. H. and Hill, M. D. 2012. Supporting very large dram caches with compound-access scheduling and missmap. IEEE Micro 32, 3, 70--78.

Digital Library

[24]

Meza, J., Chang, J., Yoon, H., Mutlu, O., and Ranganathan, P. 2012. Enabling efficient and scalable hybrid memories using fine-granularity dram cache management. IEEE Comput. Archit. Lett. 11, 2, 61--64.

Digital Library

[25]

Michaud, P. 2010. The 3p and 4p cache replacement policies. In Proceedings of the 1st JILP Worshop on Computer Architecture Competitions: Cache Replacement Championship (JWAC’10).

[26]

Molka, D., Hackenberg, D., Schone, R., and Muller, M. S. 2009. Memory performance and cache coherency effects on an Intel Nehalem multiprocessor system. In Proceedings of the 2009 18th International Conference on Parallel Architectures and Compilation Techniques (PACT’09). IEEE Computer Society, Washington, DC, 261--270.

Digital Library

[27]

Park, S.-y., Jung, D., Kang, J.-u., Kim, J.-s., and Lee, J. 2006. CFLRU: A replacement algorithm for flash memory. In Proceedings of the 2006 International Conference on Compilers, Architecture and Synthesis for Embedded Systems (CASES’06). ACM, New York, NY, 234--241.

Digital Library

[28]

Perelman, E., Hamerly, G., Van Biesbrouck, M., Sherwood, T., and Calder, B. 2003. Using SimPoint for accurate and efficient simulation. In Proceedings of the 2003 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems. 318--319.

Digital Library

[29]

Qureshi, M. K., Franceschini, M. M., and Lastras-Montano, L. A. 2010. Improving read performance of phase change memories via write cancellation and write pausing. In Proceedings of the 2010 IEEE 16th International Symposium on High Performance Computer Architecture (HPCA’10). 1--11.

[30]

Qureshi, M. K., Jaleel, A., Patt, Y. N., Steely, S. C., and Emer, J. 2007. Adaptive insertion policies for high performance caching. In Proceedings of the 34th Annual International Symposium on Computer Architecture (ISCA’07). ACM, New York, NY, 381--391.

Digital Library

[31]

Qureshi, M. K., Karidis, J., Franceschini, M., Srinivasan, V., Lastras, L., and Abali, B. 2009a. Enhancing lifetime and security of PCM-based main memory with start-gap wear leveling. In Proceedings of the 42nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’09). ACM, New York, NY, 14--23.

Digital Library

[32]

Qureshi, M. K. and Loh, G. H. 2012. Fundamental latency trade-off in architecting dram caches: Outperforming impractical sram-tags with a simple and practical design. In Proceedings of the 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’12). IEEE Computer Society, Washington, DC, 235--246.

Digital Library

[33]

Qureshi, M. K., Srinivasan, V., and Rivers, J. A. 2009b. Scalable high performance main memory system using phase-change memory technology. In Proceedings of the 36th Annual International Symposium on Computer Architecture (ISCA’09). ACM, New York, NY, 24--33.

Digital Library

[34]

Ramos, L. E., Gorbatov, E., and Bianchini, R. 2011. Page placement in hybrid memory systems. In Proceedings of the International Conference on Supercomputing (ICS’11). ACM, New York, NY, 85--95.

Digital Library

[35]

Rosenfeld, P., Cooper-Balis, E., and Jacob, B. 2011. DRAMSim2: A cycle accurate memory system simulator. IEEE Comput. Archit. Lett. 10, 1, 16--19.

Digital Library

[36]

Seong, N. H., Woo, D. H., and Lee, H.-H. S. 2010. Security refresh: Prevent malicious wear-out and increase durability for phase-change memory with dynamically randomized address mapping. In Proceedings of the 37th Annual International Symposium on Computer Architecture (ISCA’10). ACM, New York, NY, 383--394.

Digital Library

[37]

Stuecheli, J., Kaseridis, D., Daly, D., Hunter, H. C., and John, L. K. 2010. The virtual write queue: Coordinating dram and last-level cache policies. In Proceedings of the 37th Annual International Symposium on Computer Architecture (ISCA’10). ACM, New York, NY, 72--82.

Digital Library

[38]

Subramanian, R., Smaragdakis, Y., and Loh, G. H. 2006. Adaptive caches: Effective shaping of cache behavior to workloads. In Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’06). IEEE Computer Society, Washington, DC, 385--396.

Digital Library

[39]

Wang, Z., Khan, S. M., and Jiménez, D. A. 2012. Improving writeback efficiency with decoupled last-write prediction. In Proceedings of the 2012 39th Annual International Symposium on Computer Architecture (ISCA’12). 309--320.

Digital Library

[40]

Wu, X., Li, J., Zhang, L., Speight, E., Rajamony, R., and Xie, Y. 2009. Hybrid cache architecture with disparate memory technologies. In Proceedings of the 36th Annual International Symposium on Computer Architecture (ISCA’09). ACM, New York, NY, 34--45.

Digital Library

[41]

Yoon, H., Meza, J., Ausavarungnirun, R., Harding, R. A., and Mutlu, O. 2012. Row buffer locality aware caching policies for hybrid memories. In Proceedings of the 2012 IEEE 30th International Conference on Computer Design (ICCD’12). 337--344.

Digital Library

[42]

Zheng, Y., Davis, B. T., and Jordan, M. 2004. Performance evaluation of exclusive cache hierarchies. In Proceedings of the 2004 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS’04). IEEE Computer Society, Washington, DC, 89--96.

Digital Library

Cited By

Zhang YYang SHu HYang CCai PZhou X(2024)SuccinctKV: a CPU-efficient LSM-tree Based KV Store with Scan-based CompactionACM Transactions on Architecture and Code Optimization10.1145/369587321:4(1-26)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3695873
Sethumurugan SYin JSartori J(2021)Designing a Cost-Effective Cache Replacement Policy using Machine Learning2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA51647.2021.00033(291-303)Online publication date: Feb-2021
https://doi.org/10.1109/HPCA51647.2021.00033
Chaudhuri MGaur JSubramoney S(2019)Bandwidth-Aware Last-Level Caching: Efficiently Coordinating Off-Chip Read and Write Bandwidth2019 IEEE 37th International Conference on Computer Design (ICCD)10.1109/ICCD46524.2019.00022(109-118)Online publication date: Nov-2019
https://doi.org/10.1109/ICCD46524.2019.00022
Show More Cited By

Index Terms

ARI: Adaptive LLC-memory traffic management
1. Computer systems organization
2. Hardware
  1. Integrated circuits
    1. Semiconductor memory

Recommendations

Bypass and insertion algorithms for exclusive last-level caches
ISCA '11

Inclusive last-level caches (LLCs) waste precious silicon estate due to cross-level replication of cache blocks. As the industry moves toward cache hierarchies with larger inner levels, this wasted cache space leads to bigger performance losses compared ...
Introducing hierarchy-awareness in replacement and bypass algorithms for last-level caches
PACT '12: Proceedings of the 21st international conference on Parallel architectures and compilation techniques

The replacement policies for the last-level caches (LLCs) are usually designed based on the access information available locally at the LLC. These policies are inherently sub-optimal due to lack of information about the activities in the inner-levels of ...
Bypass and insertion algorithms for exclusive last-level caches
ISCA '11: Proceedings of the 38th annual international symposium on Computer architecture

Inclusive last-level caches (LLCs) waste precious silicon estate due to cross-level replication of cache blocks. As the industry moves toward cache hierarchies with larger inner levels, this wasted cache space leads to bigger performance losses compared ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Architecture and Code Optimization

ACM Transactions on Architecture and Code Optimization Volume 10, Issue 4

December 2013

1046 pages

ISSN:1544-3566

EISSN:1544-3973

DOI:10.1145/2541228

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2013

Accepted: 01 November 2013

Revised: 01 November 2013

Received: 01 February 2013

Published in TACO Volume 10, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Division of Computing and Communication Foundations

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
522
Total Downloads

Downloads (Last 12 months)61
Downloads (Last 6 weeks)8

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang YYang SHu HYang CCai PZhou X(2024)SuccinctKV: a CPU-efficient LSM-tree Based KV Store with Scan-based CompactionACM Transactions on Architecture and Code Optimization10.1145/369587321:4(1-26)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3695873
Sethumurugan SYin JSartori J(2021)Designing a Cost-Effective Cache Replacement Policy using Machine Learning2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA51647.2021.00033(291-303)Online publication date: Feb-2021
https://doi.org/10.1109/HPCA51647.2021.00033
Chaudhuri MGaur JSubramoney S(2019)Bandwidth-Aware Last-Level Caching: Efficiently Coordinating Off-Chip Read and Write Bandwidth2019 IEEE 37th International Conference on Computer Design (ICCD)10.1109/ICCD46524.2019.00022(109-118)Online publication date: Nov-2019
https://doi.org/10.1109/ICCD46524.2019.00022
Xu JFeng DHua YTong WLiu JLi CLi Z(2018)An efficient PCM-based main memory system via exploiting fine-grained dirtiness of cachelines2018 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE.2018.8342273(1616-1621)Online publication date: Mar-2018
https://doi.org/10.23919/DATE.2018.8342273
Ustiugov DDaglis APicorel JSutherland MBugnion EFalsafi BPnevmatikatos DJacob B(2018)Design guidelines for high-performance SCM hierarchiesProceedings of the International Symposium on Memory Systems10.1145/3240302.3240310(3-16)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1145/3240302.3240310
Jia YZhou FGao XWu SJin HLiao XYuan P(2018)VAILProceedings of the 9th International Workshop on Programming Models and Applications for Multicores and Manycores10.1145/3178442.3178451(79-84)Online publication date: 24-Feb-2018
https://dl.acm.org/doi/10.1145/3178442.3178451
Ma TQu JShen WTian YAl-Dhelaan AAl-Rodhaan M(2018)Weighted Greedy Dual Size Frequency Based Caching Replacement AlgorithmIEEE Access10.1109/ACCESS.2018.27903816(7214-7223)Online publication date: 2018
https://doi.org/10.1109/ACCESS.2018.2790381
Kim JTeran EGratz PJiménez DPugsley SWilkerson C(2017)Kill the Program CounterACM SIGARCH Computer Architecture News10.1145/3093337.303770145:1(737-749)Online publication date: 4-Apr-2017
https://dl.acm.org/doi/10.1145/3093337.3037701
Kim JTeran EGratz PJiménez DPugsley SWilkerson C(2017)Kill the Program CounterACM SIGPLAN Notices10.1145/3093336.303770152:4(737-749)Online publication date: 4-Apr-2017
https://dl.acm.org/doi/10.1145/3093336.3037701
Kim JTeran EGratz PJiménez DPugsley SWilkerson C(2017)Kill the Program CounterACM SIGOPS Operating Systems Review10.1145/3093315.303770151:2(737-749)Online publication date: 4-Apr-2017
https://doi.org/10.1145/3093315.3037701
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents