Article

Simple penalty-sensitive replacement policies for caches

Authors:

Per Stenström,

Michel DuboisAuthors Info & Claims

CF '06: Proceedings of the 3rd conference on Computing frontiers

Pages 341 - 352

https://doi.org/10.1145/1128022.1128068

Published: 03 May 2006 Publication History

Abstract

Classic cache replacement policies assume that miss costs are uniform. However, the correlation between miss rate and cache performance is not as straightforward as it used to be. Ultimately, the true cost measure of a miss should be the penalty, i.e. the actual processing bandwidth lost because of the miss. It is known that, contrary to loads, the penalty of stores is mostly hidden in modern processors. To take advantage of this observation, we propose simple schemes to replace load misses by store misses. We extend classic replacement algorithms such as LRU (Least Recently Used) and PLRU (Partial LRU) to reduce the aggregate miss penalty instead of the miss count.One key issue is to predict the next access type to a block, so that higher replacement priority is given to blocks that will be accessed next with a store. We introduce and evaluate various prediction schemes based on instructions, and broadly inspired from branch predictors. To guide the design we run extensive trace-driven simulations on eight Spec95 benchmarks with a wide range of cache configurations and observe that our simple penalty-sensitive policies yield positive load miss improvements over classic algorithms across most the benchmarks and cache configurations. In some cases the improvements are very large.

References

[1]

Abraham, S., Sugumar, R., Windheiser, D., Rau, B., and Gupta, R., Predictability of Load/Store Instruction Latencies. In Proceedings of the 26th International Symposium on Microarchitecture, Dec. 1993, 139-152.

Digital Library

[2]

Burger, D., and Austin, T., The SimpleScalar Tool Set, Version 2.0. Computer Sciences Dept. Tech. Report #1342, Univ. of Wisconsin-Madison, June 1997.

[3]

Chen, T., Data Prefetching for High-Performance Processors. Ph.D. Dissertation, Tech Report 93-07-01, Dept. of CSE, University of Washington, July 1993.

Digital Library

[4]

Collins, T., and Tullsen, D., Hardware Identification of Cache Conflict Misses. In Proceedings of the 32nd International Symposium on Microarchitecture, Nov. 1999.

Digital Library

[5]

Fu, J., Patel, J., and Janssens, B., Stride Directed Prefetching in Scalar Processors. In Proceedings of the 25th International Symposium on Microarchitecture, Dec. 1992, 102--110.

Digital Library

[6]

Gonzalez, A., Aliagas, C., and Valero, M., A Data Cache with Multiple Caching Strategies Tuned to Different Types of Locality. In Proceeding of International Conference on Supercomputing, July 1995, 338--347.

Digital Library

[7]

Jeong, J., and Dubois, M., Optimal Replacements in Caches with Two Miss Costs. In Proceedings of the 11th ACM Symposium on Parallel Algorithms and Architectures, June 1999, 155--164.

Digital Library

[8]

Jeong, J., Cost-Sensitive Cache Replacement Algorithms, Ph.D. Dissertation, University of Southern California, May 2002.

Digital Library

[9]

Jeong, J., and Dubois, M., Cost-Sensitive Cache Replacement Algorithms. In Proceedings of the 9th International Symposium on High-Performance Computer Architecture, Feb. 2003.

Digital Library

[10]

Johnson, T., Merten, M., and Hwu, W., Run-time Spatial Locality Detection and Optimization. In Proceedings of the 30th International Symposium on Microarchitecture, Dec. 1997.

Digital Library

[11]

Jouppi, N., Cache Write Policies and Performance. In Proceedings of the 27th International Symposium on Computer Architecture, May 1993, 191--201.

Digital Library

[12]

Kaxiras, S., and Goodman, J., Improving CC-NUMA Performance Using Instruction-Based Prediction. In Proceedings of the 5th International Symposium on High-Performance Computer Architecture, Jan. 1999, 161--170.

Digital Library

[13]

Lai, A., and Falsafi, B., Selective, Accurate, and Timely Self-Invalidation Using Last-Touch Prediction. In Proceedings of the 27th International Symposium on Computer Architecture, June 2000.

Digital Library

[14]

Mounes-Toussi, F., and Lilja, D., The Effect of Using State-Based Priority Information in a Shared-Memory Multiprocessor Cache Replacement Policy. In Proceedings of International Conference on Parallel Processing, Aug. 1998, 217--224.

Digital Library

[15]

Mowry, T., Tolerating Latency Through Software-Controlled Data Prefetching. Ph.D. Dissertation, Stanford University, Mar. 1994.

Digital Library

[16]

Mukherjee, S., and Hill, M., Using Prediction to Accelerate Coherence Protocols. In Proceedings of the 25th International Symposium on Computer Architecture, June 1998, 179--190.

Digital Library

[17]

Seznec, A., and Lloansi, F., About Effective Cache Miss Penalty on Out-of-Order Superscalar Processors. IRISA Report #970, Nov. 1995.

[18]

Skadron, K., and Clark, D., Design Issues and Tradeoffs for Write Buffers. In Proceedings of the 3rd International Symposium on High-Performance Computer Architecture, Feb. 1997, 144--155.

Digital Library

[19]

Smith, A., Cache Memories. ACM Computing Surveys, Sept. 1982, v. 3, 473--530.

Digital Library

[20]

So, K., and Rechtschaffen, R., Cache Operations by MRU Change. IEEE Transactions on Computers, June 1988, v. 37, 700--709.

Digital Library

[21]

Srivivasan, S., Ju, R., Lebeck, A., and Wilkerson, C., Locality vs. Criticality. In Proceedings of the 28th International Symposium on Computer Architecture, July 2001, 132--143.

Digital Library

[22]

Standard Performance Evaluation Corporation, http:// www.specbench.org.

[23]

Stenström, P., Brorsson, M., and Sandberg, L., An Adaptive Cache Coherence Protocol Optimized for Migratory Sharing. In Proceedings of the 20th International Symposium on Computer Architecture, May 1993, 109--118.

Digital Library

[24]

Stone, H., High-Performance Computer Architecture. 2nd Edition, Addison-Wesley Publishing Company, Nov. 1990.

Digital Library

[25]

Tyson, G., Farrens, M., Matthews, J., and Pleszkun, A., A Modified Approach to Data Cache Management. In Proceedings of the 28th International Symposium on Microarchitecture, Dec. 1995, 93--103.

Digital Library

[26]

Wong, W., and Baer, J., Modified LRU Policies for Improving Second-Level Cache Behavior. In Proceedings of the 6th International Symposium on High-Performance Computer Architecture, Jan. 2000, 49--60.

[27]

Yeh, T., and Patt, Y., Alternative Implementation of Two-Level Adaptive Branch Prediction. In Proceedings of the 19th International Symposium on Computer Architecture, May 1992, 124--134.

Digital Library

Cited By

Geanta MGhica LTapus N(2016)Leverage cache replacement policy in multicore processors2016 IEEE 12th International Conference on Intelligent Computer Communication and Processing (ICCP)10.1109/ICCP.2016.7737182(417-424)Online publication date: Sep-2016
https://doi.org/10.1109/ICCP.2016.7737182
Li LLu JCheng XBode AGerndt MStenström PRauchwerger LMiller BSchulz M(2014)Block value based insertion policy for high performance last-level cachesProceedings of the 28th ACM international conference on Supercomputing10.1145/2597652.2597653(63-72)Online publication date: 10-Jun-2014
https://dl.acm.org/doi/10.1145/2597652.2597653
Li LLu JCheng X(2014)Retention Benefit Based Intelligent Cache ReplacementJournal of Computer Science and Technology10.1007/s11390-014-1481-229:6(947-961)Online publication date: 17-Nov-2014
https://doi.org/10.1007/s11390-014-1481-2
Show More Cited By

Index Terms

Simple penalty-sensitive replacement policies for caches
1. Hardware
  1. Integrated circuits
    1. Semiconductor memory
      1. Dynamic memory

Recommendations

Improving Performance in Sub-Block Caches with Optimized Replacement Policies
Special Issues on Neuromorphic Computing and Emerging Many-Core Systems for Exascale Computing

Recent advances in computer processor design have led to the introduction of sub-blocking to cache architectures. Sub-block caches reduce the tag area and power overhead in caches without reducing the effective cache size by using fewer tags to index ...
Cache Replacement Algorithms with Nonuniform Miss Costs

Cache replacement algorithms originally developed in the context of uniprocessors executing one instruction at a time implicitly assume that all cache misses have the same cost. However, in modern systems, some cache misses are more expensive than ...
A new cache replacement algorithm for last-level caches by exploiting tag-distance correlation of cache lines

Cache memory plays a crucial role in determining the performance of processors, especially for embedded processors where area and power are tightly constrained. It is necessary to have effective management mechanisms, such as cache replacement policies, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CF '06: Proceedings of the 3rd conference on Computing frontiers

May 2006

430 pages

ISBN:1595933026

DOI:10.1145/1128022

General Chairs:
Monica Alderighi
IASF - INAF
,
Valentina Salapura
IBM
,
Program Chair:
Sally A. McKee
Cornell University

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 May 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CF06

Sponsor:

CF06: Computing Frontiers Conference

May 3 - 5, 2006

Ischia, Italy

Acceptance Rates

Overall Acceptance Rate 273 of 785 submissions, 35%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
412
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)4

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Geanta MGhica LTapus N(2016)Leverage cache replacement policy in multicore processors2016 IEEE 12th International Conference on Intelligent Computer Communication and Processing (ICCP)10.1109/ICCP.2016.7737182(417-424)Online publication date: Sep-2016
https://doi.org/10.1109/ICCP.2016.7737182
Li LLu JCheng XBode AGerndt MStenström PRauchwerger LMiller BSchulz M(2014)Block value based insertion policy for high performance last-level cachesProceedings of the 28th ACM international conference on Supercomputing10.1145/2597652.2597653(63-72)Online publication date: 10-Jun-2014
https://dl.acm.org/doi/10.1145/2597652.2597653
Li LLu JCheng X(2014)Retention Benefit Based Intelligent Cache ReplacementJournal of Computer Science and Technology10.1007/s11390-014-1481-229:6(947-961)Online publication date: 17-Nov-2014
https://doi.org/10.1007/s11390-014-1481-2
Petoumenos PKeramidas GKaxiras S(2009)Instruction-based reuse-distance prediction for effective cache management2009 International Symposium on Systems, Architectures, Modeling, and Simulation10.1109/ICSAMOS.2009.5289241(49-58)Online publication date: Jul-2009
https://doi.org/10.1109/ICSAMOS.2009.5289241
Kharbutli MSolihin Y(2008)Counter-Based Cache Replacement and Bypassing AlgorithmsIEEE Transactions on Computers10.1109/TC.2007.7081657:4(433-447)Online publication date: 1-Apr-2008
https://dl.acm.org/doi/10.1109/TC.2007.70816
Tang MLin X(2008)A Novel Scheme to Balance the Cache Sharing in High Performance Computing SystemProceedings of the 2008 10th IEEE International Conference on High Performance Computing and Communications10.1109/HPCC.2008.13(695-701)Online publication date: 25-Sep-2008
https://dl.acm.org/doi/10.1109/HPCC.2008.13
Jeong JDubois M(2006)Cache Replacement Algorithms with Nonuniform Miss CostsIEEE Transactions on Computers10.1109/TC.2006.5055:4(353-365)Online publication date: 1-Apr-2006
https://dl.acm.org/doi/10.1109/TC.2006.50

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents