research-article

When Caches Aren't Enough: Data Prefetching Techniques

Authors:

Steven P. VanderWiel,

David J. LiljaAuthors Info & Claims

Computer, Volume 30, Issue 7

Pages 23 - 30

https://doi.org/10.1109/2.596622

Published: 01 July 1997 Publication History

Publisher Site

Abstract

For the past few years, CPU performance has outpaced that of dynamic RAM, the primary component of main memory. Developers have had to use increasingly aggressive techniques to reduce or hide delays in accessing main memory. Even so, it is still not uncommon for scientific programs to spend more than half their runtimes stalled on memory requests. This poor performance is partially a result of the policies used to fetch data from main memory: Processors typically request data only when it is needed and then only if it is not first found in the cache. In contrast, data prefetching calls data into the cache before the processor needs it. Ideally, prefetching completes just in time for the processor to access the needed data. Prefetching can nearly double the performance of some scientific applications running on commercial systems. But to achieve this performance, it is critical that the most suitable prefetching technique is used. This article reviews three popular prefetching techniques and examines in which situations they are best used.

References

[1]

T.C. Mowry S. Lam and A. Gupta, "Design and Evaluation of a Compiler Algorithm for Prefetching," Proc. Fifth Int'l Conf. Architectural Support for Programming Languages and Operating Systems, ACM Press, New York, 1992, pp. 62-73.]]

Digital Library

Google Scholar

[2]

T.F. Chen and J.-L. Baer, "A Performance Study of Software and Hardware Data Prefetching Schemes," Proc. 21st Int'l Symp. Computer Architecture, IEEE CS Press, Los Alamitos, Calif., 1994, pp. 223-232.]]

Digital Library

Google Scholar

[3]

F. Dahlgren and P. Stenström, "Effectiveness of Hardware-based Stride and Sequential Prefetching in Shared-memory Multiprocessors," Proc. First IEEE Symp. High-Performance Computer Architecture, IEEE CS Press, Los Alamitos, Calif., 1995, pp. 68-77.]]

Digital Library

Google Scholar

[4]

D. Bernstein C. Doron and A. Freund, "Compiler Techniques for Data Prefetching on the PowerPC," Proc. Int'l Conf. Parallel Architectures and Compilation Techniques, ACM Press, New York, pp. 16-19.]]

Digital Library

Google Scholar

[5]

M.H. Lipasti, et al., "SPAID: Software Prefetching in Pointer and Call-Intensive Environments," Proc. 28th Int'l Symp. Microarchitecture, IEEE CS Press, Los Alamitos, Calif., 1995, pp. 231-236.]]

Digital Library

Google Scholar

[6]

C.K. Luk and T. C. Mowry, "Compiler-Based Prefetching for Recursive Data Structures," Proc. Ninth Int'l Conf. Architectural Support for Programming Languages and Operating Systems, ACM Press, New York, 1996, pp. 222-233.]]

Digital Library

Google Scholar

[7]

S. Przybylski, "The Performance Impact of Block Sizes and Fetch Strategies," Proc. 17th Int'l Symp. Computer Architecture, IEEE CS Press, Los Alamitos, Calif., 1990, pp. 160-169.]]

Digital Library

Google Scholar

[8]

A.J. Smith, "Cache Memories," Computing Surveys, Sept. 1982, pp. 473-530.]]

Digital Library

Google Scholar

[9]

F. Dahlgren M. Dubois and P. Stenström, "Fixed and Adaptive Sequential Prefetching in Shared-memory Multiprocessors," Proc. 1993 Int'l Conf. Parallel Processing, CRC Press, Boca Raton, Fla., 1993, pp. I56-I63.]]

Digital Library

Google Scholar

[10]

T.F. Chen and J.-L. Baer, "Effective Hardware-Based Data Prefetching for High Performance Processors," IEEE Trans. Computers, May 1995, pp. 609-623.]]

Digital Library

Google Scholar

[11]

K.K. Chan, et al., "Design of the HP PA 7200 CPU," Hewlett-Packard J., Feb. 1996, pp. 25-33.]]

Google Scholar

[12]

D. Hunt, "Advanced Performance Features of the 64-bit PA8000," Proc. 40th IEEE Int'l Computer Conf., IEEE CS Press, Los Alamitos, Calif., 1995, pp. 123-128.]]

Digital Library

Google Scholar

Cited By

View all

Bensalem AD'Orazio LLallet JEnrici A(2024)VG-Prefetcher Cache: Towards Edge-Based Time Series Data Management Using Visibility Graph PrefetchingProceedings of the 36th International Conference on Scientific and Statistical Database Management10.1145/3676288.3676304(1-4)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3676288.3676304
Mostofi SFalahati HMahani NLotfi-Kamran PSarbazi-Azad H(2023)Snake: A Variable-length Chain-based Prefetching for GPUsProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3613424.3623782(728-741)Online publication date: 28-Oct-2023
https://dl.acm.org/doi/10.1145/3613424.3623782
Zhang PKannan RPrasanna VMohror KArnold DBadia R(2023)Phases, Modalities, Spatial and Temporal Locality: Domain Specific ML Prefetcher for Accelerating Graph AnalyticsProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3607043(1-15)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3581784.3607043
Show More Cited By

Index Terms

When Caches Aren't Enough: Data Prefetching Techniques
1. Hardware
  1. Integrated circuits
    1. Semiconductor memory
      1. Dynamic memory

Recommendations

Fetch Caches
Performance of One's Complement Caches

On-chip caches to reduce average memory access latency are commonplace in today's commercial microprocessors. These on-chip caches generally have low associativity and small cache sizes. Cache line conflicts are the main source of cache misses, which ...
Reactive-Associative Caches
PACT '01: Proceedings of the 2001 International Conference on Parallel Architectures and Compilation Techniques

Abstract: While set-associative caches typically incur fewer misses than direct-mapped caches, set-associative caches have slower hit times. We propose the reactive-associative cache (r-a cache), which provides flexible associativity by placing most ...

Comments

Information & Contributors

Information

Published In

Computer Volume 30, Issue 7

July 1997

90 pages

ISSN:0018-9162

Issue’s Table of Contents

Publisher

IEEE Computer Society Press

Washington, DC, United States

Publication History

Published: 01 July 1997

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Bensalem AD'Orazio LLallet JEnrici A(2024)VG-Prefetcher Cache: Towards Edge-Based Time Series Data Management Using Visibility Graph PrefetchingProceedings of the 36th International Conference on Scientific and Statistical Database Management10.1145/3676288.3676304(1-4)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3676288.3676304
Mostofi SFalahati HMahani NLotfi-Kamran PSarbazi-Azad H(2023)Snake: A Variable-length Chain-based Prefetching for GPUsProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3613424.3623782(728-741)Online publication date: 28-Oct-2023
https://dl.acm.org/doi/10.1145/3613424.3623782
Zhang PKannan RPrasanna VMohror KArnold DBadia R(2023)Phases, Modalities, Spatial and Temporal Locality: Domain Specific ML Prefetcher for Accelerating Graph AnalyticsProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3607043(1-15)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3581784.3607043
Zhang PKannan RSrivastava ANori APrasanna VWolf FShende SCulhane CAlam SJagode H(2022)ReSembleProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.5555/3571885.3571992(1-14)Online publication date: 13-Nov-2022
https://dl.acm.org/doi/10.5555/3571885.3571992
Zhang PSrivastava ANori AKannan RPrasanna VSterpone LBartolini AButko A(2022)Fine-grained address segmentation for attention-based variable-degree prefetchingProceedings of the 19th ACM International Conference on Computing Frontiers10.1145/3528416.3530236(103-112)Online publication date: 17-May-2022
https://dl.acm.org/doi/10.1145/3528416.3530236
Zhang PSrivastava ABrooks BKannan RPrasanna V(2020)RAOP: Recurrent Neural Network Augmented Offset PrefetcherProceedings of the International Symposium on Memory Systems10.1145/3422575.3422807(352-362)Online publication date: 28-Sep-2020
https://dl.acm.org/doi/10.1145/3422575.3422807
Anwar AMohamed MTarasov VLittley MRupprecht LCheng YZhao NSkourtis DWarke ALudwig HHildebrand DButt AAgrawal NRangaswami R(2018)Improving docker registry design based on production workload analysisProceedings of the 16th USENIX Conference on File and Storage Technologies10.5555/3189759.3189785(265-278)Online publication date: 12-Feb-2018
https://dl.acm.org/doi/10.5555/3189759.3189785
Zhang BRoss BTripathi SBatra SKosar TBalman MByna STierney B(2013)Network-aware data caching and prefetching for cloud-hosted metadata retrievalProceedings of the Third International Workshop on Network-Aware Data Management10.1145/2534695.2534700(1-10)Online publication date: 17-Nov-2013
https://dl.acm.org/doi/10.1145/2534695.2534700
Chen YZhu HSun X(2010)An Adaptive Data Prefetcher for High-Performance ProcessorsProceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing10.1109/CCGRID.2010.61(155-164)Online publication date: 17-May-2010
https://dl.acm.org/doi/10.1109/CCGRID.2010.61
Chen YByna SSun XVerastegui B(2007)Data access history cache and associated data prefetching mechanismsProceedings of the 2007 ACM/IEEE conference on Supercomputing10.1145/1362622.1362651(1-12)Online publication date: 16-Nov-2007
https://dl.acm.org/doi/10.1145/1362622.1362651
Show More Cited By

Abstract

References

Cited By

Index Terms

Recommendations

Fetch Caches

Performance of One's Complement Caches

Reactive-Associative Caches

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations