research-article

Heterogeneous system coherence for integrated CPU-GPU systems

Authors:

Arkaprava Basu,

Sooraj Puthoor,

Bradford M. Beckmann,

Steven K. Reinhardt,

David A. WoodAuthors Info & Claims

MICRO-46: Proceedings of the 46th Annual IEEE/ACM International Symposium on Microarchitecture

Pages 457 - 467

https://doi.org/10.1145/2540708.2540747

Published: 07 December 2013 Publication History

Abstract

Many future heterogeneous systems will integrate CPUs and GPUs physically on a single chip and logically connect them via shared memory to avoid explicit data copying. Making this shared memory coherent facilitates programming and fine-grained sharing, but throughput-oriented GPUs can overwhelm CPUs with coherence requests not well-filtered by caches. Meanwhile, region coherence has been proposed for CPU-only systems to reduce snoop bandwidth by obtaining coherence permissions for large regions.

This paper develops Heterogeneous System Coherence (HSC) for CPU-GPU systems to mitigate the coherence bandwidth effects of GPU memory requests. HSC replaces a standard directory with a region directory and adds a region buffer to the L2 cache. These structures allow the system to move bandwidth from the coherence network to the high-bandwidth direct-access bus without sacrificing coherence.

Evaluation results with a subset of Rodinia benchmarks and the AMD APP SDK show that HSC can improve performance compared to a conventional directory protocol by an average of more than 2x and a maximum of more than 4.5x. Additionally, HSC reduces the bandwidth to the directory by an average of 94% and by more than 99% for four of the analyzed benchmarks.

References

[1]

3D-ICs: http://www.jedec.org/category/technology-focus-area/3d-ics-0. Accessed: 2013-09-18.

[2]

Agarwal, N. et al. 2009. In-network coherence filtering. (2009), 232.

[3]

Alisafaee, M. 2012. Spatiotemporal Coherence Tracking. (Dec. 2012), 341--350.

Digital Library

[4]

AMD Inc. 2010. AMD Fusion#8482; Family of APUs.

[5]

AMD Inc. 2008. AMD Stream SDK.

[6]

AMD Radeon#8482; HD 7970 Graphics: www.amd.com/radeonHD7970.

[7]

Archibald, J. and Baer, J.-L. 1984. An Economical Solution to the Cache Coherence Problem. Proceedings of the 11th Annual International Symposium on Computer Architecture (Jun. 1984), 355--362.

Digital Library

[8]

Basu, A. et al. 2013. CMP Directory Coherence: One Granularity Does Not Fit All. Technical Report #CS-TR-2013-1798. Univ. of Wisconsin Computer Sciences.

[9]

Binkert, N. et al. 2011. The gem5 simulator. Computer Architecture News (CAN). (2011).

Digital Library

[10]

Black, B. et al. 2006. Die Stacking (3D) Microarchitecture. (Dec. 2006), 469--479.

Digital Library

[11]

Cantin, J. F. et al. 2005. Improving Multiprocessor Performance with Coarse-Grain Coherence Tracking. Proceedings of the 32nd Annual International Symposium on Computer Architecture (Jun. 2005).

Digital Library

[12]

Che, S. et al. 2009. Rodinia: A benchmark suite for heterogeneous computing. 2009 IEEE International Symposium on Workload Characterization (IISWC) (2009).

Digital Library

[13]

GeForce GTX 780: 2013. http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780/specifications.

[14]

Gelado, I. et al. 2010. An asymmetric distributed shared memory model for heterogeneous parallel systems. (2010), 347.

Digital Library

[15]

Jerger, N. D. E. et al. 2008. Virtual tree coherence: Leveraging regions and in-network multicast trees for scalable cache coherence. MICRO 41 (Nov. 2008), 35--46.

Digital Library

[16]

Kelm, J. H. et al. 2010. Cohesion: A Hybrid Memory Model for Accelerators. Proccedings of the 37th Annual Internal Symposium on Computer Architecture (ISCA) (Jun. 2010).

Digital Library

[17]

Kim, D. et al. 2010. Subspace snooping. Proceedings of the 19th international conference on Parallel architectures and compilation techniques (2010), 111--122.

Digital Library

[18]

Lotfi-Kamran, P. et al. 2010. TurboTag: lookup filtering to reduce coherence directory power. Proceedings of the 16th ACM/IEEE international symposium on Low power electronics and design (2010), 377--382.

Digital Library

[19]

Mantor, M. 2011. Fusion and the Future of Heterogeneous Computing.

[20]

Mantor, M. and Houston, M. 2011. AMD Graphics Core Next.

[21]

Martin, M. M. K. et al. 2012. Why on-chip cache coherence is here to stay. Communications of the ACM. 55, 7 (Jul. 2012), 78.

Digital Library

[22]

Moshovos, A. et al. 2001. JETTY: Filtering Snoops for Reduced Power Consumption in SMP Servers. Proceedings of the Seventh IEEE Symposium on High-Performance Computer Architecture (Jan. 2001).

Digital Library

[23]

Moshovos, A. 2005. RegionScout: Exploiting Coarse Grain Sharing in Snoop-Based Coherence. Proceedings of the 32nd Annual International Symposium on Computer Architecture (Jun. 2005).

Digital Library

[24]

Owens, J. D. et al. 2008. GPU Computing. Proceedings of the IEEE. 96, 5 (2008).

[25]

Rogers, P. 2013. Heterogeneous System Architecture Overview. HOT CHIPS (2013).

[26]

Rossbach, C. J. et al. 2011. PTask: Operating System Abstractions To Manage GPUs as Compute Devices. Proc. of the 23nd ACM Symp. on Operating System Principles (Oct. 2011).

Digital Library

[27]

Salapura, V. et al. 2007. Improving the accuracy of snoop filtering using stream registers. Proceedings of the 2007 workshop on MEmory performance: DEaling with Applications, systems and architecture (2007), 25--32.

Digital Library

[28]

Singh, I. et al. 2013. Cache Coherence for GPU Architectures. The 19th IEEE International Symposium on High Performance Computer Architecture. (2013).

Digital Library

[29]

Tuck, J. et al. 2006. Scalable Cache Miss Handling for High Memory-Level Parallelism. Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture (Dec. 2006).

Digital Library

[30]

Wilkins, M. NVIDIA Jumps on Graphics-Enabled Microprocessor Bandwagon.

[31]

Zebchuk, J. et al. 2007. A Framework for Coarse-Grain Optimizations in the On-Chip Memory Hierarchy. Proceedings of the 40th Annual IEEE/ACM International Symposium on Microarchitecture (Dec. 2007), 314--327.

Digital Library

Cited By

Liu JEgawa RTakahashi KShimomura YTakizawa H(2024)Reuse distance-based shared LLC management mechanism for heterogeneous CPU-GPU systemsIEICE Electronics Express10.1587/elex.21.2023052021:4(20230520-20230520)Online publication date: 25-Feb-2024
https://doi.org/10.1587/elex.21.20230520
Fang JLin SYang HXu YSu X(2023)A perceptual and predictive batch-processing memory scheduling strategy for a CPU-GPU heterogeneous systemCPU-GPU异构系统感知和预测的批处理内存调度策略Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.220044924:7(994-1006)Online publication date: 28-Jul-2023
https://doi.org/10.1631/FITEE.2200449
Puthoor SLipasti M(2023)Turn-based Spatiotemporal Coherence for GPUsACM Transactions on Architecture and Code Optimization10.1145/359305420:3(1-27)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3593054
Show More Cited By

Index Terms

Heterogeneous system coherence for integrated CPU-GPU systems
1. Computer systems organization
  1. Architectures
    1. Other architectures
      1. Heterogeneous (hybrid) systems
2. Hardware
  1. Integrated circuits
    1. Semiconductor memory
      1. Dynamic memory

Recommendations

A simple cache coherence scheme for integrated CPU-GPU systems
DAC '20: Proceedings of the 57th ACM/EDAC/IEEE Design Automation Conference

This paper presents a novel approach to accelerate applications running on integrated CPU-GPU systems. Many integrated CPU-GPU systems use cache-coherent shared memory to communicate. For example, after CPU produces data for GPU, the GPU may pull the ...
Boosting performance of directory-based cache coherence protocols with coherence bypass at subpage granularity and a novel on-chip page table
CF '16: Proceedings of the ACM International Conference on Computing Frontiers

Chip multiprocessors (CMPs) require effective cache coherence protocols as well as fast virtual-to-physical address translation mechanisms for high performance. Directory-based cache coherence protocols are the state-of-the-art approaches in many-core ...
Increasing the effectiveness of directory caches by deactivating coherence for private memory blocks
ISCA '11: Proceedings of the 38th annual international symposium on Computer architecture

To meet the demand for more powerful high-performance shared-memory servers, multiprocessor systems must incorporate efficient and scalable cache coherence protocols, such as those based on directory caches. However, the limited directory cache size of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MICRO-46: Proceedings of the 46th Annual IEEE/ACM International Symposium on Microarchitecture

December 2013

498 pages

ISBN:9781450326384

DOI:10.1145/2540708

General Chair:
Matthew Farrens
UC Davis
,
Program Chair:
Christos Kozyrakis
Stanford University

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 December 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

MICRO-46

Sponsor:

SIGMICRO

MICRO-46: The 46th Annual IEEE/ACM International Symposium on Microarchitecture

December 7 - 11, 2013

California, Davis

Acceptance Rates

MICRO-46 Paper Acceptance Rate 39 of 239 submissions, 16%;

Overall Acceptance Rate 484 of 2,242 submissions, 22%

Upcoming Conference

MICRO '24

Sponsor:
sigmicro

57th Annual IEEE/ACM International Symposium on Microarchitecture

November 2 - 6, 2024

Austin , TX , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

89
Total Citations
View Citations
1,740
Total Downloads

Downloads (Last 12 months)138
Downloads (Last 6 weeks)9

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu JEgawa RTakahashi KShimomura YTakizawa H(2024)Reuse distance-based shared LLC management mechanism for heterogeneous CPU-GPU systemsIEICE Electronics Express10.1587/elex.21.2023052021:4(20230520-20230520)Online publication date: 25-Feb-2024
https://doi.org/10.1587/elex.21.20230520
Fang JLin SYang HXu YSu X(2023)A perceptual and predictive batch-processing memory scheduling strategy for a CPU-GPU heterogeneous systemCPU-GPU异构系统感知和预测的批处理内存调度策略Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.220044924:7(994-1006)Online publication date: 28-Jul-2023
https://doi.org/10.1631/FITEE.2200449
Puthoor SLipasti M(2023)Turn-based Spatiotemporal Coherence for GPUsACM Transactions on Architecture and Code Optimization10.1145/359305420:3(1-27)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3593054
Muthukrishnan HLustig DVilla OWenisch TNellans D(2023)FinePack: Transparently Improving the Efficiency of Fine-Grained Transfers in Multi-GPU Systems2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA56546.2023.10070949(516-529)Online publication date: Feb-2023
https://doi.org/10.1109/HPCA56546.2023.10070949
Li WCheng HLu ZLu YLiu W(2023)HASpMV: Heterogeneity-Aware Sparse Matrix-Vector Multiplication on Modern Asymmetric Multicore Processors2023 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER52292.2023.00025(209-220)Online publication date: 31-Oct-2023
https://doi.org/10.1109/CLUSTER52292.2023.00025
Masola ACapodieci N(2023)Optimization strategies for GPUs: an overview of architectural approachesInternational Journal of Parallel, Emergent and Distributed Systems10.1080/17445760.2023.217375238:2(140-154)Online publication date: 5-Feb-2023
https://doi.org/10.1080/17445760.2023.2173752
Loughlin KSaroiu SWolman AManerkar YKasikci BSalapura VZahran MChong FTang L(2022)MOESI-primeProceedings of the 49th Annual International Symposium on Computer Architecture10.1145/3470496.3527427(670-684)Online publication date: 18-Jun-2022
https://dl.acm.org/doi/10.1145/3470496.3527427
Nagarajan VSorin DHill MWood DNagarajan VSorin DHill MWood D(2022)Consistency and Coherence for Heterogeneous SystemsA Primer on Memory Consistency and Cache Coherence10.1007/978-3-031-01764-3_10(211-251)Online publication date: 28-Mar-2022
https://doi.org/10.1007/978-3-031-01764-3_10
Michalke AGrulich PLutz CZeuch SMarkl V(2021)An Energy-Efficient Stream Join for the Internet of ThingsProceedings of the 17th International Workshop on Data Management on New Hardware10.1145/3465998.3466005(1-6)Online publication date: 20-Jun-2021
https://dl.acm.org/doi/10.1145/3465998.3466005
Muthukrishnan HNellans DLustig DFessler JWenisch TMartínez JDuato JJohn L(2021)Efficient multi-GPU shared memory via automatic optimization of fine-grained transfersProceedings of the 48th Annual International Symposium on Computer Architecture10.1109/ISCA52012.2021.00020(139-152)Online publication date: 14-Jun-2021
https://dl.acm.org/doi/10.1109/ISCA52012.2021.00020
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents