research-article

Improving GPU Performance Through Resource Sharing

Authors:

Vishwesh Jatala,

Jayvant Anantpur,

Amey KarkareAuthors Info & Claims

HPDC '16: Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing

Pages 203 - 214

https://doi.org/10.1145/2907294.2907298

Published: 31 May 2016 Publication History

Abstract

Graphics Processing Units (GPUs) consisting of Streaming Multiprocessors (SMs) achieve high throughput by running a large number of threads and context switching among them to hide execution latencies. The number of thread blocks, and hence the number of threads that can be launched on an SM, depends on the resource usage--e.g. number of registers, amount of shared memory--of the thread blocks. Since the allocation of threads to an SM is at the thread block granularity, some of the resources may not be used up completely and hence will be wasted.

We propose an approach that shares the resources of SM to utilize the wasted resources by launching more thread blocks. We show the effectiveness of our approach for two resources: register sharing, and scratchpad (shared memory) sharing. We further propose optimizations to hide long execution latencies, thus reducing the number of stall cycles. We implemented our approach in GPGPU-Sim simulator and experimentally validated it on 19 applications from 4 different benchmark suites: GPGPU-Sim, Rodinia, CUDA-SDK, and Parboil. We observed that applications that underutilize register resource show a maximum improvement of 24% and an average improvement of 11% with register sharing. Similarly, the applications that underutilize scratchpad resource show a maximum improvement of 30% and an average improvement of 12.5% with scratchpad sharing. The remaining applications, which do not waste any resources, perform similar to the baseline approach.

References

[1]

CUDA C Programming Guide. https://docs.nvidia.com/cuda/cuda-c-programming-guide/.

[2]

CUDA-SDK. http://docs.nvidia.com/cuda/cuda-samples.

[3]

GPGPU-Sim. http://www.gpgpu-sim.org.

[4]

Parboil Benchmarks. http://impact.crhc.illinois.edu/Parboil/parboil.aspx.

[5]

M. Abdel-Majeed and M. Annavaram. Warped Register File: A Power Efficient Register File for GPGPUs. In HPCA, 2013.

Digital Library

[6]

J. Anantpur and R. Govindarajan. Taming Control Divergence in GPUs through Control Flow Linearization. In CC, 2014.

[7]

A. Bakhoda, G. Yuan, W. Fung, H. Wong, and T. Aamodt. Analyzing CUDA workloads using a detailed GPU simulator. In ISPASS, 2009.

[8]

N. Brunie, S. Collange, and G. Diamos. Simultaneous Branch and Warp Interweaving for Sustained GPU Performance. In ISCA, 2012.

Digital Library

[9]

S. Che, M. Boyer, J. Meng, D. Tarjan, J. Sheaffer, S.-H. Lee, and K. Skadron. Rodinia: A benchmark suite for heterogeneous computing. In IISWC, 2009.

Digital Library

[10]

G. Diamos, B. Ashbaugh, S. Maiyuran, A. Kerr, H. Wu, and S. Yalamanchili. SIMD Re-convergence at Thread Frontiers. In MICRO, 2011.

Digital Library

[11]

W. W. L. Fung and T. M. Aamodt. Thread Block Compaction for Efficient SIMT Control Flow. In HPCA, 2011.

Digital Library

[12]

W. W. L. Fung, I. Sham, G. Yuan, and T. M. Aamodt. Dynamic Warp Formation and Scheduling for Efficient GPU Control Flow. In MICRO, 2007.

Digital Library

[13]

M. Gebhart, D. R. Johnson, D. Tarjan, S. W. Keckler, W. J. Dally, E. Lindholm, and K. Skadron. A Hierarchical Thread Scheduler and Register File for Energy-Efficient Throughput Processors. ACM Trans. Comput. Syst., 2012.

Digital Library

[14]

M. Gebhart, S. W. Keckler, B. Khailany, R. Krashinsky, and W. J. Dally. Unifying Primary Cache, Scratch, and Register File Memories in a Throughput Processor. In MICRO, 2012.

Digital Library

[15]

T. D. Han and T. S. Abdelrahman. Reducing Branch Divergence in GPU Programs. In GPGPU-4, 2011.

Digital Library

[16]

V. Jatala, J. Anantpur, and A. Karkare. Improving GPU Performance Through Resource Sharing. CoRR, http://arxiv.org/abs/1503.05694, 2015.

[17]

H. Jeon, G. S. Ravi, N. S. Kim, and M. Annavaram. GPU Register File Virtualization. MICRO, 2015.

Digital Library

[18]

A. Jog, O. Kayiran, N. Chidambaram Nachiappan, A. K. Mishra, M. T. Kandemir, O. Mutlu, R. Iyer, and C. R. Das. OWL: Cooperative Thread Array Aware Scheduling Techniques for Improving GPGPU Performance. In ASPLOS, 2013.

Digital Library

[19]

O. Kayiran, A. Jog, M. Kandemir, and C. Das. Neither more nor less: Optimizing thread-level parallelism for GPGPUs. In PACT, 2013.

Digital Library

[20]

M. Lee, S. Song, J. Moon, J. Kim, W. Seo, Y. Cho, and S. Ryu. Improving GPGPU resource utilization through alternative thread block scheduling. In HPCA, 2014.

[21]

S.-Y. Lee, A. Arunkumar, and C.-J. Wu. Cawa: Coordinated warp scheduling and cache prioritization for critical warp acceleration of gpgpu workloads. In ISCA, 2015.

Digital Library

[22]

S.-Y. Lee and C.-J. Wu. CAWS: Criticality-aware Warp Scheduling for GPGPU Workloads. In PACT, 2014.

Digital Library

[23]

D. Li, M. Rhu, D. R. Johnson, M. O'Connor, M. Erez, D. Burger, D. S. Fussell, and S. W. Redder. Priority-based cache allocation in throughput processors. In HPCA, 2015.

[24]

T. Li, V. K. Narayana, E. El-Araby, and T. El-Ghazawi. GPU Resource Sharing and Virtualization on High Performance Computing Systems. In ICPP, 2011.

Digital Library

[25]

W. Ma and G. Agrawal. An Integer Programming Framework for Optimizing Shared Memory Use on GPUs. PACT, 2010.

Digital Library

[26]

J. Meng, D. Tarjan, and K. Skadron. Dynamic Warp Subdivision for Integrated Branch and Memory Divergence Tolerance. In ISCA, 2010.

Digital Library

[27]

V. Narasiman, M. Shebanow, C. J. Lee, R. Miftakhutdinov, O. Mutlu, and Y. N. Patt. Improving GPU Performance via Large Warps and Two-level Warp Scheduling. In MICRO, 2011.

Digital Library

[28]

M. Rhu and M. Erez. CAPRI: Prediction of Compaction-adequacy for Handling Control-divergence in GPGPU Architectures. In ISCA, 2012.

Digital Library

[29]

T. G. Rogers, M. O'Connor, and T. M. Aamodt. Cache-Conscious Wavefront Scheduling. In MICRO, 2012.

Digital Library

[30]

A. Sethia, D. A. Jamshidi, and S. Mahlke. Mascar: Speeding up GPU warps by reducing memory pitstops. In HPCA, 2015.

[31]

D. Tarjan and K. Skadron. On demand register allocation and deallocation for a multithreaded processor, 2011. US Patent App. 12/649,238.

[32]

P. Xiang, Y. Yang, and H. Zhou. Warp-level divergence in GPUs: Characterization, impact, and mitigation. In HPCA, 2014.

[33]

X. Xie, Y. Liang, X. Li, Y. Wu, G. Sun, T. Wang, and D. Fan. Enabling Coordinated Register Allocation and Thread-level Parallelism Optimization for GPUs. In MICRO, 2015.

Digital Library

[34]

Y. Yang, P. Xiang, M. Mantor, N. Rubin, and H. Zhou. Shared Memory Multiplexing: A Novel Way to Improve GPGPU Throughput. In PACT, 2012.

Digital Library

Cited By

Alavani GDesai JSaha SSarkar S(2023)Program Analysis and Machine Learning–based Approach to Predict Power Consumption of CUDA KernelACM Transactions on Modeling and Performance Evaluation of Computing Systems10.1145/36035338:4(1-24)Online publication date: 24-Jul-2023
https://dl.acm.org/doi/10.1145/3603533
Darabi SYousefzadeh-Asl-Miandoab EAkbarzadeh NFalahati HLotfi-Kamran PSadrosadati MSarbazi-Azad H(2022)OSM: Off-Chip Shared Memory for GPUsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2022.315431533:12(3415-3429)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1109/TPDS.2022.3154315
Yu CBai YSun QYang H(2018)Improving Thread-level Parallelism in GPUs Through Expanding Register File to Scratchpad MemoryACM Transactions on Architecture and Code Optimization10.1145/328084915:4(1-24)Online publication date: 16-Nov-2018
https://dl.acm.org/doi/10.1145/3280849
Show More Cited By

Index Terms

Improving GPU Performance Through Resource Sharing
1. Computer systems organization
  1. Architectures
    1. Parallel architectures
      1. Single instruction, multiple data

Recommendations

Scratchpad Sharing in GPUs

General-Purpose Graphics Processing Unit (GPGPU) applications exploit on-chip scratchpad memory available in the Graphics Processing Units (GPUs) to improve performance. The amount of thread level parallelism (TLP) present in the GPU is limited by the ...
Improving GPU performance via large warps and two-level warp scheduling
MICRO-44: Proceedings of the 44th Annual IEEE/ACM International Symposium on Microarchitecture

Due to their massive computational power, graphics processing units (GPUs) have become a popular platform for executing general purpose parallel applications. GPU programming models allow the programmer to create thousands of threads, each executing the ...
Accelerated high-performance computing through efficient multi-process GPU resource sharing
CF '12: Proceedings of the 9th conference on Computing Frontiers

The HPC field is witnessing a widespread adoption of GPUs as accelerators for traditional homogeneous HPC systems. One of the prevalent parallel programming models is the SPMD paradigm, which has been adapted for GPU-based parallel processing. Since ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HPDC '16: Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing

May 2016

302 pages

ISBN:9781450343145

DOI:10.1145/2907294

General Chair:
Hiroshi Nakashima
Kyoto University, Japan
,
Program Chairs:
Kenjiro Taura
The University of Tokyo, Japan
,
Jack Lange
University of Pittsburgh, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

University of Arizona: University of Arizona
SIGARCH: ACM Special Interest Group on Computer Architecture

In-Cooperation

SIGHPC: ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Google India Private Limited
TCS

Conference

HPDC'16

Sponsor:

University of Arizona
SIGARCH

HPDC'16: The 25th International Symposium on High-Performance Parallel and Distributed Computing

May 31 - June 4, 2016

Kyoto, Japan

Acceptance Rates

HPDC '16 Paper Acceptance Rate 20 of 129 submissions, 16%;

Overall Acceptance Rate 166 of 966 submissions, 17%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
474
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)3

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Alavani GDesai JSaha SSarkar S(2023)Program Analysis and Machine Learning–based Approach to Predict Power Consumption of CUDA KernelACM Transactions on Modeling and Performance Evaluation of Computing Systems10.1145/36035338:4(1-24)Online publication date: 24-Jul-2023
https://dl.acm.org/doi/10.1145/3603533
Darabi SYousefzadeh-Asl-Miandoab EAkbarzadeh NFalahati HLotfi-Kamran PSadrosadati MSarbazi-Azad H(2022)OSM: Off-Chip Shared Memory for GPUsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2022.315431533:12(3415-3429)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1109/TPDS.2022.3154315
Yu CBai YSun QYang H(2018)Improving Thread-level Parallelism in GPUs Through Expanding Register File to Scratchpad MemoryACM Transactions on Architecture and Code Optimization10.1145/328084915:4(1-24)Online publication date: 16-Nov-2018
https://dl.acm.org/doi/10.1145/3280849
Khorasani FEsfeden HFarmahini-Farahani AJayasena NSarkar V(2018)RegMutexProceedings of the 45th Annual International Symposium on Computer Architecture10.1109/ISCA.2018.00073(816-828)Online publication date: 2-Jun-2018
https://dl.acm.org/doi/10.1109/ISCA.2018.00073
Jatala VAnantpur JKarkare A(2017)Scratchpad Sharing in GPUsACM Transactions on Architecture and Code Optimization10.1145/307561914:2(1-29)Online publication date: 26-May-2017
https://dl.acm.org/doi/10.1145/3075619
Mittal S(2017)A Survey of Techniques for Architecting and Managing GPU Register FileIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2016.254624928:1(16-28)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.1109/TPDS.2016.2546249
Dhar AChen D(2017)Efficient GPGPU Computing with Cross-Core Resource Sharing and Core Reconfiguration2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)10.1109/FCCM.2017.59(48-55)Online publication date: Apr-2017
https://doi.org/10.1109/FCCM.2017.59

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents