research-article

GPUVerify: a verifier for GPU kernels

Authors:

Alastair Donaldson,

Paul ThomsonAuthors Info & Claims

OOPSLA '12: Proceedings of the ACM international conference on Object oriented programming systems languages and applications

Pages 113 - 132

https://doi.org/10.1145/2384616.2384625

Published: 19 October 2012 Publication History

Abstract

We present a technique for verifying race- and divergence-freedom of GPU kernels that are written in mainstream kernel programming languages such as OpenCL and CUDA. Our approach is founded on a novel formal operational semantics for GPU programming termed synchronous, delayed visibility (SDV) semantics. The SDV semantics provides a precise definition of barrier divergence in GPU kernels and allows kernel verification to be reduced to analysis of a sequential program, thereby completely avoiding the need to reason about thread interleavings, and allowing existing modular techniques for program verification to be leveraged. We describe an efficient encoding for data race detection and propose a method for automatically inferring loop invariants required for verification. We have implemented these techniques as a practical verification tool, GPUVerify, which can be applied directly to OpenCL and CUDA source code. We evaluate GPUVerify with respect to a set of 163 kernels drawn from public and commercial sources. Our evaluation demonstrates that GPUVerify is capable of efficient, automatic verification of a large number of real-world kernels.

References

[1]

AMD. AMD Accelerated Parallel Processing (APP) SDK. developer.amd.com/sdks/amdappsdk/pages/default.aspx

[2]

M. Barnett, B. Chang, R. DeLine, B. Jacobs, and K. R. M. Leino. Boogie: A modular reusable verifier for object-oriented programs. In FMCO, 2005.

Digital Library

[3]

D. Beyer, T. A. Henzinger, R. Majumdar, and A. Rybalchenko. Path invariants. In PLDI, 2007.

Digital Library

[4]

M. Boyer, K. Skadron, and W. Weimer. Automated dynamic analysis of CUDA programs. In STMCS, 2008.

[5]

C. Cadar, D. Dunbar, and D. Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In OSDI, 2008.

Digital Library

[6]

P. Collingbourne, C. Cadar, and P. H. J. Kelly. Symbolic testing of OpenCL code. In HVC, 2011.

[7]

P. Cousot and R. Cousot. Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In POPL, 1977.

Digital Library

[8]

L. de Moura and N. Bjørner. Z3: An efficient SMT solver. In TACAS, 2008.

Digital Library

[9]

A. Donaldson, D. Kroening and P. Rummer. Automatic analysis of scratch-pad memory code for heterogeneous multicore processors. In TACAS, 2010.

Digital Library

[10]

A. Donaldson, D. Kroening and P. Rummer. Automatic analysis of DMA races using model checking and k-induction. Formal Methods in System Design 39(1):83--113.

Digital Library

[11]

M. D. Ernst, J. H. Perkins, P. J. Guo, S. McCamant, C. Pacheco, M. S. Tschantz, and C. Xiao The Daikon system for dynamic detection of likely invariants. Sci. Comput. Program. 69(1-3):35--45.

Digital Library

[12]

C. Flanagan and K. R. M. Leino. Houdini, an annotation assistant for ESC/Java. In FME, 2001.

Digital Library

[13]

S. Graf and H. Sa idi. Construction of abstract state graphs with PVS. In CAV, 1997.

Digital Library

[14]

A. Habermaier and A. Knapp. On the correctness of the SIMT execution model of GPUs. In ESOP, 2012.

Digital Library

[15]

T. Kahsai, Y. Ge, and C. Tinelli. Instantiation-based invariant discovery. In NFM, 2011.

Digital Library

[16]

R. K. Karmani, P. Madhusudan, and B. Moore. Thread contracts for safe parallelism. In PPOPP, 2011.

Digital Library

[17]

Khronos OpenCL Working Group. The OpenCL specifica-tion, version 1.1, 2011. Document Revision: 44.

[18]

S. K. Lahiri and S. Qadeer. Complexity and algorithms for monomial and clausal predicate. In CADE, 2009.

Digital Library

[19]

K. R. M. Leino, G. Nelson, and J. B. Saxe. ESC/Java user's manual. Technical Note 2000-002, Compaq Systems Research Center, October 2000.

[20]

A. Leung, M. Gupta, Y. Agarwal, R. Gupta, R. Jhala, and S. Lerner. Verifying GPU kernels by test amplification. In PLDI, 2012.

Digital Library

[21]

G. Li and G. Gopalakrishnan. Scalable SMT-based verifica-tion of GPU kernel functions. In FSE, 2010.

Digital Library

[22]

G. Li, P. Li, G. Gopalakrishnan. Parametric flows: automated behaviour equivalencing for symbolic analysis of races in CUDA programs. In SC, 2012.

Digital Library

[23]

G. Li, P. Li, G. Sawaya, G. Gopalakrishnan, I. Ghosh, and S. P. Rajan. GKLEE: concolic verification and test generation for GPUs. In PPOPP, 2012.

Digital Library

[24]

llvm.org. clang: a C language family frontend for LLVM. clang.llvm.org

[25]

A. Lokhmotov. Mobile and embedded computing on Mali GPUs. In UK GPU Computing Conference, 2011.

[26]

K. L. McMillan. Lazy abstraction with interpolants. In CAV, 2006.

Digital Library

[27]

Microsoft Corporation. C++ AMP sample projects for download (MSDN blog). blogs.msdn.com/b/nativeconcurrency/archive/2012/01/30/c-amp-sample-projects-for-download.aspx

[28]

D. Moth and Y. Levanoni. Microsoft's C++ AMP unveiled. www.drdobbs.com/windows/231600761

[29]

NVIDIA. CUDA Toolkit Release Archive. developer.nvidia.com/cuda-toolkit-archive

[30]

NVIDIA. CUDA C programming guide, v4.0, 2011.

[31]

NVIDIA. PTX: Parallel thread execution ISA, v2.3, 2011.

[32]

L. Nyland. Personal communication, April 2012.

[33]

L. Nyland, M. Harris, and J. Prins. Fast N-body simulation with CUDA. GPU Gems 3, Chapter 31. Addison-Wesley, 2007.

[34]

Rightware Oy. Basemark CL. www.rightware.com/en/Benchmarking+Software/Basemark%99+CL

[35]

S. Srivastava and S. Gulwani. Program verification using templates over predicate abstraction. In PLDI, 2009.

Digital Library

[36]

B. Steensgaard. Points-to analysis in almost linear time. In POPL, 1996.

Digital Library

[37]

S. Stone, J. Haldar, S. Tsao, W. Hwu, B. Sutton, and Z. Liang. Accelerating advanced MRI reconstructions on GPUs. J. Parallel Distrib. Comput., 68(10):1307--1318, 2008.

Digital Library

[38]

S. Tripakis, C. Stergiou, and R. Lublinerman. Checking non-interference in SPMD programs. In HotPar, 2010.

Cited By

Liew DCogumbreiro TLange J(2024)Sound and Partially-Complete Static Analysis of Data-Races in GPU ProgramsProceedings of the ACM on Programming Languages10.1145/36897978:OOPSLA2(2434-2461)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689797
Hsu CHuang CLee CLee JLin P(2024)The Rewriting of DataRaceBench Benchmark for OpenCL Program ValidationsWorkshop Proceedings of the 53rd International Conference on Parallel Processing10.1145/3677333.3678148(15-22)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3677333.3678148
Köpcke BGorlatch SSteuwer M(2024)Descend: A Safe GPU Systems Programming LanguageProceedings of the ACM on Programming Languages10.1145/36564118:PLDI(841-864)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656411
Show More Cited By

Index Terms

GPUVerify: a verifier for GPU kernels
1. Theory of computation
  1. Semantics and reasoning
    1. Program reasoning

Recommendations

The Design and Implementation of a Verification Technique for GPU Kernels

We present a technique for the formal verification of GPU kernels, addressing two classes of correctness properties: data races and barrier divergence. Our approach is founded on a novel formal operational semantics for GPU kernels termed <i>synchronous, ...
GPUVerify: a verifier for GPU kernels
OOPSLA '12

We present a technique for verifying race- and divergence-freedom of GPU kernels that are written in mainstream kernel programming languages such as OpenCL and CUDA. Our approach is founded on a novel formal operational semantics for GPU programming ...
Barrier invariants: a shared state abstraction for the analysis of data-dependent GPU kernels
OOPSLA '13: Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications

Data-dependent GPU kernels, whose data or control flow are dependent on the input of the program, are difficult to verify because they require reasoning about shared state manipulated by many parallel threads. Existing verification techniques for GPU ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

OOPSLA '12: Proceedings of the ACM international conference on Object oriented programming systems languages and applications

October 2012

1052 pages

ISBN:9781450315616

DOI:10.1145/2384616

General Chair:
Gary T. Leavens
University of Central Florida
,
Program Chair:
Matthew B. Dwyer
University of Nebraska - Lincoln

ACM SIGPLAN Notices Volume 47, Issue 10
OOPSLA '12
October 2012
1011 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/2398857
Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SPLASH '12

Sponsor:

SIGPLAN

SPLASH '12: Conference on Systems, Programming, and Applications: Software for Humanity

October 19 - 26, 2012

Arizona, Tucson, USA

Acceptance Rates

Overall Acceptance Rate 268 of 1,244 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

147
Total Citations
View Citations
727
Total Downloads

Downloads (Last 12 months)64
Downloads (Last 6 weeks)14

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liew DCogumbreiro TLange J(2024)Sound and Partially-Complete Static Analysis of Data-Races in GPU ProgramsProceedings of the ACM on Programming Languages10.1145/36897978:OOPSLA2(2434-2461)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689797
Hsu CHuang CLee CLee JLin P(2024)The Rewriting of DataRaceBench Benchmark for OpenCL Program ValidationsWorkshop Proceedings of the 53rd International Conference on Parallel Processing10.1145/3677333.3678148(15-22)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3677333.3678148
Köpcke BGorlatch SSteuwer M(2024)Descend: A Safe GPU Systems Programming LanguageProceedings of the ACM on Programming Languages10.1145/36564118:PLDI(841-864)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656411
Jacobson JBurtscher MGopalakrishnan G(2024)HiRace: Accurate and Fast Data Race Checking for GPU ProgramsProceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis10.1109/SC41406.2024.00042(1-14)Online publication date: 17-Nov-2024
https://dl.acm.org/doi/10.1109/SC41406.2024.00042
Mustafa DAlkhasawneh RObeidat FShatnawi A(2024)MIMD Programs Execution Support on SIMD Machines: A Holistic SurveyIEEE Access10.1109/ACCESS.2024.337299012(34354-34377)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3372990
Cogumbreiro TLange JLiew DZicarelli H(2024)Memory access protocols: certified data-race freedom for GPU kernelsFormal Methods in System Design10.1007/s10703-023-00415-063:1-3(134-171)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s10703-023-00415-0
Armborst LBos Pvan den Haak LHuisman MRubbens RŞakar ÖTasche P(2024)The VerCors Verifier: A Progress ReportComputer Aided Verification10.1007/978-3-031-65630-9_1(3-18)Online publication date: 24-Jul-2024
https://dl.acm.org/doi/10.1007/978-3-031-65630-9_1
Altalhi SEassa FAl-Ghamdi ASharaf SAlghamdi AAlmarhabi KKhemakhem M(2023)An Architecture for a Tri-Programming Model-Based Parallel Hybrid Testing ToolApplied Sciences10.3390/app13211196013:21(11960)Online publication date: 1-Nov-2023
https://doi.org/10.3390/app132111960
Wu XTian DKim C(2023)Building GPU TEEs using CPU Secure Enclaves with GEVisorProceedings of the 2023 ACM Symposium on Cloud Computing10.1145/3620678.3624659(249-264)Online publication date: 30-Oct-2023
https://dl.acm.org/doi/10.1145/3620678.3624659
Klimis VClark JBaker ANeto DWickerson JDonaldson A(2023)Taking Back Control in an Intermediate Representation for GPU ComputingProceedings of the ACM on Programming Languages10.1145/35712537:POPL(1740-1769)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3571253
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents