Article

SPEC ACCEL: A Standard Application Suite for Measuring Hardware Accelerator Performance

High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation: 5th International Workshop, PMBS 2014, New Orleans, LA, USA, November 16, 2014. Revised Selected Papers

Pages 46 - 67

https://doi.org/10.1007/978-3-319-17248-4_3

Published: 18 April 2015 Publication History

Abstract

Hybrid nodes with hardware accelerators are becoming very common in systems today. Users often find it difficult to characterize and understand the performance advantage of such accelerators for their applications. The SPEC High Performance Group (HPG) has developed a set of performance metrics to evaluate the performance and power consumption of accelerators for various science applications. The new benchmark comprises two suites of applications written in OpenCL and OpenACC and measures the performance of accelerators with respect to a reference platform. The first set of published results demonstrate the viability and relevance of the new metrics in comparing accelerator performance. This paper discusses the benchmark suites and selected published results in great detail.

References

[1]

The OpenACC Application Programming Interface, November 2011. http://www.openacc.org/sites/default/files/OpenACC.1.0_0.pdf, version 1.0

[2]

Bailey, D., Barszcz, E., Barton, J., Browning, D., Carter, R., Dagum, L., Fatoohi, R., Fineberg, S., Lasinski, T., Schreiber, R., Simon, H., Venkatakrishnan, V., Weeratunga, S.: The NAS parallel benchmarks. Technical report RNR-94-2007, NASA (1994). http://www.nas.nasa.gov/assets/pdf/techreports/1994/rnr-94-007.pdf

[3]

Che S, Boyer M, Meng J, Tarjan D, Sheaffer JW, and Skadron KA performance study of general-purpose applications on graphics processors using CUDAJ. Parallel Distrib. Comput.200868101370-1380http://dx.doi.org/10.1016/j.jpdc.2008.05.014

[4]

Che, S., Boyer, M., Meng, J., Tarjan, D., Sheaffer, W.J., Lee, S.H., Skadron, K.: Rodinia: a benchmark suite for heterogeneous computing. In: Proceedings of the IEEE International Symposium on Workload Characterization (IISWC), pp. 44–54, October 2009

[5]

Che, S., Sheaffer, W.J., Boyer, M., Szafaryn, L.G., Wang, L., Skadron, K.: A characterization of the rodinia benchmark suite with comparison to contemporary CMP workloads. In: Proceedings of the IEEE International Symposium on Workload Characterization (IISWC), December 2010

[6]

Corrigan, A., Camelli, F., Lohner, R., Wallin, J.: Running unstructured grid CFD solvers on modern graphics hardware. In: Proceedings of the 19th AIAA Computational Fluid Dynamics Conference, June 2009

[7]

Danalis, A., Marin, G., McCurdy, C., Meredith, J.S., Roth, P.C., Spafford, K., Tipparaju, V., Vetter, J.S.: The scalable heterogeneous computing (SHOC) benchmark suite. In: Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units, GPGPU 2010, pp. 63–74. ACM, New York (2010). http://doi.acm.org/10.1145/1735688.1735702

[8]

Komatitsch, D., Martin, R.: University of Pau: SEISMIC\_CPML. http://geodynamics.org/cig/software/seismic_cpml/

[9]

Fix, J., Wilkes, A., Skadron, K.: Accelerating braided B+ tree searches on a GPU with CUDA. In: Proceedings of the 2nd Workshop on Applications for Multi and Many Core Processors: Analysis, Implementation, and Performance (A4MMC), in Conjunction with ISCA, June 2011

[10]

Hardy, D.J., Stone, J.E., Vandivort, K.L., Gohara, D., Rodrigues, C., Schulten, K.: Fast molecular electrostatics algorithms on GPUs. In: GPU Computing Gems (2010)

[11]

Herdman, J., Gaudin, W., McIntosh-Smith, S., Boulton, M., Beckingsale, D., Mallinson, A., Jarvis, S.: Accelerating hydrocodes with OpenACC, OpeCL and CUDA. In: 2012 SC Companion: High Performance Computing, Networking, Storage and Analysis (SCC), pp. 465–471, November 2012

[12]

Horowitz CJ, Berry DK, and Brown EFPhase separation in the crust of accreting neutron starsPhys. Rev. E200775066101http://link.aps.org/doi/10.1103/PhysRevE.75.066101

[13]

Huang W, Ghosh S, Velusamy S, Sankaranarayanan K, Skadron K, and Stan M HotSpot: a compact thermal modeling methodology for early-stage VLSI design IEEE Trans. Very Large Scale Integr. (VLSI) Syst. 2006 14 5 501-513

[14]

Johnson, N.: EPCC OpenACC benchmark suite. https://www.epcc.ed.ac.uk/research/computing/performance-characterisation-and-benchmarking/epcc-openacc-benchmark-suite

[15]

Khronos Group: OpenCL 1.1 API and C Language Specification, June 2011. https://www.khronos.org/registry/cl/specs/opencl-1.1.pdf, revision 44

[16]

Lange KD Identifying shades of green: the SPECpower benchmarks Computer 2009 42 95-97

[17]

Lee, S., Eigenmann, R.: OpenMPC: extended OpenMP programming and tuning for GPUs. In: Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–11. IEEE Computer Society (2010)

[18]

Lee S, Min SJ, and Eigenmann R OpenMP to GPGPU: a compiler framework for automatic translation and optimization ACM Sigplan Not. 2009 44 4 101-110

[19]

Lee, S., Vetter, J.S.: Early evaluation of directive-based gpu programming models for productive exascale computing. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, p. 23. IEEE Computer Society Press (2012)

[20]

Luo, L., Wong, M., Hwu, W.W.: An effective GPU implementation of breadth-first search. In: Proceedings of the 47th Design Automation Conference, pp. 52–55, June 2010

[21]

Müller MS et al.Chapman BM, Massaioli F, Müller MS, Rorro M, et al.SPEC OMP2012 — an application benchmark suite for parallel systems using OpenMPOpenMP in a Heterogeneous World2012HeidelbergSpringer223-236http://dx.doi.org/10.1007/978-3-642-30961-8_17

[22]

Müller MS, van Waveren M, Lieberman R, Whitney B, Saito H, Kumaran K, Baron J, Brantley WC, Parrott C, Elken T, Feng H, and Ponder CSPEC MPI2007 - an application benchmark suite for parallel systems using MPIConcurr. Comput. Pract. Exper.2010222191-205http://dx.doi.org/10.1002/cpe.v22:2

[23]

Qian YH, D’Humieres D, and Lallemand P Lattice BGK models for navier-stokes equation Europhys. Lett. 1992 17 479-484

[24]

Barrett, R.F., Vaughan, C.T., Heroux, M.A.: MiniGhost: A miniapp for exploring boundary exchange strategies using stencil computations in scientific parallel computing, Version 1.0. Techical report (2012)

[25]

Raasch, S.: Leibniz University of Hannover: PALM. http://palm.muk.uni-hannover.de/

[26]

Stone, S.S., Haldar, J.P., Tsao, S.C., Hwu, W.W., Liang, Z., Sutton, B.P.: Accelerating advanced MRI reconstructions on GPUs. In: International Conference on Computing Frontiers, pp. 261–272 (2008)

[27]

Stratton, J.A., Rodrigues, C., Sung, I.J., Obeid, N., Chang, L., Liu, G., Hwu, W.W.: Parboil: a revised benchmark suite for scientific and commercial throughput computing. Technical report IMPACT-12-01. University of Illinois at Urbana-Champaign, Urbana, March 2012

[28]

Szafaryn, L.G., Skadron, K., Saucerman, J.J.: Experiences accelerating MATLAB systems biology applications. In: Proceedings of the Workshop on Biomedicine in Computing: Systems, Architectures, and Circuits (BiC) 2009, in Conjunction with the 36th IEEE/ACM International Symposium on Computer Architecture (ISCA), June 2009

[29]

Szafaryn LG, Gamblin T, de Supinski BR, and Skadron K Trellis: portability across architectures with a high-level framework J. Parallel Distrib. Comput. 2013 73 10 1400-1413

Cited By

Afzal AHager GWellein G(2023)SPEChpc 2021 Benchmarks on Ice Lake and Sapphire Rapids Infiniband Clusters: A Performance and Energy Case StudyProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624197(1245-1254)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624197
Lambert JLee SMalony AVetter J(2019)CCAMP: OpenMP and OpenACC Interoperable FrameworkEuro-Par 2019: Parallel Processing Workshops10.1007/978-3-030-48340-1_28(357-369)Online publication date: 26-Aug-2019
https://dl.acm.org/doi/10.1007/978-3-030-48340-1_28

Index Terms

SPEC ACCEL: A Standard Application Suite for Measuring Hardware Accelerator Performance

Index terms have been assigned to the content through auto-classification.

Recommendations

Performance Portable Applications for Hardware Accelerators: Lessons Learned from SPEC ACCEL
IPDPSW '15: Proceedings of the 2015 IEEE International Parallel and Distributed Processing Symposium Workshop

The popular and diverse hardware accelerator ecosystem makes apples-to-apples comparisons between platforms rather difficult. SPEC ACCEL tries to offer a yardstick to compare different accelerator hardware and software ecosystems. This paper uses this ...
Benchmarking OpenCL, OpenACC, OpenMP, and CUDA: Programming Productivity, Performance, and Energy Consumption
ARMS-CC '17: Proceedings of the 2017 Workshop on Adaptive Resource Management and Scheduling for Cloud Computing

Many modern parallel computing systems are heterogeneous at their node level. Such nodes may comprise general purpose CPUs and accelerators (such as, GPU, or Intel Xeon Phi) that provide high performance with suitable energy-consumption characteristics. ...
Evaluation of a Directive-Based GPU Programming Approach for High-Order Unstructured Mesh Computational Fluid Dynamics
PASC '17: Proceedings of the Platform for Advanced Scientific Computing Conference

In this work we evaluate the effectiveness of using OpenACC as a paradigm for the auto-parallelization of a high-order unstructured CFD code on Graphics Processing Units (GPUs). This is in lieu of hand-written CUDA or OpenCL code for the algorithms that ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation: 5th International Workshop, PMBS 2014, New Orleans, LA, USA, November 16, 2014. Revised Selected Papers

Nov 2014

268 pages

ISBN:978-3-319-17247-7

DOI:10.1007/978-3-319-17248-4

Editors:
Stephen A. Jarvis
University of Warwick, Coventry, United Kingdom
,
Steven A. Wright
University of Warwick, Coventry, United Kingdom
,
Simon D. Hammond
Sandia National Laboratories CSRI, Albuquerque, New Mexico, USA

© Springer International Publishing Switzerland 2015.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 18 April 2015

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Afzal AHager GWellein G(2023)SPEChpc 2021 Benchmarks on Ice Lake and Sapphire Rapids Infiniband Clusters: A Performance and Energy Case StudyProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624197(1245-1254)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624197
Lambert JLee SMalony AVetter J(2019)CCAMP: OpenMP and OpenACC Interoperable FrameworkEuro-Par 2019: Parallel Processing Workshops10.1007/978-3-030-48340-1_28(357-369)Online publication date: 26-Aug-2019
https://dl.acm.org/doi/10.1007/978-3-030-48340-1_28

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

Affiliations

Guido Juckeland

SPEC High Performance Group, Gainesville, USA

Center for Information Services and High Performance Computing (ZIH), Technische Universität Dresden, 01062, Dresden, Germany

William Brantley

SPEC High Performance Group, Gainesville, USA

Advanced Micro Devices, Inc., Sunnyvale, CA, USA

Sunita Chandrasekaran

SPEC High Performance Group, Gainesville, USA

University of Houston, Houston, TX, USA

Barbara Chapman

SPEC High Performance Group, Gainesville, USA

University of Houston, Houston, TX, USA

Shuai Che

SPEC High Performance Group, Gainesville, USA

Advanced Micro Devices, Inc., Sunnyvale, CA, USA

Mathew Colgrove

SPEC High Performance Group, Gainesville, USA

NVIDIA, Santa Clara, CA, USA

Huiyu Feng

SPEC High Performance Group, Gainesville, USA

Silicon Graphics International Corp., Milpitas, CA, USA

Alexander Grund

SPEC High Performance Group, Gainesville, USA

Center for Information Services and High Performance Computing (ZIH), Technische Universität Dresden, 01062, Dresden, Germany

Robert Henschel

SPEC High Performance Group, Gainesville, USA

Indiana University, Bloomington, IN, USA

Wen-Mei W. Hwu

SPEC High Performance Group, Gainesville, USA

University of Illinois (UIUC), Champaign, IL, USA

Huian Li

SPEC High Performance Group, Gainesville, USA

Indiana University, Bloomington, IN, USA

Matthias S. Müller

SPEC High Performance Group, Gainesville, USA

RWTH Aachen University, Aachen, Germany

Wolfgang E. Nagel

SPEC High Performance Group, Gainesville, USA

Center for Information Services and High Performance Computing (ZIH), Technische Universität Dresden, 01062, Dresden, Germany

Maxim Perminov

SPEC High Performance Group, Gainesville, USA

Intel, Nizhny Novgorod, Russia

Pavel Shelepugin

SPEC High Performance Group, Gainesville, USA

Intel, Nizhny Novgorod, Russia

Kevin Skadron

SPEC High Performance Group, Gainesville, USA

University of Virginia, Charlottesville, VA, USA

John Stratton

SPEC High Performance Group, Gainesville, USA

University of Illinois (UIUC), Champaign, IL, USA

Colgate University, Hamilton, NY, USA

Alexey Titov

SPEC High Performance Group, Gainesville, USA

Advanced Micro Devices, Inc., Sunnyvale, CA, USA

Ke Wang

SPEC High Performance Group, Gainesville, USA

University of Virginia, Charlottesville, VA, USA

Matthijs van Waveren

SPEC High Performance Group, Gainesville, USA

Compilaflows, Toulouse, France

Brian Whitney

SPEC High Performance Group, Gainesville, USA

Oracle, Redwood Shores, CA, USA

Sandra Wienke

SPEC High Performance Group, Gainesville, USA

RWTH Aachen University, Aachen, Germany

Rengan Xu

SPEC High Performance Group, Gainesville, USA

University of Houston, Houston, TX, USA

Kalyan Kumaran

SPEC High Performance Group, Gainesville, USA

Argonne National Laboratory, Lemont, IL, USA

View Table of Contents