article

Free access

Supercomputer performance evaluation and the Perfect Benchmarks

Authors:

George Cybenko,

Lynn Pointer, and

David KuckAuthors Info & Claims

ACM SIGARCH Computer Architecture News, Volume 18, Issue 3b

Pages 254 - 266

https://doi.org/10.1145/255129.255163

Published: 01 June 1990 Publication History

Abstract

In the past three years, the Perfect Benchmark^TM Suite has evolved from a supercomputer performance evaluation plan, presented by Kuck and Sameh at the 1987 International Conference on Supercomputing, to a vigorous international activity. This paper surveys the current state of this supercomputer performance evaluation effort with particular focus on the adopted methodology. While there has been considerable success in achieving the goals of the plan, some issues remain unresolved, and new questions have surfaced.

References

[1]

D. BAILEY AND J. BARTON, The NAS Kernel Benchmark Program, Tech. Rep. 86711, NASA Ames Technical Memorandum, 1985.

[2]

M. BERRY ET AL., The Perfect Club Benchmarks: Effective Performance Evaluation of Supercomputers, The International Journal of Supercomputer Applications, 3 (1989), pp. 5- 40.

[3]

H. CURNOW AND B. WICHMAN, A Synthetic Benchmark~ Comput. :I., 19 (1976), pp. 43-49.

[4]

G.CYBENKO,Networking Pee- 965, Center for Supercomputing Research and Development, University of illinois at Urbana, Technical Report, 1990.

[5]

J. DONGARRA, Performance of Various Computers Using Standard Linear Equations Softwave in a Fortran Environment, Tech. Rep. 23, Argonne National Lab Technical Report, 1988.

[6]

J. DONGARRA, J, MARTIN, AND J. WORL- TON, Computer Benchmarking: Paths and Pitfalls, IEEE Spectrum, (1987).

Digital Library

[7]

R. H. HILL, The Art of Benchmarking, The Spang Robinson Report on Supercomputing and Parallel Processing, 3 (1989).

[8]

P. F. Koss, Application Performance on Supercomputers, Tech. Rep. 847, Center for Supercomputing Research and Development, University of Illinois, January 1989.

[9]

D. J. KucK AND A. H. SAMEH, 21 gupercomputing Performance Evaluation Plan, Tech. Rep. 692, Center for Supercomputing Research and Development, University of Illinois, June 1987.

[10]

A. MAZONY, Performance Measurement Intrusion and Perturbation Analysis, Tech. R.ep. 923, Center for Supercomputing Research and Development, University of Illinois at Urbana, Technical Report, 1990.

[11]

F. McMAI~ON, The Livermore Fortran Kernels: A Test of the Numerical Performance Range, tech. rep., Lawrence Livermore Lab, 1986.

[12]

P. MESSlNA ST AL., Benchmarking Advanced Architecture Computers, Tech. Rep. C3P712, Caltech Concurrent Supercomputing Facility, June 1989.

[13]

L. POINTER, Perfect: Performance Evaluation foe Cost Effective Transformations: Report 2, Tech. Rep. 964, Center for Supercomputing Research and Development, University of Illinois at Urbana, Technical Report, 1990.

[14]

R. SAAVEDRA-BARRERA, A. SMITH, AND E. MIYA, Machine Characterization Based on an Abstract High-Level Language Machine, IEEE Trans. on Computers, 38 (1989), pp. 1659-1679.

Digital Library

[15]

J. E. SMX'rH, Charactevizin9 Computer Per- (1988).

[16]

J. UNIEJEWSKI, SPEC Benchmark Suite: Designed for Today's Advanced Systems. SPEC Newsletter, Fall 1989. Volttme 1, Issue 1.

[17]

J. WORLTON, Understanding Supercomputer Benchmarks, Datamation, (1984), pp. 121- 129.

Cited By

Temam O(2005)Streaming prefetchEuro-Par'96 Parallel Processing10.1007/BFb0024775(765-772)Online publication date: 10-Jun-2005
https://doi.org/10.1007/BFb0024775
Vetter Jde Supinski BKissel LMay JVaidya S(2005)Evaluating high‐performance computersConcurrency and Computation: Practice and Experience10.1002/cpe.89217:10(1239-1270)Online publication date: 22-Mar-2005
https://doi.org/10.1002/cpe.892
McKinley KTemam O(1999)Quantifying loop nest locality using SPEC'95 and the perfect benchmarksACM Transactions on Computer Systems10.1145/329466.32948417:4(288-336)Online publication date: 1-Nov-1999
https://dl.acm.org/doi/10.1145/329466.329484
Show More Cited By

Index Terms

Recommendations

Supercomputer performance evaluation and the Perfect Benchmarks
ICS '90: Proceedings of the 4th international conference on Supercomputing

In the past three years, the Perfect Benchmark^TM Suite has evolved from a supercomputer performance evaluation plan, presented by Kuck and Sameh at the 1987 International Conference on Supercomputing, to a vigorous international activity. This paper ...
Read More
Performance evaluation of the hitachi SR8000 using SPEC OMP2001 benchmarks
Special issue: OpenMP: Experiences and implementations

This paper reports the performance of a single node of the Hitachi SR8000 when using SPEC OMP2001 benchmarks. Each processing node of the SR8000 is a shared-memory parallel computer composed of eight scalar processors with pseudo-vector processing ...
Read More
Performance evaluation of supercomputers using HPCC and IMB benchmarks
IPDPS'06: Proceedings of the 20th international conference on Parallel and distributed processing

The HPC Challenge (HPCC) benchmark suite and the Intel MPI Benchmark (IMB) are used to compare and evaluate the combined performance of processor, memory subsystem and interconnect fabric of five leading supercomputers - SGI Altix BX2, Cray X1, Cray ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM SIGARCH Computer Architecture News

ACM SIGARCH Computer Architecture News Volume 18, Issue 3b

Special Issue: Proceedings of the 4th international conference on Supercomputing

Sept. 1990

489 pages

ISSN:0163-5964

DOI:10.1145/255129

Chairmen:
Ahmed Sameh
Univ. of Illinois at Urbana-Champaign, Urbana
,
Henk van der Vorst
Delft Univ. of Technology and CWI, The Netherlands

Issue’s Table of Contents

ICS '90: Proceedings of the 4th international conference on Supercomputing
June 1990
492 pages
ISBN:0897913698
DOI:10.1145/77726
Chairmen:
Ahmed Sameh
Univ. of Illinois
,
Henk van der Vorst
Delft Univ. of Technology and CWI, The Netherlands

Copyright © 1990 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 1990

Published in SIGARCH Volume 18, Issue 3b

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

77
Total Citations
View Citations
945
Total Downloads

Downloads (Last 12 months)123
Downloads (Last 6 weeks)5

Other Metrics

View Author Metrics

Citations

Cited By

Temam O(2005)Streaming prefetchEuro-Par'96 Parallel Processing10.1007/BFb0024775(765-772)Online publication date: 10-Jun-2005
https://doi.org/10.1007/BFb0024775
Vetter Jde Supinski BKissel LMay JVaidya S(2005)Evaluating high‐performance computersConcurrency and Computation: Practice and Experience10.1002/cpe.89217:10(1239-1270)Online publication date: 22-Mar-2005
https://doi.org/10.1002/cpe.892
McKinley KTemam O(1999)Quantifying loop nest locality using SPEC'95 and the perfect benchmarksACM Transactions on Computer Systems10.1145/329466.32948417:4(288-336)Online publication date: 1-Nov-1999
https://dl.acm.org/doi/10.1145/329466.329484
Stoltz EWolfe M(1995)Detecting value-based scalar dependenceInternational Journal of Parallel Programming10.1007/BF0257777023:4(327-358)Online publication date: 1-Aug-1995
https://dl.acm.org/doi/10.1007/BF02577770
Kennedy KMcKinley K(2014)Optimizing for parallelism and data localityACM International Conference on Supercomputing 25th Anniversary Volume10.1145/2591635.2667164(151-162)Online publication date: 10-Jun-2014
https://dl.acm.org/doi/10.1145/2591635.2667164
Awan MJarvis S(2012)MalikStone — A context-aware light-weight benchmark for P2P computing2012 International Conference on High Performance Computing & Simulation (HPCS)10.1109/HPCSim.2012.6266936(350-356)Online publication date: Jul-2012
https://doi.org/10.1109/HPCSim.2012.6266936
Topham NMcDougall K(2006)Performance of the decoupled ACRI-1 architecture: The perfect clubHigh-Performance Computing and Networking10.1007/BFb0046669(472-480)Online publication date: 2-Feb-2006
https://doi.org/10.1007/BFb0046669
Stoltz EWolfe M(2005)Detecting value-based scalar dependenceLanguages and Compilers for Parallel Computing10.1007/BFb0025879(186-200)Online publication date: 9-Jun-2005
https://doi.org/10.1007/BFb0025879
Nguyen TVaswani RZahorjan J(2005)Parallel application characterization for multiprocessor scheduling policy designJob Scheduling Strategies for Parallel Processing10.1007/BFb0022294(175-199)Online publication date: 15-Jun-2005
https://doi.org/10.1007/BFb0022294
Voevodin VVoevodin V(2005)The V-Ray technology of optimizing programs to parallel computersNumerical Analysis and Its Applications10.1007/3-540-62598-4_136(546-556)Online publication date: 3-Jun-2005
https://doi.org/10.1007/3-540-62598-4_136
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents