Article

The design and development of ZPL

Author:

Lawrence SnyderAuthors Info & Claims

HOPL III: Proceedings of the third ACM SIGPLAN conference on History of programming languages

Pages 8-1 - 8-37

https://doi.org/10.1145/1238844.1238852

Published: 09 June 2007 Publication History

Abstract

ZPL is an implicitly parallel programming language, which means all instructions to implement and manage the parallelism are inserted by the compiler. It is the first implicitly parallel language to achieve performance portability, that is, consistent high performance across all (MIMD) parallel platforms. ZPL has been designed from first principles, and is founded on the CTA abstract parallel machine. A key enabler of ZPL's performance portability is its What You See Is What You Get (WYSIWYG) performance model. The paper describes the antecedent research on which ZPL was founded, the design principles used to build it incrementally, and the technical basis for its performance portability. Comparisons with other parallel programming approaches are included.

Supplementary Material

JPG File (m8-snyder-h.jpg)

Download
21.51 KB

slides.ppt (zpl.ppt)

The design and development of ZPL PowerPoint presentations

Download
1.81 MB

MOV File (m8-snyder-h.mov)

Download
246.41 MB

References

[1]

Lawrence Snyder. Type architecture, shared memory and the corollary of modest potential. Annual Review of Computer Science, 1:289--317, 1986.

[2]

Gil Lerman and Larry Rudolph. Parallel Evolution of Parallel Processors, Plenum Press, 1993.

Digital Library

[3]

G. C. Fox, M. A. Johnson, G. A. Lyzenga, S. W. Otto, J. K. Salmon and D. W. Walker. Solving Problems on Concurrent Processors, Prentice-Hall, 1988.

Digital Library

[4]

Lawrence Snyder, The Blue CHiP Project Description. Department of Computer Science Technical Report, Purdue University, 1980.

[5]

Calvin Lin and Lawrence Snyder. ZPL: An array sublanguage. In Proceedings of the Workshop on Languages and Compilers for Parallel Computing, 1993.

Digital Library

[6]

Calvin Lin and Lawrence Snyder. SIMPLE performance results in ZPL. In Proceedings of the Workshop on Languages and Compilers for Parallel Computing, 1994.

Digital Library

[7]

Lawrence Snyder. Parallel programming and the poker programming environment. Computer, 17(7):27--36, July 1984.

Digital Library

[8]

Lawrence Snyder. Introduction to the configurable, highly parallel computer. Computer, 15(1):47--56, January 1982.

Digital Library

[9]

Richard O'Keefe, The Craft of Prolog, MIT Press, 1994.

Digital Library

[10]

Simon Peyton Jones and P. L. Wadler. A static semantics for Haskell. University of Glasgow, 1992.

[11]

Jorg Keller, Christoph W. Kessler and Jesper Larsson Traff. Practical PRAM Programming, John Wiley, 2000.

Digital Library

[12]

Richard J. Anderson and Lawrence Snyder. A comparison of shared and nonshared memory models of parallel computation. Proceedings of the IEEE, 79(4):480--487, April 1991.

[13]

Alan M. Turing. On computable numbers, with an application to the Entscheidungsproblem, Proc. London Math. Soc., 2(42):230--265, 1936.

[14]

John Backus. Can programming be liberated from the von Neumann style? Communications of the ACM 21(8):613--641, 1978.

Digital Library

[15]

J. T. Schwartz. Ultracomputers. ACM Transactions on Programming Languages and Systems, 2(4):484--521, 1980.

Digital Library

[16]

I.D. Scherson and A. S. Youssef. Interconnection Networks for High-Performance Parallel Computers. IEEE Computer Society Press, 1994

Digital Library

[17]

K. Bolding, M. L. Fulgham and L. Snyder. The case for Chaotic adaptive routing. IEEE Trans. Computers 46(12): 1281--1291, 1997.

Digital Library

[18]

David Culler, Richard Karp, David Patterson, Abhijit Sahay, Klaus Erik Schauser, Eunice Santos, Ramesh Subramonian, and Thorsten von Eicken. LogP: towards a realistic model of parallel computation, ACM Symposium on Principles and Practice of Parallel Programming, 1993.

Digital Library

[19]

Benjamin Ylvisaker, Brian Van Essen and Carl Ebeling. A Type Architecture for Hybrid Micro-Parallel Computers. In IEEE Symposium on Field-Programmable Custom Computing Machines, 2006.

Digital Library

[20]

David G. Socha. Supporting Fine-Grain Computation on Distributed memory Parallel Computers. PhD Dissertation, University of Washington, 1991.

Digital Library

[21]

Calvin Lin, Jin-ling Lee and Lawrence Snyder. Programming SIMPLE for parallel portability, In U. Banerjee, D. Gelernter, A. Nicolau and D. Padua (eds.), Languages and Compilers for Parallel Computing, Springer-Verlag, pp. 84--98, 1992.

Digital Library

[22]

Calvin Lin. The Portability of Parallel Programs Across MIMD Computers. PhD Dissertation, University of Washington, 1992.

Digital Library

[23]

Ton Ahn Ngo and Lawrence Snyder. On the influence of programming models on shared memory computer performance. In Proceedings of the Scalable High Performance Computing Conference, 1992.

[24]

Ton Anh Ngo. The Role of Performance Models in Parallel Programming and Languages. PhD Dissertation, University of Washington, 1997.

[25]

Raymond Greenlaw and Lawrence Snyder. Achieving speedups for APL on an SIMD distributed memory machine. International Journal of Parallel Programming, 19(2):111---127, April 1990.

Digital Library

[26]

Walter S. Brainerd, Charles H. Goldberg and Jeanne C. Adams. Programmer's Guide to Fortran 90, 3^rd Ed. Springer, 1996.

Digital Library

[27]

High Performance Fortran Language Specification Version 1.0 (1992) High Performance Fortran Forum, May 3, 1993. {34} Bradford L. Chamberlain. The Design and Implementation of a Region-Based Parallel Language. PhD Dissertation, University of Washington, 2001.

Digital Library

[28]

Bradford L. Chamberlain, Sung-Eun Choi, and Lawrence Snyder. A compiler abstraction for machine independent parallel communication generation. In Proceedings of the Workshop on Languages and Compilers for Parallel Computing, 1997.

Digital Library

[29]

M. R. Haghighat and C. D. Polychronopoulos. Symbolic analysis for parallelizing compilers. ACM Transactions on Programming Languages and Systems, 18(4):477-518, 1996.

Digital Library

[30]

J. R. Rose and G. L. Steele Jr. C*: An extended C language for data parallel programming. In Proceedings Second International Conference on Supercomputing, 1987.

[31]

Bradford L. Chamberlain, Sung-Eun Choi, E Christopher Lewis, Calvin Lin, Lawrence Snyder, and W. Derrick Weathersby. ZPL's WYSIWYG performance model. In Proceedings of the IEEE Workshop on High-Level Parallel Programming Models and Supportive Environments, 1998.

Digital Library

[32]

Sung-Eun Choi. Machine Independent Communication Optimization. PhD Dissertation, University of Washington, March 1999.

Digital Library

[33]

Bradford L. Chamberlain. The Design and Implementation of a Region-based Parallel Programming Language. PhD Dissertation, University of Washington, 2001.

Digital Library

[34]

Bradford L. Chamberlain, E Christopher Lewis, and Lawrence Snyder. Problem space promotion and its evaluation as a technique for efficient parallel computation. In Proceedings of the ACM International Conference on Supercomputing, 1999.

Digital Library

[35]

Robert van de Geijn and Jerrell Watts. SUMMA: Scalable Universal Matrix Multiplication Algorithm. Concurrency: Practice and Experience, 1998.

[36]

R. E. Cypher. J. L. C. Sanz and L. Snyder. Algorithms for image component labeling on SIMD mesh connected computers. IEEE Transactions on Computers, 39(2):276--281, 1990.

Digital Library

[37]

MasPar Programming Language (ANSI C-compatible MPL) Reference Manual Document Part Number: 9302-0001 Revision: A3 July 1992.

[38]

Gregory R. Watson. The Design and Implementation of a Parallel Relative Debugger. PhD Dissertation, Monash University, 2000.

[39]

W. P. Crowley et al. The SIMPLE code. Technical Report UCID 17715, Lawrence Livermore Laboratory, February 1978.

[40]

Lawrence Snyder. A Programmer's Guide to ZPL. MIT Press, Cambridge, MA, 1999. (The language changed in small ways and has been extended; it is now most accurately described in Chapter 2 of Chamberlain {33}.)

Digital Library

[41]

Steven J. Deitz, Bradford L. Chamberlain, Sung-Eun Choi, and Lawrence Snyder. The design and implementation of a parallel array operator for the arbitrary remapping of data. In Proceedings of the ACM Conference on Principles and Practice of Parallel Programming, 2003.

Digital Library

[42]

Adam Beguelin, Jack Dongara, Al Geist, Robert Manchek, and Vaidy Sunderam. User guide to PVM. Oak Ridge National Laboratory, Oak Ridge TN 378 316367, 1993.

[43]

M. Snir, S. W. Otto, S. Huss-Lederman, D. W. Walker, and J. Dongarra. MPI: the Complete Reference. MIT Press, Cambridge, MA, USA, 1996.

Digital Library

[44]

Bradford L. Chamberlain, Sung-Eun Choi, and Lawrence Snyder. A compiler abstraction for machine-independent parallel communication generation. In Proceedings of the Workshop on Languages and Compilers for Parallel Computing, 1997.

Digital Library

[45]

Sung-Eun Choi. Machine Independent Communication Optimization. PhD Dissertation, University of Washington, March 1999.

Digital Library

[46]

E Christopher Lewis, Calvin Lin, and Lawrence Snyder. The implementation and evaluation of fusion and contraction in array languages. In Proceedings of the ACM Conference on Programming Language Design and Implementation, 1998.

Digital Library

[47]

E Christopher Lewis. Achieving Robust Performance in Parallel Programming Languages. PhD Dissertation, University of Washington, February 2001.

Digital Library

[48]

W. Derrick Weathersby. Machine-Independent Compiler Optimizations for Collective Communication. PhD Dissertation, University of Washington, August 1999.

Digital Library

[49]

Steven J. Deitz, Bradford L. Chamberlain, and Lawrence Snyder. Eliminating redundancies in sum-of-product array computations. In Proceedings of the ACM International Conference on Supercomputing, 2001.

Digital Library

[50]

Bradford L. Chamberlain, Sung-Eun Choi, E Christopher Lewis, Calvin Lin, Lawrence Snyder, and W. Derrick Weathersby. Factor-join: A unique approach to compiling array languages for parallel machines. In Proceedings of the Workshop on Languages and Compilers for Parallel Computing, 1996.

Digital Library

[51]

Philip S. Abrams. An APL Machine, PhD Dissertation. Stanford University, SLAC Report 114, 1970.

Digital Library

[52]

Ulrich Ruede. Mathematical and computational techniques for multilevel adaptive methods, Frontiers in Applied Mathematics, 13, SIAM, 1993.

[53]

Guy E. Blelloch. Vector Models for Data-Parallel Computing. MIT Press, 1990.

Digital Library

[54]

R. E. Ladner and M. J. Fischer. Parallel prefix computation. In Proceedings of the IEEE International Conference on Parallel Processing, 1977.

[55]

Steven J. Deitz. High-Level Programming Language Abstractions for Advanced and Dynamic Parallel Computations. PhD Dissertation, University of Washington, February 2005.

Digital Library

[56]

Steven J. Deitz, Bradford L. Chamberlain, and Lawrence Snyder. High-level language support for user-defined reductions. Journal of Supercomputing, 23(1), 2002.

Digital Library

[57]

E Christopher Lewis and Lawrence Snyder. Pipelining wavefront computations: Experiences and performance. In Proceedings of the IEEE Workshop on High-Level Parallel Programming Models and Supportive Environments, May 2000.

Digital Library

[58]

Bradford L. Chamberlain, E Christopher Lewis, and Lawrence Snyder. Problem space promotion and its evaluation as a technique for efficient parallel computation. In Proceedings of the ACM International Conference on Supercomputing, 1999.

Digital Library

[59]

Calvin Lin, Lawrence Snyder, Ruth Anderson, Brad Chamberlain, Sung-Eun Choi, George Forman, E. Christopher Lewis, and W. Derrick Weathersby. ZPL vs. HPF: A Comparison of Performance and Programming Style, TR # 95-11-05 (available online from the University of Washington CSE technical report archive).

[60]

David Callahan, Bradford L. Chamberlain, and Hans P. Zima. The Cascade High Productivity Language. In 9th International Workshop on High-Level Parallel Programming Models and Supportive Environments (HIPS 2004), pp 52--60, April 2004.

[61]

Philippe Charles, Christian Grothoff, Vijay Saraswat, Christopher Donawa, Allan Kielstra, Kemal Ebcioglu, Christoph von Praun, and Vivek Sarkar. X10: An object-oriented approach to non-uniform cluster computing. In 20th OOPSLA, pp. 519--538, 2005.

Digital Library

[62]

David Bailey, Tim Harris, William Saphir, Rob van der Wijngaart, AlexWoo, and Maurice Yarrow. The NAS parallel benchmarks 2.0. Technical Report NAS-95-020, Nasa Ames Research Center, Moffet Field, CA, December 1995.

[63]

Bradford L. Chamberlain, Steven J. Deitz, and Lawrence Snyder. A comparative study of the NAS MG benchmark across parallel languages and architectures. In Proceedings of the ACM Conference on Supercomputing, 2000.

Digital Library

[64]

R. Alverson, D. Callahan, D. Cummings, B. Koblenz, A. Porterfield, and B. Smith. The Tera Computer System. ACM SIGARCH Computer Architecture News, 18(3):1 - 6, 1990.

Digital Library

[65]

Michael A. Hiltzik. Dealers in Lightning: Xerox PARC and the Dawn of the Computer Age, Harper Collins, 1999.

Digital Library

[66]

Eric Steven Raymond and Rob W. Landley. The Art of Unix Usability, Creative Commons, 2004. http://www.catb.org/~esr/writings/taouu/html/ch02.html.

[67]

Calvin Lin and Lawrence Snyder. A comparison of programming models for shared memory multiprocessors. In Proceedings of the IEEE International Conference on Parallel Processing, 1990.

[68]

M. Kandemir, P. Banerjee, A. Choudhary, J. Ramanujam and N. Shenoy. A global communication optimization technique based on data-flow analysis and linear algebra. ACM Transactions on Programming Languages and Systems 21(6):1251--1297, 1999.

Digital Library

[69]

A. J. Wagner, L. Giraud and C. E. Scott. Simulation of a cusped bubble rising in a viscoelastic fluid with a new numerical method, Computer Physics Communications, 129(3):227--232, 2000.

[70]

Robert W. Numrich, John Reid, and Kieun Kim. Writing a multigrid solver using Co-array Fortran. In Proceedings of the Fourth International Workshop on Applied Parallel Computing, 1998.

Digital Library

[71]

S. B. Scholz. A case study: Effects of WITH-loop-folding on the NAS benchmark MG in SAC. In Proceedings of IFL '98, Springer-Verlag, 1998.

Digital Library

Cited By

Ciccozzi FAddazi LAsadollah SLisper BMasud AMubeen S(2022)A Comprehensive Exploration of Languages for Parallel ComputingACM Computing Surveys10.1145/348500855:2(1-39)Online publication date: 18-Jan-2022
https://dl.acm.org/doi/10.1145/3485008
Heisig MVerna DCooper D(2018)Petalisp: A Common Lisp Library for Data Parallel ProgrammingProceedings of the 11th European Lisp Symposium on European Lisp Symposium10.5555/3323215.3323216(4-11)Online publication date: 18-Apr-2018
https://dl.acm.org/doi/10.5555/3323215.3323216
Heisig MKöstler HScholz SShivers O(2018)Petalisp: run time code generation for operations on strided arraysProceedings of the 5th ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming10.1145/3219753.3219755(11-17)Online publication date: 19-Jun-2018
https://dl.acm.org/doi/10.1145/3219753.3219755
Show More Cited By

Index Terms

Recommendations

ZPL: A Machine Independent Programming Language for Parallel Computers
Special issue on architecture-independent languages and software tools for parallel processing

The goal of producing architecture-independent parallel programs is complicated by the competing need for high performance. The ZPL programming language achieves both goals by building upon an abstract parallel machine and by providing programming ...
Self-Consistent MPI Performance Guidelines

Message passing using the Message-Passing Interface (MPI) is at present the most widely adopted framework for programming parallel applications for distributed memory and clustered parallel systems. For reasons of (universal) implementability, the MPI ...
Control replication: compiling implicit parallelism to efficient SPMD with logical regions
SC '17: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

We present control replication, a technique for generating high-performance and scalable SPMD code from implicitly parallel programs. In contrast to traditional parallel programming models that require the programmer to explicitly manage threads and the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HOPL III: Proceedings of the third ACM SIGPLAN conference on History of programming languages

June 2007

483 pages

ISBN:9781595937667

DOI:10.1145/1238844

Conference Chairs:
Barbara Ryder
Rutgers University
,
Brent Hailpern
IBM Research

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 June 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

HOPL-III '07

Sponsor:

SIGPLAN

HOPL-III '07: ACM SIGPLAN History of Programming Languages Conference III

June 9 - 10, 2007

California, San Diego

Acceptance Rates

HOPL III Paper Acceptance Rate 12 of 12 submissions, 100%;

Overall Acceptance Rate 12 of 12 submissions, 100%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
1,198
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ciccozzi FAddazi LAsadollah SLisper BMasud AMubeen S(2022)A Comprehensive Exploration of Languages for Parallel ComputingACM Computing Surveys10.1145/348500855:2(1-39)Online publication date: 18-Jan-2022
https://dl.acm.org/doi/10.1145/3485008
Heisig MVerna DCooper D(2018)Petalisp: A Common Lisp Library for Data Parallel ProgrammingProceedings of the 11th European Lisp Symposium on European Lisp Symposium10.5555/3323215.3323216(4-11)Online publication date: 18-Apr-2018
https://dl.acm.org/doi/10.5555/3323215.3323216
Heisig MKöstler HScholz SShivers O(2018)Petalisp: run time code generation for operations on strided arraysProceedings of the 5th ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming10.1145/3219753.3219755(11-17)Online publication date: 19-Jun-2018
https://dl.acm.org/doi/10.1145/3219753.3219755
Coullon HLimet S(2016)The SIPSim implicit parallelism model and the SkelGIS libraryConcurrency and Computation: Practice & Experience10.1002/cpe.349428:7(2120-2144)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1002/cpe.3494
De Wael MMarr SDe Fraine BVan Cutsem TDe Meuter W(2015)Partitioned Global Address Space LanguagesACM Computing Surveys10.1145/271632047:4(1-27)Online publication date: 26-May-2015
https://dl.acm.org/doi/10.1145/2716320
Imam SSarkar VLeibs DKessler P(2014)Exploiting Implicit Parallelism in Dynamic Array Programming LanguagesProceedings of ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming10.1145/2627373.2627374(1-7)Online publication date: 9-Jun-2014
https://dl.acm.org/doi/10.1145/2627373.2627374
Helene CSebastien L(2013)Algorithmic skeleton library for scientific simulations: SkelGIS2013 International Conference on High Performance Computing & Simulation (HPCS)10.1109/HPCSim.2013.6641450(429-436)Online publication date: Jul-2013
https://doi.org/10.1109/HPCSim.2013.6641450
Hélène CMinh-Hoang LSébastien L(2013)Parallelization of Shallow-water Equations with the Algorithmic Skeleton Library SkelGISProcedia Computer Science10.1016/j.procs.2013.05.22318(591-600)Online publication date: 2013
https://doi.org/10.1016/j.procs.2013.05.223
González CFraguela B(2013)A framework for argument-based task synchronization with automatic detection of dependenciesParallel Computing10.1016/j.parco.2013.04.01239:9(475-489)Online publication date: Sep-2013
https://doi.org/10.1016/j.parco.2013.04.012
Diaz JMunoz-Caro CNino A(2012)A Survey of Parallel Programming Models and Tools in the Multi and Many-Core EraIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2011.30823:8(1369-1386)Online publication date: 1-Aug-2012
https://dl.acm.org/doi/10.1109/TPDS.2011.308
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents