research-article

Communication Generation for Aligned and Cyclic(K) Distributions Using Integer Lattice

Authors:

Eric Hung-Yu Tseng,

Jean-Luc GaudiotAuthors Info & Claims

IEEE Transactions on Parallel and Distributed Systems, Volume 10, Issue 2

Pages 136 - 146

https://doi.org/10.1109/71.752780

Published: 01 February 1999 Publication History

Abstract

Optimizing communication is a key issue in generating efficient SPMD codes in compiling distributed arrays on data parallel languages, such as High Performance Fortran. In HPF, the array distribution may involve alignment and cyclic(k)-distribution such that the enumeration of the local set and the enumeration of the communication set exhibit regular patterns which can be modeled as integer lattices. In the special case of unit-strided alignment, many techniques of the communication set enumeration have been proposed, while in the general case of the non-unit-strided alignment, inspector-like run-time codes are needed to build repeating pattern table or to scan over local elements such that the communication set can be constructed. Unlike other works on this problem of the general alignment and cyclic(k) distribution, our approach derives an algebraic solution for such an integer lattice that models the communication set by using the Smith-Normal-Form analysis, therefore, efficient enumeration of the communication set can be generated. Based on the integer lattice, we also present our algorithm for the SPMD code generation. In our approach, when the parameters are known, the SPMD program can be efficiently constructed without any inspector-like run-time codes.

References

[1]

A. Thirumalai and J. Ramanujam, “Efficient Computation of Address Sequences in Data Parallel Programs Using Closed Forms for Basic Vectors,” J. Parallel and Distributed Computing, vol. 38, no. 2, Nov. 1996.

Digital Library

[2]

A. Venkatachar J. Ramanujam, and A. Thirumalai, “Communication Generation for Block-Cyclic Distributions,” Parallel Processing Letters, vol. 7, no. 2, June 1997.

[3]

C. Ancourt and F. Irigoin, “Scanning Polyhedra with Do Loops,” Proc. Symp. Principles and Practice of Parallel Programming, Williamsburg, Va., 1991.

Digital Library

[4]

C. Ancourt F. Coelho F. Irigoin, and R. KerYell, “A Linear Algebra Framework for Static High Performance Fortran Code Distribution,” Scientific Programming, vol. 6, no. 1, Spring 1997.

Digital Library

[5]

E.M. Paalvast H.J. Sips, and A. van Gemund, “Automatic Parallel Program Generation and Optimization from Data Decompositions,” Proc. Int'l Conf. Parallel Processing, Aug. 1991.

[6]

M. Le Fur, “Scanning Parameterized Polyhedron Using Fourier-Motzkin Elimination,” Publication Interne 858, IRISA, Sept. 1994.

[7]

C. Germain and F. Delaplace, “Automatic Vectorization of Communications for Data-Parallel Programs,” Proc. EURO PAR, June 1995.

Digital Library

[8]

H.J. Sips K. van Reeuwijk, and W. Denissen, “Analysis of Local Enumeration and Storage Schemes in HPF,” Proc. Int'l Conf. Supercomputing, 1996.

Digital Library

[9]

J.-F. Collard and T. Risset, “Construction of do Loops from Systems of Affine Constraints,” Parallel Processing Letters, vol. 5, no. 3, 1995.

[10]

J.M. Stichnoth D. O'Hallaron, and T.R. Gross, “Generating Communication for Array Statements: Design, Implementation, and Evaluation,” J. Parallel and Distributed Computing, vol. 21, 1994.

Digital Library

[11]

K. Kennedy N. Nedeljkovic, and A. Sethi, “Communication Generation for Cyclic(k) Distributions,” Languages, Compilers, and Run-Time Systems for Scalable Computers, B. Szymanski and B. Sinharoy, eds. 1996.

[12]

K. Kennedy N. Nedeljkovic, and A. Sethi, “A Linear-Time Algorithm for Computing the Memory Access Sequence in Data-Parallel Programs,” ACM SIGPLAN Symp. Principle and Practice of Parallel Programming, Santa Barbara, Calif., July 1995.

Digital Library

[13]

K. van Reeuwijk W.D.H. Sips, and E.M. Paalvast, “An Implementation Framework for HPF Distributed Arrays on Message-Passing Parallel Computer Systems,” IEEE Trans. Parallel and Distributed Systems, vol. 7, no. 9, Sept. 1996.

Digital Library

[14]

C. Koelbel and P. Mehrotra, “Compiling Global Name-Space Parallel Loops for Distributed Execution, ” IEEE Trans. Parallel and Distributed Systems, vol. 2, no. 4, Oct. 1991.

Digital Library

[15]

J. Li and M. Chen, “Compiling Communication-Efficient Programs for Massively Parallel Machines,” IEEE Trans. Parallel and Distributed Systems, vol. 2, no. 3, July 1991.

Digital Library

[16]

M. Newman, Integral Matrices. New York: Academic Press, 1972.

[17]

S. Benkner, “Handling Block-Cyclic Distributed Arrays in Vienna Fortran 90,” Proc. Int'l Conf. Parallel Architectures and Compilation Techniques, June 1995.

Digital Library

[18]

S. Benkner P. Brezany, and H. Zima, “Processing Array Statements and Procedure Interfaces in the Prepare High Performance Fortran Compiler,” Proc. Fifth Int'l Conf. Compiler Construction, Apr. 1994.

Digital Library

[19]

S. Chatterjee J.R. Gilbert F. Long R. Schreiber, and S.H. Teng, “Generating Local Addresses and Communication Sets for Data-Parallel Programs,” J. Parallel and Distributed Computing, vol. 26, no. 1, Apr. 1995.

Digital Library

[20]

S. Gupta S. Kaushik S. Mufti S. Sharma C.H. Huang, and P. Sadayappan, “On Compiling Array Expressions for Efficient Execution on Distributed-Memory Machines,” J. Parallel and Distributed Computing, vol. 32, 1996.

Digital Library

[21]

S. Hiranandani K. Kennedy J. Mellor-Crummey, and A. Sethi, “Compilation Techniques for Block-Cyclic Distributions,” Proc. Int'l Conf. Supercomputing, Manchester, England, 11-15 July 1994.

Digital Library

[22]

S.D. Kaushik C.H. Huang, and P. Sadayappan, “Compiling Array Statements for Efficient Execution on Distributed Memory Machines: Two-Level Mappings,” Proc. Eighth Ann. Workshop Languages and Compilers for Parallel Computing, Aug 1995.

Digital Library

[23]

C.-W. Tseng, “An Optimizing Fortran D Compiler for MIMD Distributed-Memory Machines,” PhD thesis, Rice Univ., 1993.

Digital Library

Cited By

Guo MPan YLiu Z(2019)Symbolic Communication Set Generation for Irregular Parallel ApplicationsThe Journal of Supercomputing10.1023/A:102426261020125:3(199-214)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1023/A%3A1024262610201
Guo MNakata I(2019)A Framework for Efficient Data Redistribution on Distributed Memory MulticomputersThe Journal of Supercomputing10.1023/A:101160273257020:3(243-265)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1023/A%3A1011602732570
Lee PChen W(2018)Generating communication sets of array assignment statements for block-cyclic distribution on distributed memory parallel computersParallel Computing10.1016/S0167-8191(02)00113-828:9(1329-1368)Online publication date: 31-Dec-2018
https://dl.acm.org/doi/10.1016/S0167-8191%2802%2900113-8
Show More Cited By

Index Terms

Communication Generation for Aligned and Cyclic(K) Distributions Using Integer Lattice

Recommendations

An Implementation Framework for HPF Distributed Arrays on Message-Passing Parallel Computer Systems

Data parallel languages, like High Performance Fortran (HPF), support the notion of distributed arrays. However, the implementation of such distributed array structures and their access on message passing computers is not straightforward. This holds ...
Scheduling Block-Cyclic Array Redistribution

This article is devoted to the run-time redistribution of one-dimensional arrays that are distributed in a block-cyclic fashion over a processor grid. While previous studies have concentrated on efficiently generating the communication messages to be ...
LLVM-based communication optimizations for PGAS programs
LLVM '15: Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC

While Partitioned Global Address Space (PGAS) programming languages such as UPC/UPC++, CAF, Chapel and X10 provide high-level programming models for facilitating large-scale distributed-memory parallel programming, it is widely recognized that compiler ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Parallel and Distributed Systems

IEEE Transactions on Parallel and Distributed Systems Volume 10, Issue 2

February 1999

96 pages

ISSN:1045-9219

Editor:
John A. Stankovic
Univ. of Virginia, Charlottesville

Issue’s Table of Contents

Copyright © Copyright © 1999 IEEE. All Rights Reserved.

Publisher

IEEE Press

Publication History

Published: 01 February 1999

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Guo MPan YLiu Z(2019)Symbolic Communication Set Generation for Irregular Parallel ApplicationsThe Journal of Supercomputing10.1023/A:102426261020125:3(199-214)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1023/A%3A1024262610201
Guo MNakata I(2019)A Framework for Efficient Data Redistribution on Distributed Memory MulticomputersThe Journal of Supercomputing10.1023/A:101160273257020:3(243-265)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1023/A%3A1011602732570
Lee PChen W(2018)Generating communication sets of array assignment statements for block-cyclic distribution on distributed memory parallel computersParallel Computing10.1016/S0167-8191(02)00113-828:9(1329-1368)Online publication date: 31-Dec-2018
https://dl.acm.org/doi/10.1016/S0167-8191%2802%2900113-8
Tseng EGaudiot J(2018)Automatic array partitioning based on the Smith normal formInternational Journal of Parallel Programming10.1007/s10766-004-1460-233:1(35-56)Online publication date: 27-Dec-2018
https://dl.acm.org/doi/10.1007/s10766-004-1460-2

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents