article

Free access

Flattening and parallelizing irregular, recurrent loop nests

Authors:

Anwar M. Ghuloum,

Allan L. FisherAuthors Info & Claims

ACM SIGPLAN Notices, Volume 30, Issue 8

Pages 58 - 67

https://doi.org/10.1145/209937.209944

Published: 01 August 1995 Publication History

Abstract

Irregular loop nests in which the loop bounds are determined dynamically by indexed arrays are difficult to compile into expressive parallel constructs, such as segmented scans and reductions. In this paper, we describe a suite of transformations to automatically parallelize such irregular loop nests, even in the presence of recurrences. We describe a simple, general loop flattening transformation, along with new optimizations which make it a viable compiler transformation. A robust recurrence parallelization technique is coupled to the loop flattening transformation, allowing parallelization of segmented reductions, scans, and combining-sends over arbitrary associative operators. We discuss the implementation and performance results of the transformations in a parallelizing Fortran 77 compiler for the Cray C90 supercomputer. In particular, we focus on important sparse matrix-vector multiplication kernels, for one of which we are able to automatically derive an algorithm used by one of the fastest library routines available.

References

[1]

A. Aho, R. Sethi, and J. Ullman. Compilers, principles, techniques, and tools. Addison Wesley, 1988.

Digital Library

[2]

Guy E. Blelloch. Scans as primitive parallel operations. IEEE Transactions on Computers, C- 38(11):1526-1538, November 1989.

Digital Library

[3]

G.E. Blelloch, S. Chatterjee, J. C. Hardwick, J. Sipelstein, and M. Zagha. Implementation of a portable nested data-parallel language. In Proceedings 4th A CM SIGPLAN Symposium on Principles and Practice of Parallel Programming. San Diego, CA, May 1993.

Digital Library

[4]

G.E. Blelloch, S. Chatterjee, and M. Zagha. Solving linear recurrences with loop raking. In Proceedings Sixth International Parallel Processing Symposium, March 1992.

Digital Library

[5]

G.E. BIelloch, M. Heroux, and M. Zagha. Segmented operations for sparse matrix computation on vector multiprocessors. Carnegie Mellon University, School of Computer Science Technical Report CMU-CS-93- 173, August 1993.

Digital Library

[6]

R. Das, J. Saltz, and R. yon Hanxleden. Slicing analysis and indirect accesses to distributed arrays. Technical Report CS-TR-3076, University of Maryland, May 1993.

Digital Library

[7]

i.S. Duff, A. M. Erisman, and J. K. Reid. Direct Methods for Sparse Matrices. Oxford Science Publications, 1986.

Digital Library

[8]

A.L. Fisher and A. M. Ghuloum. Parallelizing complex scans and reductions. In Proceedings of the ACM SIGPLAN '94 Conference on Programming Language Design and Implementation, pages 135-146, Orlando, FL, June 1994.

Digital Library

[9]

R. von Hanxleden and K. Kennedy. Relaxing SIMD control flow constraints using loop transformations. In Proceedings of the A CM SIGPLAN '92 Conference on Programming Language Design and Implementation, pages 188-199, San Francisco, CA, June 1992.

Digital Library

[10]

R. von Hanxleden, K. Kennedy, C. KoelbeI, R. Das, and J. Saltz. Compiler Analysis for Irregular Problems in Fortran D. In Languages and Compilers for Parallel Computing, 5th International Workshop Proceedings, pages 97-111, New Haven, CT, August 1992.

Digital Library

[11]

D.R. Kincaid and T. C. Oppe. Recent vectorization and parallelization of ITPACKV. Technical report, Center for Numerical Analysis, The University of Texas at Austin, November 1984.

[12]

C.J. Kuck, R. H. Kuhn, B. Leasure, and M. Wolfe. The structure of an advanced vectorizer for pipelined processors. In the proceedings of The 4th International Computer Software and Applications Conference (COMPSAC 80), October 1980.

[13]

C. Polychronopoulos. Loop coalescing: a compiler transformation for parallel machines. In Proceedings of the 1987 International Conference on Parallel Processing, St. Charles, IL, August 1987.

[14]

T.J. Sheffler. Implementing the multiprefix operation on parallel and vector computers. Carnegie Mellon University, School of Computer Science Technical Report CMU-CS-92-173, August 1992.

Digital Library

[15]

T.J. Sheffler. Match and move, an approach to data parallel computing. Ph.D. Thesis. Carnegie Mellon University, School of Computer Science 1992.

Digital Library

[16]

J. Subhlok, D. R. O'Hallaron, T. Gross, P. A. Dinda, and J. Webb. Communication and memory requirements as the basis for mapping task and data parallel programs. In Proceedings Supercomputing '94, Washington, D.C., November 1994.

Digital Library

[17]

M. Wolfe. Optimizing Supercompilers for Supercomputers. MIT Press, 1989.

Digital Library

Cited By

Yang YHeath DHazay CKolesnikov VVenkitasubramaniam MMeng WJensen CCremers CKirda E(2023)Batchman and Robin: Batched and Non-batched Branching for Interactive ZKProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security10.1145/3576915.3623169(1452-1466)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1145/3576915.3623169
Sim HLee HSeo SLee J(2019)Mapping Imperfect Loops to Coarse-Grained Reconfigurable ArchitecturesIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2015.250491835:7(1092-1104)Online publication date: 4-Jan-2019
https://dl.acm.org/doi/10.1109/TCAD.2015.2504918
Clauss PAltintas EKuhn M(2017)Automatic Collapsing of Non-Rectangular Loops2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS.2017.34(778-787)Online publication date: May-2017
https://doi.org/10.1109/IPDPS.2017.34
Show More Cited By

Index Terms

Flattening and parallelizing irregular, recurrent loop nests
1. Computing methodologies
  1. Concurrent computing methodologies
    1. Concurrent programming languages
2. Software and its engineering
  1. Software notations and tools
    1. Compilers
    2. General programming languages
      1. Language types
        Concurrent programming languages

Recommendations

Flattening and parallelizing irregular, recurrent loop nests
PPOPP '95: Proceedings of the fifth ACM SIGPLAN symposium on Principles and practice of parallel programming

Irregular loop nests in which the loop bounds are determined dynamically by indexed arrays are difficult to compile into expressive parallel constructs, such as segmented scans and reductions. In this paper, we describe a suite of transformations to ...
Flattening-based mapping of imperfect loop nests for CGRAs
CODES '14: Proceedings of the 2014 International Conference on Hardware/Software Codesign and System Synthesis

For loop accelerators such as coarse-grained reconfigurable architectures (CGRAs) and GP-GPUs, nested loops represent an important source of parallelism. Existing solutions to mapping nested loops on CGRAs, however, are either designed for perfectly ...
Tiling imperfectly-nested loop nests
SC '00: Proceedings of the 2000 ACM/IEEE conference on Supercomputing

Tiling is one of the more important transformations for enhancing loca lity of reference in programs. Intuitively, tiling a set of loops achieves the effect of interleaving iterations of these loops. Tiling of perfectly-nested loop nests (which are loop ...

Comments

Information & Contributors

Information

Published In

Copyright © 1995 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 1995

Published in SIGPLAN Volume 30, Issue 8

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
975
Total Downloads

Downloads (Last 12 months)155
Downloads (Last 6 weeks)30

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang YHeath DHazay CKolesnikov VVenkitasubramaniam MMeng WJensen CCremers CKirda E(2023)Batchman and Robin: Batched and Non-batched Branching for Interactive ZKProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security10.1145/3576915.3623169(1452-1466)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1145/3576915.3623169
Sim HLee HSeo SLee J(2019)Mapping Imperfect Loops to Coarse-Grained Reconfigurable ArchitecturesIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2015.250491835:7(1092-1104)Online publication date: 4-Jan-2019
https://dl.acm.org/doi/10.1109/TCAD.2015.2504918
Clauss PAltintas EKuhn M(2017)Automatic Collapsing of Non-Rectangular Loops2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS.2017.34(778-787)Online publication date: May-2017
https://doi.org/10.1109/IPDPS.2017.34
Cardoso JCoutinho JDiniz P(2017)Source code transformations and optimizationsEmbedded Computing for High Performance10.1016/B978-0-12-804189-5.00005-3(137-183)Online publication date: 2017
https://doi.org/10.1016/B978-0-12-804189-5.00005-3
Mururu GKhan SChatterjee BChen CPorter CGavrilovska APande S(2023)Beacons: An End-to-End Compiler Framework for Predicting and Utilizing Dynamic Loop CharacteristicsProceedings of the ACM on Programming Languages10.1145/36228037:OOPSLA2(173-203)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3622803
Rocha RPetoumenos PFranke BBhatotia PO'Boyle MLee J(2022)Loop rolling for code size reductionProceedings of the 20th IEEE/ACM International Symposium on Code Generation and Optimization10.1109/CGO53902.2022.9741256(217-229)Online publication date: 2-Apr-2022
https://dl.acm.org/doi/10.1109/CGO53902.2022.9741256
Campos JFayollas CHarrison MMartinie CMasci PPalanque P(2020)Supporting the Analysis of Safety Critical User InterfacesACM Transactions on Computer-Human Interaction10.1145/340419927:5(1-48)Online publication date: 3-Aug-2020
https://dl.acm.org/doi/10.1145/3404199
Yaqub USharma NPabreja RChun SAtluri VVaidya J(2020)Location-based Sentiment Analyses and Visualization of Twitter Election DataDigital Government: Research and Practice10.1145/33399091:2(1-19)Online publication date: 9-Apr-2020
https://dl.acm.org/doi/10.1145/3339909
Zhang YGenkin DKatz JPapadopoulos DPapamanthou C(2018)vRAM: Faster Verifiable RAM with Program-Independent Preprocessing2018 IEEE Symposium on Security and Privacy (SP)10.1109/SP.2018.00013(908-925)Online publication date: May-2018
https://doi.org/10.1109/SP.2018.00013
Silberstein MKim SHuh SZhang XHu YWated AWitchel E(2016)GPUnetACM Transactions on Computer Systems10.1145/296309834:3(1-31)Online publication date: 17-Sep-2016
https://dl.acm.org/doi/10.1145/2963098
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents