Article

Free access

Communication and memory requirements as the basis for mapping task and data parallel programs

Authors:

Jaspal Subhlok,

David R. O'Hallaron,

Peter A. Dinda,

Jon WebbAuthors Info & Claims

Supercomputing '94: Proceedings of the 1994 ACM/IEEE conference on Supercomputing

Pages 330 - 339

Published: 14 November 1994 Publication History

Abstract

For a wide variety of applications, both task and data parallelism must be exploited to achieve the best possible performance on a multicomputer. Recent research has underlined the importance of exploiting task and data parallelism in a single compiler framework, and such a compiler can map a single source program in many different ways onto a parallel machine. The tradeoffs between task and data parallelism are complex and depend on the characteristics of the program to be executed, most significantly the memory and communication requirements, and the performance parameters of the target parallel machine. In this paper, we present a framework to isolate and examine the specific characteristics of programs that determine the performance for different mappings. Our focus is on applications that process a stream of input, and whose computation structure is fairly static and predictable. We describe three such applications that were developed with our compiler: fast Fourier transforms, narrowband tracking radar, and multibaseline stereo. We examine the tradeoffs between various mappings for them and show how the framework is used to obtain efficient mappings.

References

[1]

Agrawal, G., Sussman, A., and Saltz, J. An integrated run-time and compile-time approach for parallelizing structured and block structured applications. Tech. Rep. CS-TR-3143 and UMIACS-TR-93-94, University of Maryland, Department of Computer Science and UMIACS, Oct. 1993.]]

Digital Library

[2]

Bailey, D. H. FFTs in external or hierarchical memory. The Journal of Supercomputing 4 (1990), 23-35.]]

Digital Library

[3]

Bokhari, S. Assignment Problems in Parallel and Distributed Computing. Kluwer Academic Publishers, 1987.]]

Digital Library

[4]

Borkar, S., Cohn, R., Cox, G., Gross, T., Kung, H. T., Lam, M., Moore, M. L. B., Moore, W., Peterson, C., Susman, J., Sutton, J., Urbanski, J., and Webb, J. Supporting systolic and memory communication in iWarp. In Proceedings of the 17th Annual International Symposium on Computer Architecture (Seattle, WA, May 1990), pp. 70-81.]]

Digital Library

[5]

Chandy, M., Foster, I., Kennedy, K., Koelbel, C., and Tseng, C. Integrated support for task and data parallelism. International Journal of Supercomputer Applications 8, 2 (1994), 80-98.]]

[6]

Chapman, B., Mehrotra, P., Van Rosendale, J., and Zima, H. A software architecture for multidisciplinary applications: Integrating task and data parallelism. Tech. Rep. 94-18, ICASE, NASA Langley Research Center, Hampton, VA, Mar. 1994.]]

Digital Library

[7]

Chapman, B., Mehrotra, P., and Zima, H. Programming in Vienna Fortran. Scientific Programming 1, 1 (Aug. 1992), 31-50.]]

Digital Library

[8]

Cheung, A., and Reeves, A. Function-parallel computation in a data-parallel environment. In Proceedings of the 1993 International Conference on Parallel Processing (St Charles, IL, August 1993), vol. 2, pp. 21-24.]]

Digital Library

[9]

Choudhary, A., Narahari, B., Nicol, D., and Simha, R. Optimal processor assignment for a class of pipelined computations. IEEE Transactions on Parallel and Distributed Systems 5, 4 (April 94), 439-445.]]

Digital Library

[10]

Dinda, P., Gross, T., O'Hallaron, D., Segall, E., Stichnoth, J., Subhlok, J., Webb, J., and Yang, B. The CMU task parallel program suite. Tech. Rep. CMU-CS-94-131, School of Computer Science, Carnegie Mellon University, Mar. 1994.]]

[11]

Foster, I., and Chandy, K. Fortran M: A language for modular parallel programming. Tech. Rep. MCS-P327-0992, Argonne National Laboratory, June 1992.]]

[12]

Fox, G. The architecture of problems and portable parallel software systems. Tech. Rep. CRPC-TR91-172, Northeast Parallel Architectures Center, 1991.]]

[13]

Gentleman, W. M., and Sande, G. Fast Fourier transforms for fun and profit. In Proc. AFIPS (1966), vol. 29, pp. 563-578.]]

[14]

Gross, T., O'Hallaron, D., and Subhlok, J. Task parallelism in a high performance fortran framework. IEEE Parallel and Distributed Technology (1994). To appear.]]

Digital Library

[15]

High Performance Fortran Forum.High Performance Fortran Language Specification, Version 1.0, May 1993.]]

[16]

Lam, M., and Rinard, M. Coarse-grain parallel programming in Jade. In Proceedings of the Third ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (Williamsburg, VA, April 1991), pp. 94-105.]]

Digital Library

[17]

Nicol, D., and O'Hallaron, D. Efficient algorithms for mapping pipelined and parallel computations. IEEE Transactions on Computers 40, 3 (Mar. 1991), 295-306.]]

Digital Library

[18]

Okutomi, M., and Kanade, T. A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 15, 4 (1993), 353-363.]]

Digital Library

[19]

Printz, H. Automatic Mapping of Large Signal Processing Systems to a Parallel Machine. PhD thesis, School of Computer Science, Carnegie Mellon University, 1991. Also available as report CMU-CS-91-101.]]

Digital Library

[20]

Ramaswamy, S., and Banerjee, P. Processor allocation and scheduling of macro dataflow graphs on distributed memory multicomputers by the PARADIGM compiler. In Proceedings of the 1993 International Conference on Parallel Processing (St Charles, IL, August 1993), vol. 2, pp. 134-138.]]

Digital Library

[21]

Sarkar, V. Partitioning and Scheduling Parallel Programs for Multiprocessors. The MIT Press, Cambridge, MA, 1989.]]

Digital Library

[22]

Shaw, G., Gabel, R., Martinez, D., Rocco, A., Pohlig, S., Gerber, A., Noonan, J., and Teitelbaum, K. Multiprocessors for radar signal processing. Tech. Rep. 961, MIT Lincoln Laboratory, Nov. 1992.]]

[23]

Subhlok, J. Automatic mapping of task and data parallel programs for efficient execution on multicomputers. Tech. Rep. CMU-CS-93-212, School of Computer Science, Carnegie Mellon University, November 1993.]]

Digital Library

[24]

Subhlok, J., Stichnoth, J., O'Hallaron, D., and Gross, T. Exploiting task and data parallelism on a multicomputer. In ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (May 1993), pp. 13-22.]]

Digital Library

[25]

Tseng, C., Hiranandani, S., and Kennedy, K. Preliminary experiences with the Fortran D compiler. In Proceedings of Supercomputing '93 (Portland, OR, November 1993), pp. 338-350.]]

Digital Library

[26]

Van Loan, C. Computational Frameworks for the Fast Fourier Transform. SIAM, Philadelphia, PA, 1992.]]

Digital Library

[27]

Webb, J. Implementation and performance of fast parallel multi-baseline stereo vision. In Computer Architectures for Machine Perception (Dec. 1993), pp. 232-240.]]

[28]

Webb, J. Latency and bandwidth consideration in parallel robotics image processing. In Supercomputing '93 (Nov. 1993), pp. 230-239.]]

Digital Library

Cited By

Dai JHuang BLi LHarrison LSarkar VHall M(2005)Automatically partitioning packet processing applications for pipelined architecturesProceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation10.1145/1065010.1065039(237-248)Online publication date: 12-Jun-2005
https://dl.acm.org/doi/10.1145/1065010.1065039
Dai JHuang BLi LHarrison L(2005)Automatically partitioning packet processing applications for pipelined architecturesACM SIGPLAN Notices10.1145/1064978.106503940:6(237-248)Online publication date: 12-Jun-2005
https://dl.acm.org/doi/10.1145/1064978.1065039
Benoit ACole MGilmore SHillston J(2005)Flexible skeletal programming with eskelProceedings of the 11th international Euro-Par conference on Parallel Processing10.1007/11549468_83(761-770)Online publication date: 30-Aug-2005
https://dl.acm.org/doi/10.1007/11549468_83
Show More Cited By

Communication and memory requirements as the basis for mapping task and data parallel programs

Recommendations

Communication and Memory Requirements as the Basis for Mapping Task and Data Parallel Programs
Exploiting Distributed-Memory and Shared-Memory Parallelism on Clusters of SMPs with Data Parallel Programs

Clusters of SMPs are hybrid-parallel architectures that combine the main concepts of distributed-memory and shared-memory parallel machines. Although SMP clusters are widely used in the high performance computing community, there exists no single ...
Automatic Mapping of Task and Data Parallel Programs for Efficient Execution on Multicomputers

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Supercomputing '94: Proceedings of the 1994 ACM/IEEE conference on Supercomputing

November 1994

840 pages

ISBN:0818666056

Conference Chair:
Gary M. Johnson
George Mason University

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS: Computer Society

Publisher

IEEE Computer Society Press

Washington, DC, United States

Publication History

Published: 14 November 1994

Check for updates

Qualifiers

Article

Conference

SC '94

Sponsor:

SIGARCH
IEEE-CS

SC '94: International Conference for High Performance Computing, Networking, Storage and Analysis

November 14 - 18, 1994

Washington, D.C.

Acceptance Rates

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
133
Total Downloads

Downloads (Last 12 months)47
Downloads (Last 6 weeks)5

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dai JHuang BLi LHarrison LSarkar VHall M(2005)Automatically partitioning packet processing applications for pipelined architecturesProceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation10.1145/1065010.1065039(237-248)Online publication date: 12-Jun-2005
https://dl.acm.org/doi/10.1145/1065010.1065039
Dai JHuang BLi LHarrison L(2005)Automatically partitioning packet processing applications for pipelined architecturesACM SIGPLAN Notices10.1145/1064978.106503940:6(237-248)Online publication date: 12-Jun-2005
https://dl.acm.org/doi/10.1145/1064978.1065039
Benoit ACole MGilmore SHillston J(2005)Flexible skeletal programming with eskelProceedings of the 11th international Euro-Par conference on Parallel Processing10.1007/11549468_83(761-770)Online publication date: 30-Aug-2005
https://dl.acm.org/doi/10.1007/11549468_83
Nanda MChandra SSarkar V(2004)Decentralizing execution of composite web servicesACM SIGPLAN Notices10.1145/1035292.102899139:10(170-187)Online publication date: 1-Oct-2004
https://dl.acm.org/doi/10.1145/1035292.1028991
Nanda MChandra SSarkar VVlissides JSchmidt D(2004)Decentralizing execution of composite web servicesProceedings of the 19th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications10.1145/1028976.1028991(170-187)Online publication date: 24-Oct-2004
https://dl.acm.org/doi/10.1145/1028976.1028991
Subhlok JVondran G(2000)Optimal Use of Mixed Task and Data Parallelism for Pipelined ComputationsJournal of Parallel and Distributed Computing10.1006/jpdc.1999.159660:3(297-319)Online publication date: 1-Mar-2000
https://dl.acm.org/doi/10.1006/jpdc.1999.1596
Park SKoo M(1997)Detection of Implicit Parallelisms in the Task Parallel LanguageProceedings of the High-Performance Computing on the Information Superhighway, HPC-Asia '9710.5555/523549.822956Online publication date: 28-Apr-1997
https://dl.acm.org/doi/10.5555/523549.822956
Ghuloum AFisher A(1995)Flattening and parallelizing irregular, recurrent loop nestsACM SIGPLAN Notices10.1145/209937.20994430:8(58-67)Online publication date: 1-Aug-1995
https://dl.acm.org/doi/10.1145/209937.209944
Ghuloum AFisher AFerrante JPadua D(1995)Flattening and parallelizing irregular, recurrent loop nestsProceedings of the fifth ACM SIGPLAN symposium on Principles and practice of parallel programming10.1145/209936.209944(58-67)Online publication date: 1-Aug-1995
https://dl.acm.org/doi/10.1145/209936.209944
Gross TO'Hallaron DSubhlok J(1994)Task Parallelism in a High Performance Fortran FrameworkIEEE Parallel & Distributed Technology: Systems & Technology10.1109/M-PDT.1994.3297912:3(16-26)Online publication date: 1-Sep-1994
https://dl.acm.org/doi/10.1109/M-PDT.1994.329791

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents