Article

Free access

Where is time spent in message-passing and shared-memory programs?

Authors:

Satish Chandra,

James R. Larus,

Anne RogersAuthors Info & Claims

ASPLOS VI: Proceedings of the sixth international conference on Architectural support for programming languages and operating systems

Pages 61 - 73

https://doi.org/10.1145/195473.195501

Published: 01 November 1994 Publication History

Abstract

Message passing and shared memory are two techniques parallel programs use for coordination and communication. This paper studies the strengths and weaknesses of these two mechanisms by comparing equivalent, well-written message-passing and shared-memory programs running on similar hardware. To ensure that our measurements are comparable, we produced two carefully tuned versions of each program and measured them on closely-related simulators of a message-passing and a shared-memory machine, both of which are based on same underlying hardware assumptions.

We examined the behavior and performance of each program carefully. Although the cost of computation in each pair of programs was similar, synchronization and communication differed greatly. We found that message-passing's advantage over shared-memory is not clear-cut. Three of the four shared-memory programs ran at roughly the same speed as their message-passing equivalent, even though their communication patterns were different.

References

[1]

Anant Agarwal, Richard Simoni, Mark Horowitz, and John Hennessy. An Evaluation of Directory Schemes for Cache Coherence. In Proceedings of the 15th Annual Internatwnal Symposium on Computer Architecture, pages 280-289, 1988.

Digital Library

[2]

Matthias A. Blumrich, Kai Li, Richard Alpert, Cezary Dubnicki, Edward W. Fetten, and Jonathon Sandberg. Virtual Memory Mapped Network Interface for the SHRIMP Multicomputer. In Proceedings of the 21st Annual International Symposium on Computer Architecture, pages 142-153, April 1994.

Digital Library

[3]

D. E. Culler, A. Dusseau, S. C. Goldstein, A. Krishnamurthy, S. Lumetta, T. von Eicken, and K. Yelick. Parallel Programming in Split-C. In Proceedings of Supercomputing 93, pages 262-273, November 1993.

Digital Library

[4]

David Culler, Richard Karp, David Patterson, Abhijit Sahay, Klaus Erik Schauser, Eunice Santos, Ramesh Subramonian, and Thorsten von Eicken. LogP: Toward a Realistic Model of Parallel Computation. In Fifth A CM SIG- PLAN Symposium on Principles ~4 Practice of Parallel Programming (PPOPP), pages 1-12, May 1993.

Digital Library

[5]

Phillip M. Dickens, Philip Heidelberger, and David M. Nicol. A Distributed Memory LAPSE: Parallel Simulation of Message-Passing Programs. In Proceedings of the 8th Workshop on Parallel and D~stributed Simulation (PADS '94), pages 32-38, July 1994.

Digital Library

[6]

Babak Falsafi, Alvin Lebeck, Steven Reinhardt, Ioannis Schoinas, Mark D. Hill, James Larus, Anne Rogers, and David Wood. Application-Specific Protocols for User- Level Shared Memory. In Proceedings of Supercomputmg 9J, November 1994. To appear.

Digital Library

[7]

Edward W. Felten. Protocol Compilation: High- Performance Communication for Parallel Programs. Technical Report 93-09-09, Department of Computer Science, University of Washington, September 1993.

[8]

Mark D. Hill, James R. Larus, Steven K. Reinhardt, and David A. Wood. Cooperative Shared Memory: Software and Hardware for Scalable Multiprocessors. A CM Transactions on Computer Systems, 11(4):300-318, November 1993. Earlier version appeared in ASPLOS V, Oct. 1992.

Digital Library

[9]

W. Daniel Hillis and Lewis W. ~cker. The CM-5 Connection Machine: A Scalable Supercomputer. Communications of the A CM, 36(11):31-40, November 1993.

Digital Library

[10]

Intel Corporation. Paragon Technical Summary. Intel Supercomputer Systems Division, 1993.

[11]

Alexander C. KIaiber and Henry M. Levy. A Comparison of Message Passing and Shared Memory Architectures for Data Parallel Programs. In Proceedings of the 21st Annual International Symposium on Computer Architecture, pages 94-105, April 1994.

Digital Library

[12]

David Kranz, Kirk Johnson, Anent Agarwal, John Kubiatowicz, and Beng-Hong Lim. Integrating Message-Passing and Shared-Memory: Early Experience. In Fifth A CM SIGPLAN Symposium on Principles FJ Practice of Parallel Programming (PPOPP), pages 54-63, May 1993.

Digital Library

[13]

Thomas J. LeBlanc and Evangelos P. Markatos. Shared Memory Vs. Message Passing in Shared-Memory Multiprocessors. In Fourth IEEE Symposium on Parallel and Distributed Processing, Dallas, TX, December 1992.

[14]

R. De Leone, O.L. Mangasarian, and T.-H. Shiau. Multi- Sweep Asynchronous Parallel Successive Overrelaxation for the Nonsymmetric Linear Complementarity Problem. Annals of Operations Research, 22:43-54, 1990.

Digital Library

[15]

Calvin Lin and Lawrence Snyder. A Comparison of Programming Models for Shared Memory Multiprocessors. in Proceedings of the 1990 International Conference on Parallel Processing (Vol. H Software), pages ii-163-170, August 1990.

[16]

Margaret Martonosi and Anoop Gupta. Tradeoffs in Message Passing and Shared Memory Implementations of a Standard Cell Router. In Proceedings of the 1989 International Conference on Parallel Processing (Vol. III Algorithms and Applications), pages III88-96, August 1989.

[17]

John M. Mellor-Crummey and Michael L. Scott. Algorithms for Scalable Synchronization on Shared-Memory Multiprocessors. A CM Transactions on Computer Systems, 9(1):21-65, February 1991.

Digital Library

[18]

Ton A. Ngo and Lawrence Snyder. On the Influence of Programming Models on Shared Memory Computer Performance, in Scalable High Performance Computing Conference (SHPCC '92), April 1992.

[19]

Steven K. Reinhardt, Mark D. Hill, James R. Larus, Alvin R. Lebeck, James C. Lewis, and David A. Wood. The Wisconsin Wind Thnnel: Virtual Prototyping of Parallel Computers. In Proceedings of the 1993 A CM Sigmetrics Conference on Measurement and Modeling of Computer Systems, pages 48-60, May 1993.

Digital Library

[20]

Steven K. Reinhardt, James R. Larus, and David A. Wood. Tempest and Typhoon: User-Level Shared Memory. In Proceedings of the 21st Annual International Symposium on Computer Architecture, pages 325-337, April 1994.

Digital Library

[21]

F. Traenkle. Parallel Programming Models and Boundary Integral Equation Methods for Microstructure Electrostatics. Master's thesis, University of Wisconsin-Madison, 1993.

[22]

Thorsten yon Eicken, David E. Culler, Seth Copen Goldstein, and Klaus Erik Schauser. Active Messages: a Mechanism for Integrating Communication and Computation. In Proceedings of the 19th Annual International Symposium on Computer Architecture, pages 256-266, May 1992.

Digital Library

[23]

Steven Cameron Woo, Jaswinder Pal Singh, and John L. Hennessy. The Performance Advantages of Integrating Message Passing in Cache-Coherent Multiprocessors. Technical Report CSL-TR-93-593, Department of Computer Science, Stanford University, November 1993. To appear in ASPLOS VI.

Digital Library

Cited By

Huh JLee D(2021)Effective On-Chip Communication for Message Passing Programs on Multi-Core ProcessorsElectronics10.3390/electronics1021268110:21(2681)Online publication date: 3-Nov-2021
https://doi.org/10.3390/electronics10212681
Aguilera MBen-David NCalciu IGuerraoui RPetrank EToueg SNewport CKeidar I(2018)Passing Messages while Sharing MemoryProceedings of the 2018 ACM Symposium on Principles of Distributed Computing10.1145/3212734.3212741(51-60)Online publication date: 23-Jul-2018
https://dl.acm.org/doi/10.1145/3212734.3212741
Cucinotta TLipari GSchubert L(2017)Operating System and Scheduling for Future Multicore and Many‐Core PlatformsProgramming multi‐core and many‐core computing systems10.1002/9781119332015.ch22(451-473)Online publication date: 27-Jan-2017
https://doi.org/10.1002/9781119332015.ch22
Show More Cited By

Index Terms

Where is time spent in message-passing and shared-memory programs?

Recommendations

Where is time spent in message-passing and shared-memory programs?

Message passing and shared memory are two techniques parallel programs use for coordination and communication. This paper studies the strengths and weaknesses of these two mechanisms by comparing equivalent, well-written message-passing and shared-...
Integrating message-passing and shared-memory: early experience
PPOPP '93: Proceedings of the fourth ACM SIGPLAN symposium on Principles and practice of parallel programming

This paper discusses some of the issues involved in implementing a shared-address space programming model on large-scale, distributed-memory multiprocessors. While such a programming model can be implemented on both shared-memory and message-passing ...
Performance of hybrid message-passing and shared-memory parallelism for discrete element modeling
SC '00: Proceedings of the 2000 ACM/IEEE conference on Supercomputing

The current trend in HPC hardware is towards clusters of shared-memory (SMP) compute nodes. For applications developers the major question is how best to program these SMP clusters. To address this we study an algorithm from Discrete Element Modeling, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASPLOS VI: Proceedings of the sixth international conference on Architectural support for programming languages and operating systems

November 1994

341 pages

ISBN:0897916603

DOI:10.1145/195473

Chairmen:
Forest Baskett
Silicon Graphics
,
Douglas Clark
Princeton Univ.

ACM SIGOPS Operating Systems Review Volume 28, Issue 5
Dec. 1994
323 pages
ISSN:0163-5980
DOI:10.1145/381792
Chairman:
Henry M. Levy
Univ. of Washington, Seattle
Issue’s Table of Contents
ACM SIGPLAN Notices Volume 29, Issue 11
Nov. 1994
323 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/195470
Editor:
Richard L. Wexelblat
Washington D.C.
Issue’s Table of Contents

Copyright © 1994 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 1994

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

ASPLOS94

Sponsor:

ASPLOS94: 6th Conference on Architectural Support of Programming Languages & Operating Systems

October 5 - 7, 1994

California, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 535 of 2,713 submissions, 20%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

67
Total Citations
View Citations
907
Total Downloads

Downloads (Last 12 months)100
Downloads (Last 6 weeks)16

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huh JLee D(2021)Effective On-Chip Communication for Message Passing Programs on Multi-Core ProcessorsElectronics10.3390/electronics1021268110:21(2681)Online publication date: 3-Nov-2021
https://doi.org/10.3390/electronics10212681
Aguilera MBen-David NCalciu IGuerraoui RPetrank EToueg SNewport CKeidar I(2018)Passing Messages while Sharing MemoryProceedings of the 2018 ACM Symposium on Principles of Distributed Computing10.1145/3212734.3212741(51-60)Online publication date: 23-Jul-2018
https://dl.acm.org/doi/10.1145/3212734.3212741
Cucinotta TLipari GSchubert L(2017)Operating System and Scheduling for Future Multicore and Many‐Core PlatformsProgramming multi‐core and many‐core computing systems10.1002/9781119332015.ch22(451-473)Online publication date: 27-Jan-2017
https://doi.org/10.1002/9781119332015.ch22
El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan K(2016)SpaceJMPACM SIGARCH Computer Architecture News10.1145/2980024.287236644:2(353-368)Online publication date: 25-Mar-2016
https://dl.acm.org/doi/10.1145/2980024.2872366
El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan K(2016)SpaceJMPACM SIGPLAN Notices10.1145/2954679.287236651:4(353-368)Online publication date: 25-Mar-2016
https://dl.acm.org/doi/10.1145/2954679.2872366
Torre DLabiche YGenero MElaasar MDas THoisl BKowal M(2016)1st International Workshop on UML Consistency Rules (WUCOR 2015)ACM SIGSOFT Software Engineering Notes10.1145/2894784.289480141:2(34-37)Online publication date: 11-May-2016
https://dl.acm.org/doi/10.1145/2894784.2894801
El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan KConte TZhou Y(2016)SpaceJMPProceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/2872362.2872366(353-368)Online publication date: 25-Mar-2016
https://dl.acm.org/doi/10.1145/2872362.2872366
Rompf TSujeeth ABrown KLee HChafi HOlukotun K(2014)Surgical precision JIT compilersACM SIGPLAN Notices10.1145/2666356.259431649:6(41-52)Online publication date: 9-Jun-2014
https://dl.acm.org/doi/10.1145/2666356.2594316
Shield JDiguet JGogniat G(2012)Asymmetric Cache CoherencyACM Transactions on Reconfigurable Technology and Systems10.1145/2362374.23623765:3(1-12)Online publication date: 1-Oct-2012
https://dl.acm.org/doi/10.1145/2362374.2362376
Shield JDiguet JGogniat G(2011)Asymmetric cache coherency: Improving multicore performance for non-uniform workloads6th International Workshop on Reconfigurable Communication-Centric Systems-on-Chip (ReCoSoC)10.1109/ReCoSoC.2011.5981491(1-8)Online publication date: Jun-2011
https://doi.org/10.1109/ReCoSoC.2011.5981491
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents