article

Free access

Where is time spent in message-passing and shared-memory programs?

Authors:

Satish Chandra,

James R. Larus,

Anne RogersAuthors Info & Claims

ACM SIGPLAN Notices, Volume 29, Issue 11

Pages 61 - 73

https://doi.org/10.1145/195470.195501

Published: 01 November 1994 Publication History

Abstract

Message passing and shared memory are two techniques parallel programs use for coordination and communication. This paper studies the strengths and weaknesses of these two mechanisms by comparing equivalent, well-written message-passing and shared-memory programs running on similar hardware. To ensure that our measurements are comparable, we produced two carefully tuned versions of each program and measured them on closely-related simulators of a message-passing and a shared-memory machine, both of which are based on same underlying hardware assumptions.

We examined the behavior and performance of each program carefully. Although the cost of computation in each pair of programs was similar, synchronization and communication differed greatly. We found that message-passing's advantage over shared-memory is not clear-cut. Three of the four shared-memory programs ran at roughly the same speed as their message-passing equivalent, even though their communication patterns were different.

References

[1]

Anant Agarwal, Richard Simoni, Mark Horowitz, and John Hennessy. An Evaluation of Directory Schemes for Cache Coherence. In Proceedings of the 15th Annual Internatwnal Symposium on Computer Architecture, pages 280-289, 1988.

Digital Library

[2]

Matthias A. Blumrich, Kai Li, Richard Alpert, Cezary Dubnicki, Edward W. Fetten, and Jonathon Sandberg. Virtual Memory Mapped Network Interface for the SHRIMP Multicomputer. In Proceedings of the 21st Annual International Symposium on Computer Architecture, pages 142-153, April 1994.

Digital Library

[3]

D. E. Culler, A. Dusseau, S. C. Goldstein, A. Krishnamurthy, S. Lumetta, T. von Eicken, and K. Yelick. Parallel Programming in Split-C. In Proceedings of Supercomputing 93, pages 262-273, November 1993.

Digital Library

[4]

David Culler, Richard Karp, David Patterson, Abhijit Sahay, Klaus Erik Schauser, Eunice Santos, Ramesh Subramonian, and Thorsten von Eicken. LogP: Toward a Realistic Model of Parallel Computation. In Fifth A CM SIG- PLAN Symposium on Principles ~4 Practice of Parallel Programming (PPOPP), pages 1-12, May 1993.

Digital Library

[5]

Phillip M. Dickens, Philip Heidelberger, and David M. Nicol. A Distributed Memory LAPSE: Parallel Simulation of Message-Passing Programs. In Proceedings of the 8th Workshop on Parallel and D~stributed Simulation (PADS '94), pages 32-38, July 1994.

Digital Library

[6]

Babak Falsafi, Alvin Lebeck, Steven Reinhardt, Ioannis Schoinas, Mark D. Hill, James Larus, Anne Rogers, and David Wood. Application-Specific Protocols for User- Level Shared Memory. In Proceedings of Supercomputmg 9J, November 1994. To appear.

Digital Library

[7]

Edward W. Felten. Protocol Compilation: High- Performance Communication for Parallel Programs. Technical Report 93-09-09, Department of Computer Science, University of Washington, September 1993.

[8]

Mark D. Hill, James R. Larus, Steven K. Reinhardt, and David A. Wood. Cooperative Shared Memory: Software and Hardware for Scalable Multiprocessors. A CM Transactions on Computer Systems, 11(4):300-318, November 1993. Earlier version appeared in ASPLOS V, Oct. 1992.

Digital Library

[9]

W. Daniel Hillis and Lewis W. ~cker. The CM-5 Connection Machine: A Scalable Supercomputer. Communications of the A CM, 36(11):31-40, November 1993.

Digital Library

[10]

Intel Corporation. Paragon Technical Summary. Intel Supercomputer Systems Division, 1993.

[11]

Alexander C. KIaiber and Henry M. Levy. A Comparison of Message Passing and Shared Memory Architectures for Data Parallel Programs. In Proceedings of the 21st Annual International Symposium on Computer Architecture, pages 94-105, April 1994.

Digital Library

[12]

David Kranz, Kirk Johnson, Anent Agarwal, John Kubiatowicz, and Beng-Hong Lim. Integrating Message-Passing and Shared-Memory: Early Experience. In Fifth A CM SIGPLAN Symposium on Principles FJ Practice of Parallel Programming (PPOPP), pages 54-63, May 1993.

Digital Library

[13]

Thomas J. LeBlanc and Evangelos P. Markatos. Shared Memory Vs. Message Passing in Shared-Memory Multiprocessors. In Fourth IEEE Symposium on Parallel and Distributed Processing, Dallas, TX, December 1992.

[14]

R. De Leone, O.L. Mangasarian, and T.-H. Shiau. Multi- Sweep Asynchronous Parallel Successive Overrelaxation for the Nonsymmetric Linear Complementarity Problem. Annals of Operations Research, 22:43-54, 1990.

Digital Library

[15]

Calvin Lin and Lawrence Snyder. A Comparison of Programming Models for Shared Memory Multiprocessors. in Proceedings of the 1990 International Conference on Parallel Processing (Vol. H Software), pages ii-163-170, August 1990.

[16]

Margaret Martonosi and Anoop Gupta. Tradeoffs in Message Passing and Shared Memory Implementations of a Standard Cell Router. In Proceedings of the 1989 International Conference on Parallel Processing (Vol. III Algorithms and Applications), pages III88-96, August 1989.

[17]

John M. Mellor-Crummey and Michael L. Scott. Algorithms for Scalable Synchronization on Shared-Memory Multiprocessors. A CM Transactions on Computer Systems, 9(1):21-65, February 1991.

Digital Library

[18]

Ton A. Ngo and Lawrence Snyder. On the Influence of Programming Models on Shared Memory Computer Performance, in Scalable High Performance Computing Conference (SHPCC '92), April 1992.

[19]

Steven K. Reinhardt, Mark D. Hill, James R. Larus, Alvin R. Lebeck, James C. Lewis, and David A. Wood. The Wisconsin Wind Thnnel: Virtual Prototyping of Parallel Computers. In Proceedings of the 1993 A CM Sigmetrics Conference on Measurement and Modeling of Computer Systems, pages 48-60, May 1993.

Digital Library

[20]

Steven K. Reinhardt, James R. Larus, and David A. Wood. Tempest and Typhoon: User-Level Shared Memory. In Proceedings of the 21st Annual International Symposium on Computer Architecture, pages 325-337, April 1994.

Digital Library

[21]

F. Traenkle. Parallel Programming Models and Boundary Integral Equation Methods for Microstructure Electrostatics. Master's thesis, University of Wisconsin-Madison, 1993.

[22]

Thorsten yon Eicken, David E. Culler, Seth Copen Goldstein, and Klaus Erik Schauser. Active Messages: a Mechanism for Integrating Communication and Computation. In Proceedings of the 19th Annual International Symposium on Computer Architecture, pages 256-266, May 1992.

Digital Library

[23]

Steven Cameron Woo, Jaswinder Pal Singh, and John L. Hennessy. The Performance Advantages of Integrating Message Passing in Cache-Coherent Multiprocessors. Technical Report CSL-TR-93-593, Department of Computer Science, Stanford University, November 1993. To appear in ASPLOS VI.

Digital Library

Cited By

El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan K(2016)SpaceJMPACM SIGARCH Computer Architecture News10.1145/2980024.287236644:2(353-368)Online publication date: 25-Mar-2016
https://dl.acm.org/doi/10.1145/2980024.2872366
El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan K(2016)SpaceJMPACM SIGOPS Operating Systems Review10.1145/2954680.287236650:2(353-368)Online publication date: 25-Mar-2016
https://doi.org/10.1145/2954680.2872366
El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan K(2016)SpaceJMPACM SIGPLAN Notices10.1145/2954679.287236651:4(353-368)Online publication date: 25-Mar-2016
https://dl.acm.org/doi/10.1145/2954679.2872366
Show More Cited By

Recommendations

A scalable multi-producer multi-consumer wait-free ring buffer
SAC '15: Proceedings of the 30th Annual ACM Symposium on Applied Computing

A ring buffer or cyclical queue is a First In, First Out (FIFO) queue that stores elements on a fixed-length array. This allows for efficient O(1) operations, cache-aware optimizations, and low memory overhead. Because ring buffers are limited to only ...
A wait-free multi-producer multi-consumer ring buffer

The ring buffer is a staple data structure used in many algorithms and applications. It is highly desirable in high-demand use cases such as multimedia, network routing, and trading systems. This work presents a new ring buffer design that is, to the ...
The many faces of publish/subscribe

Well adapted to the loosely coupled nature of distributed interaction in large-scale applications, the publish/subscribe communication paradigm has recently received increasing attention. With systems based on the publish/subscribe interaction scheme, ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGPLAN Notices

ACM SIGPLAN Notices Volume 29, Issue 11

Nov. 1994

323 pages

ISSN:0362-1340

EISSN:1558-1160

DOI:10.1145/195470

Editor:
Richard L. Wexelblat
Washington D.C.

Issue’s Table of Contents

ASPLOS VI: Proceedings of the sixth international conference on Architectural support for programming languages and operating systems
November 1994
341 pages
ISBN:0897916603
DOI:10.1145/195473
Chairmen:
Forest Baskett
Silicon Graphics
,
Douglas Clark
Princeton Univ.

Copyright © 1994 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 1994

Published in SIGPLAN Volume 29, Issue 11

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

67
Total Citations
View Citations
907
Total Downloads

Downloads (Last 12 months)100
Downloads (Last 6 weeks)16

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan K(2016)SpaceJMPACM SIGARCH Computer Architecture News10.1145/2980024.287236644:2(353-368)Online publication date: 25-Mar-2016
https://dl.acm.org/doi/10.1145/2980024.2872366
El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan K(2016)SpaceJMPACM SIGOPS Operating Systems Review10.1145/2954680.287236650:2(353-368)Online publication date: 25-Mar-2016
https://doi.org/10.1145/2954680.2872366
El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan K(2016)SpaceJMPACM SIGPLAN Notices10.1145/2954679.287236651:4(353-368)Online publication date: 25-Mar-2016
https://dl.acm.org/doi/10.1145/2954679.2872366
El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan KConte TZhou Y(2016)SpaceJMPProceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/2872362.2872366(353-368)Online publication date: 25-Mar-2016
https://dl.acm.org/doi/10.1145/2872362.2872366
Talbot SKelly P(2000)Adaptive Proxies: Handling Widely-Shared Data in Shared-Memory MultiprocessorsEuro-Par 2000 Parallel Processing10.1007/3-540-44520-X_77(567-572)Online publication date: 18-Aug-2000
https://doi.org/10.1007/3-540-44520-X_77
Huh JLee D(2021)Effective On-Chip Communication for Message Passing Programs on Multi-Core ProcessorsElectronics10.3390/electronics1021268110:21(2681)Online publication date: 3-Nov-2021
https://doi.org/10.3390/electronics10212681
Aguilera MBen-David NCalciu IGuerraoui RPetrank EToueg SNewport CKeidar I(2018)Passing Messages while Sharing MemoryProceedings of the 2018 ACM Symposium on Principles of Distributed Computing10.1145/3212734.3212741(51-60)Online publication date: 23-Jul-2018
https://dl.acm.org/doi/10.1145/3212734.3212741
Cucinotta TLipari GSchubert L(2017)Operating System and Scheduling for Future Multicore and Many‐Core PlatformsProgramming multi‐core and many‐core computing systems10.1002/9781119332015.ch22(451-473)Online publication date: 27-Jan-2017
https://doi.org/10.1002/9781119332015.ch22
El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan K(2016)SpaceJMPACM SIGARCH Computer Architecture News10.1145/2980024.287236644:2(353-368)Online publication date: 25-Mar-2016
https://dl.acm.org/doi/10.1145/2980024.2872366
El Hajj IMerritt AZellweger GMilojicic DAchermann RFaraboschi PHwu WRoscoe TSchwan K(2016)SpaceJMPACM SIGPLAN Notices10.1145/2954679.287236651:4(353-368)Online publication date: 25-Mar-2016
https://dl.acm.org/doi/10.1145/2954679.2872366
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents