article

Accurate data redistribution cost estimation in software distributed shared memory systems

Authors:

Donald G. Morris,

David K. LowenthalAuthors Info & Claims

ACM SIGPLAN Notices, Volume 36, Issue 7

Pages 62 - 71

https://doi.org/10.1145/568014.379570

Published: 18 June 2001 Publication History

Abstract

Distributing data is one of the key problems in implementing efficient distributed-memory parallel programs. The problem becomes more difficult in programs where data redistribution between computational phases is considered. The global data distribution problem is to find the optimal distribution in multi-phase parallel programs. Solving this problem requires accurate knowledge of data redistribution cost.

We are investigating this problem in the context of a software distributed shared memory (SDSM) system, in which obtaining accurate redistribution cost estimates is difficult. This is because SDSM communication is implicit: It depends on access patterns, page locations, and the SDSM consistency protocol.

We have developed integrated compile- and run-time analysis for SDSM systems to determine accurate redistribution cost estimates with low overhead. Our resulting system, SUIF-Adapt, can efficiently and accurately estimate execution time, including redistribution, to within 5% of the actual time in all of our test cases and is often much closer. These precise costs enable SUIF-Adapt to find efficient global data distributions in multiple-phase programs.

References

[1]

J. Anderson and M. Lam. Global optimizations for parallelism and locality on scalable parallel machines. In Proceedings of the SIGPLAN '93 Conference on Program Language Design and Implementation, pages 112-125, June 1993.]]

Digital Library

[2]

J. B. Carter, J. K. Bennett, and W. Zwaenepoel. Implementation and performance of Munin. In Proceedings of 13th ACM Symposium On Operating Systems, pages 152-164, Oct. 1991.]]

Digital Library

[3]

S. Chandra and J. R. Larus. Optimizing communication in HPF programs on fine-grain distributed shared memory. InSixth Symposium on Principles and Practice ofParallel Programming, pages 100-111, June 1997.]]

Digital Library

[4]

M. W. Hall, J. M. Anderson, S. P. Amarasinghe, B. R. Murphy, S.-W. Liao, E. Bugnion, and M. S. Lam. Maximizing multiprocessor performance with the SUIF compiler. IEEE Computer, 29(12):84-89, Dec. 1996.]]

Digital Library

[5]

P. Havlak and K. Kennedy. An implementation of interprocedural bounded regular section analysis. IEEE Transactions on Parallel and Distributed Systems, 2(3):350-360, 1991.]]

Digital Library

[6]

G. M. Howard and D. K. Lowenthal. An integrated compiler/run-time system for global data distribution in distributed shared memory systems. In Second Workshop on Software DSM, May 2000.]]

[7]

L. Iftode. Home-Based Shared Virtual Memory. PhD thesis, Princeton University, June 1998.]]

Digital Library

[8]

S. Ioannidis and S. Dwarkadas. Compiler and run-time support for adaptive load balancing in software distributed shared memory systems. In Proceedings of the Fourth Workshop on Languages, Compilers, and Run-Time Systems for Parallel Computing, pages 107-122, May 1998.]]

Digital Library

[9]

P. Keleher, S. Dwarkadas, A. Cox, and W. Zwaenepoel. TreadMarks: Distributed shared memory on standard workstations and operating systems. In Proceedings of the 1994 Winter Usenix Conference, pages 115-131, Jan. 1994.]]

Digital Library

[10]

K. Kennedy and U. Kremer. Automatic data layout for distributed-memory machines. ACM TOPLAS, 20(4):869-916, 1998.]]

Digital Library

[11]

K. Li and P. Hudak. Memory coherence in shared virtual memory systems. ACM Transactions on Computer Systems, 7(4), Nov. 1989.]]

Digital Library

[12]

D. K. Lowenthal and G. R. Andrews. An adaptive approach to data placement. In Proceedings of the 10th International Symposium on Parallel Processing, pages 349-353, Apr. 1996.]]

Digital Library

[13]

D. K. Lowenthal, V. W. Freeh, and G. R. Andrews. Using fine-grain threads and run-time decision making in parallel computing. Journal of Parallel and Distributed Computing, 37:41-54, Nov. 1996.]]

Digital Library

[14]

D. K. Lowenthal and F. Lowenthal. Supporting regular data distributions on nondedicated parallel machines (submitted to Supercomputing '01). May 2001.]]

[15]

H.Lu, A. L.Cox, S. Dwarkadas, R. Rajamony, and W. Zwaenepoel. Compiler and distributed shared memory support for irregular applications. In Sixth Symposium on Principles and Practice of Parallel Programming, pages 48-56, June 1997.]]

Digital Library

[16]

U. Rencuzogullari and S. Dwarkadas. Dynamic adaptation to available resources for parallel computing in an autonomous network of workstations. In Eighth Conference on Principles and Practice of Parallel Programming (to appear), June 2001.]]

Digital Library

[17]

C.-W. Tseng. An Optimizing Fortran D Compiler for MIMD Distributed-Memory Machines. PhD thesis, Rice University, Jan. 1993.]]

Digital Library

[18]

K. Zhang, J. Mellor-Crummey, and R. J. Fowler. Compilation and runtime optimizations for software distributed shared memory. InFifth Workshop on Languages, Compilers, and Run-Time Systems for Scalable Computers, pages83-88, May 2000.]]

Digital Library

Cited By

Weatherly DLowenthal DNakazawa MLowenthal F(2006)Dyn-MPIJournal of Parallel and Distributed Computing10.1016/j.jpdc.2006.02.00266:6(822-838)Online publication date: 1-Jun-2006
https://dl.acm.org/doi/10.1016/j.jpdc.2006.02.002
Freeh VKappiah NLowenthal DBletsch T(2008)Just-in-time dynamic voltage scalingJournal of Parallel and Distributed Computing10.1016/j.jpdc.2008.04.00768:9(1175-1185)Online publication date: 1-Sep-2008
https://dl.acm.org/doi/10.1016/j.jpdc.2008.04.007
Weatherly DLowenthal DNakazawa MLowenthal F(2006)Dyn-MPIJournal of Parallel and Distributed Computing10.1016/j.jpdc.2006.02.00266:6(822-838)Online publication date: 1-Jun-2006
https://dl.acm.org/doi/10.1016/j.jpdc.2006.02.002
Show More Cited By

Index Terms

Accurate data redistribution cost estimation in software distributed shared memory systems

Recommendations

Accurate data redistribution cost estimation in software distributed shared memory systems
PPoPP '01: Proceedings of the eighth ACM SIGPLAN symposium on Principles and practices of parallel programming

Distributing data is one of the key problems in implementing efficient distributed-memory parallel programs. The problem becomes more difficult in programs where data redistribution between computational phases is considered. The global data ...
Compiling shared-memory applications for distributed-memory systems
Scalable data redistribution services for distributed-memory machines

Comments

Information & Contributors

Information

Published In

cover image ACM SIGPLAN Notices

ACM SIGPLAN Notices Volume 36, Issue 7

July 2001

143 pages

ISSN:0362-1340

EISSN:1558-1160

DOI:10.1145/568014

Issue’s Table of Contents

PPoPP '01: Proceedings of the eighth ACM SIGPLAN symposium on Principles and practices of parallel programming
June 2001
142 pages
ISBN:1581133464
DOI:10.1145/379539
Chairmen:
Michael Heath
Univ. of Illinois, Illinois, IN
,
Andrew Lumsdaine
Indiana Univ.

Copyright © 2001 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2001

Published in SIGPLAN Volume 36, Issue 7

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
405
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Weatherly DLowenthal DNakazawa MLowenthal F(2006)Dyn-MPIJournal of Parallel and Distributed Computing10.1016/j.jpdc.2006.02.00266:6(822-838)Online publication date: 1-Jun-2006
https://dl.acm.org/doi/10.1016/j.jpdc.2006.02.002
Freeh VKappiah NLowenthal DBletsch T(2008)Just-in-time dynamic voltage scalingJournal of Parallel and Distributed Computing10.1016/j.jpdc.2008.04.00768:9(1175-1185)Online publication date: 1-Sep-2008
https://dl.acm.org/doi/10.1016/j.jpdc.2008.04.007
Weatherly DLowenthal DNakazawa MLowenthal F(2006)Dyn-MPIJournal of Parallel and Distributed Computing10.1016/j.jpdc.2006.02.00266:6(822-838)Online publication date: 1-Jun-2006
https://dl.acm.org/doi/10.1016/j.jpdc.2006.02.002
Kwok YMaciejewski ASiegel HAhmad IGhafoor A(2006)A semi-static approach to mapping dynamic iterative tasks onto heterogeneous computing systemsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2005.06.01566:1(77-98)Online publication date: 1-Jan-2006
https://dl.acm.org/doi/10.1016/j.jpdc.2005.06.015
Freeh VLowenthal DPingali KYelick KGrimshaw A(2005)Using multiple energy gears in MPI programs on a power-scalable clusterProceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming10.1145/1065944.1065967(164-173)Online publication date: 15-Jun-2005
https://dl.acm.org/doi/10.1145/1065944.1065967
Nakazawa MLowenthal DZhou WKramer W(2005)The MHETA Execution Model for Heterogeneous ClustersProceedings of the 2005 ACM/IEEE conference on Supercomputing10.1109/SC.2005.73Online publication date: 12-Nov-2005
https://dl.acm.org/doi/10.1109/SC.2005.73
Kappiah NFreeh VLowenthal DKramer W(2005)Just In Time Dynamic Voltage ScalingProceedings of the 2005 ACM/IEEE conference on Supercomputing10.1109/SC.2005.39Online publication date: 12-Nov-2005
https://dl.acm.org/doi/10.1109/SC.2005.39
Weatherly DLowenthal DNakazawa MLowenthal FMcGraw J(2003)Dyn-MPIProceedings of the 2003 ACM/IEEE conference on Supercomputing10.1145/1048935.1050156Online publication date: 15-Nov-2003
https://dl.acm.org/doi/10.1145/1048935.1050156
Rencuzogullari UDwardadas S(2001)Dynamic adaptation to available resources for parallel computing in an autonomous network of workstationsACM SIGPLAN Notices10.1145/568014.37957236:7(72-81)Online publication date: 18-Jun-2001
https://dl.acm.org/doi/10.1145/568014.379572
Rencuzogullari UDwardadas SHeath MLumsdaine A(2001)Dynamic adaptation to available resources for parallel computing in an autonomous network of workstationsProceedings of the eighth ACM SIGPLAN symposium on Principles and practices of parallel programming10.1145/379539.379572(72-81)Online publication date: 18-Jun-2001
https://dl.acm.org/doi/10.1145/379539.379572

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents