research-article

An Analysis of Scatter Decomposition

Authors:

David M. Nicol,

Joel H. SaltzAuthors Info & Claims

IEEE Transactions on Computers, Volume 39, Issue 11

Pages 1337 - 1345

https://doi.org/10.1109/12.61043

Published: 01 November 1990 Publication History

Abstract

A formal analysis of a powerful mapping technique known as scatter decomposition is provided. Scatter decomposition divides an irregular computational domain into a large number of equally sized pieces and distributes them modularly among processors. A probabilistic model of workload in one dimension is used to formally explain why and when scatter decomposition works. The first result is that if a correlation in workload is a convex function of distance, then scattering a more finely decomposed domain yields a lower average processor workload variance. The second result shows that if the workload process is a stationary Gaussian and the correlation function decreases linearly in distance until becoming zero and then remain zero, scattering a more finely decomposed domain yields a lower expected maximum processor workload. It is shown that if the correlation function decreases linearly across the entire domain, then among all mappings that assign an equal number of domain pieces to each processor, scatter decomposition minimizes the average processor workload variance. The dependence of these results on the assumption of decreasing correlation is illustrated with situations where a coarser granularity actually achieves better load balance.

References

[1]

{1} G. Fox, M. Johnson, G. Lyzenga, S. Otto, J. Salmon, and D. Walker, Solving Problems on Concurrent Computers. Englewood Cliffs, NJ: Prentice-Hall, 1988.

Digital Library

[2]

{2} G. A. Geist and M. T. Heath, "Matrix factorization on a hypercube multiprocessor," in Proc. Hypercube Microprocessors Conf., Knoxville, TN, Sept. 1986, pp. 161-180.

[3]

{3} P. Heidelberger, "Discrete-event simulations and parallel processing: Statistical properties," SIAM J. Scientif. Statistical Comput., vol. 9, no. 6, pp. 1114-1132, Nov. 1988.

Digital Library

[4]

{4} T. Hoshino, Pax Computer. Reading, MA: Addison-Wesley, 1989.

[5]

{5} I. Ipsen, Y. Saad, and M. H. Schultz, "Complexity of dense linear system solution on a multiprocessor ring," Lin. Algebra Appl., vol. 77, pp. 205-239, 1986.

[6]

{6} C. P. Kruscal and A. Weiss, "Allocating independent subtasks on parallel processors," IEEE Trans. Software Eng., vol. SE-11, no. 10, pp. 1001-1015, Oct. 1985.

Digital Library

[7]

{7} H. J. Larson and B. O. Shubert, Probabilistic Models in Engineering Sciences, Vol. 1. New York: Wiley, 1979.

[8]

{8} M. R. Leadbetter, G. Lindgren, and H. Rootzén, Extremes and Related Properties of Random Sequences and Processes. New York: Springer-Verlag, 1983.

[9]

{9} R. Morison and S. W. Otto, "The scattered decomposition for finite element problems," J. Scientif. Comput., vol. 2, no. 1, pp. 59-76, Mar. 1987.

Digital Library

[10]

{10} D. M. Nicol, "Mapping a battlefield simulation onto parallel message-passing architectures," in Proc. 1988 SCS Conf. Distributed Simulation, Feb. 1988, pp. 141-146.

[11]

{11} D. M. Nicol and P. F. Reynolds, Jr., "Optimal dynamic remapping of parallel computations," IEEE Trans. Comput., vol. 39, no. 2, pp. 206-219, Feb. 1990.

Digital Library

[12]

{12} R. S. Pindyck and D. L. Rubinfeld, Econometric Models and Economic Forecasts. New York: McGraw-Hill, 1976.

[13]

{13} H. S. Ross, Stochastic Processes. New York: Wiley, 1983.

[14]

{14} Y. Saad, "Communication complexity of the Gaussian elimination algorithm on multiprocessors," Lin. Algebra Appl., vol. 77, pp. 315-340, 1986.

[15]

{15} J. Salmon, "A mathematical analysis of the scattered decomposition," in Proc. Third Conf. Hypercube Concurrent Comput. Appl., Vol. 1, ACM, 1988, pp. 239-240.

Digital Library

[16]

{16} J. H. Saltz, V. K. Naik, and D. M. Nicol, "Reduction of the effects of the communication delays in scientific algorithms on message passing MIMD architectures," SIAM J. Sci. Statist. Comput, vol. 8, no. 1, p. s118, 1987.

Digital Library

[17]

{17} Y. Won and S. Sahni, "Maze routing on a hypercube multiprocessor computer," in Proc. 1987 Int. Conf. Parallel Processing , St. Charles, IL, Aug. 1987, pp. 630-637.

Cited By

Giordano ADe Rango ARongo RD’Ambrosio DSpataro W(2019)A Dynamic Load Balancing Technique for Parallel Execution of Structured Grid ModelsNumerical Computations: Theory and Algorithms10.1007/978-3-030-39081-5_25(278-290)Online publication date: 15-Jun-2019
https://dl.acm.org/doi/10.1007/978-3-030-39081-5_25
Thulasidasan SKasiviswanathan SEidenbenz SRomero P(2010)Explicit Spatial Scattering for Load Balancing in Conservatively Synchronized Parallel Discrete Event SimulationsProceedings of the 2010 IEEE Workshop on Principles of Advanced and Distributed Simulation10.1109/PADS.2010.5471664(150-158)Online publication date: 17-May-2010
https://dl.acm.org/doi/10.1109/PADS.2010.5471664
Yin GXu CWang L(2003)Optimal Remapping in Dynamic Bulk Synchronous Computations via a Stochastic Control ApproachIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2003.116737014:1(51-62)Online publication date: 1-Jan-2003
https://dl.acm.org/doi/10.1109/TPDS.2003.1167370
Show More Cited By

Index Terms

An Analysis of Scatter Decomposition

Recommendations

The Fast Hartley Transform Algorithm

The fast Hartley transform (FHT) is similar to the Cooley-Tukey fast Fourier transform (FFT) but performs much faster because it requires only real arithmetic computations compared to the complex arithmetic computations required by the FFT. Through use ...
New correlation analysis method for nonstationary signal

This paper proposes a new correlation analysis method for nonstationary and energy-limited continual signals, which works out a formula similar to time correlation theory; that is to say when the operator, ƒ, meets a certain condition, its auto-...
Similarity-Based Correlation Functions for Binary Data
Advances in Computational Intelligence
Abstract
The purpose of this study is to survey the correlation and association coefficients introduced previously on the set of binary n-tuples and to determine coefficients satisfying the properties of correlation functions. These functions were recently ...

Reviews

Reviewer: Andrzej P. Niemiec

Scatter decomposition is an intuitive new method of parallel computation for partial differential equations. The authors investigate the probabilistic characteristics of the processor workload. Their simplifying assumptions—a one-dimensional process, and stationary Gaussian workload with convex covariance—enable them to derive simple formulas for expected workload, workload variance, and covariance. Scatter decomposition is inherently probabilistic, which justifies their approach. The paper lacks even short explanations of the assumptions. For example, for what class of partial differential equations does the workload process have the assumed properties__?__ This technical paper takes an interesting approach to the analysis of algorithms. The reader should have a background in advanced probability and analysis of algorithms. While the paper is a bit long and the proofs are simple, the work is a good example of modern algorithm analysis.

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Computers

IEEE Transactions on Computers Volume 39, Issue 11

November 1990

102 pages

ISSN:0018-9340

Editor:
Ming T. Liu
Ohio State Univ., Columbus

Issue’s Table of Contents

Copyright © Copyright © 1990 IEEE. All Rights Reserved.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 November 1990

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Giordano ADe Rango ARongo RD’Ambrosio DSpataro W(2019)A Dynamic Load Balancing Technique for Parallel Execution of Structured Grid ModelsNumerical Computations: Theory and Algorithms10.1007/978-3-030-39081-5_25(278-290)Online publication date: 15-Jun-2019
https://dl.acm.org/doi/10.1007/978-3-030-39081-5_25
Thulasidasan SKasiviswanathan SEidenbenz SRomero P(2010)Explicit Spatial Scattering for Load Balancing in Conservatively Synchronized Parallel Discrete Event SimulationsProceedings of the 2010 IEEE Workshop on Principles of Advanced and Distributed Simulation10.1109/PADS.2010.5471664(150-158)Online publication date: 17-May-2010
https://dl.acm.org/doi/10.1109/PADS.2010.5471664
Yin GXu CWang L(2003)Optimal Remapping in Dynamic Bulk Synchronous Computations via a Stochastic Control ApproachIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2003.116737014:1(51-62)Online publication date: 1-Jan-2003
https://dl.acm.org/doi/10.1109/TPDS.2003.1167370
Yin GXu CWang L(2002)Optimal Remapping in Dynamic Bulk Synchronous Computations via a Stochastic Control ApproachProceedings of the 16th International Parallel and Distributed Processing Symposium10.5555/645610.661716Online publication date: 15-Apr-2002
https://dl.acm.org/doi/10.5555/645610.661716
Tomko KDavidson EYew P(1996)Profile driven weighted decompositionProceedings of the 10th international conference on Supercomputing10.1145/237578.237600(165-172)Online publication date: 1-Jan-1996
https://dl.acm.org/doi/10.1145/237578.237600
Grama AKumar VSameh AJohnson G(1994)Scalable parallel formulations of the barnes-hut method for n-body simulationsProceedings of the 1994 ACM/IEEE conference on Supercomputing10.5555/602770.602846(439-448)Online publication date: 14-Nov-1994
https://dl.acm.org/doi/10.5555/602770.602846
Varadarajan RHwang IBerghel HHlengl TUrban J(1994)An efficient dynamic load balancing algorithm for adaptive mesh refinementProceedings of the 1994 ACM symposium on Applied computing10.1145/326619.326814(467-472)Online publication date: 6-Apr-1994
https://dl.acm.org/doi/10.1145/326619.326814
Grama AKumar V(1992)Scalability analysis of partitioning strategies for finite element graphsProceedings of the 1992 ACM/IEEE conference on Supercomputing10.5555/147877.147912(83-92)Online publication date: 1-Dec-1992
https://dl.acm.org/doi/10.5555/147877.147912
Deo NMedidi MPrasad SCrain R(1992)Processor allocation in parallel battlefield simulationProceedings of the 24th conference on Winter simulation10.1145/167293.167690(718-725)Online publication date: 1-Dec-1992
https://dl.acm.org/doi/10.1145/167293.167690

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents