research-article

Scalability of Parallel Algorithm-Machine Combinations

Authors:

D. T. RoverAuthors Info & Claims

IEEE Transactions on Parallel and Distributed Systems, Volume 5, Issue 6

Pages 599 - 613

https://doi.org/10.1109/71.285606

Published: 01 June 1994 Publication History

Abstract

Scalability has become an important consideration in parallel algorithm and machinedesigns. The word scalable, or scalability, has been widely and often used in the parallelprocessing community. However, there is no adequate, commonly accepted definition ofscalability available. Scalabilities of computer systems and programs are difficult toquantify, evaluate, and compare. In this paper, scalability is formally defined foralgorithm-machine combinations. A practical method is proposed to provide a quantitative measurement of the scalability. The relation between the newly proposed scalability and other existing parallel performance metrics is studied. A harmony between speedup and scalability has been observed. Theoretical results show that a large class ofalgorithm-machine combinations is scalable and the scalability can be predicted throughpremeasured machine parameters. Two algorithms have been studied on an nCUBE 2multicomputer and on a MasPar MP-1 computer. These case studies have shown howscalabilities can be measured, computed, and predicted. Performance instrumentation andvisualization tools also have been used and developed to understand the scalabilityrelated behavior.

References

[1]

{1} J. Gustafson, G. Montry, and R. Benner, "Development of parallel methods for a 1024-processor hypercube," SIAM J. SSTC, vol. 9, July 1988.

[2]

{2} J. Gustafson, "Reevaluating Amdahl's law," Commun. ACM, vol. 31, pp. 532-533, May 1988.

Digital Library

[3]

{3} V. Kumar and A. Gupta, "Analysis of scalability of parallel algorithms and architectures: A survey," in Proc. Int. Conf. on Supercomputing, June 1991.

Digital Library

[4]

{4} V. Kumar and V. Singh, "Scalability of parallel algorithms for the allpairs shortest path problem: A summary of results," in Proc. of Conf. on Parallel Processing, Chicago, IL, 1990, pp. III 136-140.

[5]

{5} M. Willebeek-LeMair, A. P. Reeves, and C. H. Ning, "Characterization of multicomputer system: a transfer ration approach," in Proc. Int. Conf. on Parallel Processing, Chicago, IL, 1990, pp. II 171-178.

[6]

{6} D. Nussbaum and A. Agarwal, "Scalability of parallel machines," Commun. the ACM, vol. 34, no. 3, pp. 57-61, 1991.

Digital Library

[7]

{7} D. Rover, V. Tsai, Y. Chow, and J. Gustafson, "Signal processing algorithms on parallel architectures: A performance update," J. Parallel Distrib. Computing, vol. 13, pp. 237-245, Nov. 1991.

Digital Library

[8]

{8} C. Goral, K. Torrance, D. Greenberg, and B. Battaile, "Modeling the interaction of light between diffuse surfaces," Comput. Graphics, vol. 18, pp. 213-222, July 1984.

Digital Library

[9]

{9} X.-H. Sun and L. Ni, "Another view on parallel speedup," in Proc. of Supercomputing'90, NY, 1990, pp. 324-333.

[10]

{10} X.-H. Sun and L. Ni, "Scalable problems and memory-bounded speedup," J. Parallel Distrib. Computing, vol. 19, pp. 27-37, Sept. 1993.

Digital Library

[11]

{11} G. Amdahl, "Validity of the single-processor approach to achieving large scale computing capabilities," in Proc. AFIPS Conf., 1967, pp. 483-485.

[12]

{12} X.-H. Sun, L. Ni, F. Salam, and S. Guo, "Compute-exchange computation for solving power flow problems: The model and application," in Proc. 4th SIAM Conf. Parallel Processing for Scientific Computing, 1989, pp. 198-203.

[13]

{13} R. W. Hackney, "Parametrization of computer performance," Parellel Computing, vol. 5, 1987.

[14]

{14} X.-H. Sun and J. Gustafson, "Toward a better parallel performance metric," Parallel Computing, vol. 17, Dec. 1991.

[15]

{15} J. Gustafson, D. Rover, S. Elbert, and M. Carter, "The design of a scalable, fixed-time computer benchmark," J. Parallel Distrib. Computing, vol. 11, Aug. 1991.

[16]

{16} G. Geist, M. Heath, B. Peyton, and P. Worley, "A machine-independent communication library," in Proc. 4th Conf. Hypercubes, Concurrent Computers, and Applications, 1989, pp. 565-568.

[17]

{17} M. T. Heath and J. A. Etheridge, "Visualizing the performance of parallel programs," IEEE Software, vol. 8, pp. 29-39, Sept. 1991.

Digital Library

Cited By

Sun XLu X(2023)The Memory-Bounded Speedup Model and Its Impacts in ComputingJournal of Computer Science and Technology10.1007/s11390-022-2911-138:1(64-79)Online publication date: 31-Jan-2023
https://dl.acm.org/doi/10.1007/s11390-022-2911-1
Li YOu DZhou XJiang CCérin C(2022)Scalability and performance analysis of BDPS in cloudsComputing10.1007/s00607-022-01056-7104:6(1425-1460)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s00607-022-01056-7
Jiang QLee YZomaya A(2020)The Limit of Horizontal Scaling in Public CloudsACM Transactions on Modeling and Performance Evaluation of Computing Systems10.1145/33733565:1(1-22)Online publication date: 4-Feb-2020
https://dl.acm.org/doi/10.1145/3373356
Show More Cited By

Recommendations

Mapping onto three classes of parallel machines: a case study using the cyclic reduction algorithm
IPPS '93: Proceedings of the 1993 Seventh International Parallel Processing Symposium

Mapping cyclic reduction, a known approach for the parallel solution of tridiagonal systems of equations, onto the MasPar MP-1, nCUBE 2, and PASM parallel machines is discussed. Each of these represents a different mode of parallelism. Issues addressed ...
Parallel Image Correlation: Case Study to Examine Trade-Offs in Algorithm-to-Machine Mappings

Performance of a parallel algorithm on a parallel machine depends not only on the time complexity of the algorithm, but also on how the underlying machine supports the fundamental operations used by the algorithm. This study analyzes various mappings of ...
Implementing a 3D multigrid algorithm on Fujitsu's vector parallel supercomputer
PAS '95: Proceedings of the First Aizu International Symposium on Parallel Algorithms/Architecture Synthesis

Multigrid algorithms are well adopted by many engineering applications programs on the traditional sequential computers due to their good convergence behavior. In order to convert those existing resources to the field of parallel computing in a timely ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Parallel and Distributed Systems

IEEE Transactions on Parallel and Distributed Systems Volume 5, Issue 6

June 1994

115 pages

ISSN:1045-9219

Issue’s Table of Contents

Copyright © Copyright © 1994 IEEE. All Rights Reserved.

Publisher

IEEE Press

Publication History

Published: 01 June 1994

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 29 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sun XLu X(2023)The Memory-Bounded Speedup Model and Its Impacts in ComputingJournal of Computer Science and Technology10.1007/s11390-022-2911-138:1(64-79)Online publication date: 31-Jan-2023
https://dl.acm.org/doi/10.1007/s11390-022-2911-1
Li YOu DZhou XJiang CCérin C(2022)Scalability and performance analysis of BDPS in cloudsComputing10.1007/s00607-022-01056-7104:6(1425-1460)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s00607-022-01056-7
Jiang QLee YZomaya A(2020)The Limit of Horizontal Scaling in Public CloudsACM Transactions on Modeling and Performance Evaluation of Computing Systems10.1145/33733565:1(1-22)Online publication date: 4-Feb-2020
https://dl.acm.org/doi/10.1145/3373356
Menouer TKhedimi ACérin CJiang C(2020)Cloud Allocation and Consolidation Based on a Scalability MetricAlgorithms and Architectures for Parallel Processing10.1007/978-3-030-60248-2_26(381-395)Online publication date: 2-Oct-2020
https://dl.acm.org/doi/10.1007/978-3-030-60248-2_26
Karakus MDurresi A(2019)An economic framework for analysis of network architecturesJournal of Network and Computer Applications10.1016/j.jnca.2019.02.032136:C(132-146)Online publication date: 15-Jun-2019
https://dl.acm.org/doi/10.1016/j.jnca.2019.02.032
Karakus MDurresi A(2018)Economic Viability of Software Defined Networking (SDN)Computer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2018.02.015135:C(81-95)Online publication date: 22-Apr-2018
https://dl.acm.org/doi/10.1016/j.comnet.2018.02.015
Budish ECachon GKessler JOthman A(2017)Course MatchOperations Research10.1287/opre.2016.154465:2(314-336)Online publication date: 1-Apr-2017
https://dl.acm.org/doi/10.1287/opre.2016.1544
Karakus MDurresi A(2017)A surveyComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2016.11.017112:C(279-293)Online publication date: 15-Jan-2017
https://dl.acm.org/doi/10.1016/j.comnet.2016.11.017
Zeng GLiu W(2017)An iso-time scaling method for big data tasks executing on parallel computing systemsThe Journal of Supercomputing10.1007/s11227-017-2029-373:10(4493-4516)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1007/s11227-017-2029-3
Huang CChen CLai C(2016)Evaluation and analysis of incorporating Fuzzy Expert System approach into test suite reductionInformation and Software Technology10.1016/j.infsof.2016.07.00579:C(79-105)Online publication date: 1-Nov-2016
https://dl.acm.org/doi/10.1016/j.infsof.2016.07.005
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents