Measuring the scalability of heterogeneous parallel systems
Pages 1066 - 1073
Abstract
A parallel algorithm cannot be evaluated apart from the architecture it is implemented on. So, we define a parallel system as the combination of a parallel algorithm and a parallel architecture. The paper is devoted to the extension of well-known isoefficiency scalability metrics to heterogeneous parallel systems. Based on this extension the scalability of SUMMA (Scalable Universal Matrix Multiplication Algorithm) on parallel architecture with homogeneous communication system supporting simultaneous point-to-point communications is evaluated. Two strategies of data distribution are considered: (i) homogeneous – data are distributed between processors evenly; (ii) data are distributed between processors according to their performance. It is shown that under some assumption both strategies ensure the same scalability of heterogeneous parallel system. This theoretical result is corroborated with experiment.
References
[1]
A. Grama, A. Gupta and V. Kumar. Isoefficiency: Measuring the Scalability of Parallel Algorithms and Architectures, IEEE Parallel & Distributed Technology, 1993, Vol.1, No. 3, pp. 12-21
[2]
R. van de Geijn and J. Watts. SUMMA: Scalable Universal Matrix Multiplication Algorithm. Concurrency: Practice and Experience, 9(4) 255-274 (1997).
[3]
A. Kalinov. Heterogeneous two-dimensional block-cyclic data distribution for solving linear algebra problems on heterogeneous networks of computers. Programming and Computer Software, Vol. 25, No. 2, 1999, pp. 3-11. Translated from Programmirovanie, Vol 25, No.2
[4]
M. Quinn. Parallel Programming in C with MPI and OpenMP, McGraw-Hill, 2004
[5]
L. Pastor and J. L. Bosque, An efficiency and scalability model for heterogeneous clusters. Proceedings of Cluster 2001, 8-11 October 2001, Newport Beach, CA, USA. IEEE Computer Society pp.427-434
[6]
A. Kalinov, Scalability Analysis of Matrix-Matrix Multiplication on Heterogeneous Clusters, Proceedings of 3rd ISPDC/HeteroPar'04, Cork, Ireland, July 05 - 07, 2004, IEEE CS Press, pp. 303-309
[7]
J. Dongarra, R. van de Geun, and D. Walker. Scalability Issues Affecting the Design of a Dense Linear Algebra Library, Journal of Parallel and Distributed Computing, 22, 523-537, 1994
[8]
Olivier Beaumont, Vincent Boudet, Antoine Petitet, Fabrice Rastello, and Yves Robert. A Proposal for a Heterogeneous Cluster ScaLAPACK (Dense Linear Solvers). IEEE Trans. Computers, 2001, Vol.50, No.10, pp.1052-1070
[9]
E. Dovolnov, A. Kalinov, and S. Klimov, Natural Block Data Decomposition for Heterogeneous Clusters, Proceedings of 17th International Parallel and Distributed Processing Symposium, IEEE CS, Nice, France, April 2003, CD-ROM
Index Terms
- Measuring the scalability of heterogeneous parallel systems
Index terms have been assigned to the content through auto-classification.
Recommendations
Scalability of Parallel Algorithm-Machine Combinations
Scalability has become an important consideration in parallel algorithm and machinedesigns. The word scalable, or scalability, has been widely and often used in the parallelprocessing community. However, there is no adequate, commonly accepted ...
A Heterogeneous Mixed-Mode Execution Model for Massively Parallel Systems
In this paper, we consider a massively parallel system that is composed of heterogeneous processors, that is, processors with different processing power, and that combines the advantages of the SIMD and MIMD architectures. The heterogeneous mixed-mode (...
Measuring the scalability of parallel computer systems
Supercomputing '89: Proceedings of the 1989 ACM/IEEE conference on SupercomputingThis paper discusses scalability and outlines a specific approach to measuring the scalability of parallel computer systems. The relationship between scalability and speedup is described. It is shown that a parallel system is scalable for a given ...
Comments
Information & Contributors
Information
Published In
September 2005
1121 pages
ISBN:3540341412
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 11 September 2005
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 29 Jul 2024