research-article

Partitioning Techniques for Large-Grained Parallelism

Authors:

H. V. JagadishAuthors Info & Claims

IEEE Transactions on Computers, Volume 37, Issue 12

Pages 1627 - 1634

https://doi.org/10.1109/12.9739

Published: 01 December 1988 Publication History

Abstract

A model is presented for parallel processing in loosely coupled multiprocessing environments, such as networks of computer workstations, that are amenable to large-grained parallelism. The model takes into account the overhead involved in data communication to and from a remote processor and can be used to partition a large class of computations optimally, consisting of computations that can be organized as a one-level tree and are homogeneous and separable. The optimal partition can be determined for a given number processors, and, if required, the optimal number of processors to use can also be derived. Experimental results validate the model and demonstrate its effectiveness.

References

[1]

{1} R. Agrawal and A. K. Ezzat, "Processor sharing in NEST: A network of computer workstations," in Proc. IEEE 1st Int. Conf. Computer Workstations, San Jose, CA, Nov. 1985, pp. 198-208.

[2]

{2} P. Agrawal and R. Agrawal, "Software implementation of a recursive fault tolerance algorithm on a network of computers," in Proc. 13th Int. Symp. Comput. Architecture, Tokyo, Japan, June 1986, pp. 65- 72.

Digital Library

[3]

{3} R. Agrawal and A. K. Ezzat, "Location independent remote execution in NEST," IEEE Trans. Software Eng., vol. 13, pp. 905-912, Aug. 1987.

Digital Library

[4]

{4} E. H. Baalbergen, "Parallel and distributed compilations in loosely-coupled systems: A case study," presented at IEEE Workshop Large Grained Parallelism, Providence, RI, Oct. 1986.

[5]

{5} F. Baskett, J. H. Howard, and J. T. Montague, "Task communication in DEMOS," in Proc. ACM-SIGOPS 6th Symp. Oper. Syst. Principles, Nov. 1977, pp. 23-31.

[6]

{6} M. Beck, D. Bitton, and W. K. Wilkinson, "Sorting large files on a backend multiprocessor," in Proc. 1986 Int. Conf. Parallel Processing , St. Charles, IL, Aug. 1986.

[7]

{7} H. J. Boehm and W. Zwaenepoel, "Parallel attribute evaluation on a local network," presented at IEEE Workshop Large Grained Parallelism, Providence, RI, Oct. 1986.

[8]

{8} W. A. Burnette, R. H. Canaday, and D. H. Fishman, "MAX: A distributed system," Tech. Memo. AT&T Bell Lab., Murray Hill, NJ, 1983.

[9]

{9} G. B. Dantzig, Linear Programming and Extensions. Princeton, NJ: Princeton Univ. Press, 1963.

[10]

{10} D. J. DeWitt, R. Finkel, and M. Solomon, "The CRYSTAL multicomputer: Design and implementation experience," IEEE Trans. Software Eng., vol. 13, Aug. 1987.

[11]

{11} F. Douglis and J. Ousterhout, "Process migration in the Sprite operating system," in Proc. 7th Int. Conf. Distributed Comput. Syst., Berlin, West Germany, Sept. 1987, pp. 18-25.

[12]

{12} A. K. Ezzat and R. Agrawal, "Making oneself known in a distributed world," in Proc. 1985 Int. Conf. Parallel Processing, St. Charles, IL, Aug. 1985, pp. 139-142.

[13]

{13} D. H. Fishman, M. Y. Lai, and W. K. Wilkinson, "Overview of the Jasmin database machine," in Proc. 1984 ACM SIGMOD Int. Conf. Management Data, Boston, MA, June 1984, pp. 234-239.

Digital Library

[14]

{14} E. J. Gilbert, "Algorithm partitioning tools for a high-performance multiprocessor," Ph.D. dissertation, STAN-CS-83-946, Comput. Sci. Dep., Stanford Univ., Stanford, CA, Feb. 1983.

[15]

{15} H. V. Jagadish, "Techniques for the design of parallel and pipelined VLSI systems for numerical computations," Ph.D. dissertation, Inform. Syst. Lab., Stanford Univ., 1985.

[16]

{16} H. V. Jagadish, S. K. Rao, and T. Kailath, "Multiprocessor architectures for iterative algorithms," Proc. IEEE, Aug. 1987.

[17]

{17} L. Kleinrock, "Distributed systems," IEEE Computer, vol. 18, pp. 90-103, Nov. 1985.

Digital Library

[18]

{18} D. Kuck, R. Kuhn, B. Leasure, and M. Wolfe, "The structure of an advanced retargettable vectorizer," in Proc. COMPSAC '80, 1980.

[19]

{19} T. J. Leblanc, "The design and performance of high-level language primitives for distributed programming," Ph.D. dissertation, Comput. Sci. Dep., Univ. Wisconsin, Madison, 1982.

[20]

{20} O. L. Mangasarian, Non-Linear Programming. New York: Mc-Graw-Hill, 1969.

[21]

{21} T. A. Marsland, M. Olafsson, and J. Schaeffer, "Multiprocessor tree-search experiments," in Advances in Computer Chess 4, D. Beal, Ed. Oxford, England: Pergamon, 1985, pp. 37-51.

[22]

{22} D. A. Nichols, "Using idle workstations in a shared computing environment," in Proc. ACM-SIGOPS 11th Symp. Oper. Syst. Principles, Nov. 1987, pp. 5-12.

[23]

{23} S. K. Rao, "Regular iterative algorithms and their implementation on processor arrays," Ph.D. dissertation, Inform. Syst. Lab., Stanford Univ., 1985.

[24]

{24} G. R. Sager, J. A. Melber, and K. T. Fong, "The Oryx/Pecos operating system," AT&T Tech. J., vol. 64, pp. 251-268, Jan. 1985.

[25]

{25} S. Sahni, "Scheduling multipipeline and multiprocessor computers," IEEE Trans. Comput., vol. C-33, pp. 637-645, July 1984.

[26]

{26} J. F. Shoch and J. A. Hupp, "The "Worm" Programs--Early experience with a distributed computation," Commun. ACM, vol. 25, pp. 172-180, Mar. 1982.

[27]

{27} M. H. Solomon and R. A. Finkel, "The Roscoe distributed operating system," in Proc. ACM-SIGOPS 7th Symp. Oper. Syst. Principles, Dec. 1979, pp. 108-114.

[28]

{28} M. Stumm and D. Cheriton, "Distributed parallel computations under V," presented at IEEE Workshop Large Grained Parallelism, Providence, RI, Oct. 1986.

[29]

{29} M. Theimer, K. Lantz, and D. Cheriton, "Preemptable remote execution facilities for the V-System," in Proc. ACM-SIGOPS 10th Symp. Oper. Syst. Principles, Orcas Island, WA, Dec. 1985, pp. 2- 12.

Cited By

Drozdowski MShakhlevich N(2021)Scheduling divisible loads with time and cost constraintsJournal of Scheduling10.1007/s10951-019-00626-624:5(507-521)Online publication date: 1-Oct-2021
https://dl.acm.org/doi/10.1007/s10951-019-00626-6
Chen CChu C(2016)A Novel Computational Model for Non-Linear Divisible Loads on a Linear NetworkIEEE Transactions on Computers10.1109/TC.2015.241965465:1(53-65)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1109/TC.2015.2419654
Ghanbari SOthman MAbu Bakar MLeong W(2016)Multi-objective method for divisible load scheduling in multi-level tree networkFuture Generation Computer Systems10.1016/j.future.2015.03.01554:C(132-143)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1016/j.future.2015.03.015
Show More Cited By

Index Terms

Partitioning Techniques for Large-Grained Parallelism

Recommendations

Utilizing Multidimensional Loop Parallelism on Large Scale Parallel Processor Systems

Program parallelism and processor allocation issues for parallel processor systems are discussed. Optimal processor assignment algorithms are presented for simple and complex nested parallel loops. These processor assignment schemes can be used by the ...
Component-based programming techniques for coarse-grained parallelism
HPC '11: Proceedings of the 19th High Performance Computing Symposia

Components can be used to implement coarse-grained parallelism on large parallel systems. A parallel component is a piece of parallel code that can be executed in parallel on a set of processors or cores and has a predefined interface to be coupled with ...
Dynamic Fault Reconfiguration in a Mesh-Connected MIMD Environment

The near-neighbor problem is characterized by many iterations of a parallel matrix operation in which each matrix element is recomputed as a function of itself and its immediately adjacent near neighbors. Several highly parallel computer systems have ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Computers

IEEE Transactions on Computers Volume 37, Issue 12

December 1988

213 pages

ISSN:0018-9340

Editor:
Ming T. Liu

Issue’s Table of Contents

Copyright © Copyright © 1988 IEEE. All Rights Reserved.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 December 1988

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Drozdowski MShakhlevich N(2021)Scheduling divisible loads with time and cost constraintsJournal of Scheduling10.1007/s10951-019-00626-624:5(507-521)Online publication date: 1-Oct-2021
https://dl.acm.org/doi/10.1007/s10951-019-00626-6
Chen CChu C(2016)A Novel Computational Model for Non-Linear Divisible Loads on a Linear NetworkIEEE Transactions on Computers10.1109/TC.2015.241965465:1(53-65)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1109/TC.2015.2419654
Ghanbari SOthman MAbu Bakar MLeong W(2016)Multi-objective method for divisible load scheduling in multi-level tree networkFuture Generation Computer Systems10.1016/j.future.2015.03.01554:C(132-143)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1016/j.future.2015.03.015
Marszałkowski JDrozdowski MMarszałkowski J(2016)Time and Energy Performance of Parallel Systems with Hierarchical MemoryJournal of Grid Computing10.1007/s10723-015-9345-814:1(153-170)Online publication date: 1-Mar-2016
https://dl.acm.org/doi/10.1007/s10723-015-9345-8
Berlińska JDrozdowski M(2011)Scheduling divisible MapReduce computationsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2010.12.00471:3(450-459)Online publication date: 1-Mar-2011
https://dl.acm.org/doi/10.1016/j.jpdc.2010.12.004
Berlińska JDrozdowski M(2010)Heuristics for multi-round divisible loads scheduling with limited memoryParallel Computing10.1016/j.parco.2010.02.00436:4(199-211)Online publication date: 1-Apr-2010
https://dl.acm.org/doi/10.1016/j.parco.2010.02.004
Shih WYang CTseng S(2010)Performance-based data distribution for data mining applications on grid computing environmentsThe Journal of Supercomputing10.1007/s11227-009-0286-552:2(171-198)Online publication date: 1-May-2010
https://dl.acm.org/doi/10.1007/s11227-009-0286-5
Drozdowski M(2009)Energy considerations for divisible load processingProceedings of the 8th international conference on Parallel processing and applied mathematics: Part II10.5555/1893586.1893600(92-101)Online publication date: 13-Sep-2009
https://dl.acm.org/doi/10.5555/1893586.1893600
Berlińska J(2009)Fully polynomial time approximation schemes for scheduling divisible loadsProceedings of the 8th international conference on Parallel processing and applied mathematics: Part II10.5555/1893586.1893588(1-10)Online publication date: 13-Sep-2009
https://dl.acm.org/doi/10.5555/1893586.1893588
Viswanathan SVeeravalli BRobertazzi T(2007)Resource-Aware Distributed Scheduling Strategies for Large-Scale Computational Cluster/Grid SystemsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2007.107318:10(1450-1461)Online publication date: 1-Oct-2007
https://dl.acm.org/doi/10.1109/TPDS.2007.1073
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents