Abstract
We propose a new theoretical model for parallelism. The model is explictly based on data and work distributions, a feature missing from other theoretical models. The major theoretic result is that data movement can then be derived by formal reasoning. While the model has an immediate interpretation in distributed memory parallelism, we show that it can also accomodate shared memory and hybrid architectures such as clusters with accelerators.The model gives rise in a natural way to objects appearing in widely different parallel programming systems such as the PETSc library or the Quark task scheduler. Thus we argue that the model offers the prospect of a high productivity programming system that can be compiled down to proven high-performance environments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
We can argue that this is no limitation, as any object will have a linearization of some sort.
- 2.
We make the common dentification of \(N=\{0,\ldots ,N-1\}\) and \(P=\{0,\ldots ,P-1\}\); likewise \(N^M\) is the set of mappings from \(M\) to \(N\), and thereby \(2^N\) is the set of mappings from \(N\) to \(\{0,1\}\); in effect the set of all subsets of \(\{0,\ldots ,N-1\}\).
- 3.
We use parentheses for indicating distributions; actual vector subsections are denoted with square brackets.
References
Adiga AK, Browne JC (1986) A graph model for parallel computations expressed in the computation structures language. In: ICPP, pp 880–886
Barnes J, Hut P (1986) A hierarchical O(N log N) force-calculation algorithm. Nature 324:446–449. http://dx.doi.org/10.1038/324446a0
Chan E, Quintana-Ortà ES, Quintana-Ortà G, van de Geijn R (2007) SuperMatrix out-of-order scheduling of matrix operations for SMP and multi-core architectures. In: SPAA ’07: Proceedings of the 19th ACM symposium on parallelism in algorithms and architectures, San Diego, CA, USA, pp 116–125.
Chapel programming language homepage. http://chapel.cray.com/
Cray Research: Cray T3E\(^{TM}\) Fortran optimization guide. http://docs.cray.com/books/004-2518-002/html-004-2518-002/004-2518-002-toc.html
Eijkhout V (2012) A unified approach to parallel programming. In: Ao S, Douglas C, Grundfest W, Burgstone J (eds) Lecture Notes in engineering and computer science: proceedings of the world congress on engineering and computer science 2012, WCECS 2012, 24–26 October, 2012, San Francisco, USA, pp 78–83. Newswood Limited, International Association of Engineers. ISBN (vol I): 978-988-19251-6-9, ISBN (vol II): 978-988-19252-4-4, ISSN: 2078-0958 (Print), ISSN: 2078-0966 (Online)
Gao G, Sterling T, Stevens R, Hereld M, Zhu W (2007) ParalleX: a study of a new parallel computation model. In: Parallel and distributed processing symposium, 2007. IPDPS 2007. IEEE International, pp 1–6. doi:10.1109/IPDPS.2007.370484.
Greengard L, Rokhlin V (1987) A fast algorithm for particle simulations. J Comput Phys 73:325
Gropp W, Lusk E, Skjellum A (1994) Using MPI. The MIT Press, Cambridge
Gropp WD, Smith BF (1994) Scalable, extensible, and portable numerical libraries. In: Proceedings of the scalable parallel libraries conference, IEEE pp 87–93.
Heroux MA, Bartlett RA, Howle VE, Hoekstra RJ, Hu JJ, Kolda TG, Lehoucq RB, Long KR, Pawlowski RP, Phipps ET, Salinger AG, Thornquist HK, Tuminaro RS, Willenbring JM, Williams A, Stanley KS (2005) An overview of the trilinos project. ACM Trans Math Softw 31(3):397–423. http://doi.acm.org/10.1145/1089014.1089021
Kale LV, Krishnan S (1996) Charm++: Parallel programming with message-driven objects. In: Wilson GV, Lu P (eds) Parallel programming using C++, MIT Press, Cambridge, pp. 175–213
Karp RM, Miller RE (1966) Properties of a model for parallel computations: Determinacy, termination, queueing. SIAM J Appl Math 14:1390–1411
Katzenelson J (1989) Computational structure of the n-body problem. SIAM J Sci Stat Comput 10:787–815
Kulkarni M, Pingali K, Walter B, Ramanarayanan G, Bala K, Chew LP (2007) Optimistic parallelism requires abstractions. SIGPLAN Not. (Proceedings of PLDI) 42(6):211–222. http://doi.acm.org/10.1145/1273442.1250759. http://iss.ices.utexas.edu/Publications/Papers/PLDI2007.pdf
Lublinerman R, Chaudhuri S, Cerny P (2009) Parallel programming with object assemblies. In: International conference on object oriented programming, systems, languages and applications (OOPSLA)
Newton P, Browne JC (1992) The code 2.0 graphical parallel programming language. In: Proceedings of the 6th international conference on supercomputing, ICS ’92, pp 167–177. ACM, New York, NY, USA. doi:10.1145/143369.143405. http://doi.acm.org/10.1145/143369.143405
Nieplocha J, Harrison R, Littlefield R (1996) Global arrays: A nonuniform memory access programming model for high-performance computers. J Supercomput 10:197–220
Poulson J, Marker B, Hammond JR, van de Geijn R Elemental: a new framework for distributed memory dense matrix computations. ACM Trans Math Softw Submitted
Quintana-Ortà G, Quintana-Ortà ES, van de Geijn RA, Van Zee FG, Chan E (2009) Programming matrix algorithms-by-blocks for thread-level parallelism. ACM Trans Math Softw 36(3):14:1–14:26. http://doi.acm.org/10.1145/1527286.1527288
Salmon JK, Warren MS, Winckelmans GS (1986) Fast parallel tree codes for gravitational and fluid dynamical n-body problems. Int J Supercomput Appl 8:129–142
Sussman A, Saltz J, Das R, Gupta S, Mavriplis D, Ponnusamy R (1992) Parti primitives for unstructured and block structured problems
Valiant LG (1990) A bridging model for parallel computation. Commun ACM 33:103–111. doi: http://doi.acm.org/10.1145/79173.79181. http://doi.acm.org/10.1145/79173.79181
YarKhan A, Kurzak J, Dongarra J (2011) QUARK users’ guide: queueing and runtime for kernels. Technical Report ICL-UT-11-02, University of Tennessee Innovative Computing Laboratory
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Eijkhout, V. (2014). A Unifying Framework for Parallel Computing. In: Kim, H., Ao, SI., Amouzegar, M., Rieger, B. (eds) IAENG Transactions on Engineering Technologies. Lecture Notes in Electrical Engineering, vol 247. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-6818-5_29
Download citation
DOI: https://doi.org/10.1007/978-94-007-6818-5_29
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-6817-8
Online ISBN: 978-94-007-6818-5
eBook Packages: EngineeringEngineering (R0)