research-article

A pattern language for parallelizing irregular algorithms

Authors:

Pedro Monteiro,

Miguel P. MonteiroAuthors Info & Claims

ParaPLoP '10: Proceedings of the 2010 Workshop on Parallel Programming Patterns

Article No.: 13, Pages 1 - 14

https://doi.org/10.1145/1953611.1953624

Published: 30 March 2010 Publication History

Abstract

In irregular algorithms, data set's dependences and distributions cannot be statically predicted. This class of algorithms tends to organize computations in terms of data locality instead of parallelizing control in multiple threads. Thus, opportunities for exploiting parallelism vary dynamically, according to how the algorithm changes data dependences. This paper presents the first part of a pattern language for creating parallel implementations of irregular algorithms and applications. Four patterns are proposed: Amorphous Data-Parallelism, Data-Parallel Graph, Optimistic Iteration and In-Order Iteration.

References

[1]

Antonopoulos, C., Ding, X., Chernikov, A., Blagojevic, F., Nikolopoulos, D. and Chrisochoides, N., Multigrain parallel delaunay mesh generation: challenges and opportunities for multithreaded architectures. in, (2005), ACM, 376.

Digital Library

[2]

Asenjo, R., Corbera, F., Guti&#233;rrez, E., Navarro, M., Plata, O. and Zapata, E., Optimization Techniques for Irregular and Pointer-Based Programs, 12th Euromicro Conference on Parallel, Distributed and Network-Based Processing (PDP'04), 2004, pp. 2

[3]

Avgeriou, P., and Zdun, U., Architectural patterns revisited -- A pattern language, In Proceedings of 10th European Conference on Pattern Languages of Programs (EuroPloP 2005), Irsee, Germany, July 2005.

[4]

Avgeriou, P. and Zdun, U., Architectural patterns revisited--a pattern language. in Proc. 10th European Conference on Pattern Languages of Programs (EuroPlop 2005), Irsee, Germany, pages 1--39, July, 2005.

[5]

Bisseling, R. Parallel Scientific Computation: A structured approach using BSP and MPI. Oxford University Press, USA, 2004.

Digital Library

[6]

Biswas, R., Oliker, L. and Shan, H. Parallel computing strategies for irregular algorithms. Annual Review of Scalable Computing.

[7]

Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P. and Stal, M. A system of patterns: Pattern-oriented software architecture, Wiley New York, 1996.

Digital Library

[8]

Capretz, L. A brief history of the object-oriented approach. ACM SIGSOFT Software Engineering Notes, 28 (2).

Digital Library

[9]

Carothers, C., Perumalla, K. and Fujimoto, R., The effect of state-saving in optimistic simulation on a cache-coherent non-uniform memory access architecture, in Proc. 31st conference on Winter simulation: Simulation - a bridge to the future -- Vol. 2, Phoenix, Arizona, United States, 1999, pp. 1624--1633.

Digital Library

[10]

Chrisochoides, N., Lee, C., Lowekamp, B. et al., Multigrain parallel delaunay mesh generation: challenges and opportunities for multithreaded architectures., in Proceedings of the 19th annual international Conference on Supercomputing, ICS 2005, Cambridge, Massachusetts, USA, pp. 376, 2005.

Digital Library

[11]

Codrescu, L., Wills, D. and Meindl, J. Architecture of the Atlas chip-multiprocessor: Dynamically parallelizing irregular applications. IEEE Transactions on Computers, 50 (1). 67--82.

Digital Library

[12]

Coplien, J. and Woolf, B. A pattern language for writers' workshops. C PLUS PLUS REPORT, 9. pp. 51--60, 1997.

[13]

Cormen, T.H., Leiserson, C.E. and Rivest, R.L. Introduction to algorithms. MIT Press, Cambridge, Mass.; London, 1990.

Digital Library

[14]

Das, R., Uysal, M., Saltz, J. and Hwang, Y. Communication optimizations for irregular scientific computations on distributed memory architectures. Journal of Parallel and Distributed Computing, 22 (3) pp. 462--478, 1994.

Digital Library

[15]

Das, S., Adaptive protocols for parallel discrete event simulation, in Proceedings of the 28th conference on Winter simulation, Coronado, California, United States, 1996, pp. 186--193.

Digital Library

[16]

Dieter, W. and Lumpp, J., A User-level Checkpointing Library for POSIX Threads Programs, in Proceedings of the Twenty-Ninth Annual International Symposium on Fault-Tolerant Computing, 1999, pp. 224.

Digital Library

[17]

Dig, D. Johnson, R. and Snir, M., N-Body Pattern Language, February, 2010; http://parlab.eecs.berkeley.edu/wiki/patterns/n-body_methods

[18]

Dijkstra, E. A note on two problems in connexion with graphs. Numerische mathematik, 1 (1). 269--271.

[19]

Dorojevets, M. and Oklobdzija, V. Multithreaded decoupled architecture. International Journal of High Speed Computing, 7 (3). 465.

[20]

Fisher, J., Very Long Instruction Word architectures and the ELI-512, in Proceedings of the 10th annual international symposium on Computer architecture, Stockholm, Sweden, 1983, pp. 140--150.

Digital Library

[21]

Gamma, E. Helm, E. Johnson R. et al., Design patterns: Elements of reusable object-oriented software. Addison Wesley, Reading, MA.

Digital Library

[22]

Goodyear P., Avgeriou P., Baggetun, R. et al., Towards a pattern language for networked learning. Networked learning 2004 (pp. 449--455). Lancaster: Lancaster University.

[23]

Gross, J. and Yellen, J. Graph theory and its applications. CRC press, 2006.

Digital Library

[24]

Gupta, M. and Nim, R., Techniques for speculative run-time parallelization of loops, in Proceedings of the 1998 ACM/IEEE conference on Supercomputing (CDROM), San Jose, CA, 1998, pp. 1--12.

Digital Library

[25]

Gustafson, J. Reevaluating Amdahl's Law. Communications of the ACM, 31 (5), 1988

Digital Library

[26]

Gutierrez, E., Asenjo, R., Plata, O. and Zapata, E. Automatic parallelization of irregular applications. Parallel Computing, 26 (13--14). 1709--1738.

Digital Library

[27]

Hennessy, J., Patterson, D., Goldberg, D. and Asanovic, K. Computer architecture: a quantitative approach. Morgan Kaufmann, 2003.

Digital Library

[28]

Hermenegildo, M. Parallelizing irregular and pointer-based computations automatically: Perspectives from logic and constraint programming. Parallel Computing, 26 (13--14). 1685--1708.

Digital Library

[29]

Hudson, B., Miller, G.L. and Phillips, T. Sparse parallel Delaunay mesh refinement Proceedings of the nineteenth annual ACM symposium on Parallel algorithms and architectures, ACM, San Diego, California, USA, 2007, 339--347.

Digital Library

[30]

Jefferson, D. Virtual time. ACM Transactions on Programming Languages and Systems (TOPLAS), 7 (3). 425.

Digital Library

[31]

Karniadakis, G. and Kirby, R. Parallel scientific computing in C++ and MPI. Cambridge University Press Cambridge, 2003.

Digital Library

[32]

Keutzer, K. and Mattson, T. Our Pattern Language (OPL): A Design Pattern Language for Engineering (Parallel) Software, ParaPLoP 2009, Santa Cruz, CA,USA, June 2009.

[33]

Kolingerov&#225;, I. and Kohout, J. Optimistic parallel Delaunay triangulation. The Visual Computer, 18 (8). 511--529.

[34]

Kulkarni, M. The Galois System: Optimistic Parallelization of Irregular Programs, Cornell University, 2008.

[35]

Kulkarni, M., Burtscher, M., Pingali, K. and Cascaval, C., Lonestar: A suite of parallel irregular programs. in IEEE International Symposium on Performance Analysis of Systems and Software, (2009), pp. 65--76.

[36]

Lamport, L. Time, clocks, and the ordering of events in a distributed system, Communications ACM, vol. 21, no. 7, pp. 558--565, 1978.

Digital Library

[37]

Launchbury, J. Graph algorithms with a functional flavour. Lecture Notes in Computer Science, 925. 308.

[38]

Lublinerman, R., Chaudhuri, S. and Cerny, P., Parallel programming with object assemblies, in Proceeding of the 24th ACM SIGPLAN conference on Object oriented programming systems languages and applications, Orlando, Florida, USA, 2009, pp. 61--80.

Digital Library

[39]

Marcuello, P. and Gonz&#225;lez, A., Control and Data Dependence Speculation in Multithreaded Processors, MTEAC '98 Conference, 1998.

[40]

Marcuello, P., Gonz&#225;lez, A., A Quantitative Assessment of Thread-Level Speculation Techniques, in Proceedings of the 14th International Symposium on Parallel and Distributed Processing, 2000, pp. 595.

Digital Library

[41]

Mattson, T., Sanders, B. and Massingill, B. Patterns for parallel programming. Addison-Wesley Professional, 2004.

Digital Library

[42]

Meszaros, G., Doble, J., MetaPatterns: A Pattern Language for Pattern Writing, in Proceedings International Conference on Pattern Languages of Programs (PLoP 96), Monticello, Illinois, September 1996

Digital Library

[43]

Monteiro P., Monteiro, M., A Pattern Language for Parallelizing Irregular Algorithms, CITI - Departamento de Inform&#225;tica, Faculdade de Ci&#234;ncias e Tecnologia da Universidade Nova de Lisboa, Almada, 2010.

[44]

Navabi, A., Zhang, X. and Jagannathan, S., Quasi-static scheduling for safe futures, in Proceedings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming, Salt Lake City, UT, USA, 2008, pp. 23--32.

Digital Library

[45]

Nikolopoulos, D., Polychronopoulos, C. and Ayguad&#233;, E., Scaling irregular parallel codes with minimal programming effort. In Proceedings of Supercomputing, 2001

Digital Library

[46]

Noble, J. and Sydney, A. Towards a pattern language for object oriented design. Proc. of Technology of Object-Oriented Languages and Systems (TOOLS Pacific), 28. 2--13.

Digital Library

[47]

Oplinger, J., Heine, D., Liao, S., Nayfeh, B., Lam, M. and Olukotun, K. Software and hardware for exploiting speculative parallelism with a multiprocessor. Computer Systems Laboratory Technical Report CSL-TR-97-715, Stanford University.

Digital Library

[48]

Pingali, K., Kulkarni, M., Nguyen, et al. Amorphous data-parallelism in irregular algorithms, Technical Report TR-09-05, Department of Computer Science, The University of Texas at Austin, February 2009.

[49]

Prabhu, M. and Olukotun, K., Using thread-level speculation to simplify manual parallelization, in Proceedings of the ninth ACM SIGPLAN symposium on Principles and practice of parallel programming, San Diego, California, USA, 2003, pp. 1--12.

Digital Library

[50]

Gupta, M. and Nim, R., Techniques for speculative run-time parallelization of loops, in Proceedings of the 1998 ACM/IEEE conference on Supercomputing (CDROM), San Jose, CA, 1998, pp. 1--12.

Digital Library

[51]

Roberts, D. and Johnson, R. Evolving frameworks: A pattern language for developing object-oriented frameworks. Pattern Languages of Program Design, 3. 471--486.

Digital Library

[52]

Schmidt, D. and Buschmann, F., Patterns, frameworks, and middleware: Their synergistic relationships. In 25^th International Conference on Software Engineering, pp. 694--704, May 2003.

Digital Library

[53]

Schwartz, J., Dewar, R., Schonberg, E. and Dubinsky, E. Programming with sets; an introduction to SETL. Springer-Verlag New York, Inc. New York, NY, USA, 1986.

Digital Library

[54]

Shewchuk, J. Delaunay refinement algorithms for triangular mesh generation. Computational Geometry: Theory and Applications, 22 (1--3). 21--74.

Digital Library

[55]

Spinellis, D. and Raptis, K. Component mining: A process and its pattern language. Information and Software Technology, 42 (9). 609--617.

[56]

Steffan, J., Colohan, C., Zhai, A. and Mowry, T. A scalable approach to thread-level speculation. ACM SIGARCH Computer Architecture News, 28 (2). 1--12.

Digital Library

[57]

Taylor, S., Watts, J., Rieffel, M. and Palmer, M. The concurrent graph: basic technology for irregular problems. IEEE Parallel and Distributed Technology. 15--25.

Digital Library

[58]

Tomasulo, R. An efficient algorithm for exploiting multiple arithmetic units. IBM Journal of research and Development, 11 (1). 25--33.

Digital Library

[59]

Tsai, J. and Yew, P. The superthreaded architecture: Thread pipelining with run-time data dependence checking and control speculation. Urbana, 51. 61801--61351.

[60]

Verma, C., Multithreaded Delaunay Triangulation., The College of William and Mary, Williamsburg, Virginia, 2004.

[61]

von Praun, C., Ceze, L. and Cascaval, C., Implicit parallelism with ordered transactions, in Proceedings of the 12th ACM SIGPLAN symposium on Principles and practice of parallel programming, San Jose, California, USA, 2007, pp. 79--89.

Digital Library

[62]

Wang, S., Dai, X., Yellajyosula, K., Zhai, A. and Yew, P. Loop selection for thread-level speculation. Lecture Notes in Computer Science, 4339. 289.

Digital Library

[63]

Wolf, K. and Liu, C. New clients with old servers: A pattern language for client/server frameworks. Pattern Languages of Program Design. 51--64.

Digital Library

[64]

Zdun, U. Pattern language for the design of aspect languages and aspect composition frameworks. IEE Proceedings-Software, 151 (2). 67--84.

[65]

Zhang, Y., Rauchwerger, L. and Torrellas, J., Hardware for speculative run-time parallelization in distributed shared-memory multiprocessors. in, (1998), IEEE Computer Society Washington, DC, USA, 162.

Digital Library

[66]

Zhang, Z. and Torrellas, J. Speeding up irregular applications in shared-memory multiprocessors: Memory binding and group prefetching. ACM SIGARCH Computer Architecture News, 23 (2). 188--199.

Digital Library

Cited By

Kao CHsu W(2015)An Adaptive Heterogeneous Runtime Framework for Irregular ApplicationsJournal of Signal Processing Systems10.1007/s11265-014-0916-x80:3(245-259)Online publication date: 1-Sep-2015
https://dl.acm.org/doi/10.1007/s11265-014-0916-x
Kowalkowski J(2014)Data processing in the wake of massive multi-core processorsJournal of Physics: Conference Series10.1088/1742-6596/513/5/052015513:5(052015)Online publication date: 11-Jun-2014
https://doi.org/10.1088/1742-6596/513/5/052015
Monteiro P(2012)HARPPIE: Hyper algorithmic recipe for productive parallelism intensive endeavors2012 34th International Conference on Software Engineering (ICSE)10.1109/ICSE.2012.6227037(1559-1562)Online publication date: Jun-2012
https://doi.org/10.1109/ICSE.2012.6227037
Show More Cited By

Index Terms

A pattern language for parallelizing irregular algorithms
1. Computing methodologies
  1. Parallel computing methodologies
    1. Parallel programming languages
2. Software and its engineering
  1. Software notations and tools
    1. General programming languages
      1. Language features
        Patterns
      2. Language types
        Parallel programming languages
  2. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles
        Client-server architectures
      2. Software architectures

Recommendations

Parallelizing irregular algorithms: a pattern language
PLoP '11: Proceedings of the 18th Conference on Pattern Languages of Programs

Outside of the high-performance computing domain, many applications are irregular in the sense that opportunities to exploit parallelism change throughout the computation, due to the use of complex, pointer-based data structures such as lists and ...
Flattening and parallelizing irregular, recurrent loop nests
PPOPP '95: Proceedings of the fifth ACM SIGPLAN symposium on Principles and practice of parallel programming

Irregular loop nests in which the loop bounds are determined dynamically by indexed arrays are difficult to compile into expressive parallel constructs, such as segmented scans and reductions. In this paper, we describe a suite of transformations to ...
Flattening and parallelizing irregular, recurrent loop nests

Irregular loop nests in which the loop bounds are determined dynamically by indexed arrays are difficult to compile into expressive parallel constructs, such as segmented scans and reductions. In this paper, we describe a suite of transformations to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ParaPLoP '10: Proceedings of the 2010 Workshop on Parallel Programming Patterns

March 2010

141 pages

ISBN:9781450301275

DOI:10.1145/1953611

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 March 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Fundação para a Ciência e a Tecnologia

Conference

ParaPLoP '10

ParaPLoP '10: 2010 Workshop on Parallel Programming Patterns

March 30 - 31, 2010

Arizona, Carefree, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
135
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kao CHsu W(2015)An Adaptive Heterogeneous Runtime Framework for Irregular ApplicationsJournal of Signal Processing Systems10.1007/s11265-014-0916-x80:3(245-259)Online publication date: 1-Sep-2015
https://dl.acm.org/doi/10.1007/s11265-014-0916-x
Kowalkowski J(2014)Data processing in the wake of massive multi-core processorsJournal of Physics: Conference Series10.1088/1742-6596/513/5/052015513:5(052015)Online publication date: 11-Jun-2014
https://doi.org/10.1088/1742-6596/513/5/052015
Monteiro P(2012)HARPPIE: Hyper algorithmic recipe for productive parallelism intensive endeavors2012 34th International Conference on Software Engineering (ICSE)10.1109/ICSE.2012.6227037(1559-1562)Online publication date: Jun-2012
https://doi.org/10.1109/ICSE.2012.6227037
Monteiro PMonteiro MPingali KHvatum L(2011)Parallelizing irregular algorithmsProceedings of the 18th Conference on Pattern Languages of Programs10.1145/2578903.2579141(1-18)Online publication date: 21-Oct-2011
https://dl.acm.org/doi/10.1145/2578903.2579141

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents