Article

Feedback directed implicit parallelism

Authors:

Satnam SinghAuthors Info & Claims

ICFP '07: Proceedings of the 12th ACM SIGPLAN international conference on Functional programming

Pages 251 - 264

https://doi.org/10.1145/1291151.1291192

Published: 01 October 2007 Publication History

Abstract

In this paper we present an automated way of using spare CPU resources within a shared memory multi-processor or multi-core machine. Our approach is (i) to profile the execution of a program, (ii) from this to identify pieces of work which are promising sources of parallelism, (iii) recompile the program with this work being performed speculatively via a work-stealing system and then (iv) to detect at run-time any attempt to perform operations that would reveal the presence of speculation.

We assess the practicality of the approach through an implementation based on GHC 6.6 along with a limit study based on the execution profiles we gathered. We support the full Concurrent Haskell language compiled with traditional optimizations and including I/O operations and synchronization as well as pure computation. We use 20 of the larger programs from the 'nofib' benchmark suite. The limit study shows that programs vary a lot in the parallelism we can identify: some have none, 16 have a potential 2x speed-up, 4 have 32x. In practice, on a 4-core processor, we get 10-80% speed-ups on 7 programs. This is mainly achieved at the addition of a second core rather than beyond this.

This approach is therefore not a replacement for manual parallelization, but rather a way of squeezing extra performance out of the threads of an already-parallel program or out of a program that has not yet been parallelized.

References

[1]

Arvind, David E. Culler, and Gino K. Maa. Assessing the benefits of fine-grained parallelism in dataflow programs. Technical Report 279, Computation Structures Group, MIT, March 1988.

[2]

K. E. Batcher. Sorting networks and their applications. In Proceedings of the AFIPS Spring Joint Computing Conference, 1969.

[3]

R. Blumofe and C. Leiserson. Scheduling multithreaded computations by work stealing. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, Santa Fe, New Mexico., pages 356--368, November 1994.

Digital Library

[4]

Robert Ennals. Adaptive Evaluation of Non-Strict Programs. PhD thesis, Cambridge University Computer Laboratory, 2004.

[5]

Robert Ennals and Simon Peyton Jones. Optimistic evaluation: an adaptive evaluation strategy for non-strict programs. In ICFP '03: Proceedings of the eighth ACM SIGPLAN international conference on Functional programming, pages 287--298. ISBN 1-58113-756-7.

Digital Library

[6]

Kevin Hammond. Parallel functional programming: An introduction. In International Symposium on Parallel Symbolic Computation, Hagenberg/Linz, Austria, September 1994.

[7]

Tim Harris. Dynamic adaptive pre-tenuring. In International Symposium on Memory Management (ISMM,'00), volume 36(1) of ACM SIGPLAN Notices, pages 127--136, January 2001.

Digital Library

[8]

Tim Harris, Simon Marlow, and Simon Peyton Jones. Haskell on a shared-memory multiprocessor. In Haskell '05: Proceedings of the 2005 ACM SIGPLAN workshop on Haskell, pages 49--61, September 2005.

Digital Library

[9]

Danny Hendler, Yossi Lev, Mark Moir, and Nir Shavit. A dynamic-sized nonblocking work stealing deque. Technical Report TR-2005-144, Sun Microsystems Laboratories, 2005.

[10]

Raj Jain. The art of computer systems performance analysis. Wiley, 1991.

[11]

H-W. Loidl. Granularity in Large-Scale Parallel Functional Programming. PhD thesis, Department of Computing Science, University of Glasgow, March 1998.

[12]

James Stewart Mattson. An effective speculative evaluation technique for parallel supercombinator graph reduction. PhD thesis, University of California at San Diego.

[13]

Rishiyur S Nikhil and Arvind. Implicit Parallel Programming in pH. Morgan Kaufman, 2001.

Digital Library

[14]

Randy B. Osborne. Speculative computation in multilisp. In LFP '90: Proceedings of the 1990 ACM conference on LISP and functional programming, pages 198--208. ISBN 0-89791-368-X.

Digital Library

[15]

Simon Peyton Jones, Andrew Gordon, and Sigbjorn Finne. Concurrent haskell. In POPL '96: Proceedings of the 23rd ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pages 295--308, New York, NY, USA, 1996. ISBN 0-89791-769-3.

Digital Library

[16]

Paul Roe. Parallel Programming Using Functional Languages. PhD thesis, Department of Computing Science, University of Glasgow, 1991.

[17]

Kenneth R. Traub. Implementation of non-strict functional programming languages. MIT Press, Cambridge, MA, USA, 1991. ISBN 0-262-70042-5.

Digital Library

[18]

Kenneth R. Traub, Gregory M. Papadopoulos, Michael J. Beckerle, James E. Hicks, and Jonathan Young. Overview of the Monsoon project. In ICCD '91: Proceedings of the 1991 IEEE International Conference on Computer Design on VLSI in Computer & Processors, pages 150--155. ISBN 0-8186-2270-9.

Digital Library

[19]

P. W. Trinder, K. Hammond, H.-W. Loidl, and S. L. Peyton Jones. Algorithm + strategy = parallelism. J. Funct. Program., 8 (1): 23--60, 1998. ISSN 0956-7968.

Digital Library

Cited By

Tan ZChon YKruse MDoerfert JXu ZHomerding BCampanoni SAugust DAamodt TJerger NSwift M(2023)SPLENDID: Supporting Parallel LLVM-IR Enhanced Natural Decompilation for Interactive DevelopmentProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3582016.3582058(679-693)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3582016.3582058
Koparkar CRainey MVollmer MKulkarni MNewton R(2021)Efficient tree-traversals: reconciling parallelism and dense data representationsProceedings of the ACM on Programming Languages10.1145/34735965:ICFP(1-29)Online publication date: 19-Aug-2021
https://dl.acm.org/doi/10.1145/3473596
Helm DKübler FKölzer JHaller PEichberg MSalvaneschi GMezini MKhurshid SPăsăreanu C(2020)A programming model for semi-implicit parallelization of static analysesProceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3395363.3397367(428-439)Online publication date: 18-Jul-2020
https://dl.acm.org/doi/10.1145/3395363.3397367
Show More Cited By

Index Terms

Feedback directed implicit parallelism
1. Computing methodologies
  1. Parallel computing methodologies
    1. Parallel programming languages
2. Software and its engineering
  1. Software notations and tools
    1. General programming languages
      1. Language types
        Parallel programming languages

Recommendations

Feedback directed implicit parallelism
Proceedings of the ICFP '07 conference

In this paper we present an automated way of using spare CPU resources within a shared memory multi-processor or multi-core machine. Our approach is (i) to profile the execution of a program, (ii) from this to identify pieces of work which are promising ...
Comparing Parallel Functional Languages: Programming and Performance

This paper presents a practical evaluation and comparison of three state-of-the-art parallel functional languages. The evaluation is based on implementations of three typical symbolic computation programs, with performance measured on a Beowulf-class ...
Exploiting Implicit Parallelism in Dynamic Array Programming Languages
ARRAY'14: Proceedings of ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming

We have built an interpreter for the array programming language J. The interpreter exploits implicit data parallelism in the language to achieve good parallel speedups on a variety of benchmark applications.

Many array programming languages operate on ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICFP '07: Proceedings of the 12th ACM SIGPLAN international conference on Functional programming

October 2007

346 pages

ISBN:9781595938152

DOI:10.1145/1291151

General Chair:
Ralf Hinze
Universität Bonn, Germany
,
Program Chair:
Norman Ramsey
Harvard University, USA

ACM SIGPLAN Notices Volume 42, Issue 9
Proceedings of the ICFP '07 conference
September 2007
331 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/1291220
Issue’s Table of Contents

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICFP07

Sponsor:

ICFP07: ACM SIGPLAN International Conference on Functional Programming

October 1 - 3, 2007

Freiburg, Germany

Acceptance Rates

Overall Acceptance Rate 333 of 1,064 submissions, 31%

Upcoming Conference

ICFP '25

Sponsor:
sigplan

ACM SIGPLAN International Conference on Functional Programming

October 12 - 18, 2025

Singapore , Singapore

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
390
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 10 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tan ZChon YKruse MDoerfert JXu ZHomerding BCampanoni SAugust DAamodt TJerger NSwift M(2023)SPLENDID: Supporting Parallel LLVM-IR Enhanced Natural Decompilation for Interactive DevelopmentProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3582016.3582058(679-693)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3582016.3582058
Koparkar CRainey MVollmer MKulkarni MNewton R(2021)Efficient tree-traversals: reconciling parallelism and dense data representationsProceedings of the ACM on Programming Languages10.1145/34735965:ICFP(1-29)Online publication date: 19-Aug-2021
https://dl.acm.org/doi/10.1145/3473596
Helm DKübler FKölzer JHaller PEichberg MSalvaneschi GMezini MKhurshid SPăsăreanu C(2020)A programming model for semi-implicit parallelization of static analysesProceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3395363.3397367(428-439)Online publication date: 18-Jul-2020
https://dl.acm.org/doi/10.1145/3395363.3397367
Ertel SAdam JRink NGoens ACastrillon JEisenberg R(2019)STCLang: state thread composition as a foundation for monadic dataflow parallelismProceedings of the 12th ACM SIGPLAN International Symposium on Haskell10.1145/3331545.3342600(146-161)Online publication date: 8-Aug-2019
https://dl.acm.org/doi/10.1145/3331545.3342600
Wang YNunez DFisher K(2016)Autobahn: using genetic algorithms to infer strictness annotationsACM SIGPLAN Notices10.1145/3241625.297600951:12(114-126)Online publication date: 8-Sep-2016
https://dl.acm.org/doi/10.1145/3241625.2976009
Surendran RSarkar V(2016)Automatic parallelization of pure method calls via conditional future synthesisACM SIGPLAN Notices10.1145/3022671.298403551:10(20-38)Online publication date: 19-Oct-2016
https://dl.acm.org/doi/10.1145/3022671.2984035
Surendran RSarkar VVisser ESmaragdakis Y(2016)Automatic parallelization of pure method calls via conditional future synthesisProceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications10.1145/2983990.2984035(20-38)Online publication date: 19-Oct-2016
https://dl.acm.org/doi/10.1145/2983990.2984035
Wang YNunez DFisher KMainland G(2016)Autobahn: using genetic algorithms to infer strictness annotationsProceedings of the 9th International Symposium on Haskell10.1145/2976002.2976009(114-126)Online publication date: 8-Sep-2016
https://dl.acm.org/doi/10.1145/2976002.2976009
Trilla JRunciman C(2015)Improving implicit parallelismACM SIGPLAN Notices10.1145/2887747.280430850:12(153-164)Online publication date: 30-Aug-2015
https://dl.acm.org/doi/10.1145/2887747.2804308
Ertel SFetzer CFelber PStansifer RKrall A(2015)OhuaProceedings of the Principles and Practices of Programming on The Java Platform10.1145/2807426.2807431(51-64)Online publication date: 8-Sep-2015
https://dl.acm.org/doi/10.1145/2807426.2807431
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents