Article

Stream fusion: from lists to streams to nothing at all

Authors:

Roman Leshchinskiy,

Don StewartAuthors Info & Claims

ICFP '07: Proceedings of the 12th ACM SIGPLAN international conference on Functional programming

Pages 315 - 326

https://doi.org/10.1145/1291151.1291199

Published: 01 October 2007 Publication History

Abstract

This paper presents an automatic deforestation system, stream fusion, based on equational transformations, that fuses a wider range of functions than existing short-cut fusion systems. In particular, stream fusion is able to fuse zips, left folds and functions over nested lists, including list comprehensions. A distinguishing feature of the framework is its simplicity: by transforming list functions to expose their structure, intermediate values are eliminated by general purpose compiler optimisations.

We have reimplemented the Haskell standard List library on top of our framework, providing stream fusion for Haskell lists. By allowing a wider range of functions to fuse, we see an increase in the number of occurrences of fusion in typical Haskell programs. We present benchmarks documenting time and space improvements.

References

[1]

Olaf Chitil. Type inference builds a short cut to deforestation. In ICFP '99: Proceedings of the fourth ACM SIGPLAN International Conference on Functional programming, pages 249--260, New York, NY, USA, 1999. ACM Press.

Digital Library

[2]

Olaf Chitil. Promoting non-strict programming. In Draft Proceedings of the 18th International Symposium on Implementation and Application of Functional Languages, IFL 2006, pages 512--516, Budapest, Hungary, September 2006. Eotvos Lorand University.

[3]

Duncan Coutts, Don Stewart, and Roman Leshchinskiy. Rewriting Haskell strings. In Practical Aspects of Declarative Languages 8th International Symposium, PADL 2007, pages 50--64. Springer-Verlag, January 2007.

Digital Library

[4]

Nils Anders Danielsson and Patrik Jansson. Chasing bottoms, a case study in program verification in the presence of partial and infinite values. In Dexter Kozen, editor, Proceedings of the 7th International Conference on Mathematics of Program Construction, MPC 2004, volume 3125 of LNCS, pages 85--109. Springer-Verlag, July 2004.

[5]

Jeremy Gibbons. Streaming representation-changers. In D. Kozen, editor, Mathematics of Program Construction, pages 142--168. Springer-Verlag, 2004. LNCS 523.

[6]

Jeremy Gibbons and Geraint Jones. The under-appreciated unfold. In ICFP '98: Proceedings of the third ACM SIGPLAN International Conference on Functional programming, pages 273--279, New York, NY, USA, 1998. ACM Press.

Digital Library

[7]

Andrew Gill. Cheap Deforestation for Non-strict Functional Languages. PhD thesis, University of Glasgow, January 1996.

[8]

Andrew Gill, John Launchbury, and Simon Peyton Jones. A shortcut to deforestation. In Conference on Functional Programming Languages and Computer Architecture, pages 223--232, June 1993.

Digital Library

[9]

Zhenjiang Hu, Hideya Iwasaki, and Masato Takeichi. Deriving structural hylomorphisms from recursive definitions. In Proceedings 1st ACM SIGPLAN International Conference on Functional Programming, volume 31(6), pages 73--82. ACM Press, New York, 1996.

Digital Library

[10]

Patricia Johann. Short cut fusion: Proved and improved. In SAIG 2001: Proceedings of the Second International Workshop on Semantics, Applications, and Implementation of Program Generation, pages 47--71, London, UK, 2001. Springer-Verlag.

Digital Library

[11]

Thomas Johnsson. Lambda lifting: transforming programs to recursive equations. In Functional programming languages and computer architecture. Proc. of a conference (Nancy, France, Sept.1985), New York, NY, USA, 1985. Springer-Verlag Inc.

Digital Library

[12]

John Launchbury and Ross Paterson. Parametricity and unboxing with unpointed types. In European Symposium on Programming, pages 204--218, 1996.

Digital Library

[13]

John Launchbury and Tim Sheard. Warm fusion: deriving build-catas from recursive definitions. In FPCA '95: Proceedings of the seventh international conference on Functional programming languages and computer architecture, pages 314--323, New York, NY, USA, 1995. ACM Press.

Digital Library

[14]

Erik Meijer, Maarten Fokkinga, and Ross Paterson. Functional programming with bananas, lenses, envelopes and barbed wire. In J. Hughes, editor, Proceedings 5th ACM Conf. on Functional Programming Languages and Computer Architecture, FPCA'91, Cambridge, MA, USA, 26-30 Aug 1991, volume 523, pages 124--144. Springer-Verlag, Berlin, 1991.

Digital Library

[15]

Will Partain. The nofib benchmark suite of Haskell programs. In Functional Programming, pages 195--202, 1992.

Digital Library

[16]

Simon Peyton Jones. Constructor Specialisation for Haskell Programs, 2007. Submitted for publication.

[17]

Simon Peyton Jones and André L. M. Santos. A transformation-based optimiser for Haskell. Sci. Comput. Program., 32(1-3): 3--47, 1998. ISSN 0167-6423.

Digital Library

[18]

Simon Peyton Jones, Andrew Tolmach, and Tony Hoare. Playing by the rules: rewriting as a practical optimisation technique in GHC. In Ralf Hinze, editor, 2001 Haskell Workshop. ACM SIGPLAN, September 2001.

[19]

Simon Peyton Jones et al. The Haskell 98 language and libraries: The revised report. Journal of Functional Programming, 13(1): 0--255, Jan 2003.

[20]

Colin Runciman. SmallCheck 0.2: another lightweight testing library in Haskell. http://article.gmane.org/gmane.comp.lang.haskell.general/14461, 2006.

[21]

Josef Svenningsson. Shortcut fusion for accumulating parameters & zip-like functions. In ICFP '02: Proceedings of the seventh ACM SIGPLAN International Conference on Functional programming, pages 124--132, New York, NY, USA, 2002. ACM Press.

Digital Library

[22]

Akihiko Takano and Erik Meijer. Shortcut deforestation in calculational form. In Conf. Record 7th ACM SIGPLAN/SIGARCH Int. Conf. on Functional Programming Languages and Computer Architecture, FPCA'95, pages 306--313. ACM Press, New York, 1995.

Digital Library

[23]

The GHC Team. The Glasgow Haskell Compiler (GHC). http://haskell.org/ghc, 2007.

[24]

Philip Wadler. List comprehensions. In Simon Peyton Jones, editor, The implementation of functional programming languages. Prentice Hall, 1987. Chapter 15.

[25]

Philip Wadler. Deforestation: transforming programs to eliminate trees. Theoretical Computer Science, (Special issue of selected papers from 2nd European Symposium on Programming), 73(2):231--248, 1990.

Digital Library

Cited By

Chen YParreaux L(2024)The Long Way to Deforestation: A Type Inference and Elaboration Technique for Removing Intermediate Data StructuresProceedings of the ACM on Programming Languages10.1145/36746348:ICFP(249-283)Online publication date: 15-Aug-2024
https://dl.acm.org/doi/10.1145/3674634
Wang ZPailoor SPrakash AWang YDillig I(2024)From Batch to Stream: Automatic Generation of Online AlgorithmsProceedings of the ACM on Programming Languages10.1145/36564188:PLDI(1014-1039)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656418
Ji RZhao YPolikarpova NXiong YHu Z(2024)Superfusion: Eliminating Intermediate Data Structures via Inductive SynthesisProceedings of the ACM on Programming Languages10.1145/36564158:PLDI(939-964)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656415
Show More Cited By

Index Terms

Stream fusion: from lists to streams to nothing at all
1. Software and its engineering
  1. Software notations and tools
    1. Compilers
    2. General programming languages
      1. Language types
        Functional languages

Recommendations

Stream fusion: from lists to streams to nothing at all
Proceedings of the ICFP '07 conference

This paper presents an automatic deforestation system, stream fusion, based on equational transformations, that fuses a wider range of functions than existing short-cut fusion systems. In particular, stream fusion is able to fuse zips, left folds and ...
The HERMIT in the stream: fusing stream fusion's concatMap
PEPM '14: Proceedings of the ACM SIGPLAN 2014 Workshop on Partial Evaluation and Program Manipulation

Stream Fusion, a popular deforestation technique in the Haskell community, cannot fuse the concatMap combinator. This is a serious limitation, as concatMap represents computations on nested streams. The original implementation of Stream Fusion used the ...
A library writer's guide to shortcut fusion
Haskell '11: Proceedings of the 4th ACM symposium on Haskell

There are now a variety of shortcut fusion techniques in the wild for removing intermediate data structures in Haskell. They are often presented, however, specialised to a specific data structure and interface. This can make it difficult to transfer ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICFP '07: Proceedings of the 12th ACM SIGPLAN international conference on Functional programming

October 2007

346 pages

ISBN:9781595938152

DOI:10.1145/1291151

General Chair:
Ralf Hinze
Universität Bonn, Germany
,
Program Chair:
Norman Ramsey
Harvard University, USA

ACM SIGPLAN Notices Volume 42, Issue 9
Proceedings of the ICFP '07 conference
September 2007
331 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/1291220
Issue’s Table of Contents

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICFP07

Sponsor:

ICFP07: ACM SIGPLAN International Conference on Functional Programming

October 1 - 3, 2007

Freiburg, Germany

Acceptance Rates

Overall Acceptance Rate 333 of 1,064 submissions, 31%

Upcoming Conference

ICFP '24

Sponsor:
sigplan

29th ACM SIGPLAN International Conference on Functional Programming

September 2 - 7, 2024

Milan , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

194
Total Citations
View Citations
925
Total Downloads

Downloads (Last 12 months)67
Downloads (Last 6 weeks)6

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen YParreaux L(2024)The Long Way to Deforestation: A Type Inference and Elaboration Technique for Removing Intermediate Data StructuresProceedings of the ACM on Programming Languages10.1145/36746348:ICFP(249-283)Online publication date: 15-Aug-2024
https://dl.acm.org/doi/10.1145/3674634
Wang ZPailoor SPrakash AWang YDillig I(2024)From Batch to Stream: Automatic Generation of Online AlgorithmsProceedings of the ACM on Programming Languages10.1145/36564188:PLDI(1014-1039)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656418
Ji RZhao YPolikarpova NXiong YHu Z(2024)Superfusion: Eliminating Intermediate Data Structures via Inductive SynthesisProceedings of the ACM on Programming Languages10.1145/36564158:PLDI(939-964)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656415
Singhal VSakka LSundararajah KNewton RKulkarni M(2024)Orchard: Heterogeneous Parallelism and Fine-grained Fusion for Complex Tree TraversalsACM Transactions on Architecture and Code Optimization10.1145/365260521:2(1-25)Online publication date: 21-May-2024
https://dl.acm.org/doi/10.1145/3652605
Keating FGale MMcDonell TVazou N(2023)This Is Driving Me Loopy: Efficient Loops in Arrowized Functional Reactive ProgramsProceedings of the 16th ACM SIGPLAN International Haskell Symposium10.1145/3609026.3609726(3-17)Online publication date: 30-Aug-2023
https://dl.acm.org/doi/10.1145/3609026.3609726
Yallop JXie NKrishnaswami N(2023)flap: A Deterministic Parser with Fused LexingProceedings of the ACM on Programming Languages10.1145/35912697:PLDI(1194-1217)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1145/3591269
Bolotina ACulpepper RScholz BKameyama Y(2022)Composable Sequence Macros for Fast IterationProceedings of the 21st ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences10.1145/3564719.3568696(59-71)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3564719.3568696
Kovács A(2022)Staged compilation with two-level type theoryProceedings of the ACM on Programming Languages10.1145/35476416:ICFP(540-569)Online publication date: 31-Aug-2022
https://dl.acm.org/doi/10.1145/3547641
Shaikhha AHuot MSmith JOlteanu D(2022)Functional collection programming with semi-ring dictionariesProceedings of the ACM on Programming Languages10.1145/35273336:OOPSLA1(1-33)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3527333
Westrick SRainey MAnderson DBlelloch GLee JAgrawal KSpear M(2022)Parallel block-delayed sequencesProceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming10.1145/3503221.3508434(61-75)Online publication date: 2-Apr-2022
https://dl.acm.org/doi/10.1145/3503221.3508434
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents