research-article

Open access

Static interpretation of higher-order modules in Futhark: functional GPU programming in the large

Authors:

Troels Henriksen,

Danil Annenkov,

Cosmin E. OanceaAuthors Info & Claims

Proceedings of the ACM on Programming Languages, Volume 2, Issue ICFP

Article No.: 97, Pages 1 - 30

https://doi.org/10.1145/3236792

Published: 30 July 2018 Publication History

Abstract

We present a higher-order module system for the purely functional data-parallel array language Futhark. The module language has the property that it is completely eliminated at compile time, yet it serves as a powerful tool for organizing libraries and complete programs. The presentation includes a static and a dynamic semantics for the language in terms of, respectively, a static type system and a provably terminating elaboration of terms into terms of an underlying target language. The development is formalised in Coq using a novel encoding of semantic objects based on products, sets, and finite maps. The module language features a unified treatment of module type abstraction and core language polymorphism and is rich enough for expressing practical forms of module composition.

Supplementary Material

WEBM File (a97-elsman.webm)

Download
128.07 MB

References

[1]

Andreas Abel. 2008. Normalization for the Simply-Typed Lambda-Calculus in Twelf. Electron. Notes Theor. Comput. Sci. 199 (Feb. 2008), 3–16.

Digital Library

[2]

Christian Andreetta, Vivien Bégot, Jost Berthold, Martin Elsman, Fritz Henglein, Troels Henriksen, Maj-Britt Nordfang, and Cosmin E. Oancea. 2016. FinPar: A Parallel Financial Benchmark. ACM Trans. Archit. Code Optim. 13, 2, Article 18 (June 2016), 27 pages.

Digital Library

[3]

Danil Annenkov and Martin Elsman. 2018. Certified Compilation of Financial Contracts. In Proceedings of the 20th International Symposium on Principles and Practice of Declarative Programming (PPDP ’18). ACM.

Digital Library

[4]

Brian Aydemir, Aaron Bohannon, and Stephanie Weirich. 2007. Nominal Reasoning Techniques in Coq. Electron. Notes Theor. Comput. Sci. 174, 5 (June 2007), 69–77.

Digital Library

[5]

Nick Benton, Chung-Kil Hur, Andrew Kennedy, and Conor McBride. 2012. Strongly Typed Term Representations in Coq. J. Autom. Reasoning 49, 2 (2012), 141–159.

Digital Library

[6]

Guy E. Blelloch. 1989. Scans as Primitive Parallel Operations. Computers, IEEE Transactions 38, 11 (1989), 1526–1538.

Digital Library

[7]

Guy E Blelloch. 1990. Vector models for data-parallel computing. Vol. 75. MIT press Cambridge.

Digital Library

[8]

Manuel MT Chakravarty, Gabriele Keller, Sean Lee, Trevor L McDonell, and Vinod Grover. 2011. Accelerating Haskell array codes with multicore GP Us. In Proceedings of the Sixth Workshop on Declarative Aspects of Multicore Programming (DAMP ’11). ACM, 3–14.

Digital Library

[9]

Adam Chlipala. 2013. Certified Programming with Dependent Types: A Pragmatic Introduction to the Coq Proof Assistant. MIT Press.

Digital Library

[10]

Koen Claessen, Mary Sheeran, and Bo Joel Svensson. 2012. Expressive Array Constructs in an Embedded GP U Kernel Programming Language. In Work. on Decl. Aspects of Multicore Prog DAMP. 21–30.

Digital Library

[11]

Ranald Clouston. 2013. Generalised Name Abstraction for Nominal Sets. In Foundations of Software Science and Computation Structures, Frank Pfenning (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 434–449.

Digital Library

[12]

Karl Crary. 2017. Modules, Abstraction, and Parametric Polymorphism. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL 2017). ACM, New York, NY, USA, 100–113.

Digital Library

[13]

Kevin Donnelly and Hongwei Xi. 2007. A Formalization of Strong Normalization for Simply-Typed Lambda-Calculus and System F. Electron. Notes Theor. Comput. Sci. 174, 5 (June 2007), 109–125.

Digital Library

[14]

Derek Dreyer, Robert Harper, Manuel M. T. Chakravarty, and Gabriele Keller. 2007. Modular Type Classes. In Proceedings of the 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’07). ACM, New York, NY, USA, 63–70.

Digital Library

[15]

Martin Elsman. 1999. Static Interpretation of Modules. In Procedings of Fourth International Conference on Functional Programming (ICFP ’99). ACM Press, 208–219.

Digital Library

[16]

Martin Elsman and Martin Dybdal. 2014. Compiling a Subset of APL Into a Typed Intermediate Language. In Procs. Int. Workshop on Lib. Lang. and Compilers for Array Prog. (ARRAY). ACM.

Digital Library

[17]

Matthew Fluet and Stephen Weeks. 2001. Contification Using Dominators. In Proceedings of the Sixth ACM SIGPLAN International Conference on Functional Programming (ICFP ’01). ACM, New York, NY, USA, 2–13.

Digital Library

[18]

Murdoch J. Gabbay and Andrew M. Pitts. 2002. A New Approach to Abstract Syntax with Variable Binding. Formal Aspects of Computing 13, 3 (01 Jul 2002), 341–363.

Digital Library

[19]

Jean Yves Girard. 1971. Interpretation Fonctionnelle et Elimination des Coupures de l’Arithmetique d’Ordre Superieur. In Proceedings of the Second Scandinavian Logic Symposium. North-Holland, 63–92.

[20]

Jing Guo, Jeyarajan Thiyagalingam, and Sven-Bodo Scholz. 2011. Breaking the GP U Programming Barrier with the Autoparallelising SAC Compiler. In Proceedings of the Sixth Workshop on Declarative Aspects of Multicore Programming (DAMP ’11). ACM, New York, NY, USA, 15–24.

Digital Library

[21]

Robert Harper and Mark Lillibridge. 1994. A Type-Theoretic Approach to Higher-Order Modules with Sharing. In Proceedings of the 21 ACM SIGPLAN Symposium on Principles of Programming Languages (POPL ’94). Portland, OR, 123–137.

Digital Library

[22]

Robert Harper, John C. Mitchell, and Eugenio Moggi. 1990. Higher-order Modules and the Phase Distinction. In Proceedings of the 17th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’90). ACM, New York, NY, USA, 341–354.

Digital Library

[23]

Robert Harper and Christopher Stone. 2000. Proof, Language, and Interaction. MIT Press, Cambridge, MA, USA, Chapter A Type-theoretic Interpretation of Standard ML, 341–387.

Digital Library

[24]

Troels Henriksen, Martin Dybdal, Henrik Urms, Anna Sofie Kiehn, Daniel Gavin, Hjalte Abelskov, Martin Elsman, and Cosmin Oancea. 2016a. APL on GP Us: A TAIL from the Past, Scribbled in Futhark. In Procs. of the 5th Int. Workshop on Functional High-Performance Computing (FHPC ’16). ACM, New York, NY, USA, 38–43.

Digital Library

[25]

Troels Henriksen, Martin Elsman, and Cosmin E Oancea. 2014. Size slicing: a hybrid approach to size inference in Futhark. In Proceedings of the 3rd ACM SIGPLAN workshop on Functional high-performance computing. ACM, 31–42.

Digital Library

[26]

Troels Henriksen, Ken Friis Larsen, and Cosmin E. Oancea. 2016b. Design and GPGP U Performance of Futhark’s Redomap Construct. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming (ARRAY 2016). ACM, New York, NY, USA, 17–24.

Digital Library

[27]

Troels Henriksen and Cosmin E Oancea. 2014. Bounds checking: An instance of hybrid analysis. In Proceedings of ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming. ACM, 88.

Digital Library

[28]

Troels Henriksen, Niels G. W. Serup, Martin Elsman, Fritz Henglein, and Cosmin E. Oancea. 2017. Futhark: Purely Functional GP U-programming with Nested Parallelism and In-place Array Updates. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2017). ACM, New York, NY, USA, 556–571.

Digital Library

[29]

Anders Kiel Hovgaard. 2018. Higher-Order Functions for a High-Performance Programming Language for GPUs. Master’s thesis. Department of Computer Science, University of Copenhagen, Universitetsparken 5, DK-2100, Denmark.

[30]

Kenneth E. Iverson. 1962. A Programming Language. John Wiley and Sons, Inc.

Digital Library

[31]

Stephen Joe and Frances Y. Kuo. 2003. Remark on Algorithm 659: Implementing Sobol’s Quasirandom Sequence Generator. ACM Trans. Math. Softw. 29, 1 (March 2003), 49–57.

Digital Library

[32]

Thomas Johnsson. 1985. Lambda Lifting: Transforming Programs to Recursive Equations. In Proceedings of the international conference on Functional Programming Languages and Computer Architecture. Springer-Verlag New York, Inc., New York, NY, USA, 190–203. http://dl.acm.org/citation.cfm?id=5280.5292

Digital Library

[33]

Rasmus Wriedt Larsen and Troels Henriksen. 2017. Strategies for Regular Segmented Reductions on GP U. In Proceedings of the 6th ACM SIGPLAN International Workshop on Functional High-Performance Computing (FHPC 2017). ACM, New York, NY, USA, 42–52.

Digital Library

[34]

Daniel K. Lee, Karl Crary, and Robert Harper. 2007. Towards a Mechanized Metatheory of Standard ML. In Proceedings of the 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’07). ACM, New York, NY, USA, 173–184.

Digital Library

[35]

Xavier Leroy. 1995. Applicative Functors and Fully Transparent Higher-order Modules. In Proceedings of the 22nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’95). ACM, New York, NY, USA, 142–153.

Digital Library

[36]

Xavier Leroy. 2000. A Modular Module System. Journal of Functional Programming 10, 3 (May 2000), 269–303.

Digital Library

[37]

Geoffrey Mainland and Greg Morrisett. 2010. Nikola: Embedding Compiled GP U Functions in Haskell. In Proceedings of the Third ACM Haskell Symposium on Haskell (Haskell ’10). ACM, New York, NY, USA, 67–78.

Digital Library

[38]

Trevor L. McDonell, Manuel MT Chakravarty, Gabriele Keller, and Ben Lippmeier. 2013. Optimising Purely Functional GP U Programs. In Proceedings of the 18th ACM SIGPLAN International Conference on Functional Programming (ICFP ’13).

Digital Library

[39]

Robin Milner, Mads Tofte, Robert Harper, and David MacQueen. 1997. The Definition of Standard ML (Revised). MIT Press.

Digital Library

[40]

Shayan Najd, Sam Lindley, Josef Svenningsson, and Philip Wadler. 2016. Everything Old is New Again: Quoted Domainspecific Languages. In Proceedings of the 2016 ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation (PEPM ’16). ACM, New York, NY, USA, 25–36.

Digital Library

[41]

Nathaniel Nystrom, Derek White, and Kishen Das. 2011. Firepile: Run-time Compilation for GP Us in Scala. In Proceedings of the 10th ACM International Conference on Generative Programming and Component Engineering (GPCE ’11). ACM, New York, NY, USA, 107–116.

Digital Library

[42]

Simon Peyton Jones, Jean-Marc Eber, and Julian Seward. 2000. Composing Contracts: an Adventure in Financial Engineering (functional pearl). In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming (ICFP ’00). ACM, 280–292.

Digital Library

[43]

Benjamin C. Pierce, Arthur Azevedo de Amorim, Chris Casinghino, Marco Gaboardi, Michael Greenberg, C ˇ at ˇ alin Hriţcu, Vilhelm Sjöberg, and Brent Yorgey. 2016. Software Foundations. Electronic textbook. Version 4.0. http://www.cis.upenn. edu/~bcpierce/sf .

[44]

Andrew M. Pitts. 2013. Nominal Sets: Names and Symmetry in Computer Science. Cambridge University Press, New York, NY, USA.

[45]

Norman Ramsey, Kathleen Fisher, and Paul Govereau. 2005. An Expressive Language of Signatures. In Proceedings of the Tenth ACM SIGPLAN International Conference on Functional Programming (ICFP ’05). ACM, New York, NY, USA, 27–40.

Digital Library

[46]

John C. Reynolds. 1995. Using Functor Categories to Generate Intermediate Code. In Proceedings of the 22Nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’95). ACM, New York, NY, USA, 25–36.

Digital Library

[47]

Andreas Rossberg and Derek Dreyer. 2013. Mixin’ Up the ML Module System. ACM Transactions on Programming Languages and Systems 35, 1, Article 2 (April 2013), 2:1–2:84 pages.

Digital Library

[48]

Andreas Rossberg, Claudio V. Russo, and Derek Dreyer. 2014. F-ing modules. Journal of Functional Programming 24, 5 (2014), 529–607.

[49]

Claudio V. Russo. 1999. Non-dependent Types for Standard ML Modules. In Proceedings of the International Conference PPDP’99 on Principles and Practice of Declarative Programming (PPDP ’99). Springer-Verlag, London, UK, UK, 80–97.

Digital Library

[50]

Zhong Shao. 1999. Transparent Modules with Fully Syntatic Signatures. In Proceedings of the Fourth ACM SIGPLAN International Conference on Functional Programming (ICFP ’99). ACM, New York, NY, USA, 220–232.

Digital Library

[51]

Zhong Shao and Andrew W. Appel. 1993. Smartest Recompilation. In Proceedings of the 20th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’93). ACM, New York, NY, USA, 439–450.

Digital Library

[52]

Michel Steuwer, Christian Fensch, Sam Lindley, and Christophe Dubach. 2015. Generating Performance Portable Code Using Rewrite Rules: From High-level Functional Expressions to High-performance OpenCL Code. SIGPLAN Not. 50, 9 (Aug. 2015), 205–217.

Digital Library

[53]

Michel Steuwer, Toomas Remmelg, and Christophe Dubach. 2017. Lift: A Functional Data-parallel IR for High-performance GP U Code Generation. In Proceedings of the 2017 International Symposium on Code Generation and Optimization (CGO 2017). IEEE Press, Piscataway, NJ, USA, 74–85.

[54]

Joel Svensson. 2011. Obsidian: GPU Kernel Programming in Haskell. Ph.D. Dissertation. Chalmers University of Technology.

[55]

David Swasey, Tom Murphy VII, Karl Crary, and Robert Harper. 2006. A Separate Compilation Extension to Standard ML. In Proceedings of the ACM SIGPLAN Workshop on ML (ML ’06). ACM.

Digital Library

[56]

Walid Taha and Tim Sheard. 2000. MetaML and multi-stage programming with explicit annotations. Theoretical Computer Science 248, 1 (2000), 211 – 242. PEPM’97.

Digital Library

[57]

William W. Tait. 1967. Intensional interpretations of functionals of finite type. Journal of symbolic logic 32 (1967), 198–212.

[58]

Yong Kiam Tan, Magnus O. Myreen, Ramana Kumar, Anthony Fox, Scott Owens, and Michael Norrish. 2016. A New Verified Compiler Backend for CakeML. In Proceedings of the 21st ACM SIGPLAN International Conference on Functional Programming (ICFP 2016). ACM, New York, NY, USA, 60–73.

Digital Library

[59]

Todd L. Veldhuizen. 1999. C++ Templates as Partial Evaluation. In Proceedings of the 1999 ACM SIGPLAN Workshop on Partial Evaluation and Semantics-Based Program Manipulation. 13–18. Technical report BRICS-NS-99-1.

[60]

P. Wadler. 1987. Views: A Way for Pattern Matching to Cohabit with Data Abstraction. In Proceedings of the 14th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages (POPL ’87). ACM, New York, NY, USA, 307–313.

Digital Library

[61]

P. Wadler and S. Blott. 1989. How to Make Ad-hoc Polymorphism Less Ad Hoc. In Proceedings of the 16th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’89). ACM, New York, NY, USA, 60–76.

Digital Library

[62]

Leo White, Frédéric Bour, and Jeremy Yallop. 2015. Modular implicits. EPTCS 198, 2 (December 2015), 22–63. In Proceedings of ML/OCaml Workshop 2014. arXiv:1512.01438.

Cited By

Suwa TIgarashi A(2024)An ML-Style Module System for Cross-Stage Type Abstraction in Multi-stage ProgrammingFunctional and Logic Programming10.1007/978-981-97-2300-3_13(237-272)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1007/978-981-97-2300-3_13
Bailly LHenriksen TElsman MKeller GWestrick S(2023)Shape-Constrained Array Programming with Size-Dependent TypesProceedings of the 11th ACM SIGPLAN International Workshop on Functional High-Performance and Numerical Computing10.1145/3609024.3609412(29-41)Online publication date: 30-Aug-2023
https://dl.acm.org/doi/10.1145/3609024.3609412
Schenck RRønning OHenriksen TOancea CWolf FShende SCulhane CAlam SJagode H(2022)AD for an array language with nested parallelismProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.5555/3571885.3571962(1-15)Online publication date: 13-Nov-2022
https://dl.acm.org/doi/10.5555/3571885.3571962
Show More Cited By

Index Terms

Static interpretation of higher-order modules in Futhark: functional GPU programming in the large
1. Software and its engineering
  1. Software notations and tools
    1. Formal language definitions
      1. Semantics
    2. General programming languages
      1. Language features
        Modules / packages
      2. Language types
        Functional languages
        Parallel programming languages
  2. Software organization and properties
    1. Software system structures
      1. Abstraction, modeling and modularity

Recommendations

Futhark: purely functional GPU-programming with nested parallelism and in-place array updates
PLDI '17

Futhark is a purely functional data-parallel array language that offers a machine-neutral programming model and an optimising compiler that generates OpenCL code for GPUs.

This paper presents the design and implementation of three key features of ...
Futhark: purely functional GPU-programming with nested parallelism and in-place array updates
PLDI 2017: Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation

Futhark is a purely functional data-parallel array language that offers a machine-neutral programming model and an optimising compiler that generates OpenCL code for GPUs.

This paper presents the design and implementation of three key features of ...
Definitional Interpreters for Higher-Order Programming Languages

Higher-order programming languages (i.e., languages in which procedures or labels can occur as values) are usually defined by interpreters that are themselves written in a programming language based on the lambda calculus (i.e., an applicative language ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages

Proceedings of the ACM on Programming Languages Volume 2, Issue ICFP

September 2018

1133 pages

EISSN:2475-1421

DOI:10.1145/3243631

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 July 2018

Published in PACMPL Volume 2, Issue ICFP

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
698
Total Downloads

Downloads (Last 12 months)139
Downloads (Last 6 weeks)35

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Suwa TIgarashi A(2024)An ML-Style Module System for Cross-Stage Type Abstraction in Multi-stage ProgrammingFunctional and Logic Programming10.1007/978-981-97-2300-3_13(237-272)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1007/978-981-97-2300-3_13
Bailly LHenriksen TElsman MKeller GWestrick S(2023)Shape-Constrained Array Programming with Size-Dependent TypesProceedings of the 11th ACM SIGPLAN International Workshop on Functional High-Performance and Numerical Computing10.1145/3609024.3609412(29-41)Online publication date: 30-Aug-2023
https://dl.acm.org/doi/10.1145/3609024.3609412
Schenck RRønning OHenriksen TOancea CWolf FShende SCulhane CAlam SJagode H(2022)AD for an array language with nested parallelismProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.5555/3571885.3571962(1-15)Online publication date: 13-Nov-2022
https://dl.acm.org/doi/10.5555/3571885.3571962
Schenck RRϕnning OHenriksen TOancea C(2022)AD for an Array Language with Nested ParallelismSC22: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41404.2022.00063(1-15)Online publication date: Nov-2022
https://doi.org/10.1109/SC41404.2022.00063
Henriksen TElsman MLow TGibbons J(2021)Towards size-dependent types for array programmingProceedings of the 7th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming10.1145/3460944.3464310(1-14)Online publication date: 17-Jun-2021
https://dl.acm.org/doi/10.1145/3460944.3464310
Oancea CRobroek TGieseke F(2020)Approximate Nearest-Neighbour Fields via Massively-Parallel Propagation-Assisted K-D Trees2020 IEEE International Conference on Big Data (Big Data)10.1109/BigData50022.2020.9378426(5172-5181)Online publication date: 10-Dec-2020
https://doi.org/10.1109/BigData50022.2020.9378426
Tran DHenriksen TElsman MZocca M(2019)Compositional deep learning in FutharkProceedings of the 8th ACM SIGPLAN International Workshop on Functional High-Performance and Numerical Computing10.1145/3331553.3342617(47-59)Online publication date: 18-Aug-2019
https://dl.acm.org/doi/10.1145/3331553.3342617
Henriksen TThorøe FElsman MOancea CHollingsworth JKeidar I(2019)Incremental flattening for nested data parallelismProceedings of the 24th Symposium on Principles and Practice of Parallel Programming10.1145/3293883.3295707(53-67)Online publication date: 16-Feb-2019
https://dl.acm.org/doi/10.1145/3293883.3295707
Hovgaard AHenriksen TElsman M(2019)High-Performance Defunctionalisation in FutharkTrends in Functional Programming10.1007/978-3-030-18506-0_7(136-156)Online publication date: 24-Apr-2019
https://doi.org/10.1007/978-3-030-18506-0_7

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents