research-article

Open access

Indexed Streams: A Formal Intermediate Representation for Fused Contraction Programs

Authors:

Praneeth Kolichala,

Fredrik KjolstadAuthors Info & Claims

Proceedings of the ACM on Programming Languages, Volume 7, Issue PLDI

Article No.: 154, Pages 1169 - 1193

https://doi.org/10.1145/3591268

Published: 06 June 2023 Publication History

Abstract

We introduce indexed streams, a formal operational model and intermediate representation that describes the fused execution of a contraction language that encompasses both sparse tensor algebra and relational algebra. We prove that the indexed stream model is correct with respect to a functional semantics. We also develop a compiler for contraction expressions that uses indexed streams as an intermediate representation. The compiler is only 540 lines of code, but we show that its performance can match both the TACO compiler for sparse tensor algebra and the SQLite and DuckDB query processing libraries for relational algebra.

References

[1]

Christopher R. Aberger, Andrew Lamb, Kunle Olukotun, and Christopher Ré. 2018. LevelHeaded: A unified engine for business intelligence and linear algebra querying. In 2018 IEEE 34th International Conference on Data Engineering (ICDE). IEEE, 449–460. https://doi.org/10.1109/ICDE.2018.00048

[2]

Christopher R. Aberger, Andrew Lamb, Susan Tu, Andres Nötzli, Kunle Olukotun, and Christopher Ré. 2017. EmptyHeaded: A relational engine for graph processing. ACM Transactions on Database Systems (TODS), 42, 4 (2017), 1–44. https://doi.org/10.1145/3129246

Digital Library

[3]

Mahmoud Abo Khamis, Hung Q. Ngo, and Atri Rudra. 2016. FAQ: Questions Asked Frequently. In Proceedings of the 35th ACM SIGMOD–SIGACT–SIGAI Symposium on Principles of Database Systems. ACM, 13–28. https://doi.org/10.1145/2902251.2902280

Digital Library

[4]

Peter Ahrens, Fredrik Kjolstad, and Saman Amarasinghe. 2022. Autoscheduling for Sparse Tensor Algebra with an Asymptotic Cost Model. In Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation (PLDI 2022). Association for Computing Machinery, New York, NY, USA. 269–285. isbn:9781450392655 https://doi.org/10.1145/3519939.3523442

Digital Library

[5]

Srinivas M. Aji and Robert J. McEliece. 2000. The generalized distributive law. IEEE Transactions on Information Theory, 46, 2 (2000), March, 325–343. https://doi.org/10.1109/18.825794

Digital Library

[6]

Morton M. Astrahan, Mike W. Blasgen, Donald D. Chamberlin, Kapali P. Eswaran, Jim N Gray, Patricia P. Griffiths, W Frank King, Raymond A. Lorie, Paul R. McJones, and James W. Mehl. 1976. System R: Relational Approach to Database Management. ACM Trans. Database Syst., 1, 2 (1976), June, 97–137. issn:0362-5915 https://doi.org/10.1145/320455.320457

Digital Library

[7]

Aart Bik, Penporn Koanantakool, Tatiana Shpeisman, Nicolas Vasilache, Bixia Zheng, and Fredrik Kjolstad. 2022. Compiler Support for Sparse Tensor Computations in MLIR. ACM Trans. Archit. Code Optim., 19, 4 (2022), Article 50, Sept., 25 pages. issn:1544-3566 https://doi.org/10.1145/3544559

Digital Library

[8]

Guy E Blelloch. 1992. Nesl: A Nested Data-Parallel Language. Carnegie Mellon Univ.

Digital Library

[9]

Guy E Blelloch and John Greiner. 1996. A provable time and space efficient implementation of NESL. ACM SIGPLAN Notices, 31, 6 (1996), 213–225. https://doi.org/10.1145/232629.232650

Digital Library

[10]

Peter Boncz, Thomas Neumann, and Orri Erling. 2014. TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark. In Performance Characterization and Benchmarking, Raghunath Nambiar and Meikel Pöss (Eds.). Springer International Publishing, Cham. 61–76. isbn:978-3-319-04936-6 https://doi.org/10.1007/978-3-319-04936-6_5

Digital Library

[11]

Paul Caspi and Marc Pouzet. 1995. A Functional Extension to Lustre. In Eighth International Symposium on Languages for Intentional Programming, M. A. Orgun and E. A. Ashcroft (Eds.) (ISLIP ’95). World Scientific, Sydney, Australia.

[12]

Manuel MT Chakravarty, Roman Leshchinskiy, Simon Peyton Jones, Gabriele Keller, and Simon Marlow. 2007. Data parallel Haskell: a status report. In Proceedings of the 2007 Workshop on Declarative Aspects of Multicore Programming (DAMP ’07). 10–18.

Digital Library

[13]

Stephen Chou, Fredrik Kjolstad, and Saman Amarasinghe. 2018. Format Abstraction for Sparse Tensor Algebra Compilers. Proc. ACM Program. Lang., 2, OOPSLA (2018), Article 123, Oct., 30 pages. issn:2475-1421 https://doi.org/10.1145/3276493

Digital Library

[14]

Edgar Frank Codd. 1970. A Relational Model of Data for Large Shared Data Banks. Commun. ACM, 13, 6 (1970), June, 377–387. issn:0001-0782 https://doi.org/10.1145/362384.362685

Digital Library

[15]

Jean-Louis Colaço, Grégoire Hamon, and Marc Pouzet. 2006. Mixing Signals and Modes in Synchronous Data-Flow Systems. In Proceedings of the 6th ACM/IEEE International Conference on Embedded Software (EMSOFT ’06). Association for Computing Machinery, New York, NY, USA. 73–82. isbn:1595935428 https://doi.org/10.1145/1176887.1176899

Digital Library

[16]

Duncan Coutts, Roman Leshchinskiy, and Don Stewart. 2007. Stream Fusion: From Lists to Streams to Nothing at All. SIGPLAN Not., 42, 9 (2007), Oct., 315–326. issn:0362-1340 https://doi.org/10.1145/1291220.1291199

Digital Library

[17]

Timothy A Davis and Yifan Hu. 2011. The University of Florida sparse matrix collection. ACM Transactions on Mathematical Software (TOMS), 38, 1 (2011), 1–25. https://doi.org/10.1145/2049662.2049663

Digital Library

[18]

Conal Elliott. 2019. Generalized Convolution and Efficient Language Recognition. arXiv:1903.10677.

[19]

Todd J. Green, Grigoris Karvounarakis, and Val Tannen. 2007. Provenance Semirings. In Proceedings of the Twenty-Sixth ACM SIGMOD–SIGACT–SIGART Symposium on Principles of Database Systems (PODS ’07). Association for Computing Machinery, New York, NY, USA. 31–40. isbn:9781595936851 https://doi.org/10.1145/1265530.1265535

Digital Library

[20]

Nicholas Halbwachs, Paul Caspi, Pascal Raymond, and Daniel Pilaud. 1991. The synchronous data flow programming language Lustre. Proc. IEEE, 79, 9 (1991), 1305–1320. https://doi.org/10.1109/5.97300

[21]

Patrick Hall, Peter Hitchcock, and Stephen Todd. 1975. An Algebra of Relations for Machine Computation. In Proceedings of the 2nd ACM SIGACT–SIGPLAN Symposium on Principles of Programming Languages (POPL ’75). Association for Computing Machinery, New York, NY, USA. 225–232. isbn:9781450373517 https://doi.org/10.1145/512976.512998

Digital Library

[22]

G. D. Held, M. R. Stonebraker, and E. Wong. 1975. INGRES: A Relational Data Base System. In Proceedings of the May 19–22, 1975, National Computer Conference and Exposition (AFIPS ’75). Association for Computing Machinery, New York, NY, USA. 409–416. isbn:9781450379199 https://doi.org/10.1145/1499949.1500029

Digital Library

[23]

Troels Henriksen, Niels GW Serup, Martin Elsman, Fritz Henglein, and Cosmin E Oancea. 2017. Futhark: purely functional GPU-programming with nested parallelism and in-place array updates. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation. 556–571. https://doi.org/10.1145/3062341.3062354

Digital Library

[24]

Rawn Henry, Olivia Hsu, Rohan Yadav, Stephen Chou, Kunle Olukotun, Saman Amarasinghe, and Fredrik Kjolstad. 2021. Compilation of sparse array programming models. Proceedings of the ACM on Programming Languages, 5, OOPSLA (2021), 1–29. https://doi.org/10.1145/3485505

Digital Library

[25]

Richard D Hipp. 2020. SQLite. https://www.sqlite.org/index.html

[26]

2009. Intel Math Kernel Library: Reference Manual.

[27]

Simon Peyton Jones. 2003. Haskell 98 language and libraries: the revised report. Cambridge University Press.

[28]

Alfons Kemper and Thomas Neumann. 2011. HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots. In 2011 IEEE 27th International Conference on Data Engineering. 195–206. https://doi.org/10.1109/ICDE.2011.5767867

Digital Library

[29]

Timo Kersten, Viktor Leis, Alfons Kemper, Thomas Neumann, Andrew Pavlo, and Peter Boncz. 2018. Everything You Always Wanted to Know about Compiled and Vectorized Queries but Were Afraid to Ask. Proc. VLDB Endow., 11, 13 (2018), Sept., 2209–2222. issn:2150-8097 https://doi.org/10.14778/3275366.3284966

Digital Library

[30]

Oleg Kiselyov, Aggelos Biboudis, Nick Palladinos, and Yannis Smaragdakis. 2017. Stream fusion, to completeness. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL ’17). ACM, 285–299. https://doi.org/10.1145/3093333.3009880

Digital Library

[31]

Fredrik Kjolstad, Peter Ahrens, Shoaib Kamil, and Saman Amarasinghe. 2019. Tensor Algebra Compilation with Workspaces. In 2019 IEEE/ACM International Symposium on Code Generation and Optimization (CGO). ACM/IEEE, 180–192. https://doi.org/10.1109/CGO.2019.8661185

[32]

Fredrik Kjolstad, Shoaib Kamil, Stephen Chou, David Lugato, and Saman Amarasinghe. 2017. The Tensor Algebra Compiler. Proceedings of the ACM on Programming Languages, 1, OOPSLA (2017), 1–29. https://doi.org/10.1145/3133901

Digital Library

[33]

Scott Kovach, Praneeth Kolichala, Tiancheng Gu, and Fredrik Kjolstad. 2023. Artifact for “Indexed Streams: A Formal Intermediate Representation for Fused Contraction Programs”. https://doi.org/10.5281/zenodo.7809339

Digital Library

[34]

Viktor Leis, Alfons Kemper, and Thomas Neumann. 2013. The adaptive radix tree: ARTful indexing for main-memory databases. In 2013 IEEE 29th International Conference on Data Engineering (ICDE). 38–49. https://doi.org/10.1109/ICDE.2013.6544812

Digital Library

[35]

Amanda Liu, Gilbert Louis Bernstein, Adam Chlipala, and Jonathan Ragan-Kelley. 2022. Verified Tensor-Program Optimization via High-Level Scheduling Rewrites. Proc. ACM Program. Lang., 6, POPL (2022), Article 55, Jan., 28 pages. https://doi.org/10.1145/3498717

Digital Library

[36]

Tim Mattson, David Bader, Jon Berry, Aydin Buluc, Jack Dongarra, Christos Faloutsos, John Feo, John Gilbert, Joseph Gonzalez, and Bruce Hendrickson. 2013. Standards for graph algorithm primitives. In 2013 IEEE High Performance Extreme Computing Conference (HPEC). 1–2. https://doi.org/10.1109/HPEC.2013.6670338

[37]

Wes McKinney. 2010. Data Structures for Statistical Computing in Python. In Proceedings of the 9th Python in Science Conference, Stéfan van der Walt and Jarrod Millman (Eds.). 56–61. https://doi.org/10.25080/Majora-92bf1922-00a

[38]

Prashanth Menon, Todd C. Mowry, and Andrew Pavlo. 2017. Relaxed Operator Fusion for In-Memory Databases: Making Compilation, Vectorization, and Prefetching Work Together at Last. Proc. VLDB Endow., 11, 1 (2017), Sept., 1–13. issn:2150-8097 https://doi.org/10.14778/3151113.3151114

Digital Library

[39]

Yasuhiko Minamide. 1998. A functional representation of data structures with a hole. In Proceedings of the 25th ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL ’98). 75–84. https://doi.org/10.1145/268946.268953

Digital Library

[40]

Leonardo de Moura, Soonho Kong, Jeremy Avigad, Floris van Doorn, and Jakob von Raumer. 2015. The Lean theorem prover (system description). In International Conference on Automated Deduction. Springer, 378–388. https://doi.org/10.1007/978-3-319-21401-6_26

[41]

Leonardo de Moura and Sebastian Ullrich. 2021. The Lean 4 Theorem Prover and Programming Language. In International Conference on Automated Deduction. 625–635. https://doi.org/10.1007/978-3-030-79876-5_37

Digital Library

[42]

Hung Q Ngo, Ely Porat, Christopher Ré, and Atri Rudra. 2018. Worst-case optimal join algorithms. Journal of the ACM (JACM), 65, 3 (2018), 1–40. https://doi.org/10.1145/3180143

Digital Library

[43]

Hung Q Ngo, Christopher Ré, and Atri Rudra. 2014. Skew strikes back: New developments in the theory of join algorithms. ACM SIGMOD Record, 42, 4 (2014), 5–16. https://doi.org/10.1145/2590989.2590991

Digital Library

[44]

Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The PageRank citation ranking: Bringing order to the web. Stanford InfoLab.

[45]

Mark Raasveldt and Hannes Mühleisen. 2019. DuckDB: An Embeddable Analytical Database. In Proceedings of the 2019 International Conference on Management of Data (SIGMOD ’19). Association for Computing Machinery, New York, NY, USA. 1981–1984. isbn:9781450356435 https://doi.org/10.1145/3299869.3320212

Digital Library

[46]

Jonathan Ragan-Kelley, Andrew Adams, Sylvain Paris, Marc Levoy, Saman Amarasinghe, and Frédo Durand. 2012. Decoupling algorithms from schedules for easy optimization of image processing pipelines. ACM Transactions on Graphics (TOG), 31, 4 (2012), 1–12. https://doi.org/10.1145/2185520.2185528

Digital Library

[47]

Maximilian Schleich, Dan Olteanu, Mahmoud Abo Khamis, Hung Q. Ngo, and XuanLong Nguyen. 2019. A layered aggregate engine for analytics workloads. In Proceedings of the 2019 International Conference on Management of Data. 1642–1659. https://doi.org/10.1145/3299869.3324961

Digital Library

[48]

Ryan Senanayake, Changwan Hong, Ziheng Wang, Amalee Wilson, Stephen Chou, Shoaib Kamil, Saman Amarasinghe, and Fredrik Kjolstad. 2020. A Sparse Iteration Space Transformation Framework for Sparse Tensor Algebra. Proc. ACM Program. Lang., 4, OOPSLA (2020), Article 158, Nov., 30 pages. https://doi.org/10.1145/3428226

Digital Library

[49]

Amir Shaikhha, Mohammad Dashti, and Christoph Koch. 2018. Push versus pull-based loop fusion in query engines. Journal of Functional Programming, 28 (2018), https://doi.org/10.1017/S0956796818000102

[50]

Amir Shaikhha, Andrew Fitzgibbon, Simon Peyton Jones, and Dimitrios Vytiniotis. 2017. Destination-Passing Style for Efficient Memory Management. In Proceedings of the 6th ACM SIGPLAN International Workshop on Functional High-Performance Computing (FHPC 2017). Association for Computing Machinery, New York, NY, USA. 12–23. isbn:9781450351812 https://doi.org/10.1145/3122948.3122949

Digital Library

[51]

Amir Shaikhha, Mathieu Huot, Jaclyn Smith, and Dan Olteanu. 2022. Functional collection programming with semi-ring dictionaries. Proceedings of the ACM on Programming Languages, 6, OOPSLA1 (2022), 1–33. https://doi.org/10.1145/3527333

Digital Library

[52]

Shaden Smith, Niranjay Ravindran, Nicholas D Sidiropoulos, and George Karypis. 2015. SPLATT: Efficient and parallel sparse tensor-matrix multiplication. In 2015 IEEE International Parallel and Distributed Processing Symposium. 61–70. https://doi.org/10.1109/IPDPS.2015.27

Digital Library

[53]

Michel Steuwer, Toomas Remmelg, and Christophe Dubach. 2017. Lift: a functional data-parallel IR for high-performance GPU code generation. In 2017 IEEE/ACM International Symposium on Code Generation and Optimization (CGO). 74–85. https://doi.org/10.1109/CGO.2017.7863730

[54]

William Thies, Michal Karczmarek, and Saman Amarasinghe. 2002. StreamIt: A language for streaming applications. In International Conference on Compiler Construction. 179–196. https://doi.org/10.1007/3-540-45937-5_14

[55]

Ruiqin Tian, Luanzheng Guo, Jiajia Li, Bin Ren, and Gokcen Kestor. 2021. A High-Performance Sparse Tensor Algebra Compiler in Multi-Level IR. arXiv:2102.05187.

[56]

Transaction Processing Performance Council. TPCH. 2022. TPC Benchmark H Standard Specification. Transaction Processing Performance Council. https://www.tpc.org/tpch/

[57]

Todd L Veldhuizen. 2014. Leapfrog Triejoin: A Simple, Worst-Case Optimal Join Algorithm. In Proc. 17th International Conference on Database Theory (ICDT). OpenProceedings.org, 96–106. isbn:978-3-89318066-1 https://doi.org/10.5441/002/icdt.2014.13

[58]

Binhang Yuan, Dimitrije Jankov, Jia Zou, Yuxin Tang, Daniel Bourgeois, and Chris Jermaine. 2021. Tensor Relational Algebra for Distributed Machine Learning System Design. Proc. VLDB Endow., 14, 8 (2021), April, 1338–1350. issn:2150-8097 https://doi.org/10.14778/3457390.3457399

Digital Library

Cited By

Root AYan BLiu PGyurgyik CBik AKjolstad F(2024)Compilation of Shape Operators on Sparse ArraysProceedings of the ACM on Programming Languages10.1145/36897528:OOPSLA2(1162-1188)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689752
Raje SXu YRountev AValeev ESadayappan P(2024)CoNST: Code Generator for Sparse Tensor NetworksACM Transactions on Architecture and Code Optimization10.1145/368934221:4(1-24)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3689342
Zhang GHsu OKjolstad F(2024)Compilation of Modular and General Sparse WorkspacesProceedings of the ACM on Programming Languages10.1145/36564268:PLDI(1213-1238)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656426
Show More Cited By

Index Terms

Indexed Streams: A Formal Intermediate Representation for Fused Contraction Programs
1. Software and its engineering
  1. Software notations and tools
    1. Compilers
      1. Source code generation
    2. Context specific languages
      1. Domain specific languages

Recommendations

Programming with Streams in a Pascal-Like Language

A description is given of features which were added to a conventional programming language that will manipulate streams of values. A stream is a sequence of values of a certain fixed type. The number of elements of a stream may be determined at ...
Decoding Lua: formal semantics for the developer and the semanticist
DLS 2017: Proceedings of the 13th ACM SIGPLAN International Symposium on on Dynamic Languages

We provide formal semantics for a large subset of the Lua programming language, in its version 5.2. We validate our model by mechanizing it and testing it against the test suite of the reference interpreter of Lua, obtaining evidence that our model ...
Decoding Lua: formal semantics for the developer and the semanticist
DLS '17

We provide formal semantics for a large subset of the Lua programming language, in its version 5.2. We validate our model by mechanizing it and testing it against the test suite of the reference interpreter of Lua, obtaining evidence that our model ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages

Proceedings of the ACM on Programming Languages Volume 7, Issue PLDI

June 2023

2020 pages

EISSN:2475-1421

DOI:10.1145/3554310

Editor:
Michael Hicks
Amazon, USA

Issue’s Table of Contents

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution 4.0 International License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 June 2023

Published in PACMPL Volume 7, Issue PLDI

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
732
Total Downloads

Downloads (Last 12 months)364
Downloads (Last 6 weeks)33

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Root AYan BLiu PGyurgyik CBik AKjolstad F(2024)Compilation of Shape Operators on Sparse ArraysProceedings of the ACM on Programming Languages10.1145/36897528:OOPSLA2(1162-1188)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689752
Raje SXu YRountev AValeev ESadayappan P(2024)CoNST: Code Generator for Sparse Tensor NetworksACM Transactions on Architecture and Code Optimization10.1145/368934221:4(1-24)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3689342
Zhang GHsu OKjolstad F(2024)Compilation of Modular and General Sparse WorkspacesProceedings of the ACM on Programming Languages10.1145/36564268:PLDI(1213-1238)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656426
Gladshtein VZhao QAhrens WAmarasinghe SSergey I(2024)Mechanised Hypersafety Proofs about Structured DataProceedings of the ACM on Programming Languages10.1145/36564038:PLDI(647-670)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656403
Liu ABernstein GChlipala ARagan-Kelley J(2024)A Verified Compiler for a Functional Tensor LanguageProceedings of the ACM on Programming Languages10.1145/36563908:PLDI(320-342)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656390

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents