research-article

Open access

Fixpoints for the masses: programming with first-class Datalog constraints

Authors:

Ondřej LhotákAuthors Info & Claims

Proceedings of the ACM on Programming Languages, Volume 4, Issue OOPSLA

Article No.: 125, Pages 1 - 28

https://doi.org/10.1145/3428193

Published: 13 November 2020 Publication History

Abstract

Datalog is a declarative logic programming language that has been used in a variety of applications, including big-data analytics, language processing, networking and distributed systems, and program analysis.

In this paper, we propose first-class Datalog constraints as a mechanism to construct, compose, and solve Datalog programs at run time. The benefits are twofold: We gain the full power of a functional programming language to operate on Datalog constraints-as-values, while simultaneously we can use Datalog where it really shines: to declaratively express and solve fixpoint problems.

We present an extension of the lambda calculus with first-class Datalog constraints, including its semantics and a type system with row polymorphism based on Hindley-Milner. We prove soundness of the type system and implement it as an extension of the Flix programming language.

Supplementary Material

Auxiliary Presentation Video (oopsla20main-p5-p-video.mp4)

Datalog is a declarative logic programming language that has been used in a variety of applications, including big-data analytics, language processing, networking and distributed systems, and program analysis. In this paper, we propose first-class Datalog constraints as a mechanism to construct, compose, and solve Datalog programs at run time. The benefits are twofold: We gain the full power of a functional programming language to operate on Datalog constraints-as-values, while simultaneously we can use Datalog where it really shines: to declaratively express and solve fixpoint problems. We present an extension of the lambda calculus with first-class Datalog constraints, including its semantics and a type system with row polymorphism based on Hindley-Milner. We prove soundness of the type system and implement it as an extension of the Flix programming language.

Download
123.68 MB

References

[1]

Peter Alvaro, Neil Conway, Joseph M Hellerstein, and William R Marczak. 2011. Consistency Analysis in Bloom: a CALM and Collected Approach. In Proc. Conference on Innovative Data Systems (CIDR).

[2]

Peter Alvaro, William R Marczak, Neil Conway, Joseph M Hellerstein, David Maier, and Russell Sears. 2010. Dedalus: Datalog in time and space. In International Datalog 2.0 Workshop.

[3]

Michael Arntzenius and Neel Krishnaswami. 2019. Seminaïve evaluation for a higher-order functional language. Proc. of ACM on Programming Languages Principles of Programming Languages (POPL) ( 2019 ).

[4]

Michael Arntzenius and Neelakantan R Krishnaswami. 2016. Datafun: a functional Datalog. In Proc. International Conference on Functional Programming.

Digital Library

[5]

Pavel Avgustinov, Oege De Moor, Michael Peyton Jones, and Max Schäfer. 2016. QL: Object-oriented queries on relational data. In Proc. European Conference on Object-Oriented Programming (ECOOP 2016 ).

[6]

Francois Bancilhon, David Maier, Yehoshua Sagiv, and Jefrey D Ullman. 1985. Magic Sets and Other Strange Ways to Implement Logic Programs. In Proc. Principles of Database Systems (PODS). https://doi.org/10.1145/6012.15399

Digital Library

[7]

Aaron Bembenek and Stephen Chong. 2018. FormuLog: Datalog for static analysis involving logical formulae. arXiv preprint arXiv: 1809. 06274 ( 2018 ).

[8]

Martin Bravenboer and Yannis Smaragdakis. 2009. Strictly Declarative Specification of Sophisticated Points-To Analyses. In Proc. Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). https://doi.org/10.1145/1640089. 1640108

Digital Library

[9]

Stefano Ceri, Georg Gottlob, and Letizia Tanca. 1989. What You Always Wanted to Know About Datalog (and Never Dared to Ask). IEEE Transactions on Knowledge and Data Engineering (TKDE) ( 1989 ). https://doi.org/10.1109/69.43410

Digital Library

[10]

Stefano Ceri, Georg Gottlob, and Letizia Tanca. 2012. Logic programming and databases. Springer Science & Business Media.

Digital Library

[11]

Jacques Cohen. 1990. Constraint Logic Programming Languages. Commun. ACM ( 1990 ). https://doi.org/10.1145/79204.79209

Digital Library

[12]

Neil Conway, William R Marczak, Peter Alvaro, Joseph M Hellerstein, and David Maier. 2012. Logic and Lattices for Distributed Programming. In Proc. Symposium on Cloud Computing (SoCC). https://doi.org/10.1145/2391229.2391230

Digital Library

[13]

Luis Damas and Robin Milner. 1982. Principal Type-Schemes for Functional Programs. In Proc. Symposium on Principles of Programming Languages (POPL).

Digital Library

[14]

O de Moor, G Gottlob, T Furche, and AJ Sellers (Eds.). 2011. Datalog Reloaded-First International Workshop, Datalog 2010. https://doi.org/10.1007/978-3-642-24206-9

[15]

Matthias Felleisen, Robert Bruce Findler, and Matthew Flatt. 2009. Semantics Engineering with PLT Redex.

Digital Library

[16]

Melvin Fitting. 2002. Fixpoint Semantics for Logic Programming a Survey. Theoretical Computer Science (TCS) ( 2002 ). https://doi.org/10.1016/S0304-3975 ( 00 ) 00330-3

Digital Library

[17]

Michael Gelfond and Vladimir Lifschitz. 1988. The Stable Model Semantics for Logic Programming. In Proc. International Conference on Logic Programming (ICLP/SLP).

[18]

Michael Gelfond and Vladimir Lifschitz. 1991. Classical Negation in Logic Programs and Disjunctive Databases. New Generation Computing ( 1991 ). https://doi.org/10.1007/BF03037169

[19]

Goetz Graefe. 1993. Query Evaluation Techniques for Large Databases. ACM Computing Surveys (CSUR) ( 1993 ). https: //doi.org/10.1145/152610.152611

Digital Library

[20]

Steve Gregory. 1987. Parallel Logic Programming in PARLOG: The Language and its Implementation. Addison-Wesley.

[21]

Elnar Hajiyev, Mathieu Verbaere, and Oege De Moor. 2006. codeQuest: Scalable Source Code Queries with Datalog. In Proc. European Conference on Object-Oriented Programming (ECOOP). https://doi.org/10.1007/11785477_2

Digital Library

[22]

Daniel Halperin, Victor Teixeira de Almeida, Lee Lee Choo, Shumo Chu, Paraschos Koutris, Dominik Moritz, Jennifer Ortiz, Vaspol Ruamviboonsuk, Jingjing Wang, Andrew Whitaker, et al. 2014. Demonstration of the Myria big data management service. In Proc. International Conference on Management of Data.

Digital Library

[23]

Fergus Henderson, Thomas Conway, Zoltan Somogyi, David Jefery, Peter Schachte, Simon Taylor, Chris Speirs, Tyson Dowd, Ralph Becket, and Mark Brown. 1996. The Mercury language reference manual. ( 1996 ).

[24]

Shan Shan Huang, Todd Jefrey Green, and Boon Thau Loo. 2011. Datalog and Emerging Applications: An Interactive Tutorial. In Proc. Management of Data (SIGMOD). https://doi.org/10.1145/1989323.1989456

Digital Library

[25]

Joxan Jafar and Jean-Louis Lassez. 1987. Constraint Logic Programming. In Proc. Principles of Programming Languages (POPL). https://doi.org/10.1145/41625.41635

Digital Library

[26]

Joxan Jafar and Michael J Maher. 1994. Constraint Logic Programming: A Survey. Journal of Logic Programming ( 1994 ). https://doi.org/10.1016/ 0743-1066 ( 94 ) 90033-7

[27]

Herbert Jordan, Bernhard Scholz, and Pavle Subotić. 2016. Souflé: On Synthesis of Program Analyzers. In International Conference on Computer Aided Verification.

[28]

Herbert Jordan, Bernhard Scholz, and Pavle Subotic. 2018. Two concurrent data structures for eficient datalog query processing. In Proc. Symposium on Principles and Practice of Parallel Programming.

[29]

George Kastrinis and Yannis Smaragdakis. 2013. Hybrid context-sensitivity for points-to analysis. Proc. International Conference on Programming Language Design and Implementation (PLDI) ( 2013 ).

Digital Library

[30]

Ross D King. 2004. Applying Inductive Logic Programming to Predicting Gene Function. AI Magazine ( 2004 ).

[31]

Kenneth Kunen. 1987. Negation in Logic Programming. Journal of Logic Programming ( 1987 ). https://doi.org/10.1016/ 0743-1066 ( 87 ) 90007-0

Digital Library

[32]

Monica S Lam, John Whaley, V Benjamin Livshits, Michael C Martin, Dzintars Avots, Michael Carbin, and Christopher Unkel. 2005. Context-sensitive Program Analysis as Database Queries. In Proc. Principles of Database Systems (PODS). https://doi.org/10.1145/1065167.1065169

Digital Library

[33]

Daan Leijen. 2005. Extensible records with scoped labels. Trends in Functional Programming ( 2005 ).

[34]

Ninghui Li and John C Mitchell. 2003. Datalog with Constraints: A Foundation for Trust Management Languages. In Proc. Practical Aspects of Declarative Languages (PADL). https://doi.org/10.1007/3-540-36388-2_6

[35]

Boon Thau Loo, Tyson Condie, Minos Garofalakis, David E Gay, Joseph M Hellerstein, Petros Maniatis, Raghu Ramakrishnan, Timothy Roscoe, and Ion Stoica. 2009. Declarative networking. Commun. ACM ( 2009 ).

[36]

Magnus Madsen and Ondřej Lhoták. 2020. Fixpoints for the Masses: Programming with First-class Datalog Constraints. Technical Report CS-2020-05. University of Waterloo. https://cs.uwaterloo.ca/sites/ca.computer-science/files/uploads/ ifles/cs-2020-05.pdf

[37]

Magnus Madsen, Ming-Ho Yee, and Ondřej Lhoták. 2016. From Datalog to Flix: A Declarative Language for Fixed Points on Lattices. In Programming Language Design and Implementation (PLDI).

[38]

Jack Minker. 1988. Foundations of Deductive Databases and Logic Programming. Morgan Kaufmann.

[39]

Raymond J Mooney. 1996. Inductive Logic Programming for Natural Language Processing. In International Conference on Inductive Logic Programming.

[40]

Christos H. Papadimitriou. 1985. A note the expressive power of Prolog. Bulletin of the European Association for Theoretical Computer Science (EATCS) ( 1985 ).

[41]

Thomas Reps, Susan Horwitz, and Mooly Sagiv. 1995. Precise Interprocedural Dataflow Analysis via Graph Reachability. In Proc. Principles of Programming Languages (POPL). https://doi.org/10.1145/199448.199462

Digital Library

[42]

Mooly Sagiv, Thomas Reps, and Susan Horwitz. 1996. Precise Interprocedural Dataflow Analysis with Applications to Constant Propagation. Theoretical Computer Science (TCS) ( 1996 ). https://doi.org/10.1016/ 0304-3975 ( 96 ) 00072-2

Digital Library

[43]

Max Schäfer and Oege de Moor. 2010. Type inference for datalog with complex type hierarchies. In Proc. Symposium on Principles of Programming Languages (POPL).

Digital Library

[44]

Bernhard Scholz, Herbert Jordan, Pavle Subotić, and Till Westmann. 2016. On fast large-scale program analysis in datalog. In Proc. International Conference on Compiler Construction (CC).

Digital Library

[45]

Jiwon Seo, Stephen Guo, and Monica S Lam. 2013. SociaLite: Datalog extensions for eficient social network analysis. In International Conference Data Engineering (ICDE).

[46]

Alexander Shkapsky, Mohan Yang, Matteo Interlandi, Hsuan Chiu, Tyson Condie, and Carlo Zaniolo. 2016. Big Data Analytics with Datalog Queries on Spark. In Proc. International Conference on Management of Data.

Digital Library

[47]

Yannis Smaragdakis, George Balatsouras, and George Kastrinis. 2013. Set-based pre-processing for points-to analysis. In Proc. International Conference on Object Oriented Programming Systems Languages & Applications (OOPSLA).

Digital Library

[48]

Yannis Smaragdakis and Martin Bravenboer. 2011. Using Datalog for Fast and Easy Program Analysis. In Datalog Reloaded. https://doi.org/10.1145/1926385.1926390

Digital Library

[49]

Yannis Smaragdakis, Martin Bravenboer, and Ondrej Lhoták. 2011. Pick Your Contexts Well: Understanding Object-Sensitivity. In Proc. Principles of Programming Languages (POPL). https://doi.org/10.1145/1925844.1926390

Digital Library

[50]

Yannis Smaragdakis, George Kastrinis, and George Balatsouras. 2014. Introspective analysis: context-sensitivity, across the board. Proc. International Conference on Programming Language Design and Implementation (PLDI).

Digital Library

[51]

Zoltan Somogyi, Fergus Henderson, and Thomas Conway. 1996. The Execution Algorithm of Mercury, an Eficient Purely Declarative Logic Programming Language. The Journal of Logic Programming ( 1996 ).

[52]

Zoltan Somogyi, Fergus J Henderson, and Thomas Charles Conway. 1995. Mercury, an eficient purely declarative logic programming language. Australian Computer Science Communications ( 1995 ).

[53]

Soufle Authors. 2018. Soufle. https://soufle-lang.github.io/ [Online; accessed 18-October-2018].

[54]

Pavle Subotic, Herbert Jordan, Lijun Chang, Alan Fekete, and Bernhard Scholz. 2018. Automatic Index Selection for Large-Scale Datalog Computation. ( 2018 ).

[55]

Jefrey D Ullman. 1984. Principles of Database Systems. Galgotia publications.

[56]

Jefrey D. Ullman. 1988. Principles of Database and Knowledge-Base Systems.

[57]

Todd L Veldhuizen. 2012. Leapfrog Triejoin: a worst-case optimal join algorithm. arXiv preprint arXiv:1210.0481 ( 2012 ).

[58]

Andrew K Wright and Matthias Felleisen. 1994. A Syntactic Approach to Type Soundness. Information and computation ( 1994 ).

[59]

David Zook, Emir Pasalic, and Beata Sarna-Starosta. 2009. Typed datalog. In International Symposium on Practical Aspects of Declarative Languages (PADL).

Cited By

Klopp DErdweg SPacak A(2024)A Typed Multi-level Datalog IR and Its Compiler FrameworkProceedings of the ACM on Programming Languages10.1145/36897678:OOPSLA2(1586-1614)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689767
Klopp DErdweg SPacak A(2024)Object-Oriented Fixpoint Programming with DatalogProceedings of the ACM on Programming Languages10.1145/36897138:OOPSLA2(60-86)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689713
Lutze MMadsen M(2024)Associated Effects: Flexible Abstractions for Effectful ProgrammingProceedings of the ACM on Programming Languages10.1145/36563938:PLDI(394-416)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656393
Show More Cited By

Index Terms

Fixpoints for the masses: programming with first-class Datalog constraints
1. Software and its engineering
  1. Software notations and tools
    1. General programming languages
      1. Language types
        Constraint and logic languages
        Functional languages

Recommendations

Datafun: a functional Datalog
ICFP '16

Datalog may be considered either an unusually powerful query language or a carefully limited logic programming language. Datalog is declarative, expressive, and optimizable, and has been applied successfully in a wide variety of problem domains. ...
Datafun: a functional Datalog
ICFP 2016: Proceedings of the 21st ACM SIGPLAN International Conference on Functional Programming

Datalog may be considered either an unusually powerful query language or a carefully limited logic programming language. Datalog is declarative, expressive, and optimizable, and has been applied successfully in a wide variety of problem domains. ...
Logic Programming with Arrays

Nial is a programming language designed around a mathematical treatment of data as nested arrays. A goal of the research described is to integrate within Nial a functional style of programming based on the theory of arrays with the declarative ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages

Proceedings of the ACM on Programming Languages Volume 4, Issue OOPSLA

November 2020

3108 pages

EISSN:2475-1421

DOI:10.1145/3436718

Issue’s Table of Contents

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 November 2020

Published in PACMPL Volume 4, Issue OOPSLA

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
1,463
Total Downloads

Downloads (Last 12 months)382
Downloads (Last 6 weeks)31

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Klopp DErdweg SPacak A(2024)A Typed Multi-level Datalog IR and Its Compiler FrameworkProceedings of the ACM on Programming Languages10.1145/36897678:OOPSLA2(1586-1614)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689767
Klopp DErdweg SPacak A(2024)Object-Oriented Fixpoint Programming with DatalogProceedings of the ACM on Programming Languages10.1145/36897138:OOPSLA2(60-86)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689713
Lutze MMadsen M(2024)Associated Effects: Flexible Abstractions for Effectful ProgrammingProceedings of the ACM on Programming Languages10.1145/36563938:PLDI(394-416)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656393
Krishna SLal APavlogiannis ATuppe O(2024)On-the-Fly Static Analysis via Dynamic Bidirected Dyck ReachabilityProceedings of the ACM on Programming Languages10.1145/36328848:POPL(1239-1268)Online publication date: 5-Jan-2024
https://dl.acm.org/doi/10.1145/3632884
Madsen Mvan de Pol JHenriksen T(2023)Fast and Efficient Boolean Unification for Hindley-Milner-Style Type and Effect SystemsProceedings of the ACM on Programming Languages10.1145/36228167:OOPSLA2(516-543)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3622816
Madsen MScholliers CSinger J(2022)The Principles of the Flix Programming LanguageProceedings of the 2022 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3563835.3567661(112-127)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3563835.3567661
Sahebolamri AGilray TMicinski KEgger BSmith A(2022)Seamless deductive inference via macrosProceedings of the 31st ACM SIGPLAN International Conference on Compiler Construction10.1145/3497776.3517779(77-88)Online publication date: 19-Mar-2022
https://dl.acm.org/doi/10.1145/3497776.3517779
KÖRNER PLEUSCHEL MBARBOSA JCOSTA VDAHL VHERMENEGILDO MMORALES JWIELEMAKER JDIAZ DABREU SCIATTO G(2022)Fifty Years of Prolog and BeyondTheory and Practice of Logic Programming10.1017/S147106842200010222:6(776-858)Online publication date: 17-May-2022
https://doi.org/10.1017/S1471068422000102
Madsen Mvan de Pol J(2021)Relational nullable types with Boolean unificationProceedings of the ACM on Programming Languages10.1145/34854875:OOPSLA(1-28)Online publication date: 15-Oct-2021
https://dl.acm.org/doi/10.1145/3485487
Riouak IReichenbach CHedin GFors N(2021)A Precise Framework for Source-Level Control-Flow Analysis2021 IEEE 21st International Working Conference on Source Code Analysis and Manipulation (SCAM)10.1109/SCAM52516.2021.00009(1-11)Online publication date: Sep-2021
https://doi.org/10.1109/SCAM52516.2021.00009

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents