research-article

Factorized Databases

Authors:

Maximilian SchleichAuthors Info & Claims

ACM SIGMOD Record, Volume 45, Issue 2

Pages 5 - 16

https://doi.org/10.1145/3003665.3003667

Published: 28 September 2016 Publication History

Abstract

This paper overviews factorized databases and their application to machine learning. The key observation underlying this work is that state-of-the-art relational query processing entails a high degree of redundancy in the computation and representation of query results. This redundancy can be avoided and is not necessary for subsequent analytics such as learning regression models.

References

[1]

S. Abiteboul, R. Hull, and V. Vianu. Foundations of Databases. Addison-Wesley, 1995.

Digital Library

[2]

A. Atserias, M. Grohe, and D. Marx. Size bounds and query plans for relational joins. In FOCS, pages 739--748, 2008.

Digital Library

[3]

N. Bakibayev, T. Kociský, D. Olteanu, and J. Závodný. Aggregation and ordering in factorised databases. PVLDB, 6(14):1990--2001, 2013.

Digital Library

[4]

N. Bakibayev, D. Olteanu, and J. Závodný. FDB: A query engine for factorised relational databases. PVLDB, 5(11):1232--1243, 2012.

Digital Library

[5]

C. M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics), 2006.

Digital Library

[6]

R. Ciucanu and D. Olteanu. Worst-case optimal join at a time. Technical report, Oxford, Nov 2015.

[7]

G. Gottlob. On minimal constraint networks. Artif. Intell., 191-192:42--60, 2012.

Digital Library

[8]

T. J. Green, G. Karvounarakis, and V. Tannen. Provenance semirings. In PODS, pages 31--40, 2007.

Digital Library

[9]

M. A. Khamis, H. Q. Ngo, and A. Rudra. FAQ: Questions Asked Frequently, CoRR:1504.04044, April 2015.

[10]

P. Koutris, P. Beame, and D. Suciu. Worst-case optimal algorithms for parallel query processing. In ICDT, pages 8:1--8:18, 2016.

[11]

H. Q. Ngo, E. Porat, C. Ré, and A. Rudra. Worst-case optimal join algorithms. In PODS, pages 37--48, 2012.

Digital Library

[12]

D. Olteanu and J. Huang. Using OBDDs for efficient query evaluation on probabilistic databases. In SUM, pages 326--340, 2008.

Digital Library

[13]

D. Olteanu, C. Koch, and L. Antova. World-set decompositions: Expressiveness and efficient algorithms. TCS, 403(2-3):265--284, 2008.

Digital Library

[14]

D. Olteanu and M. Schleich. F: Regression models over factorized views. PVLDB, 9(10), 2016.

Digital Library

[15]

D. Olteanu and J. Závodný. On factorisation of provenance polynomials. In TaPP, 2011.

[16]

D. Olteanu and J. Závodný. Factorised representations of query results: size bounds and readability. In ICDT, pages 285--298, 2012.

Digital Library

[17]

D. Olteanu and J. Závodný. Size bounds for factorised representations of query results. TODS, 40(1):2, 2015.

Digital Library

[18]

J. Pearl. Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann, 1989.

Digital Library

[19]

M. Schleich, D. Olteanu, and R. Ciucanu. Learning linear regression models over factorized joins. In SIGMOD, 2016.

Digital Library

[20]

P. Sen, A. Deshpande, and L. Getoor. Read-once functions and query evaluation in probabilistic databases. PVLDB, 3(1):1068--1079, 2010.

Digital Library

[21]

J. Shute, R. Vingralek, B. Samwel, B. Handy, C. Whipkey, E. Rollins, M. Oancea, K. Littlefield, D. Menestrina, S. Ellner, J. Cieslewicz, I. Rae, T. Stancescu, and H. Apte. F1: A distributed SQL database that scales. PVLDB, 6(11):1068--1079, 2013.

Digital Library

[22]

Simons Institute for the Theory of Computing, UC Berkeley. Workshop on "Succinct Data Representations and Applications", September 2013.

[23]

T. L. Veldhuizen. Triejoin: A simple, worst-case optimal join algorithm. In ICDT, pages 96--106, 2014.

Cited By

Kakaraparthy APatel J(2024)SplitDF: Splitting Dataframes for Memory-Efficient Data AnalysisProceedings of the VLDB Endowment10.14778/3665844.366584917:9(2175-2184)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.14778/3665844.3665849
Makhija NGatterbauer W(2024)Minimally Factorizing the Provenance of Self-join Free Conjunctive QueriesProceedings of the ACM on Management of Data10.1145/36516052:2(1-24)Online publication date: 14-May-2024
https://dl.acm.org/doi/10.1145/3651605
Arroyuelo DGómez-Brandón AHogan ANavarro GReutter JRojas-Ledesma JSoto A(2024)The Ring: Worst-case Optimal Joins in Graph Databases using (Almost) No Extra SpaceACM Transactions on Database Systems10.1145/364482449:2(1-45)Online publication date: 23-Mar-2024
https://dl.acm.org/doi/10.1145/3644824
Show More Cited By

Recommendations

Demonstration of the FDB query engine for factorised databases

FDB is an in-memory query engine for factorised databases, which are relational databases that use compact factorised representations at the physical layer to reduce data redundancy and boost query performance.

We demonstrate FDB using real data sets ...
Aggregation and ordering in factorised databases

A common approach to data analysis involves understanding and manipulating succinct representations of data. In earlier work, we put forward a succinct representation system for relational data called factorised databases and reported on the main-memory ...
Integrated non-factorized variational inference
NIPS'13: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2

We present a non-factorized variational method for full posterior inference in Bayesian hierarchical models, with the goal of capturing the posterior variable dependencies via efficient and possibly parallel computation. Our approach unifies the ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGMOD Record

ACM SIGMOD Record Volume 45, Issue 2

June 2016

66 pages

ISSN:0163-5808

DOI:10.1145/3003665

Editors:
Yanlei Diao
University of Massachusetts Amherst
,
Vanessa Braganholo
Universidade Federal Fluminense
,
Marco Brambilla
Politecnico di Milano
,
Chee Yong Chan
National University of Singapore
,
Rada Chirkova
North Carolina State University
,
Zackary Ives
University of Pennsylvania
,
Anastasios Kementsietsidis
Google Research
,
Jeffrey Naughton
University of Wisconsin-Madison
,
Frank Neven
Hasselt University
,
Olga Papaemmanoui
Brandeis Univesity
,
Aditya Parameswaran
University of Illinois
,
Anish Das Sarma
Google Research
,
Alkis Simitsis
HP Labs
,
Wang-Chiew Tan
University of California Santa Cruz
,
Nesime Tatbul
MIT CSAIL
,
Marianne Winslett
University of Illinois
,
Jun Yang
Duke University

Issue’s Table of Contents

Copyright © 2016 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 September 2016

Published in SIGMOD Volume 45, Issue 2

Check for updates

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

56
Total Citations
View Citations
477
Total Downloads

Downloads (Last 12 months)87
Downloads (Last 6 weeks)7

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kakaraparthy APatel J(2024)SplitDF: Splitting Dataframes for Memory-Efficient Data AnalysisProceedings of the VLDB Endowment10.14778/3665844.366584917:9(2175-2184)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.14778/3665844.3665849
Makhija NGatterbauer W(2024)Minimally Factorizing the Provenance of Self-join Free Conjunctive QueriesProceedings of the ACM on Management of Data10.1145/36516052:2(1-24)Online publication date: 14-May-2024
https://dl.acm.org/doi/10.1145/3651605
Arroyuelo DGómez-Brandón AHogan ANavarro GReutter JRojas-Ledesma JSoto A(2024)The Ring: Worst-case Optimal Joins in Graph Databases using (Almost) No Extra SpaceACM Transactions on Database Systems10.1145/364482449:2(1-45)Online publication date: 23-Mar-2024
https://dl.acm.org/doi/10.1145/3644824
Shi CLi HSui YLu JLi LXue J(2024)Pearl: A Multi-Derivation Approach to Efficient CFL-Reachability SolvingIEEE Transactions on Software Engineering10.1109/TSE.2024.343768450:9(2379-2397)Online publication date: 5-Aug-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3437684
Olteanu DVortmeier NŽivanović Ɖ(2024)Givens rotations for QR decomposition, SVD and PCA over database joinsThe VLDB Journal — The International Journal on Very Large Data Bases10.1007/s00778-023-00818-933:4(1013-1037)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s00778-023-00818-9
Huang ZSen RLiu JWu E(2023)JoinBoost: Grow Trees over Normalized Data Using Only SQLProceedings of the VLDB Endowment10.14778/3611479.361150916:11(3071-3084)Online publication date: 24-Aug-2023
https://dl.acm.org/doi/10.14778/3611479.3611509
Wang JTrummer IKara AOlteanu D(2023)ADOPT: Adaptively Optimizing Attribute Orders for Worst-Case Optimal Join Algorithms via Reinforcement LearningProceedings of the VLDB Endowment10.14778/3611479.361148916:11(2805-2817)Online publication date: 24-Aug-2023
https://dl.acm.org/doi/10.14778/3611479.3611489
Wang YWillsey MSuciu D(2023)Free Join: Unifying Worst-Case Optimal and Traditional JoinsProceedings of the ACM on Management of Data10.1145/35892951:2(1-23)Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1145/3589295
Zhao HDeep SKoutris PGeerts FNgo HSintos S(2023)Space-Time Tradeoffs for Conjunctive Queries with Access PatternsProceedings of the 42nd ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems10.1145/3584372.3588675(59-68)Online publication date: 18-Jun-2023
https://dl.acm.org/doi/10.1145/3584372.3588675
Carmeli NTziavelis NGatterbauer WKimelfeld BRiedewald M(2023)Tractable Orders for Direct Access to Ranked Answers of Conjunctive QueriesACM Transactions on Database Systems10.1145/357851748:1(1-45)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3578517
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents