research-article

Adaptive join processing in pipelined plans

Authors:

Kwanchai Eurviriyanukul,

Norman W. Paton,

Alvaro A. A. Fernandes,

Steven J. LyndenAuthors Info & Claims

EDBT '10: Proceedings of the 13th International Conference on Extending Database Technology

Pages 183 - 194

https://doi.org/10.1145/1739041.1739066

Published: 22 March 2010 Publication History

Abstract

In adaptive query processing, the way in which a query is evaluated is changed in the light of feedback obtained from the environment during query evaluation. Such feedback may, for example, establish that misleading selectivity estimates were used when the query was compiled, leading to the optimizer choosing an inappropriate join order or unsuitable join algorithms. This paper describes how joins can be reordered, and the join algorithms used replaced, while they are being evaluated in pipelined plans. Where joins are reordered and/or replaced during their evaluation, the approach avoids duplicating work that has already been carried out, by resuming from where the previous plan left off. The approach has been evaluated empirically, and shown to be effective for improving query performance in the light of misleading selectivity estimates.

References

[1]

R. Avnur and J. Hellerstein. Eddies: Continuously Adaptive Query Processing. In ACM SIGMOD, pages 261--272, 2000.

Digital Library

[2]

S. Babu, P. Bizarro, and D. DeWitt. Proactive Re-Optimization. In Proc. ACM SIGMOD, pages 107--118, 2005.

Digital Library

[3]

B. Chandramouli, C. N. Bond, S. Babu, and J. Yang. On suspending and resuming dataflows. In ICDE, pages 1289--1291, 2007.

[4]

S. Chaudhuri, R. Kaushik, and R. Ramamurthy. When can we trust progress estimators for sql queries? In SIGMOD Conference, pages 575--586, 2005.

Digital Library

[5]

R. L. Cole and G. Graefe. Optimization of dynamic query evaluation plans. In SIGMOD Conference, pages 150--160, 1994.

Digital Library

[6]

R. L. Cole and G. Graefe. Optimization of dynamic query evaluation plans. In Proc. SIGMOD, pages 150--160, 1994.

Digital Library

[7]

J. V. den Bercken, B. Blohsfeld, J. Dittrich, J.ämer, T. Schäfer, M. Schneider, and B. Seeger. Xxl - a library approach to supporting efficient implementations of advanced database queries. In Proc. VLDB, pages 39--48, 2001.

Digital Library

[8]

A. Deshpande and J. M. Hellerstein. Lifting the burden of history from adaptive query processing. In Proc. VLDB, pages 948--959, 2004.

Digital Library

[9]

A. Deshpande, Z. G. Ives, and V. Raman. Adaptive query processing. Foundations and Trends in Databases, 1(1):1--140, 2007.

Digital Library

[10]

K. Eurviriyanukul, A. A. Fernandes, and N. W. Paton. A Foundation for the Replacement of Pipelined Physical Join Operators in Adaptive Query Processing. In Proc. EDBT Workshops, pages 589--600. Springer-Verlag, 2006.

Digital Library

[11]

H. Garcia-Molina, J. Widom, and J. Ullman. Database System Implementation. Prentice-Hall, Inc., 1999.

Digital Library

[12]

G. Graefe. Query Evaluation Techniques for Large Databases. ACM Comput. Surv., 25(2):73--170, 1993.

Digital Library

[13]

Z. Ives, A. Halevy, and D. Weld. Adapting to Source Properties in Data Integration Queries. In Proc. SIGMOD, pages 395--406, 2004.

Digital Library

[14]

N. Kabra and D. J. DeWitt. Efficient mid-query re-optimization of sub-optimal query execution plans. In Proc. SIGMOD, pages 106--117, 1998.

Digital Library

[15]

J. Krämer, Y. Yang, M. Cammert, B. Seeger, and D. Papadias. Dynamic plan migration for snapshot-equivalent continuous queries in data stream systems. In EDBT Workshops, pages 497--516. Springer, 2006.

Digital Library

[16]

Q. Li, M. Shao, V. Markl, K. Beyer, L. Colby, and G. Lohman. Adaptively Reordering Joins during Query Execution. In Proc. ICDE, pages 26--35, 2007.

[17]

V. Markl, V. Raman, D. Simmen, G. Lohman, and H. Pirahesh. Robust query processing through progressive optimization. In Proc. ACM SIGMOD, pages 659--670, 2004.

Digital Library

[18]

V. Raman, A. Deshpande, and J. M. Hellerstein. Using State Modules for Adaptive Query Processing. In Proc. ICDE, pages 353--364, 2003.

[19]

Y. Zhu, E. A. Rundensteiner, and G. T. Heineman. Dynamic plan migration for continuous queries over data streams. In SIGMOD Conference, pages 431--442, 2004.

Digital Library

Cited By

Zhang YChronis YPatel JRekatsinas T(2023)Simple Adaptive Query Processing vs. Learned Query Optimizers: Observations and AnalysisProceedings of the VLDB Endowment10.14778/3611479.361150116:11(2962-2975)Online publication date: 24-Aug-2023
https://dl.acm.org/doi/10.14778/3611479.3611501
Faltín TTrigonakis VBerdai AFusco LIorgulescu CHong SChafi HHartig OYoshida Y(2023)Better Distributed Graph Query Planning With Scouting QueriesProceedings of the 6th Joint Workshop on Graph Data Management Experiences & Systems (GRADES) and Network Data Analytics (NDA)10.1145/3594778.3594884(1-9)Online publication date: 18-Jun-2023
https://dl.acm.org/doi/10.1145/3594778.3594884
Borovica-Gajic RIdreos SAilamaki AZukowski MFraser C(2018)Smooth ScanThe VLDB Journal — The International Journal on Very Large Data Bases10.1007/s00778-018-0507-827:4(521-545)Online publication date: 1-Aug-2018
https://dl.acm.org/doi/10.1007/s00778-018-0507-8
Show More Cited By

Index Terms

Adaptive join processing in pipelined plans
1. Information systems
  1. Data management systems
    1. Database management system engines
      1. Database query processing
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query processing and optimization (theory)

Recommendations

Processing multi-join queries
Distributed stream join query processing with semijoins

This paper addresses the distributed stream processing of window-based multi-way join queries considering the semijoin as a key join operator. In distributed stream processing, data streams arriving at remote sites need to be shipped to the processing ...
Adaptive Multi-join Query Processing in PDBMS
ICDE '09: Proceedings of the 2009 IEEE International Conference on Data Engineering

Traditionally, distributed databases assume that the small) set of nodes participating in a query is known apriori, the data is well placed, and the statistics are readily available. However, these assumptions are no longer valid in a Peer-based ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

EDBT '10: Proceedings of the 13th International Conference on Extending Database Technology

March 2010

741 pages

ISBN:9781605589459

DOI:10.1145/1739041

Editors:
Ioana Manolescu
INRIA, France
,
Stefano Spaccapietra
EPFL, Switzerland
,
Jens Teubner
ETH Zurich, Switzerland
,
Masaru Kitsuregawa
Tokyo University, Japan
,
Alain Leger
Orange - France Telecom R&D, France
,
Felix Naumann
Hasso Plattner Institute, Germany
,
Anastasia Ailamaki
EPFL, Switzerland
,
Fatma Ozcan
IBM Almaden Research Center

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 March 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

EDBT/ICDT '10

EDBT/ICDT '10: EDBT/ICDT '10 joint conference

March 22 - 26, 2010

Lausanne, Switzerland

Acceptance Rates

Overall Acceptance Rate 7 of 10 submissions, 70%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
220
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang YChronis YPatel JRekatsinas T(2023)Simple Adaptive Query Processing vs. Learned Query Optimizers: Observations and AnalysisProceedings of the VLDB Endowment10.14778/3611479.361150116:11(2962-2975)Online publication date: 24-Aug-2023
https://dl.acm.org/doi/10.14778/3611479.3611501
Faltín TTrigonakis VBerdai AFusco LIorgulescu CHong SChafi HHartig OYoshida Y(2023)Better Distributed Graph Query Planning With Scouting QueriesProceedings of the 6th Joint Workshop on Graph Data Management Experiences & Systems (GRADES) and Network Data Analytics (NDA)10.1145/3594778.3594884(1-9)Online publication date: 18-Jun-2023
https://dl.acm.org/doi/10.1145/3594778.3594884
Borovica-Gajic RIdreos SAilamaki AZukowski MFraser C(2018)Smooth ScanThe VLDB Journal — The International Journal on Very Large Data Bases10.1007/s00778-018-0507-827:4(521-545)Online publication date: 1-Aug-2018
https://dl.acm.org/doi/10.1007/s00778-018-0507-8
Yarygina A(2013)Execution and optimization techniques for approximate queries in heterogeneous systemsProgramming and Computer Software10.1134/S036176881306006639:6(309-317)Online publication date: 19-Nov-2013
https://doi.org/10.1134/S0361768813060066
Gounaris ATsamoura EManolopoulos Y(2013)Adaptive Query Processing in Distributed SettingsAdvanced Query Processing10.1007/978-3-642-28323-9_9(211-236)Online publication date: 2013
https://doi.org/10.1007/978-3-642-28323-9_9
Catania BGuerrini G(2013)Approximate Queries with Adaptive ProcessingAdvanced Query Processing10.1007/978-3-642-28323-9_10(237-269)Online publication date: 2013
https://doi.org/10.1007/978-3-642-28323-9_10
Braga DGrossniklaus MPaton N(2011)Run-time adaptivity for search computingSearch computing10.5555/1983774.1983794(156-166)Online publication date: 1-Jan-2011
https://dl.acm.org/doi/10.5555/1983774.1983794
Braga DGrossniklaus MPaton N(2011)Run-Time Adaptivity for Search ComputingSearch Computing10.1007/978-3-642-19668-3_15(156-166)Online publication date: 2011
https://doi.org/10.1007/978-3-642-19668-3_15

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents