Article

Applying the golden rule of sampling for query estimation

Authors:

Divyakant Agrawal,

Amr El AbbadiAuthors Info & Claims

SIGMOD '01: Proceedings of the 2001 ACM SIGMOD international conference on Management of data

Pages 449 - 460

https://doi.org/10.1145/375663.375724

Published: 01 May 2001 Publication History

Abstract

Query size estimation is crucial for many database system components. In particular, query optimizers need efficient and accurate query size estimation when deciding among alternative query plans. In this paper we propose a novel sampling technique based on the golden rule of sampling, introduced by von Neumann in 1947, for estimating range queries. The proposed technique randomly samples the frequency domain using the cumulative frequency distribution and yields good estimates without any a priori knowledge of the actual underlying distribution of spatial objects. We show experimentally that the proposed sampling technique gives smaller approximation error than the Min-Skew histogram based and wavelet based approaches for both synthetic and real datasets. Moreover, the proposed technique can be easily extended for higher dimensional datasets.

References

[1]

Swarup Acharya, Viswanath Poosala, and Sridhar Ramaswamy. Selectivity estimation in spatial databases. In SIGMOD 1999, Proceedings ACM SIGMOD International Conference on Management of Data, June 1-3, 1999, Philadephia, Pennsylvania, USA, pages 13-24. ACM Press, 1999.

Digital Library

[2]

H. Ahrens and U. Dieter. Sequential random sampling. ACM Transaction Mathematical Software, 11(2):157 - 169, June 1985.

Digital Library

[3]

Surajit Chandhuri, Rajeev Motwani, and Vivek Narasyya. On Random Sampling over Joins. In Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, pages 263-274, Philadelphia, Pennsylvania, June 1999.

Digital Library

[4]

Computational Science Education Project. Introduction to Monte Carlo Methods. http://csep1.phy.ornl.gov/mc/mc.html, 1995.

[5]

Roger Eckhardt. Stan Ulam, John Von Neumann, and the Monte Carlo Method. Los Alamos Science, (15, Spacial Issue):135, 1987.

[6]

P. B. Gibbons, V. Poosala, S. Acharya, Y. Matias Y. Bartal, S. Muthukrishnan, S. Ramaswamy, and T. Suel. AQUA: System and Techniques for Approximate Query Answering. Technical report, Bell Laboratories, Murray Hill, N/I, February 1998.

[7]

Phillip B. Gibbons and Yossi Matias. New sampling-based summary statistics for improving approximate query answers. In SIGMOD 1998, Proceedings ACM SIGMOD International Conference on Management of Data, June 2-4, 1998, Seattle, Washington, USA, pages 331-342. ACM Press, 1998.

Digital Library

[8]

Ching-Tien Ho, Rakesh Agrawal, Nimrod Megiddo, and Ramakrishnan Srikant. Range Queries in OLAP Data Cubes. In Proceedings of the 1997 ACM SIGMOD International Conference on Management of Data, pages 73-88, Arizona, 1997.

Digital Library

[9]

Yannis E. Ioannidis and Viswanath Poosala. Balancing Histogram Optimality and Practicality for Query Result Size Estimation. In Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, pages 233-244, March 1995.

Digital Library

[10]

Richard J. Lipton, Jeffrey F. Nanghton, and Donovan A. Schneider. Practical selectivity estimation through adaptive sampling. In Proceedings of 1990 ACM SIGMOD international conference on Management of data, pages 1 - 11, 1990.

Digital Library

[11]

Yossi Matias, Jeffrey Scott Vitter, and Min Wang. Wavelet-Based Histograms for Selectivity Estimation. In Proceedings of the 1998 ACM SIGMOD International Conference on Management of Data, pages 448-459, Seattle, 1998.

Digital Library

[12]

V. Poosala, Y. E. Ioannidis, P.J. Haas, and E. Shekita. Improved Histograms for Selectivity Estimation of Range Predicates. In Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data, Montreal, Canada, May 1996.

Digital Library

[13]

Viswanath Poosala. Private communication. 2000.

[14]

T. R. Smith. A digital library for geographically referenced materials. Computers, 29(5):54-60, May 1996.

Digital Library

[15]

TIGER. 1997 TIGER/Line Files (machine-readable data files). Technical report, U.S. Bureau Of the Census, Washington, DC, 1997.

[16]

Jeffrey Scott Vitter, Min Wang, and Bala Iyer. Data Cube Approximation and Histograms via Wavelets. In Proceedings of Seventh International Conference on Information and Knowledge Management - CIKM'98, pages 96-104, Bethesda, Maryland, November 1998.

Digital Library

[17]

Yi-Leh Wu, Divyakant Agrawal, and Amr E1 Abbadi. Applying the Golden Rule of Sampling for Query Estimation. Technical Report TRCS01-05, Computer Science Department, University of California at Santa Barbara, Santa Barbara, California, March 2001.

Digital Library

[18]

G. K. Zipf. Human Behavior and the Principle of Least Effort. Addison-Wesley, Reading, MA, 1949.

Cited By

Bai LZhu L(2023)Interpolation and Prediction of Spatiotemporal XML Data Integrated With Grey Dynamic ModelUncertain Spatiotemporal Data Management for the Semantic Web10.4018/978-1-6684-9108-9.ch011(193-210)Online publication date: 15-Dec-2023
https://doi.org/10.4018/978-1-6684-9108-9.ch011
Sun LLi CJi TChen H(2021)MOSE: A Monotonic Selectivity Estimator Using Learned CDFIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.3112753(1-1)Online publication date: 2021
https://doi.org/10.1109/TKDE.2021.3112753
Dutt AWang CNazi AKandula SNarasayya VChaudhuri S(2019)Selectivity estimation for range predicates using lightweight modelsProceedings of the VLDB Endowment10.14778/3329772.332978012:9(1044-1057)Online publication date: 1-May-2019
https://dl.acm.org/doi/10.14778/3329772.3329780
Show More Cited By

Index Terms

Applying the golden rule of sampling for query estimation
1. Information systems
  1. Data management systems
    1. Database management system engines
      1. Database query processing
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query processing and optimization (theory)

Recommendations

Applying the golden rule of sampling for query estimation

Query size estimation is crucial for many database system components. In particular, query optimizers need efficient and accurate query size estimation when deciding among alternative query plans. In this paper we propose a novel sampling technique ...
Query Estimation by Adaptive Sampling
ICDE '02: Proceedings of the 18th International Conference on Data Engineering

The ability to provide accurate and efficient result estimations of user queries is very important for the query optimizer in database systems. In this paper, we show that the traditional estimation techniques with data reduction points of view do not ...
Optimized stratified sampling for approximate query processing

The ability to approximately answer aggregation queries accurately and efficiently is of great benefit for decision support and data mining tools. In contrast to previous sampling-based studies, we treat the problem as an optimization problem where, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '01: Proceedings of the 2001 ACM SIGMOD international conference on Management of data

May 2001

630 pages

ISBN:1581133324

DOI:10.1145/375663

ACM SIGMOD Record Volume 30, Issue 2
June 2001
625 pages
ISSN:0163-5808
DOI:10.1145/376284
Editors:
Timos Sellis
National Technical Univ. of Athens
,
Sharad Mehrotra
Univ. of California at Irvine
Issue’s Table of Contents

Copyright © 2001 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 May 2001

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGMOD/PODS01

Sponsor:

SIGMOD

SIGMOD/PODS01: ACM SIGMOD International Conference on Management of Data

May 21 - 24, 2001

California, Santa Barbara, USA

Acceptance Rates

SIGMOD '01 Paper Acceptance Rate 44 of 293 submissions, 15%;

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
577
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bai LZhu L(2023)Interpolation and Prediction of Spatiotemporal XML Data Integrated With Grey Dynamic ModelUncertain Spatiotemporal Data Management for the Semantic Web10.4018/978-1-6684-9108-9.ch011(193-210)Online publication date: 15-Dec-2023
https://doi.org/10.4018/978-1-6684-9108-9.ch011
Sun LLi CJi TChen H(2021)MOSE: A Monotonic Selectivity Estimator Using Learned CDFIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.3112753(1-1)Online publication date: 2021
https://doi.org/10.1109/TKDE.2021.3112753
Dutt AWang CNazi AKandula SNarasayya VChaudhuri S(2019)Selectivity estimation for range predicates using lightweight modelsProceedings of the VLDB Endowment10.14778/3329772.332978012:9(1044-1057)Online publication date: 1-May-2019
https://dl.acm.org/doi/10.14778/3329772.3329780
Bai LYan LMa Z(2017)Interpolation and Prediction of Spatiotemporal Data Based on XML Integrated with Grey Dynamic ModelISPRS International Journal of Geo-Information10.3390/ijgi60401136:4(113)Online publication date: 7-Apr-2017
https://doi.org/10.3390/ijgi6040113
Riondato MAkdere MÇetintemel UZdonik SUpfal E(2011)The VC-dimension of SQL queries and selectivity estimation through samplingProceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part II10.5555/2034117.2034160(661-676)Online publication date: 5-Sep-2011
https://dl.acm.org/doi/10.5555/2034117.2034160
Riondato MAkdere MÇetintemel UZdonik SUpfal E(2011)The VC-dimension of SQL queries and selectivity estimation through samplingProceedings of the 2011th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II10.1007/978-3-642-23783-6_42(661-676)Online publication date: 5-Sep-2011
https://dl.acm.org/doi/10.1007/978-3-642-23783-6_42
Liu DHua KVu KYu N(2009)Fast Query Point Movement Techniques for Large CBIR SystemsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2008.18821:5(729-743)Online publication date: 1-May-2009
https://dl.acm.org/doi/10.1109/TKDE.2008.188
Gupta SGarg D(2009)Survey on Query Estimation in Data Streams2009 IEEE International Advance Computing Conference10.1109/IADCC.2009.4809224(1417-1422)Online publication date: Mar-2009
https://doi.org/10.1109/IADCC.2009.4809224
Lu XGuan J(2009)A new approach to building histogram for selectivity estimation in query processing optimizationComputers & Mathematics with Applications10.1016/j.camwa.2008.10.05657:6(1037-1047)Online publication date: 1-Mar-2009
https://dl.acm.org/doi/10.1016/j.camwa.2008.10.056
Luo JZhou XZhang YShen HLi J(2007)Selectivity estimation by batch-query based histogram and parametric methodProceedings of the eighteenth conference on Australasian database - Volume 6310.5555/1273730.1273741(93-102)Online publication date: 30-Mar-2007
https://dl.acm.org/doi/10.5555/1273730.1273741
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents