article

Free access

Query caching and optimization in distributed mediator systems

Authors:

Y. Papakonstantinou,

V. S. SubrahmanianAuthors Info & Claims

ACM SIGMOD Record, Volume 25, Issue 2

Pages 137 - 146

https://doi.org/10.1145/235968.233327

Published: 01 June 1996 Publication History

Abstract

Query processing and optimization in mediator systems that access distributed non-proprietary sources pose many novel problems. Cost-based query optimization is hard because the mediator does not have access to source statistics information and furthermore it may not be easy to model the source's performance. At the same time, querying remote sources may be very expensive because of high connection overhead, long computation time, financial charges, and temporary unavailability. We propose a cost-based optimization technique that caches statistics of actual calls to the sources and consequently estimates the cost of the possible execution plans based on the statistics cache. We investigate issues pertaining to the design of the statistics cache and experimentally analyze various tradeoffs. We also present a query result caching mechanism that allows us to effectively use results of prior queries when the source is not readily available. We employ the novel invariants mechanism, which shows how semantic information about data sources may be used to discover cached query results of interest.

References

[1]

{1} S. Abiteboul and A. Bonner. (1991) Obiects and view., In Proc. of the ACM SIGMOD Conf. on Management of Data, pp. 238-247, 1991.

Digital Library

[2]

{2} S. Abiteboul, S. Cluet, and T. Mile. (1993) Querying and updating the file., In Proc. Int. Conf. on Very Large Data Bases (VLDB), pp. 73-84, 1993.

Digital Library

[3]

{3} S. Adah, K.S. Candan, Su-Shing Chen, K. Erol and V.S. Subrahmanian. (1995) Advanced Video Information System: Data Structures and Query Processing., Accepted for publication in the ACM-Springer Multi-media Systems Journal.

Digital Library

[4]

{4} S. Adah and V.S. Subrahmanian. (1994) Amalgamating Knowledge Bases, III: Algorithms, data structures and query processing. Technical Report CS-TR-3124, Computer Science Department, University of Maryland, Aug. 1993. Accepted for publication in Journal of Logic Programming. (http://www.cs.umd.edu/projects/hermes/publications/abstracts/akbiii.ps)

[5]

{5} S. Adah and R. Emery. (1995) A Uniform Framework For Integrating Knowledge In Heterogeneous Knowledge Systems, Proc. of the Eleventh International Conference on Data Engineering, pp. 513- 520, (http://www.cs.umd.edu/projects/hermes/publications/abstracts/cons.ps)

Digital Library

[6]

{6} S. Adah and V.S. Subrahmanian. (1995) Intelligent Caching in Heterogeneous Reasoning and Mediator Systems, Proc. of the Second International Conference on Building and Sharing of Very Large-Scale Knowledge Bases (cd. N. Mars), pps 247-256, IOS Press, Twente, The Netherlands, May 1995.

[7]

{7} J. Blakeley, N. Coburn, and P.-A. Larson. (1989) Updating derived relations: Detecting irrelevant and autonomously computable updates., ACM Trans. on Database Systems, 14(3):369-400, 1989.

Digital Library

[8]

{8} Stefano Ceri and Jennifer Widom. Deriving Production Rules for Incremental View Maintenance., Proc. of the 17th VLDB Conference, 1991.

Digital Library

[9]

{9} U. Dayal. (1989) Queries and view in a object-oriented databases., In Int. Workshop on Database Programming Languages, 1989.

Digital Library

[10]

{10} U. Dayal and H. Hwang. (1984) View definition and generalization for database integration in a multidatabase system., IEEE Trans. Software Eng., SE- 10(6):628-644, 1984.

[11]

{11} N. Gehani, H. Jagadish, and W. Roome. (1994) OdeFS: A file system interface to an object-oriented database., Proc. Int. Conf. on Very Large Databases (VLDB), pp. 249-260, 1994.

Digital Library

[12]

{12} Ashish Gupta, Dinesh Katiyar, and Inderpal Singh Mumick. (1992) Counting Solutions to the View Maintenance Problem., In Workshop on Deductive Databases, JICSLP, 1992.

[13]

{13} A. Gupta, I.S. Mumick and V.S. Subrahmanian. (1993) Maintaining Views Incrementally., Proc. 1993 ACM SIGMOD Conf. on Management of Data, Washington, DC.

Digital Library

[14]

{14} E. Hanson. (1987) A performance analysis of view materialization strategies., In Proc. of the ACM SIGMOD Conf. on Management of Data, pp. 440-453, 1987.

Digital Library

[15]

{15} A. Kemper, C. Kilger, G. Moerkotte. (1994) Function Materialization in Object Bases: Design, Realization, and Evaluation., IEEE Transactions on Knowledge and Data Engineering, Vol. 6, No. 4, August 1994.

Digital Library

[16]

{16} Laks V.S. Lakshmanan, F. Sadri amd I.N. Subramanian. (1993) On the logical foundations of schema integration and evolution in Heterogeneous Database Systems. , Proc. DOOD-93, Phoenix, Arizona.

[17]

{17} Laks V.S. Lakshmanan, F. Sadri and I.N. Subramanian. (1995) Logic and Algebraic Languages for Interoperability in Multidatabase Systems, submitted to Journal of Logic Programming.

[18]

{18} S. Leach and J. Lu. (1994) Computing Annotated Logic Programs., Proceedings of the 11th International Conference on Logic Programming (ed. P. Van Hentenryck), MIT Press, pps 257-271.

Digital Library

[19]

{19} J. Lu, G. Moerkotte, J. Schue, V.S. Subrahmanian. (1995) Efficient Maintenance of Materialized Mediated Views, Proc. 1995 ACM SIGMOD Conf. on Management of Data, San Jose, CA, May 1995.

Digital Library

[20]

{20} J. Lu, A. Nerode and V.S. Subrahmanian. (1993) Hybrid Knowledge Bases, Accepted for publication in: IEEE Trans. on Knowledge and Data Engineering.

Digital Library

[21]

{21} A. Motro. (1987) Superviews: Virtual integration of multiple databases., IEEE Trans. Software Eng., 13(7):785-798, 1987.

Digital Library

[22]

{22} I. S. Mumick. (1991) Query Optimization in Deductive and Relational Databases., Ph.D. Thesis, Stanford University, CA 94305, 1991.

Digital Library

[23]

{23} M. Scholl, C. Laasch, and M. Tresch. (1991) Updatable views in object-oriented databases., In Proc. Int. Conf. on Deductive and Object-Oriented Databases (DOOD), 1991.

[24]

{24} A. Sheth and J. Larson. (1990) Federated database systems for managing distributed, heterogeneous and autonomous databases., ACM Computing Surveys, 22(3):183-236, 1990.

Digital Library

[25]

{25} Oded Shmueli and Alon Itai. (1984) Maintenance of Views. In Sigmod Record, 14(2):240-255, 1984.

Digital Library

[26]

{26} M. Stonebraker, A. Jhingran, J. Goh, and S. Potamianos. (1990) On rules, procedures, caching and views in data base systems., In Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 281-290, 1990.

Digital Library

[27]

{27} V.S. Subrahmanian. (1994) Amalgamating Knowledge Bases., ACM Trans, on Database Systems, 19, 2, pps 291-331, 1994.

Digital Library

[28]

{28} V.S. Subrahmanian, S. Adah, A. Brink, R. Emery, J. Lu, A. Rajput, T.J. Rogers, R. Ross. (1994) HERMES: A Heterogeneous Reasoning and Mediator System, submitted for publication. (http://www.cs.umd.edu/projects/hermes/ overview/paper)

[29]

{29} G. Wiederhold. (1992) Mediators in the Architecture of Future Information Systems, IEEE Computer, March 1992, pps 38-49.

Digital Library

[30]

{30} G. Wiederhold, S. Jajodia, and W. Litwin. (1993) Integrating temporal data in a heterogeneous environment. , In Temporal Databases Benjamin/Cummings, Jan. 1993.

[31]

{31} P. Selinger, M. Astrahan, D. Chamberlin, R. Lorie and T. Price. (1979) Access Path Selection in a Relational Database Management System., Proc. of the 1979 ACM SIGMOD International Conference on Management of Data, pp. 22-34.

Digital Library

[32]

{32} D. Chimenti, R. Gamboa and R. Krishnamurty. (1989) Towards an open architecture for LDL., Proc. of the 15th International VLDB Conference, pp. 195-203, Amsterdam, The Netherlands, August 1989.

Digital Library

[33]

{33} S. Chaudhuri and K. Shim. (1993) Query optimization in the Presence of Foreign Functions., Proc. of the 19th VLDB Conference.

Digital Library

[34]

{34} S. Chaudhuri, R. Krishnamurty, S. Potamianos and K. Shim. (1995) Optimizing Queries with Materialized Views, Proc. of International Conference on Data Engineering, pp. 109-200, 1995.

Digital Library

[35]

{35} J. D. Ullman. Principles of Database and knowledge Base Systems, volume 2. Computer Science Press, 1989.

Digital Library

[36]

{36} C. M. Chen and N. Rousopoulos. (1994) Adaptive Selectivity Estimation Using Query Feedback., Proc. of the 1994 ACM-SIGMOD Conference on Management of Data, pp. 161-172.

Digital Library

[37]

{37} H. Tamaki and T. Sato. (1986) OLD Resolution with Tabulation, Proc. 3rd Intl. Conf. on Logic Programming (ed. E. Shapiro), pps 84-98, Springer.

Digital Library

[38]

{38} D. S. Warren. (1992) Memoing for Logic Programs, Comm. of the ACM, 35, 3, pps 94-111.

Digital Library

[39]

{39} X. Qian. (1995) Query folding., To appear in the Proc. of the 1996 IEEE Data Engineering Conf., Technical Report SRI-CSL-95-09, Computer Science Laboratory, SRI International, July 1995.

Digital Library

[40]

{40} S. Adah and X. Qian. (1995) Query Transformation in Heterogeneous Reasoning Systems., Submitted for publication.

[41]

{41} S. Chawathe, H. Garcia-Molina, J. Hammer, K. Ireland, Y. Papakonstantinou, J. Ullman, and J. Widom. (1994) The TSIMMIS Project: Integration of Heterogeneous Information Sources., In Proceedings of IPSJ Conference, Tokyo, Japan, October 1994. (Also available via anonymous FTP from host db.stanford.edu, file/pub/chawathe/1994/tsimmis-overview.ps.)

[42]

{42} W. Du and R. Krishnamurthy and M.-C. Shan. (1992) Query Optimization in Heterogeneous Database Management Systems., In Proc. VLDB Conference, pp. 277-291, Vancouver, Canada, 1992.

Digital Library

[43]

{43} Q. Zhu and P.-A. Larson. (1994) A Query Sampling Method for Estimating Local Cost Parameters in a Multidatabase System., Proc. IEEE Data Engineering Conf., pp. 144-153, 1994.

Digital Library

[44]

{44} H. Lu and B.-C. Ooi and C.-H. Goh. (1993) Multidatabase Query Optimization: Issues and Solutions., Proc. RIDE-IMS '93, pp. 137-143, 1993.

Cited By

Monir SZhao D(2024)Efficient Feature Extraction for Image Analysis through Adaptive Caching in Vector Databases2024 7th International Conference on Information and Computer Technologies (ICICT)10.1109/ICICT62343.2024.00036(193-198)Online publication date: 15-Mar-2024
https://doi.org/10.1109/ICICT62343.2024.00036
Ji HJiang SZhao YWu GWang GYuan G(2022)Bs-join: A novel and efficient mixed batch-stream join method for spatiotemporal data management in FlinkFuture Generation Computer Systems10.1016/j.future.2022.11.016Online publication date: Nov-2022
https://doi.org/10.1016/j.future.2022.11.016
Özsu MValduriez PÖzsu MValduriez P(2019)Peer-to-Peer Data ManagementPrinciples of Distributed Database Systems10.1007/978-3-030-26253-2_9(395-448)Online publication date: 3-Dec-2019
https://doi.org/10.1007/978-3-030-26253-2_9
Show More Cited By

Index Terms

Query caching and optimization in distributed mediator systems

Recommendations

Query caching and optimization in distributed mediator systems
SIGMOD '96: Proceedings of the 1996 ACM SIGMOD international conference on Management of data

Query processing and optimization in mediator systems that access distributed non-proprietary sources pose many novel problems. Cost-based query optimization is hard because the mediator does not have access to source statistics information and ...
Intelligent data caching in mediator-based systems
Query optimization in distributed database systems

Comments

Information & Contributors

Information

Published In

cover image ACM SIGMOD Record

ACM SIGMOD Record Volume 25, Issue 2

June 1996

557 pages

ISSN:0163-5808

DOI:10.1145/235968

Chairman:
T. H. Merrett
McGill Univ.
,
Editors:
H. V. Jagadish,
Inderpal Singh Mumick

Issue’s Table of Contents

SIGMOD '96: Proceedings of the 1996 ACM SIGMOD international conference on Management of data
June 1996
560 pages
ISBN:0897917944
DOI:10.1145/233269
Editor:
Jennifer Widom
Stanford Univ., Stanford, CT

Copyright © 1996 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 1996

Published in SIGMOD Volume 25, Issue 2

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

242
Total Citations
View Citations
1,170
Total Downloads

Downloads (Last 12 months)146
Downloads (Last 6 weeks)19

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Monir SZhao D(2024)Efficient Feature Extraction for Image Analysis through Adaptive Caching in Vector Databases2024 7th International Conference on Information and Computer Technologies (ICICT)10.1109/ICICT62343.2024.00036(193-198)Online publication date: 15-Mar-2024
https://doi.org/10.1109/ICICT62343.2024.00036
Ji HJiang SZhao YWu GWang GYuan G(2022)Bs-join: A novel and efficient mixed batch-stream join method for spatiotemporal data management in FlinkFuture Generation Computer Systems10.1016/j.future.2022.11.016Online publication date: Nov-2022
https://doi.org/10.1016/j.future.2022.11.016
Özsu MValduriez PÖzsu MValduriez P(2019)Peer-to-Peer Data ManagementPrinciples of Distributed Database Systems10.1007/978-3-030-26253-2_9(395-448)Online publication date: 3-Dec-2019
https://doi.org/10.1007/978-3-030-26253-2_9
Özsu MValduriez PÖzsu MValduriez P(2019)Parallel Database SystemsPrinciples of Distributed Database Systems10.1007/978-3-030-26253-2_8(349-394)Online publication date: 3-Dec-2019
https://doi.org/10.1007/978-3-030-26253-2_8
Özsu MValduriez PÖzsu MValduriez P(2019)Database Integration—Multidatabase SystemsPrinciples of Distributed Database Systems10.1007/978-3-030-26253-2_7(281-347)Online publication date: 3-Dec-2019
https://doi.org/10.1007/978-3-030-26253-2_7
Özsu MValduriez PÖzsu MValduriez P(2019)Data ReplicationPrinciples of Distributed Database Systems10.1007/978-3-030-26253-2_6(247-280)Online publication date: 3-Dec-2019
https://doi.org/10.1007/978-3-030-26253-2_6
Özsu MValduriez PÖzsu MValduriez P(2019)Distributed Transaction ProcessingPrinciples of Distributed Database Systems10.1007/978-3-030-26253-2_5(183-246)Online publication date: 3-Dec-2019
https://doi.org/10.1007/978-3-030-26253-2_5
Özsu MValduriez PÖzsu MValduriez P(2019)Distributed Query ProcessingPrinciples of Distributed Database Systems10.1007/978-3-030-26253-2_4(129-182)Online publication date: 3-Dec-2019
https://doi.org/10.1007/978-3-030-26253-2_4
Özsu MValduriez PÖzsu MValduriez P(2019)Distributed Data ControlPrinciples of Distributed Database Systems10.1007/978-3-030-26253-2_3(91-127)Online publication date: 3-Dec-2019
https://doi.org/10.1007/978-3-030-26253-2_3
Özsu MValduriez PÖzsu MValduriez P(2019)Distributed and Parallel Database DesignPrinciples of Distributed Database Systems10.1007/978-3-030-26253-2_2(33-89)Online publication date: 3-Dec-2019
https://doi.org/10.1007/978-3-030-26253-2_2
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents