article

Query optimization in distributed networks of autonomous database systems

Authors:

Fragkiskos Pentaris,

Yannis IoannidisAuthors Info & Claims

ACM Transactions on Database Systems (TODS), Volume 31, Issue 2

Pages 537 - 583

https://doi.org/10.1145/1138394.1138397

Published: 01 June 2006 Publication History

Abstract

Large-scale distributed environments, where each node is completely autonomous and offers services to its peers through external communication, pose significant challenges to query processing and optimization. Autonomy is the main source of the problem, as it results in lack of knowledge about any particular node with respect to the information it can produce and its characteristics, for example, cost of production or quality of produced results. In this article, inspired by e-commerce technology, we recognize queries as commodities and model query optimization as a trading negotiation process. Subquery answers and subquery operator execution jobs are traded between nodes until deals are struck with some nodes for all of them. Such trading may also occur recursively, in the sense that some nodes may play the role of intermediaries between other nodes (subcontracting). We identify the key parameters of the overall framework and suggest several potential alternatives for each one. In comparison to trading negotiations for e-commerce, query optimization faces unique new challenges that stem primarily from the fact that queries have a complex structure and can be broken into smaller parts. We address these challenges through a particular instantiation of our framework focusing primarily on the optimization algorithms run on “buying” and “selling” nodes, the evaluation metrics of the queries, and the negotiation strategy. Finally, we present the results of several experiments that demonstrate the performance characteristics of our approach compared to those of traditional query optimization.

Supplementary Material

Pentaris Appendix (p537-pentaris-apndx.pdf)

Online appendix to designing mediation for context-aware applications. The appendix supports the information on page 537.

Download
395.97 KB

References

[1]

Bichler, M., Kaukal, M., and Segev, A. 1999. Multi-attribute auctions for electronic procurement. In Proceedings of the 1st IBM IAC Workshop on Internet Based Negotiation Technologies (Yorktown Heights, NY).

[2]

Collins, J., Tsvetovat, M., Sundareswara, R., van Tonder, J., Gini, M. L., and Mobasher, B. 1999. Evaluating risk: Flexibility and feasibility in multi-agent contracting. In Proceedings of the 3rd Annual Conference on Autonomous Agents (Seattle, WA). ACM, New York.

Digital Library

[3]

Conitzer, V. and Sandholm, T. 2003. Complexity results about nash equilibria. In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI-03). Morgan Kaufmann, San Francisco, CA.

Digital Library

[4]

Deshpande, A. and Hellerstein, J. M. 2002. Decoupled query optimization for federated database systems. In Proceedings of the 18th International Conference of Data Engineering (Son Jose, CA). IEEE Computer Society, Los Alamitos, CA, 716--732.

Digital Library

[5]

Franklin, M. J., Jónsson, B. T., and Kossmann, D. 1996. Performance tradeoffs for client-server query processing. In Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data (Montreal, Queb. Canada, June 4--6), H. V. Jagadish and I. S. Mumick, Eds. ACM, New York, 149--160.

Digital Library

[6]

Ganguly, S., Hasan, W., and Krishnamurthy, R. 1992. Query optimization for parallel execution. In Proceedings of the ACM SIGMOD International Conference on Management of Data, M. Stonebraker, Ed. ACM, New York, 9--18.

Digital Library

[7]

Gravelle, H. and Rees, R. 2004. Microeconomics (3rd ed). Pearson Education, England.

[8]

Halevy, A. Y. 2001. Answering queries using views: A survey. VLDB J. 10, 4, 270--294.

Digital Library

[9]

Ioannidis, Y. E. and Kang, Y. C. 1990. Randomized algorithms for optimizing large join queries. In Proceedings of the 1990 ACM SIGMOD Conference (Atlantic City, NJ). H. Garcia-Molina and H. V. Jagadish, Eds. ACM, New York, 312--321.

Digital Library

[10]

Ioannidis, Y. E., Ng, R. T., Shim, K., and Sellis, T. K. 1997. Parametric query optimization. VLDB J. 6, 2, 132--151.

Digital Library

[11]

Kagel, J. H. 1995. Auctions: A survey of experimental research. In The Handbook of Experimental Economics, J. H. Kagel and A. E. Roth, Eds. Princeton University Press, Princeton, NJ.

[12]

Kossmann, D. 2000. The state of the art in distributed query processing. ACM Comput. Surv. 34, 4 (Sept.), 422--469.

Digital Library

[13]

Kossmann, D. and Stocker, K. 2000. Iterative dynamic programming: A new class of query optimization algorithms. ACM Trans. Datab. Syst. 25, 1, 43--82.

Digital Library

[14]

Kraus, S. 2001. Strategic Negotiation in Multiagent Environments (Intelligent Robotics and Autonomous Agents). Bradford Book, Cambridge, MA, USA.

Digital Library

[15]

Liu, B. and Rundensteiner, E. 2005. Revisiting the role of pipelined parallelism in multi-join query processing. In Proceedings of the 31st International Conference on VLDB. VLDB Endowment, Trondheim, Norway, 829--840.

Digital Library

[16]

Mariposa. 2002. Mariposa Distributed Database Management Systems, User's Manual. Mariposa, Available at http://s2k-ftp.cs.berkeley.edu:8000/mariposa/ /src/alpha-1/mariposa-manual.pdf.

[17]

Mas-Colell, A., Whinston, M. D., and Green, J. R. 1995. Microeconomic Theory. Oxford University Press, New York.

[18]

Navas, J. C. and Wynblatt, M. 2001. The network is the database: Data management for highly distributed systems. In Proceedings of the ACM SIGMOD'01 Conference (Santa Barbara, CA). ACM, New York.

Digital Library

[19]

Ogston, E. and Vassiliadis, S. 2002. A peer-to-peer agent auction. In 1st International Joint Conference on Autonomous Agents and Multi-Agent Systems (Bologna, Italy). ACM, New York.

Digital Library

[20]

Papadimitriou, C. H. and Yannakakis, M. 2001. Multiobjective query optimization. In Proceedings of the 20th ACM SIGACT-SIGMOD-SIGART Symposium on PODS (Santa Barbara, CA, May 21--23). ACM, New York.

Digital Library

[21]

Parunak, H. V. D. 1987. Manufacturing experience with the contract net. In Distributed Artificial Intelligence, Research Notes in Artificial Intelligence, M. N. Huhns, Ed. vol. 1. Pitman, London, England, 285--310.

[22]

Pentaris, F. and Ioannidis, Y. E. 2004. Distributed query optimization by query trading. In EDBT, E. Bertino, S. Christodoulakis, D. Plexousakis, V. Christophides, M. Koubarakis, K. Böhm, and E. Ferrari, Eds. Lecture Notes in Computer Science, vol. 2992. Springer-Verlag, New York, 532--550.

[23]

Pottinger, R. and Levy, A. Y. 2000. A scalable algorithm for answering queries using views. In Proceedings of the 26th International Conference on VLDB, (Cairo, Egypt, Sept. 10--14), A. E. Abbadi, M. L. Brodie, S. Chakravarthy, U. Dayal, N. Kamel, G. Schlageter, and K.-Y. Whang, Eds. Morgan-Kaufmann, San Francisco, CA, 484--495.

Digital Library

[24]

Rosenchein, J. S. and Zlotkin, G. 1994. Rules of Encounter : Designing conventions for automated negotiation among computers. The MIT Press series in artificial intelligence, Cambridge, MA.

Digital Library

[25]

Sandholm, T. 2002. Algorithm for optimal winner determination in combinatorial auctions. Artif. Intell. 135, 1--54.

Digital Library

[26]

Selinger, P. G., Astrahan, M. M., Chamberlin, D. D., Lorie, R. A., and Price, T. G. 1979. Access path selection in a relational database management system. In Proceedings of the 1979 ACM SIGMOD International Conference on Management of Data (Boston, MA), ACM, New York, 22--34.

Digital Library

[27]

Smith, R. G. 1980. The contract net protocol: High-level communication and control in a distributed problem solver. IEEE Trans. Comput. 29, 12 (Dec.), 1104--1113.

[28]

Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., and Balakrishnan, H. 2001. Chord: A scalable peer-to-peer lookup service for internet applications. In SIGCOMM '01: Proceedings of the 2001 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications. ACM, New York, 149--160.

Digital Library

[29]

Stonebraker, M., Aoki, P. M., Devine, R., Litwin, W., and Olson, M. A. 1994. Mariposa: A new architecture for distributed data. In Proceedings of the 10th International Conference on Data Engineering (Houston, TX, Feb. 14--18). IEEE Computer Society Press, Los Alamitos, CA, 54--65.

Digital Library

[30]

Stonebraker, M., Aoki, P. M., Litwin, W., Pfeller, A., Sah, A., Sidell, J., Staelin, C., and Yu, A. 1996. Mariposa: A wide-area distributed database system. VLDB J. 5, 1, 48--63.

Digital Library

[31]

Su, S. Y., Huang, C., Hammer, J., Huang, Y., Li, H., Wang, L., Liu, Y., Pluempitiwiriyawej, C., Lee, M., and Lam, H. 2001. An internet-based negotiation server for e-commerce. VLDB J. 10, 72--90.

Digital Library

[32]

Winoto, P., McCalla, G., and Vassileva, J. 2002. An extended alternating-offers bargaining protocol for automated negotiation in multi-agent systems. In Proceedings of the 10th International Conference on CoopIS (Irvine, CA). Springer-Verlag, New York.

Digital Library

[33]

Zaharioudakis, M., Cochrane, R., Lapis, G., Pirahesh, H., and Urata, M. 2000. Answering complex sql queries using automatic summary tables. In Proceedings of the ACM SIGMOD'00 Conference (Dallas, TX). ACM, New York, 105--116.

Digital Library

Cited By

Xu RPalanisamy BJoshi J(2018)QueryGuard: Privacy-Preserving Latency-Aware Query Optimization for Edge Computing2018 17th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/ 12th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE)10.1109/TrustCom/BigDataSE.2018.00153(1097-1106)Online publication date: Aug-2018
https://doi.org/10.1109/TrustCom/BigDataSE.2018.00153
Tan K(2018)Distributed Database SystemsEncyclopedia of Database Systems10.1007/978-1-4614-8265-9_701(1174-1177)Online publication date: 7-Dec-2018
https://doi.org/10.1007/978-1-4614-8265-9_701
Yadav ASairam AMazumdar A(2017)Selecting stable route in multipath routing protocols2017 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT)10.1109/ICCCNT.2017.8204139(1-7)Online publication date: Jul-2017
https://doi.org/10.1109/ICCCNT.2017.8204139
Show More Cited By

Index Terms

Query optimization in distributed networks of autonomous database systems
1. Information systems
  1. Data management systems
    1. Database management system engines
      1. Database query processing
      2. Parallel and distributed DBMSs
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query processing and optimization (theory)

Recommendations

Query Optimization for Ontology-Mediated Query Answering
WWW '24: Proceedings of the ACM Web Conference 2024

Ontology-mediated query answering (OMQA) consists in asking database queries on knowledge bases (KBs); a KB is a set of facts called the KB's database, which is described by domain knowledge called the KB's ontology. A widely-investigated OMQA technique ...
Improvement Algorithms for Semijoin Query Processing Programs in Distributed Database Systems

The problem of optimal query processing in distributed database systems was shown to be NP-hard. This means that heuristic algorithms are necessary to solve the query processing problem. In this paper, we describe algorithms to improve the solutions ...
Query optimization in distributed database systems

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Database Systems

ACM Transactions on Database Systems Volume 31, Issue 2

June 2006

329 pages

ISSN:0362-5915

EISSN:1557-4644

DOI:10.1145/1138394

Issue’s Table of Contents

Copyright © 2006 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 2006

Published in TODS Volume 31, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tag

Query optimization

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
2,993
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)1

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xu RPalanisamy BJoshi J(2018)QueryGuard: Privacy-Preserving Latency-Aware Query Optimization for Edge Computing2018 17th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/ 12th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE)10.1109/TrustCom/BigDataSE.2018.00153(1097-1106)Online publication date: Aug-2018
https://doi.org/10.1109/TrustCom/BigDataSE.2018.00153
Tan K(2018)Distributed Database SystemsEncyclopedia of Database Systems10.1007/978-1-4614-8265-9_701(1174-1177)Online publication date: 7-Dec-2018
https://doi.org/10.1007/978-1-4614-8265-9_701
Yadav ASairam AMazumdar A(2017)Selecting stable route in multipath routing protocols2017 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT)10.1109/ICCCNT.2017.8204139(1-7)Online publication date: Jul-2017
https://doi.org/10.1109/ICCCNT.2017.8204139
Yadav PRizvi S(2017)Performance analysis of randomized algorithm for optimal query plan generation in distributed environment2017 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT)10.1109/ICCCNT.2017.8204044(1-7)Online publication date: Jul-2017
https://doi.org/10.1109/ICCCNT.2017.8204044
Yadav PRizvi S(2017)Query Optimization: Issues and Challenges in Mining of Distributed DataBig Data Analytics10.1007/978-981-10-6620-7_67(693-698)Online publication date: 4-Oct-2017
https://doi.org/10.1007/978-981-10-6620-7_67
Tan K(2017)Distributed Database SystemsEncyclopedia of Database Systems10.1007/978-1-4899-7993-3_701-2(1-4)Online publication date: 6-May-2017
https://doi.org/10.1007/978-1-4899-7993-3_701-2
Zhao MLiu PLobo J(2015)Towards Collaborative Query Planning in Multi-party Database NetworksData and Applications Security and Privacy XXIX10.1007/978-3-319-20810-7_2(19-34)Online publication date: 23-Jun-2015
https://doi.org/10.1007/978-3-319-20810-7_2
Zhu MMahmood KRisch T(2015)Scalable Queries Over Log Database CollectionsData Science10.1007/978-3-319-20424-6_17(173-185)Online publication date: 11-Jun-2015
https://doi.org/10.1007/978-3-319-20424-6_17
Liang XWang SWei CShen C(2014)University Heterogeneous Data Source Integration Middleware Design Based on XMLApplied Mechanics and Materials10.4028/www.scientific.net/AMM.543-547.2937543-547(2937-2940)Online publication date: Mar-2014
https://doi.org/10.4028/www.scientific.net/AMM.543-547.2937
Farnan NLee AChrysanthis PYu T(2014)PAQO: Preference-aware query optimization for decentralized database systems2014 IEEE 30th International Conference on Data Engineering10.1109/ICDE.2014.6816670(424-435)Online publication date: Mar-2014
https://doi.org/10.1109/ICDE.2014.6816670
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents