A Game-theoretic Approach to Data Interaction
Authors: 
Ben McCamish, Vahid Ghadakchi, Arash Termehchy, Behrouz Touri, Eduardo Cotilla-Sanchez, Liang Huang, Soravit ChangpinyoAuthors Info & Claims
Ben McCamish, Vahid Ghadakchi, Arash Termehchy, Behrouz Touri, Eduardo Cotilla-Sanchez, Liang Huang, Soravit ChangpinyoAuthors Info & ClaimsArticle No.: 1, Pages 1 - 44
Abstract
As most users do not precisely know the structure and/or the content of databases, their queries do not exactly reflect their information needs. The database management system (DBMS) may interact with users and use their feedback on the returned results to learn the information needs behind their queries. Current query interfaces assume that users do not learn and modify the way they express their information needs in the form of queries during their interaction with the DBMS. Using a real-world interaction workload, we show that users learn and modify how to express their information needs during their interactions with the DBMS and their learning is accurately modeled by a well-known reinforcement learning mechanism. As current data interaction systems assume that users do not modify their strategies, they cannot discover the information needs behind users’ queries effectively. We model the interaction between the user and the DBMS as a game with identical interest between two rational agents whose goal is to establish a common language for representing information needs in the form of queries. We propose a reinforcement learning method that learns and answers the information needs behind queries and adapts to the changes in users’ strategies and proves that it improves the effectiveness of answering queries, stochastically speaking. We propose two efficient implementations of this method over large relational databases. Our extensive empirical studies over real-world query workloads indicate that our algorithms are efficient and effective.
References
[1]
Serge Abiteboul, Richard Hull, and Victor Vianu. 1994. Foundations of Databases: The Logical Level. Addison-Wesley.
[2]
Azza Abouzied, Dana Angluin, Christos H. Papadimitriou, Joseph M. Hellerstein, and Avi Silberschatz. 2013. Learning and verifying quantified Boolean queries by example. In Proceedings of the ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS’13).
[3]
Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. 2002. Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47, 2--3 (2002), 235--256.
[4]
Peter Auer, Nicolo Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. 2002. The nonstochastic multiarmed bandit problem. SIAM J. Comput. 32, 1 (2002), 48--77.
[5]
Paolo Avesani and Marco Cova. 2005. Shared lexicon for distributed annotations on the Web. In Proceedings of the International World Wide Web Conferences (WWW’05).
[6]
J. A. Barrett and K. Zollman. 2008. The role of forgetting in the evolution and learning of language. J. Exper. Theoret. Artific. Intell. 21, 4 (2008), 293--309.
[7]
Angela Bonifati, Radu Ciucanu, and Slawomir Staworko. 2015. Learning join queries from user examples. ACM Trans. Datab. Syst. 40, 4 (2015).
[8]
Robert R. Bush and Frederick Mosteller. 1953. A stochastic model with applications to learning. Ann. Math. Stat. 24, 4 (1953), 559--585.
[9]
Yonghua Cen, Liren Gan, and Chen Bai. 2013. Reinforcement learning in information searching. Inf. Res.: Int. Elect. J. 18, 1 (2013).
[10]
Gloria Chatzopoulou, Magdalini Eirinaki, and Neoklis Polyzotis. 2009. Query recommendations for interactive database exploration. In Proceedings of the 21st International Conference on Scientific and Statistical Database Management (SSDBM’09). Springer-Verlag, 3--18.
[11]
Surajit Chaudhuri, Gautam Das, Vagelis Hristidis, and Gerhard Weikum. 2006. Probabilistic information retrieval approach for ranking of database query results. ACM Trans. Datab. Syst. 31, 3 (2006).
[12]
Surajit Chaudhuri, Bolin Ding, and Srikanth Kandula. 2017. Approximate query processing: No silver bullet. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD’17). 511--519.
[13]
Surajit Chaudhuri, Rajeev Motwani, and Vivek Narasayya. 1999. On random sampling over joins. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD’99). ACM, New York, NY, 263--274.
[14]
R. Chen and Hani S. Mahmassani. 2009. Learning and risk attitudes in route choice dynamics. In The Expanding Sphere of Travel Behavior Research: Selected Papers from the 11th International Conference on Travel Behavior Research. Emerald Publishing Ltd.
[15]
Yi Chen, Wei Wang, Ziyang Liu, and Xuemin Lin. 2009. Keyword search on structured and semi-structured data. In Proceedings of the International Conference on Management of Data (SIGMOD’09).
[16]
I. Cho and D. Kreps. 1987. Signaling games and stable equilibria. Quart. J. Econ. 102 (1987).
[17]
James J. Choi, David Laibson, Brigitte C. Madrian, and Andrew Metrick. 2009. Reinforcement learning and savings behavior. J. Finance 64, 6 (2009), 2515--2534.
[18]
John G. Cross. 1973. A stochastic learning model of economic behavior. Quart. J. Econ. 87, 2 (1973), 239--266.
[19]
Sanmay Das and Allen Lavoie. 2014. The effects of feedback on human behavior in social media: An inverse reinforcement learning model. In Proceedings of the International Conference on Autonomous Agents and Multi-agent Systems (AAMAS’14). International Foundation for Autonomous Agents and Multiagent Systems, 653--660. Retrieved from: http://dl.acm.org/citation.cfm?id=2615731.2615837.
[20]
Constantinos Daskalakis, Rafael Frongillo, Christos H. Papadimitriou, George Pierrakos, and Gregory Valiant. 2010. On learning algorithms for Nash equilibria. In Proceedings of the 3rd International Conference on Algorithmic Game Theory (SAGT’10). Springer-Verlag, 114--125. Retrieved from: http://dl.acm.org/citation.cfm?id=1929237.1929248.
[21]
Kyriaki Dimitriadou, Olga Papaemmanouil, and Yanlei Diao. 2014. Explore-by-example: An automatic query steering framework for interactive data exploration. In Proceedings of the International Conference on Management of Data (SIGMOD’14).
[22]
Matina C. Donaldson, Michael Lachmannb, and Carl T. Bergstroma. 2007. The evolution of functionally referential meaning in a structured world. J. Math. Biol. 246 (2007).
[23]
Rick Durrett. 2010. Probability: Theory and Examples. Cambridge University Press.
[24]
Elena Demidova, Xuan Zhou, Irina Oelze, and Wolfgang Nejdl. 2010. Evaluating evidences for keyword query disambiguation in entity centric database search. In Proceedings of the International Conference on Database and Expert Systems Applications (DEXA’10).
[25]
Ido Erev and Alvin E. Roth. 1995. On the Need for Low Rationality, Cognitive Game Theory: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria.
[26]
Ronald Fagin, Amnon Lotem, and Moni Naor. 2001. Optimal aggregation algorithms for middleware. In Proceedings of the 20th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS’01). ACM, New York, NY, 102--113.
[27]
Arjita Ghosh and Sandip Sen. 2004. Learning TOMs: Towards non-myopic equilibria. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’04).
[28]
Laura A. Granka, Thorsten Joachims, and Geri Gay. 2004. Eye-tracking analysis of user behavior in WWW search. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’04).
[29]
Artem Grotov and Maarten de Rijke. 2016. Online learning to rank for information retrieval: SIGIR 2016 tutorial. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’16). ACM, New York, NY, 1215--1218.
[30]
Teck Ho. 2008. Individual learning in games. In The New Palgrave Dictionary of Economics: Design of Experiments and Behavioral Economics, L. Blume and S. Durlauf (Eds.). Palgrave Macmillan.
[31]
Katja Hofmann, Shimon Whiteson, and Maarten de Rijke. 2013. Balancing exploration and exploitation in listwise and pairwise online learning to rank for information retrieval. Inf. Retr. 16, 1 (2013), 63--90.
[32]
Vagelis Hristidis, Luis Gravano, and Yannis Papakonstantinou. 2003. Efficient IR-style keyword search over relational databases. In Proceedings of the Very Large Data Bases Conference (VLDB’03).
[33]
Yilei Hu, Brian Skyrms, and Pierre Tarrès. 2011. Reinforcement learning in signaling game. arXiv preprint arXiv:1103.5818 (2011).
[34]
Jeff Huang, Ryen White, and Georg Buscher. 2012. User see, user point: Gaze and cursor alignment in web search. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI’12).
[35]
Stratos Idreos, Olga Papaemmanouil, and Surajit Chaudhuri. 2015. Overview of data exploration techniques. In Proceedings of the International Conference on Management of Data (SIGMOD’15).
[36]
H. V. Jagadish, Adriane Chapman, Aaron Elkiss, Magesh Jayapandian, Yunyao Li, Arnab Nandi, and Cong Yu. 2007. Making database systems usable. In Proceedings of the International Conference on Management of Data (SIGMOD’07).
[37]
Srikanth Kandula, Anil Shanbhag, Aleksandar Vitorovic, Matthaios Olma, Robert Grandl, Surajit Chaudhuri, and Bolin Ding. 2016. Quickr: Lazily approximating complex Ad hoc queries in big data clusters. In Proceedings of the International Conference on Management of Data (SIGMOD’16). 631--646.
[38]
Nodira Khoussainova, Yong Chul Kwon, Magdalena Balazinska, and Dan Suciu. 2010. SnipSuggest: Context-aware autocompletion for SQL. Proc. VLDB Endow. 4, 1 (2010).
[39]
Daphne Koller, Nir Friedman, Sašo Džeroski, Charles Sutton, Andrew McCallum, Avi Pfeffer, Pieter Abbeel, Ming-Fai Wong, David Heckerman, Chris Meek et al. 2007. Introduction to Statistical Relational Learning. The MIT Press.
[40]
Harold J. Larson. 1969. Introduction to Probability Theory and Statistical Inference. Vol. 12. Wiley New York.
[41]
David Lewis. 1969. Convention. Harvard University Press, Cambridge, MA.
[42]
Hao Li, Chee-Yong Chan, and David Maier. 2015. Query from examples: An iterative, data-driven approach to query construction. Proc. VLDB Endow. 8, 13 (2015).
[43]
Erietta Liarou and Stratos Idreos. 2014. dbTouch in action database kernels for touch-based data exploration. In Proceedings of the IEEE 30th International Conference on Data Engineering (ICDE’14). 1262--1265.
[44]
Jiyun Luo, Sicong Zhang, and Hui Yang. 2014. Win-win search: Dual-agent stochastic game in session search. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’14).
[45]
Yi Luo, Xumein Lin, Wei Wang, and Xiaofang Zhou. 2007. SPARK: Top-k keyword query in relational databases. In Proceedings of the International Conference on Management of Data (SIGMOD’07).
[46]
Michael W. Macy and Andreas Flache. 2002. Learning dynamics in social dilemmas. Proc. Nat. Acad. Sci. 99, suppl 3 (2002), 7229--7236.
[47]
Christopher Manning, Prabhakar Raghavan, and Hinrich Schutze. 2008. An Introduction to Information Retrieval. Cambridge University Press.
[48]
Ben McCamish, Vahid Ghadakchi, Arash Termehchy, Behrouz Touri, and Liang Huang. 2018. The data interaction game. In Proceedings of the International Conference on Management of Data (SIGMOD’18). ACM, New York, NY, 83--98.
[49]
Ben McCamish, Arash Termehchy, and Behrouz Touri. 2016. A signaling game approach to databases querying and interaction. arXiv preprint arXiv:1603.04068 (2016).
[50]
Taesup Moon, Wei Chu, Lihong Li, Zhaohui Zheng, and Yi Chang. 2012. An online learning framework for refining recency search results with user click feedback. ACM Trans. Inf. Syst. 30, 4 (2012), 20.
[51]
Yael Niv. 2009. The neuroscience of reinforcement learning. In Proceedings of the International Conference on Machine Learning (ICML’09).
[52]
Martin A. Nowak and David C. Krakauer. 1999. The evolution of language. Proc. Nat. Acad. Sci. 96, 14 (1999).
[53]
Frank Olken. 1993. Random Sampling from Databases. Ph.D. Dissertation. University of California, Berkeley.
[54]
Filip Radlinski, Robert Kleinberg, and Thorsten Joachims. 2008. Learning diverse rankings with multi-armed bandits. In Proceedings of the 25th International Conference on Machine Learning. ACM, 784--791.
[55]
Herbert Robbins and David Siegmund. 1985. A convergence theorem for non negative almost supermartingales and some applications. In Herbert Robbins Selected Papers. Springer.
[56]
Alvin E. Roth and Ido Erev. 1995. Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term. Games Econ. Behav. 8, 1 (1995), 164--212.
[57]
Lloyd S. Shapley et al. 1964. Some topics in two-person games. Adv. Game Theor. 52, 1--29 (1964), 1--2.
[58]
Yoav Shoham, Rob Powers, and Trond Grenager. 2003. Multi-agent reinforcement learning: A critical survey. Web Manuscript (2003). https://www.cc.gatech.edu/∼isbell/classes/2009/cs7641_spring/handouts/MALearning_ACriticalSurvey_2003_0516.pdf.
[59]
Hanan Shteingart and Yonatan Loewenstein. 2014. Reinforcement learning and human behavior. Curr. Opin. Neurobiol. 25 (04/2014 2014), 93--98.
[60]
Aleksandrs Slivkins, Filip Radlinski, and Sreenivas Gollapudi. 2013. Ranked bandits in metric spaces: Learning diverse rankings over large document collections. J. Mach. Learn. Res. 14, Feb. (2013), 399--436.
[61]
Richard S. Sutton and Andrew G. Barto. 1998. Introduction to Reinforcement Learning (1st ed.). The MIT Press, Cambridge, MA.
[62]
Steve Tadelis. 2013. Game Theory: An Introduction. Princeton University Press.
[63]
Q. Tran, C. Chan, and S. Parthasarathy. 2009. Query by output. In Proceedings of the International Conference on Management of Data (SIGMOD’09).
[64]
Peter Trapa and Martin Nowak. 2000. Nash equilibria for an evolutionary language game. J. Math. Biol. 41 (2000).
[65]
Aleksandr Vorobev, Damien Lefortier, Gleb Gusev, and Pavel Serdyukov. 2015. Gathering additional feedback on search results by multi-armed bandits with respect to production ranking. In Proceedings of the International World Wide Web Conferences (WWW’15). International World Wide Web Conferences Steering Committee, 1177--1187.
[66]
Robert L. Wolpert. 2010. Introduction to martingales. (2010). https://www2.stat.duke.edu/courses/Fall10/sta205/lec/topics/mg.pdf.
[67]
Yahoo!. 2011. Yahoo! Webscope Dataset Anonymized Yahoo! Search Logs with Relevance Judgments Version 1.0. Retrieved from http://labs.yahoo.com/Academic_Relations.
[68]
Zhepeng Yan, Nan Zheng, Zachary G. Ives, Partha Pratim Talukdar, and Cong Yu. 2013. Actively soliciting feedback for query answers in keyword search-based data integration. In Proceedings of the VLDB Endowment, Vol. 6. VLDB Endowment, 205--216.
[69]
Ozlem Yanmaz-Tuzel and Kaan Ozbay. 2009. Modeling Learning Impacts on Day-to-day Travel Choice. Springer US, Boston, MA, 387--401.
[70]
H. H. Peyton Young. 2008. Adaptive heuristics. In The New Palgrave Dictionary of Economics: Design of Experiments and Behavioral Economics, L. Blume and S. Durlauf (Eds.). Palgrave Macmillan.
[71]
H. Peyton Young. 2010. Strategic Learning and Its Limits. Oxford University Press.
[72]
Yisong Yue, Josef Broder, Robert Kleinberg, and Thorsten Joachims. 2012. The K-armed dueling bandits problem. J. Comput. Syst. Sci. 78, 5 (2012).
[73]
Yinan Zhang and ChengXiang Zhai. 2015. Information retrieval as card playing: A formal model for optimizing interactive retrieval interface. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’15).
Recommendations
The Data Interaction Game
SIGMOD '18: Proceedings of the 2018 International Conference on Management of DataAs many users do not precisely know the structure and/or the content of databases, their queries do not exactly reflect their information needs. The database management systems (DBMS) may interact with users and leverage their feedback on the returned ...
A Game-theoretic Approach to Data Interaction: A Progress Report
HILDA '17: Proceedings of the 2nd Workshop on Human-In-the-Loop Data AnalyticsAs most database users cannot precisely express their information needs in the form of database queries, it is challenging for database query interfaces to understand and satisfy their intents. Database systems usually improve their understanding of ...
Improving New Users’ Query Performance: Deterring Premature Stopping of Query Revision with Information for Forming Ex Ante Expectations
As the volume of data in organizational databases grows, organizations are seeking to use this data to improve organizational success. To this end, users are being asked to query these databases to provide information to help answer questions posed by ...
Comments
Information & Contributors
Information
Published In
March 2020
177 pages
ISSN:0362-5915
EISSN:1557-4644
DOI:10.1145/3382758
- Editor:
- Christian S. Jensen
Copyright © 2020 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 February 2020
Accepted: 01 July 2019
Revised: 01 May 2019
Received: 01 November 2018
Published in TODS Volume 45, Issue 1
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 519Total Downloads
- Downloads (Last 12 months)22
- Downloads (Last 6 weeks)0
Reflects downloads up to 22 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format