Learning Relevant Questions for Conversational Product Search using Deep Reinforcement Learning
Pages 746 - 754
Abstract
We propose RelQuest, a conversational product search model based on reinforcement learning to generate questions from product descriptions in each round of the conversation, directly maximizing any desired metrics (i.e., the ultimate goal of the conversation), objectives, or even an arbitrary user satisfaction signal. By enabling systems to ask questions about user needs, conversational product search has gained increasing attention in recent years. Asking the right questions through conversations helps the system collect valuable feedback to create better user experiences and ultimately increase sales. In contrast, existing conversational product search methods are based on an assumption that there is a set of effectively pre-defined candidate questions for each product to be asked. Moreover, they make strong assumptions to estimate the value of questions in each round of the conversation. Estimating the true value of questions in each round of the conversation is not trivial since it is unknown. Experiments on real-world user purchasing data show the effectiveness of RelQuest at generating questions that maximize standard evaluation measures such as NDCG.
Supplementary Material
This is a Presentation video for the "Learning Relevant Questions for Conversational Product Search using Deep Reinforcement Learning" paper.
- Download
- 58.58 MB
References
[1]
Qingyao Ai, Yongfeng Zhang, Keping Bi, Xu Chen, and W Bruce Croft. 2017. Learning a hierarchical embedding model for personalized product search. In SIGIR'17 . 645--654.
[2]
Mozhdeh Ariannezhad, Ali Montazeralghaem, Hamed Zamani, and Azadeh Shakery. 2017. Improving retrieval performance for verbose queries via axiomatic analysis of term discrimination heuristic. In SIGIR'17. 1201--1204.
[3]
Nicholas J Belkin, Colleen Cool, Adelheit Stein, and Ulrich Thiel. 1995. Cases, scripts, and information-seeking strategies: On the design of interactive information retrieval systems. Expert systems with applications, Vol. 9, 3 (1995), 379--395.
[4]
Keping Bi, Qingyao Ai, Yongfeng Zhang, and W Bruce Croft. 2019. Conversational product search based on negative feedback. In CIKM'19. 359--368.
[5]
Haw-Shiuan Chang, Ziyun Wang, Luke Vilnis, and Andrew McCallum. 2017. Distributional inclusion vector embedding for unsupervised hypernymy detection. arXiv preprint arXiv:1710.00880 (2017).
[6]
Christiane Fellbaum. 2010. WordNet. In Theory and applications of ontology: computer applications. Springer, 231--243.
[7]
Jiafeng Guo, Yixing Fan, Qingyao Ai, and W Bruce Croft. 2016. A deep relevance matching model for ad-hoc retrieval. In CIKM'16. 55--64.
[8]
Ayyoob Imani, Amir Vakili, Ali Montazer, and Azadeh Shakery. 2019 a. An Axiomatic Study of Query Terms Order in Ad-hoc Retrieval. In ECIR'19. Springer, 196--202.
[9]
Ayyoob Imani, Amir Vakili, Ali Montazer, and Azadeh Shakery. 2019 b. Deep neural networks for query expansion using word embeddings. In ECIR'19. Springer, 203--210.
[10]
Kalervo J"arvelin and Jaana Kek"al"ainen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), Vol. 20, 4 (2002), 422--446.
[11]
Maryam Karimzadehgan and ChengXiang Zhai. 2011. Improving retrieval accuracy of difficult queries through generalizing negative document language models. In CIKM'11. 27--36.
[12]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR'15 (San Diego, CA, USA).
[13]
Vijay R Konda and John N Tsitsiklis. 2000. Actor-critic algorithms. In Advances in neural information processing systems. 1008--1014.
[14]
Karol Kurach, Anton Raichuk, Piotr Sta'nczyk, Micha? Zajka c, Olivier Bachem, Lasse Espeholt, Carlos Riquelme, Damien Vincent, Marcin Michalski, Olivier Bousquet, et al. 2020. Google research football: A novel reinforcement learning environment. In AAAI'20, Vol. 34. 4501--4510.
[15]
John Lafferty and Chengxiang Zhai. 2001. Document language models, query models, and risk minimization for information retrieval. In SIGIR'01. 111--119.
[16]
Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, and Tat-Seng Chua. 2020. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In WSDM'20. 304--312.
[17]
Berenike Litz, Hagen Langer, and Rainer Malaka. 2009. Sequential supervised learning for hypernym discovery from Wikipedia. In IC3K'09. Springer, 68--80.
[18]
Zhengdong Lu and Hang Li. 2013. A deep architecture for matching short texts. Advances in neural information processing systems, Vol. 26 (2013), 1367--1375.
[19]
Ali Montazeralghaem, James Allan, and Philip S Thomas. 2021. Large-scale Interactive Conversational Recommendation System using Actor-Critic Framework. In RecSys'21. 220--229.
[20]
Ali Montazeralghaem, Razieh Rahimi, and James Allan. 2020 a. Relevance Ranking Based on Query-Aware Context Analysis. Advances in Information Retrieval, Vol. 12035 (2020), 446.
[21]
Ali Montazeralghaem, Hamed Zamani, and James Allan. 2020 b. A reinforcement learning framework for relevance feedback. In SIGIR'20. 59--68.
[22]
Ali Montazeralghaem, Hamed Zamani, and Azadeh Shakery. 2016. Axiomatic analysis for improving the log-logistic feedback model. In SIGIR'16 . 765--768.
[23]
Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Jingfang Xu, and Xueqi Cheng. 2017. Deeprank: A new deep architecture for relevance ranking in information retrieval. In CIKM'17 . 257--266.
[24]
Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304 (2017).
[25]
Ivaylo Popov, Nicolas Heess, Timothy Lillicrap, Roland Hafner, Gabriel Barth-Maron, Matej Vecerik, Thomas Lampe, Yuval Tassa, Tom Erez, and Martin Riedmiller. 2017. Data-efficient deep reinforcement learning for dexterous manipulation. arXiv preprint arXiv:1704.03073 (2017).
[26]
Peng Qi, Yuhao Zhang, and Christopher D Manning. 2020. Stay hungry, stay focused: Generating informative and specific questions in information-seeking conversations. arXiv preprint arXiv:2004.14530 (2020).
[27]
Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011. Introduction to recommender systems handbook. In Recommender systems handbook . Springer, 1--35.
[28]
Stephen E Robertson and Steve Walker. 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In SIGIR'94. Springer, 232--241.
[29]
J. J. Rocchio. 1971. Relevance feedback in information retrieval. In The Smart retrieval system - experiments in automatic document processing, G. Salton (Ed.). Englewood Cliffs, NJ: Prentice-Hall, 313--323.
[30]
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1986. Learning representations by back-propagating errors. nature, Vol. 323, 6088 (1986), 533.
[31]
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016. Mastering the game of Go with deep neural networks and tree search. nature, Vol. 529, 7587 (2016), 484.
[32]
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. 2017a. Mastering the game of Go without human knowledge. Nature, Vol. 550, 7676 (2017), 354.
[33]
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. 2017b. Mastering the game of go without human knowledge. Nature, Vol. 550, 7676 (2017), 354--359.
[34]
Yueming Sun and Yi Zhang. 2018. Conversational recommender system. In SIGIR'18. 235--244.
[35]
Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction .MIT press.
[36]
Christophe Van Gysel, Maarten de Rijke, and Evangelos Kanoulas. 2016. Learning latent vector spaces for product search. In CIKM'16. 165--174.
[37]
Lakshmi Vikraman, Ali Montazeralghaem, Helia Hashemi, W Bruce Croft, and James Allan. 2021. Passage Similarity and Diversification in Non-factoid Question Answering. In ICTIR'21 . 271--280.
[38]
Yansen Wang, Chenyi Liu, Minlie Huang, and Liqiang Nie. 2018. Learning to ask questions in open-domain conversational systems with typed decoders. arXiv preprint arXiv:1805.04843 (2018).
[39]
Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, Vol. 8, 3--4 (1992), 229--256.
[40]
Liu Yang, Hamed Zamani, Yongfeng Zhang, Jiafeng Guo, and W Bruce Croft. 2017. Neural matching models for question retrieval and next question prediction in conversation. arXiv preprint arXiv:1707.05409 (2017).
[41]
Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V Le. 2018. Qanet: Combining local convolution with global self-attention for reading comprehension. arXiv preprint arXiv:1804.09541 (2018).
[42]
Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W Bruce Croft. 2018. Towards conversational search and recommendation: System ask, user respond. In CIKM'18 . 177--186.
[43]
Jie Zou, Yifan Chen, and Evangelos Kanoulas. 2020. Towards question-based recommender systems. In SIGIR'20. 881--890.
[44]
Jie Zou and Evangelos Kanoulas. 2019. Learning to ask: Question-based sequential Bayesian product search. In CIKM'19 . 369--378.
Index Terms
- Learning Relevant Questions for Conversational Product Search using Deep Reinforcement Learning
Recommendations
Extracting Relevant Information from User's Utterances in Conversational Search and Recommendation
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data MiningConversational search and recommendation systems can ask clarifying questions through the conversation and collect valuable information from users. However, an important question remains: how can we extract relevant information from the user's utterances ...
Learning to Ask: Conversational Product Search via Representation Learning
Online shopping platforms, such as Amazon and AliExpress, are increasingly prevalent in society, helping customers purchase products conveniently. With recent progress in natural language processing, researchers and practitioners shift their focus from ...
Ask or Recommend: An Empirical Study on Conversational Product Search
CIKM '24: Proceedings of the 33rd ACM International Conference on Information and Knowledge ManagementConversational Product Search (CPS) provides an engaging way for users to find products through effective natural language conversations. However, understanding the effect of conversational characteristics on user search performance and when to ask ...
Comments
Information & Contributors
Information
Published In
February 2022
1690 pages
ISBN:9781450391320
DOI:10.1145/3488560
- General Chairs:
- K. Selcuk Candan,
- Huan Liu,
- Program Chairs:
- Leman Akoglu,
- Xin Luna Dong,
- Jiliang Tang
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 15 February 2022
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Center for Intelligent Information Retrieval - University of Massachusetts Amherst
- Amazon.com
Conference
WSDM '22: The Fifteenth ACM International Conference on Web Search and Data Mining
February 21 - 25, 2022
AZ, Virtual Event, USA
Acceptance Rates
Overall Acceptance Rate 498 of 2,863 submissions, 17%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 307Total Downloads
- Downloads (Last 12 months)38
- Downloads (Last 6 weeks)4
Reflects downloads up to 24 Dec 2024
Other Metrics
Citations
Cited By
View allView Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in