research-article

Knowledge-Based Conversational Recommender Systems Enhanced by Dialogue Policy Learning

Authors:

Shiliang SunAuthors Info & Claims

IJCKG '21: Proceedings of the 10th International Joint Conference on Knowledge Graphs

Pages 10 - 18

https://doi.org/10.1145/3502223.3502225

Published: 24 January 2022 Publication History

Abstract

The conversational recommender system (CRS) provides personalized recommendations for users through dialogues. Knowledge-based CRS, which applies external knowledge graphs into the CRS, can provide knowledge-aware recommendations, and has proved successful in many fields. However, existing models suffer from two limitations. First, previous knowledge-based CRSs ignore the transfer of topics in the conversation and cannot handle multi-task recommendation dialogues. Second, it takes many inquiries for traditional models to obtain knowledge-based user profiles, which affect the user’s interactive experience. In this work, we use the dialogue policy learning to tackle these issues, and propose a model called CRSDP, standing for knowledge-based Conversational Recommender Systems enhanced by Dialogue Policy learning. We leverage the actor-critic framework to learn a dialogue policy in the reinforcement learning paradigm. The optimized dialogue policy leads the conversation strategically in multi-task dialogue scenarios, determines the user preference in fewer turns, and makes effective recommendations. We evaluate the performance of the model in the recommendation task and the conversation task. Experimental results validate the effectiveness of our model.

References

[1]

Wanling Cai and Li Chen. 2020. Predicting user intents and satisfaction with dialogue-based conversational recommendations. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. 33–42.

Digital Library

[2]

Qibin Chen, Junyang Lin, Yichang Zhang, Ming Ding, Yukuo Cen, Hongxia Yang, and Jie Tang. 2019. Towards knowledge-based recommender dialog system. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 1803–1813.

[3]

Qian Chen, Zhu Zhuo, and Wen Wang. 2019. Bert for joint intent classification and slot filling. arXiv preprint arXiv:1902.10909(2019).

[4]

Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. 2016. Towards conversational recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 815–824.

Digital Library

[5]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 18th Conference of the Association for Computational Linguistics. 4171–4186.

[6]

Zuohui Fu, Yikun Xian, Yaxin Zhu, Yongfeng Zhang, and Gerard de Melo. 2020. Cookie: A dataset for conversational recommendation over knowledge graphs in e-commerce. arXiv preprint arXiv:2008.09237(2020).

[7]

Donghoon Ham, Jeong-Gwan Lee, Youngsoo Jang, and Kee-Eung Kim. 2020. End-to-end neural pipeline for goal-oriented dialogue systems using gpt-2. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 583–592.

[8]

Shirley Anugrah Hayati, Dongyeop Kang, Qingxiaoyang Zhu, Weiyan Shi, and Zhou Yu. 2020. Inspired: Toward sociable recommendation dialog systems. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 8142–8152.

[9]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web. 173–182.

Digital Library

[10]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In Proceedings of the 4th International Conference on Learning Representations. 1–10.

[11]

Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In Proceedings of IEEE International Conference on Data Mining. 197–206.

[12]

Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1746–1751.

[13]

Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef, Sören Auer, 2015. Dbpedia–a large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web 6, 2 (2015), 167–195.

[14]

Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, and Tat-Seng Chua. 2020. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining. 304–312.

Digital Library

[15]

Wenqiang Lei, Gangyi Zhang, Xiangnan He, Yisong Miao, Xiang Wang, Liang Chen, and Tat-Seng Chua. 2020. Interactive path reasoning on graph for conversational recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2073–2083.

Digital Library

[16]

Raymond Li, Samira Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards deep conversational recommendations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 9748–9758.

[17]

Shijun Li, Wenqiang Lei, Qingyun Wu, Xiangnan He, Peng Jiang, and Tat-Seng Chua. 2020. Seamlessly unifying attributes and items: Conversational recommendation for cold-start users. arXiv preprint arXiv:2005.12979(2020).

[18]

Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, and Asli Celikyilmaz. 2017. End-to-end task-completion neural dialogue systems. In Proceedings of the 8th International Joint Conference on Natural Language Processing. 733–743.

[19]

Lizi Liao, Ryuichi Takanobu, Yunshan Ma, Xun Yang, Minlie Huang, and Tat-Seng Chua. 2019. Deep conversational recommender in travel. arXiv preprint arXiv:1907.00710(2019).

[20]

Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence. 2181–2187.

[21]

Zeming Liu, Haifeng Wang, Zheng-Yu Niu, Hua Wu, Wanxiang Che, and Ting Liu. 2020. Towards conversational recommendation over multi-type dialogs. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 1036–1049.

[22]

Wenchang Ma, Ryuichi Takanobu, Minghao Tu, and Minlie Huang. 2020. Bridging the gap between conversational reasoning and interactive recommendation. arXiv preprint arXiv:2010.10333(2020).

[23]

Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. 2019. Opendialkg: Explainable conversational reasoning with attention-based walks over knowledge graphs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 845–854.

[24]

Florian Pecune, Shruti Murali, Vivian Tsai, Yoichi Matsuyama, and Justine Cassell. 2019. A model of social explanations for a conversational movie recommendation system. In Proceedings of the 7th International Conference on Human-Agent Interaction. 135–143.

Digital Library

[25]

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 1–24.

[26]

Xuhui Ren, Hongzhi Yin, Tong Chen, Hao Wang, Nguyen Quoc Viet Hung, Zi Huang, and Xiangliang Zhang. 2020. Crsal: Conversational recommender systems with adversarial learning. ACM Transactions on Information Systems 38, 4 (2020), 1–40.

Digital Library

[27]

Rajdeep Sarkar, Koustava Goswami, Mihael Arcan, and John Philip McCrae. 2020. Suggest me a movie for tonight: Leveraging knowledge graphs for conversational recommendation. In Proceedings of the 28th International Conference on Computational Linguistics. 4179–4189.

[28]

Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International Conference on World Wide Web. 285–295.

Digital Library

[29]

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347(2017).

[30]

Iulian Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In Proceedings of the AAAI Conference on Artificial Intelligence. 3295–3301.

[31]

Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the AAAI Conference on Artificial Intelligence. 4444–4451.

[32]

Yueming Sun and Yi Zhang. 2018. Conversational recommender system. In Proceeding of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 235–244.

Digital Library

[33]

Daisuke Tsumita and Tomohiro Takagi. 2019. Dialogue based recommender system that flexibly mixes utterances and recommendations. In Proceedings of IEEE International Conference on Web Intelligence. 51–58.

Digital Library

[34]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of Advances in Neural Information Processing Systems. 5998–6008.

[35]

Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. Kgat: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 950–958.

Digital Library

[36]

Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. Persuasion for good: Towards a personalized persuasive dialogue system for social good. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 5635–5649.

[37]

Hu Xu, Seungwhan Moon, Honglei Liu, Bing Liu, Pararth Shah, and S Yu Philip. 2020. User memory reasoning for conversational recommendation. In Proceedings of the 28th International Conference on Computational Linguistics. 5288–5308.

[38]

Kun Zhou, Xiaolei Wang, Yuanhang Zhou, Chenzhan Shang, Yuan Cheng, Wayne Xin Zhao, Yaliang Li, and Ji-Rong Wen. 2021. Crslab: An open-Source toolkit for building conversational recommender system. arXiv preprint arXiv:2101.00939(2021).

[39]

Kun Zhou, Wayne Xin Zhao, Shuqing Bian, Yuanhang Zhou, Ji-Rong Wen, and Jingsong Yu. 2020. Improving conversational recommender systems via knowledge graph based semantic fusion. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1006–1014.

Digital Library

[40]

Kun Zhou, Yuanhang Zhou, Wayne Xin Zhao, Xiaoke Wang, and Ji-Rong Wen. 2020. Towards topic-guided conversational recommender system. In Proceedings of the 28th International Conference on Computational Linguistics. 4128–4139.

[41]

Li Zhou, Kevin Small, Oleg Rokhlenko, and Charles Elkan. 2017. End-to-end offline goal-oriented dialog policy learning via policy gradient. arXiv preprint arXiv:1712.02838(2017).

Cited By

Zou JSun ALong CKanoulas E(2024)Knowledge-Enhanced Conversational Recommendation via Transformer-Based Sequential ModelingACM Transactions on Information Systems10.1145/367737642:6(1-27)Online publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1145/3677376
Zhao YYin KWang ZDastani MWang S(2024)Decomposed Deep Q-Network for Coherent Task-Oriented Dialogue Policy LearningIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2024.335703832(1380-1391)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1109/TASLP.2024.3357038
Zhang QChen Y(2022)A Survey of Literature Analysis Methods Based on Representation LearningImage and Graphics Technologies and Applications10.1007/978-981-19-5096-4_19(249-263)Online publication date: 22-Jul-2022
https://doi.org/10.1007/978-981-19-5096-4_19

Index Terms

Knowledge-Based Conversational Recommender Systems Enhanced by Dialogue Policy Learning
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
    1. Learning settings
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
  2. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Conversational recommender systems (CRS) enable the traditional recommender systems to explicitly acquire user preferences towards items and attributes through interactive conversations. Reinforcement learning (RL) is widely adopted to learn ...
Conversational Recommender System
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

A personalized conversational sales agent could have much commercial potential. E-commerce companies such as Amazon, eBay, JD, Alibaba etc. are piloting such kind of agents with their users. However, the research on this topic is very limited and ...
Learning a Voice-based Conversational Recommender using Offline Policy Optimization
RecSys '21: Proceedings of the 15th ACM Conference on Recommender Systems

Voice-based conversational recommenders offer a natural way to improve recommendation quality by asking the user for missing information. This talk details how we use offline policy optimization to learn a dialog manager that determines what items to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

IJCKG '21: Proceedings of the 10th International Joint Conference on Knowledge Graphs

December 2021

204 pages

ISBN:9781450395656

DOI:10.1145/3502223

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 January 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IJCKG'21

IJCKG'21: The 10th International Joint Conference on Knowledge Graphs

December 6 - 8, 2021

Virtual Event, Thailand

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
252
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zou JSun ALong CKanoulas E(2024)Knowledge-Enhanced Conversational Recommendation via Transformer-Based Sequential ModelingACM Transactions on Information Systems10.1145/367737642:6(1-27)Online publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1145/3677376
Zhao YYin KWang ZDastani MWang S(2024)Decomposed Deep Q-Network for Coherent Task-Oriented Dialogue Policy LearningIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2024.335703832(1380-1391)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1109/TASLP.2024.3357038
Zhang QChen Y(2022)A Survey of Literature Analysis Methods Based on Representation LearningImage and Graphics Technologies and Applications10.1007/978-981-19-5096-4_19(249-263)Online publication date: 22-Jul-2022
https://doi.org/10.1007/978-981-19-5096-4_19

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten