research-article

Interacting with Non-Cooperative User: A New Paradigm for Proactive Dialogue Policy

Authors:

Zhenglu Yang, and

Tat-Seng ChuaAuthors Info & Claims

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2022

Pages 212 - 222

https://doi.org/10.1145/3477495.3532001

Published: 07 July 2022 Publication History

Abstract

Proactive dialogue system is able to lead the conversation to a goal topic and has advantaged potential in bargain, persuasion, and negotiation. Current corpus-based learning manner limits its practical application in real-world scenarios. To this end, we contribute to advancing the study of the proactive dialogue policy to a more natural and challenging setting, i.e., interacting dynamically with users. Further, we call attention to the non-cooperative user behavior - the user talks about off-path topics when he/she is not satisfied with the previous topics introduced by the agent. We argue that the targets of reaching the goal topic quickly and maintaining a high user satisfaction are not always converged, because the topics close to the goal and the topics user preferred may not be the same. Towards this issue, we propose a new solution named I-Pro that can learn Proactive policy in the Interactive setting. Specifically, we learn the trade-off via a learned goal weight, which consists of four factors (dialogue turn, goal completion difficulty, user satisfaction estimation, and cooperative degree). The experimental results demonstrate I-Pro significantly outperforms baselines in terms of effectiveness and interpretability.

Supplementary Material

MP4 File (SIGIR22-fp0284.mp4)

Presentation video.

Download
11.90 MB

References

[1]

Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. 2020. Towards a Human-like Open-Domain Chatbot. CoRR, Vol. abs/2001.09977 (2020).

[2]

Jiaqi Bai, Ze Yang, Xinnian Liang, Wei Wang, and Zhoujun Li. 2021. Learning to Copy Coherent Knowledge for Response Generation. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI '21, 14). 12535--12543.

[3]

Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017. Learning End-to-End Goal-Oriented Dialog. In Proceedings of the 5th International Conference on Learning Representations (ICLR '17).

[4]

Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the Properties of Neural Machine Translation: Encoder--Decoder Approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation. 103--111.

[5]

Ritam Dutt, Sayan Sinha, Rishabh Joshi, Surya Shekhar Chakraborty, Meredith Riggs, Xinru Yan, Haogang Bao, and Carolyn Rose. 2021. ResPer: Computationally Modelling Resisting Strategies in Persuasive Conversations. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL '21). 78--90.

[6]

Asma Ghandeharioun, Judy Hanwen Shen, Natasha Jaques, Craig Ferguson, Noah Jones, Agata Lapedriza, and Rosalind Picard. 2019. Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems. In Advances in Neural Information Processing Systems (NeurIPS '19). Curran Associates, Inc. https://proceedings.neurips.cc/paper/2019/file/fc9812127bf09c7bd29ad6723c683fb5-Paper.pdf

[7]

He He, Anusha Balakrishnan, Mihail Eric, and Percy Liang. 2017. Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL '17). 1766--1776.

[8]

He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. 2018. Decoupling Strategy and Generation in Negotiation Dialogues. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP '18). 2333--2343.

[9]

Dongyeop Kang, Anusha Balakrishnan, Pararth Shah, Paul A Crook, Y-Lan Boureau, and Jason Weston. 2019. Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialogue. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP '19). 1951--1961.

[10]

Veton Këpuska and Gamal Bohouta. 2018. Next-generation of virtual personal assistants (microsoft cortana, apple siri, amazon alexa and google home). In 2018 IEEE 8th annual computing and communication workshop and conference (CCWC). IEEE, 99--103.

[11]

Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, and Tat-Seng Chua. 2020. Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems. In Proceedings of the 13th International Conference on Web Search and Data Mining (WSDM '20). 304--312.

Digital Library

[12]

Mike Lewis, Denis Yarats, Yann Dauphin, Devi Parikh, and Dhruv Batra. 2017. Deal or No Deal? End-to-End Learning of Negotiation Dialogues. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP '17). 2443--2453.

[13]

Dawen Liang, Jaan Altosaar, Laurent Charlin, and David M. Blei. 2016. Factorization Meets the Item Embedding: Regularizing Matrix Factorization with Item Co-Occurrence. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys '16). 59--66.

[14]

Bing Liu, Gokhan Tur, Dilek Hakkani-Tur, Pararth Shah, and Larry Heck. 2018b. Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL '18). 2060--2069.

[15]

Shuman Liu, Hongshen Chen, Zhaochun Ren, Yang Feng, Qun Liu, and Dawei Yin. 2018a. Knowledge diffusion for neural dialogue generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL '18). 1489--1498.

[16]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et almbox. 2015. Human-level control through deep reinforcement learning. Nature, Vol. 518, 7540 (2015), 529--533.

[17]

Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. 2019. Opendialkg: Explainable conversational reasoning with attention-based walks over knowledge graphs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL '19). 845--854.

[18]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD '14). 701--710.

Digital Library

[19]

Yue Shi, Martha Larson, and Alan Hanjalic. 2014. Collaborative Filtering beyond the User-Item Matrix: A Survey of the State of the Art and Future Challenges. ACM Comput. Surv., Vol. 47, 1 (2014), bibinfonumpages45 pages.

Digital Library

[20]

Ning Su, Jiyin He, Yiqun Liu, Min Zhang, and Shaoping Ma. 2018. User intent, behaviour, and perceived satisfaction in product search. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (WSDM '18). 547--555.

Digital Library

[21]

Xiaoyuan Su and Taghi M Khoshgoftaar. 2009. A survey of collaborative filtering techniques. Advances in artificial intelligence, Vol. 2009 (2009).

[22]

R Sutton and A Barto. 1998. Reinforcement Learning:An Introduction. MIT Press.

[23]

Youzhi Tian, Weiyan Shi, Chen Li, and Zhou Yu. 2020. Understanding User Resistance Strategies in Persuasive Conversations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings (EMNLP '20). 4794--4798.

[24]

Yi-Lin Tuan, Yun-Nung Chen, and Hung-yi Lee. 2019. DyKgChat: Benchmarking Dialogue Generation Grounding on Dynamic Knowledge Graphs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP '19). 1855--1865.

[25]

Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL '19). 5635--5649.

[26]

Junjie Wu and Hao Zhou. 2021. Augmenting Topic Aware Knowledge-Grounded Conversations with Dynamic Built Knowledge Graphs. In Proceedings of Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures. 31--39.

[27]

Wenquan Wu, Zhen Guo, Xiangyang Zhou, Hua Wu, Xiyuan Zhang, Rongzhong Lian, and Haifeng Wang. 2019. Proactive Human-Machine Conversation with Explicit Conversation Goal. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL '19). 3794--3804.

[28]

Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP '17). 564--573.

[29]

Zhao Yan, Nan Duan, Peng Chen, Ming Zhou, Jianshe Zhou, and Zhoujun Li. 2017. Building Task-Oriented Dialogue Systems for Online Shopping. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI'17). AAAI Press, 4618--4625.

Digital Library

[30]

Runzhe Yang, Jingxiao Chen, and Karthik Narasimhan. 2021. Improving Dialog Systems for Negotiation with Personality Modeling. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP '21). 681--693.

[31]

Hao Yuan and Jinqi An. 2020. Multi-Hop Memory Network with Graph Neural Networks Encoding for Proactive Dialogue. In Proceedings of the 2020 6th International Conference on Computing and Artificial Intelligence (ICCAI '20). 24--29.

Digital Library

[32]

Chen Zhang, Yiming Chen, Luis Fernando D'Haro, Yan Zhang, Thomas Friedrichs, Grandee Lee, and Haizhou Li. 2021. DynaEval: Unifying Turn and Dialogue Level Evaluation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP '21). Association for Computational Linguistics, 5676--5689.

[33]

Hao Zhou, Chujie Zheng, Kaili Huang, Minlie Huang, and Xiaoyan Zhu. 2020. KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL '20). 7098--7108.

[34]

Yiheng Zhou, Yulia Tsvetkov, Alan W Black, and Zhou Yu. 2019. Augmenting Non-Collaborative Dialog Systems with Explicit Semantic and Strategic Dialog History. In Proceedings of the 7th International Conference on Learning Representations (ICLR '19).

[35]

Yutao Zhu, Jian-Yun Nie, Kun Zhou, Pan Du, Hao Jiang, and Zhicheng Dou. 2021. Proactive Retrieval-Based Chatbots Based on Relevant Knowledge and Goals. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '21). 2000--2004.

Digital Library

[36]

Lixin Zou, Long Xia, Yulong Gu, Xiangyu Zhao, Weidong Liu, Jimmy Xiangji Huang, and Dawei Yin. 2020. Neural Interactive Collaborative Filtering. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '20). 749--758.

Digital Library

Cited By

Chua T(2024)Towards Generative Search and Recommendation: A keynote at RecSys 2023ACM SIGIR Forum10.1145/3642979.364298657:2(1-14)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1145/3642979.3642986
Deng YLei WLam WChua TElkind E(2023)A survey on proactive dialogue systemsProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/738(6583-6591)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/738
Chen XSong XWei YNie LChua TChen HDuh WHuang HKato MMothe JPoblete B(2023)Dual Semantic Knowledge Composed Multimodal Dialog SystemsProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591673(1518-1527)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591673

Index Terms

Interacting with Non-Cooperative User: A New Paradigm for Proactive Dialogue Policy

Recommendations

Cooperative or vindictive: bidding strategies in sponsored search auction
WINE'07: Proceedings of the 3rd international conference on Internet and network economics

We analyze the economic stability and dynamic manipulation of vindictive strategies in conjunction with forward-looking cooperative bidders in Sponsored Search Auction. We investigate different vindictive strategies of different rationalities : ...
Read More
Interacting oligopolistic and oligopsonistic Cournot markets

This paper considers the model of several interacting Cournot markets. Some of them are final goods markets, while the others are resource markets. The markets interact by sharing the same set of economic agents (producers), so that the latter are ...
Read More
User's expertise differences when interacting with simple medical user interfaces
USAB'07: Proceedings of the 3rd Human-computer interaction and usability engineering of the Austrian computer society conference on HCI and usability for medicine and health care

In order to provide helpful proposals for future redesign of insulin pump interfaces, a study was carried out to investigate the expertise difference between novice users and expert users when interacting with a simple user interface of insulin pumps. ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2022

3569 pages

ISBN:9781450387323

DOI:10.1145/3477495

General Chairs:
Enrique Amigo
UNED
,
Pablo Castells
UAM and Amazon
,
Julio Gonzalo
UNED
,
Program Chairs:
Ben Carterette
Spotify
,
J. Shane Culpepper
RMIT University
,
Gabriella Kazai
Waseda University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Fundamental Research Funds for the Central Universities of China
Beijing Academy of Artificial Intelligence (BAAI)
China Postdoctoral Science Foundation
China Scholarship Council
China Postdoctoral Science Foundation

Conference

SIGIR '22

Sponsor:

SIGIR

SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2022

Madrid, Spain

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
266
Total Downloads

Downloads (Last 12 months)108
Downloads (Last 6 weeks)4

Other Metrics

View Author Metrics

Citations

Cited By

Chua T(2024)Towards Generative Search and Recommendation: A keynote at RecSys 2023ACM SIGIR Forum10.1145/3642979.364298657:2(1-14)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1145/3642979.3642986
Deng YLei WLam WChua TElkind E(2023)A survey on proactive dialogue systemsProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/738(6583-6591)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/738
Chen XSong XWei YNie LChua TChen HDuh WHuang HKato MMothe JPoblete B(2023)Dual Semantic Knowledge Composed Multimodal Dialog SystemsProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591673(1518-1527)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591673

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents