Article

Towards Automated Customer Support

Authors:

Momchil Hardalov,

Preslav NakovAuthors Info & Claims

Artificial Intelligence: Methodology, Systems, and Applications: 18th International Conference, AIMSA 2018, Varna, Bulgaria, September 12–14, 2018, Proceedings

Pages 48 - 59

https://doi.org/10.1007/978-3-319-99344-7_5

Published: 12 September 2018 Publication History

Abstract

Recent years have seen growing interest in conversational agents, such as chatbots, which are a very good fit for automated customer support because the domain in which they need to operate is narrow. This interest was in part inspired by recent advances in neural machine translation, esp. the rise of sequence-to-sequence (seq2seq) and attention-based models such as the Transformer, which have been applied to various other tasks and have opened new research directions in question answering, chatbots, and conversational systems. Still, in many cases, it might be feasible and even preferable to use simple information retrieval techniques. Thus, here we compare three different models: (i) a retrieval model, (ii) a sequence-to-sequence model with attention, and (iii) Transformer. Our experiments with the Twitter Customer Support Dataset, which contains over two million posts from customer support services of twenty major brands, show that the seq2seq model outperforms the other two in terms of semantics and word overlap.

References

[1]

Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)

[2]

Banerjee, S., Lavie, A.: METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, Michigan, pp. 65–72 (2005)

[3]

Boyanov, M., Nakov, P., Moschitti, A., Da San Martino, G., Koychev, I.: Building chatbots from forum data: model selection using question answering metrics. In: Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, Varna, Bulgaria, pp. 121–129 (2017)

[4]

Cui, L., Huang, S., Wei, F., Tan, C., Duan, C., Zhou, M.: SuperAgent: a customer service chatbot for e-commerce websites. In: Proceedings of the Association for Computational Linguistics 2017, System Demonstrations, ACL 2017, Vancouver, Canada, pp. 97–102 (2017)

[5]

Forgues, G., Pineau, J., Larchevêque, J.M., Tremblay, R.: Bootstrapping dialog systems with word embeddings. In: Proceedings of the NIPS Workshop on Modern Machine Learning and Natural Language Processing, Montreal, Canada (2014)

[6]

Hochreiter S and Schmidhuber J Long short-term memory Neural Comput. 1997 9 8 1735-1780

Digital Library

[7]

Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the 2015 International Conference on Learning Representations, ICLR 2015, San Diego, California (2015)

[8]

Lin, C.Y.: ROUGE: a package for automatic evaluation of summaries. In: Proceedings of the ACL Workshop on Text Summarization Branches Out, Barcelona, Spain, pp. 74–81 (2004)

[9]

Lin, C.Y., Och, F.J.: Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. In: Proceedings of the 42nd Annual Conference of the Association for Computational Linguistics, ACL 2004, Barcelona, Spain, pp. 605–612 (2004)

[10]

Liu, C.W., Lowe, R., Serban, I., Noseworthy, M., Charlin, L., Pineau, J.: How NOT to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, pp. 2122–2132 (2016)

[11]

Lowe, R., Noseworthy, M., Serban, I.V., Angelard-Gontier, N., Bengio, Y., Pineau, J.: Towards an automatic Turing test: learning to evaluate dialogue responses. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, pp. 1116–1126 (2017)

[12]

Lowe, R., Pow, N., Serban, I., Pineau, J.: The Ubuntu dialogue corpus: a large dataset for research in unstructured multi-turn dialogue systems. In: Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL 2015, Prague, Czech Republic, pp. 285–294 (2015)

[13]

Lowe RT, Pow N, Serban IV, Charlin L, Liu CW, and Pineau J Training end-to-end dialogue systems with the Ubuntu dialogue corpus Dialogue Discourse 2017 8 1 31-65

[14]

Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, ACL 2014, Baltimore, Maryland, pp. 55–60 (2014)

[15]

Nakov, P., et al.: SemEval-2016 task 3: community question answering. In: Proceedings of the 10th International Workshop on Semantic Evaluation, SemEval 2016, San Diego, California, pp. 525–545 (2016)

[16]

Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, ACL 2002, Philadelphia, Pennsylvania, pp. 311–318 (2002)

[17]

Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, Doha, Qatar, pp. 1532–1543 (2014)

[18]

Qiu, M., et al.: AliMe Chat: a sequence to sequence and rerank based chatbot engine. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, pp. 498–503 (2017)

[19]

Robertson S and Zaragoza H The probabilistic relevance framework: BM25 and beyond Found. Trends Inf. Retr. 2009 3 4 333-389

Digital Library

[20]

Rus, V., Lintean, M.: A comparison of greedy and optimal assessment of natural language student input using word-to-word similarity metrics. In: Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, Montreal, Canada, pp. 157–162 (2012)

[21]

Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, pp. 1715–1725 (2016)

[22]

Serban IV, Lowe R, Henderson P, Charlin L, and Pineau J A survey of available corpora for building data-driven dialogue systems: the journal version Dialogue Discourse 2018 9 1 1-49

[23]

Serban, I.V., Sordoni, A., Bengio, Y., Courville, A.C., Pineau, J.: Hierarchical neural network generative models for movie dialogues. CoRR, abs/1507.04808 (2015)

[24]

Shang, L., Lu, Z., Li, H.: Neural responding machine for short-text conversation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, ACL-IJCNLP 2015, Beijing, China, pp. 1577–1586 (2015)

[25]

Sordoni, A., et al.: A neural network approach to context-sensitive generation of conversational responses. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2015, Denver, Colorado, pp. 196–205 (2015)

[26]

Srivastava N, Hinton G, Krizhevsky A, Sutskever I, and Salakhutdinov R Dropout: a simple way to prevent neural networks from overfitting J. Mach. Learn. Res. 2014 15 1 1929-1958

Digital Library

[27]

Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Proceedings of the 27th Annual Conference on Neural Information Processing Systems, NIPS 2014, Montreal, Canada, pp. 3104–3112 (2014)

[28]

Vaswani, A., et al.: Attention is all you need. In: Proceedings of the 30th Annual Conference on Neural Information Processing Systems, NIPS 2017, Long Beach, California, pp. 5998–6008 (2017)

[29]

Vinyals, O., Le, Q.V.: A neural conversational model. CoRR abs/1506.05869 (2015)

Cited By

Karren KSchmitz MSchaffer S(2024)Improving Conversational User Interfaces for Citizen Complaint Management through enhanced Contextual FeedbackProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665562(1-11)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665562
Zhuang YSong JSadagopan NBeniwal A(2023)Self-supervised Pre-training and Semi-supervised Learning for Extractive Dialog SummarizationCompanion Proceedings of the ACM Web Conference 202310.1145/3543873.3587680(1069-1076)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543873.3587680
Alias SSainin MSoo Fun TDaut N(2019)Identification of Conversational Intent Pattern Using Pattern-Growth Technique for Academic ChatbotMulti-disciplinary Trends in Artificial Intelligence10.1007/978-3-030-33709-4_24(263-270)Online publication date: 17-Nov-2019
https://dl.acm.org/doi/10.1007/978-3-030-33709-4_24

Recommendations

Conversational Agents: Acting on the Wave of Research and Development
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

In the last five years, work on software that interacts with people via typed or spoken natural language, called chatbots, intelligent assistants, social bots, virtual companions, non-human players, and so on, increased dramatically. Chatbots burst into ...
Predicting Customer Satisfaction in Customer Support Conversations in Social Media Using Affective Features
UMAP '16: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization

Providing customer support through social media channels is gaining popularity. In such a context, predicting customer satisfaction in an early stage of a service conversation is important. Such an analysis can help personalize agent assignment to ...
User Engagement with Chatbots: A Discursive Psychology Approach
CUI '20: Proceedings of the 2nd Conference on Conversational User Interfaces

Conversational agents have transcended into multiple industries with increased ability for user engagement in intelligent conversation. Conversations with chatbots are different from interpersonal communication in terms of turn-taking, intentions, and ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

Artificial Intelligence: Methodology, Systems, and Applications: 18th International Conference, AIMSA 2018, Varna, Bulgaria, September 12–14, 2018, Proceedings

Sep 2018

296 pages

ISBN:978-3-319-99343-0

DOI:10.1007/978-3-319-99344-7

Editors:
Gennady Agre
Institute of Information and Communication Technologies, Bulgarian Academy of Sciences, Sofia, Bulgaria
,
Josef van Genabith
Universität des Saarlandes, Saarbrücken, Germany
,
Thierry Declerck
DFKI GmbH, Saarbrücken, Germany

© Springer Nature Switzerland AG 2018.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 12 September 2018

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Karren KSchmitz MSchaffer S(2024)Improving Conversational User Interfaces for Citizen Complaint Management through enhanced Contextual FeedbackProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665562(1-11)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665562
Zhuang YSong JSadagopan NBeniwal A(2023)Self-supervised Pre-training and Semi-supervised Learning for Extractive Dialog SummarizationCompanion Proceedings of the ACM Web Conference 202310.1145/3543873.3587680(1069-1076)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543873.3587680
Alias SSainin MSoo Fun TDaut N(2019)Identification of Conversational Intent Pattern Using Pattern-Growth Technique for Academic ChatbotMulti-disciplinary Trends in Artificial Intelligence10.1007/978-3-030-33709-4_24(263-270)Online publication date: 17-Nov-2019
https://dl.acm.org/doi/10.1007/978-3-030-33709-4_24

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents