research-article

Conco-ERNIE: Complex User Intent Detect Model for Smart Healthcare Cognitive Bot

Authors:

Xiaofei XuAuthors Info & Claims

ACM Transactions on Internet Technology, Volume 23, Issue 1

Article No.: 21, Pages 1 - 24

https://doi.org/10.1145/3574135

Published: 23 February 2023 Publication History

Abstract

The outbreak of Covid-19 has exposed the lack of medical resources, especially the lack of medical personnel. This results in time and space restrictions for medical services, and patients cannot obtain health information all the time and everywhere. Based on the medical knowledge graph, healthcare bots alleviate this burden effectively by providing patients with diagnosis guidance, pre-diagnosis, and post-diagnosis consultation services in the way of human-machine dialogue. However, the medical utterance is more complicated in language structure, and there are complex intention phenomena in semantics. It is a challenge to detect the single intent, multi-intent, and implicit intent of a patient’s utterance. To this end, we create a high-quality annotated Chinese Medical query (utterance) dataset, CMedQ (about 16.8k queries in medical domain which includes single, multiple, and implicit intents). It is hard to detect intent on such a complex dataset through traditional text classification models. Thus, we propose a novel detect model Conco-ERNIE, using concept co-occurrence patterns to enhance the representation of pre-trained model ERNIE. These patterns are mined using Apriori algorithm and will be embedded via Node2Vec. Their features will be aggregated with semantic features into Conco-ERNIE by using an attention module, which can catch user explicit intents and also predict user implicit intents. Experiments on CMedQ demonstrates that Conco-ERNIE achieves outstanding performance over baseline. Based on Conco-ERNIE, we develop an intelligent healthcare bot, MedicalBot. To provide knowledge support for MedicalBot, we also build a Chinese medical graph, CMedKG (about 45k entities and 283k relationships).

References

[1]

Klaus-Peter Adlassnig. 1986. Fuzzy set theory in medical diagnosis. IEEE Transactions on Systems, Man, and Cybernetics 16, 2 (1986), 260–265.

Digital Library

[2]

Rakesh Agrawal, Ramakrishnan Srikant, et al. 1994. Fast algorithms for mining association rules. In Proc. 20th Int. Conf. Very Large Data Bases, VLDB, Vol. 1215. 487–499.

[3]

Guirong Bai, Shizhu He, Kang Liu, and Jun Zhao. 2022. Incremental intent detection for medical domain with contrast replay networks. In Findings of the Association for Computational Linguistics: ACL, Dublin, Ireland, May 22–27. 3549–3556.

[4]

Jovan Carlo S. Ca ballero, Carl Benedick D. Ching, Sean Patrick S. Co, Hazel O. Noble, and Arne B. Barcelo. 2021. LifeDoc: Availability and monitoring system of online medical consultation. In 11th IEEE International Conference on Control System, Computing and Engineering, ICCSCE 2021, Penang, Malaysia, August 27–28, 2021. 103–108.

[5]

Ruichu Cai, Binjun Zhu, Lei Ji, Tianyong Hao, Jun Yan, and Wenyin Liu. [n.d.]. An CNN-LSTM attention approach to understanding user query intent from online health communities. In 2017 IEEE International Conference on Data Mining Workshops, ICDM Workshops 2017, New Orleans, LA, USA, November 18–21, 2017. 430–437.

[6]

Nan Chen, Xiangdong Su, Tongyang Liu, Qizhi Hao, and Ming Wei. 2020. A benchmark dataset and case study for Chinese medical question intent classification. BMC Medical Informatics Decis. Mak. 20-S, 3 (2020), 125.

[7]

Ziheng Chen and Jiangtao Ren. 2021. Multi-label text classification with latent word-wise label information. Appl. Intell. 51, 2 (2021), 966–979.

Digital Library

[8]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[9]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 855–864.

Digital Library

[10]

Zhaohan Daniel Guo, Gökhan Tür, Wen-tau Yih, and Geoffrey Zweig. [n.d.]. Joint semantic utterance classification and slot filling with recursive neural networks. In 2014 IEEE Spoken Language Technology Workshop, SLT 2014, South Lake Tahoe, NV, USA, December 7–10, 2014. 554–559.

[11]

E. Haihong, Peiqing Niu, Zhongfu Chen, and Meina Song. 2019. A novel bi-directional interrelated model for joint intent detection and slot filling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 5467–5471.

[12]

Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. CoRR abs/1508.01991 (2015). http://arxiv.org/abs/1508.01991.

[13]

Rie Johnson and Tong Zhang. 2016. Supervised and semi-supervised text categorization using LSTM for region embeddings. In Proceedings of the 33rd International Conference on Machine Learning, ICML 2016, Maria-Florina Balcan and Kilian Q. Weinberger (Eds.), Vol. 48. 526–534.

[14]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[15]

Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent convolutional neural networks for text classification. In Twenty-ninth AAAI Conference on Artificial Intelligence.

Digital Library

[16]

C. Y. Lin, Y. H. Wu, and Alp Chen. 2021. Selecting the most helpful answers in online health question answering communities. Journal of Intelligent Information Systems3 (2021).

[17]

Haruhisa Maeda, Sachio Saiki, Masahide Nakamura, and Kiyoshi Yasuda. 2019. Recording daily health status with chatbot on mobile phone - A preliminary study. In Twelfth International Conference on Mobile Computing and Ubiquitous Network, ICMU 2019, Kathmandu, Nepal, November 4–6, 2019. IEEE, 1–6.

[18]

Song Mao, Lu-Lu Zhang, and Zhen-Guo Guan. 2021. An LSTM&Topic-CNN model for classification of online Chinese medical questions. IEEE Access (2021), 52580–52589.

[19]

Shervin Minaee, Nal Kalchbrenner, Erik Cambria, Narjes Nikzad, Meysam Chenaghlu, and Jianfeng Gao. 2021. Deep learning-based text classification: A comprehensive review. ACM Comput. Surv. 54, 3 (2021), 62:1–62:40.

[20]

Antoine Neuraz, Leonardo Campillos Llanos, Anita Burgun, and Sophie Rosset. 2018. Natural language understanding for task oriented dialog in the biomedical domain in a low resources context. arXiv preprint arXiv:1811.09417 (2018).

[21]

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1–6, 2018, Volume 1 (Long Papers). 2227–2237.

[22]

Heereen Shim, Dietwig Lowet, Stijn Luca, and Bart Vanrumste. 2021. Building blocks of a task-oriented dialogue system in the healthcare domain. In Proceedings of the Second Workshop on Natural Language Processing for Medical Conversations. 47–57.

[23]

Isabelle Stanton, Samuel Ieong, and Nina Mishra. 2014. Circumlocution in diagnostic medical queries. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval. 133–142.

Digital Library

[24]

Yu Sun, Shuohuan Wang, Yu-Kun Li, Shikun Feng, Hao Tian, Hua Wu, and Haifeng Wang. 2020. ERNIE 2.0: A continual pre-training framework for language understanding. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020. AAAI Press, 8968–8975.

[25]

Jixuan Wang, Kai Wei, Martin Radfar, Weiwei Zhang, and Clement Chung. 2021. Encoding syntactic knowledge in transformer encoder for intent detection and slot filling. In Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021. 13943–13951.

[26]

Zhongyu Wei, Qianlong Liu, Baolin Peng, Huaixiao Tou, Ting Chen, Xuan-Jing Huang, Kam-Fai Wong, and Xiang Dai. 2018. Task-oriented dialogue system for automatic diagnosis. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 201–207.

[27]

Chaochen Wu, Guan Luo, Chao Guo, Yin Ren, Anni Zheng, and Cheng Yang. 2020. An attention-based multi-task model for named entity recognition and intent analysis of Chinese online medical questions. J. Biomed. Informatics 108 (2020), 103511.

[28]

Di Wu, Liang Ding, Fan Lu, and Jian Xie. 2020. SlotRefine: A fast non-autoregressive model for joint intent detection and slot filling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16–20, 2020. 1932–1937.

[29]

Lin Xu, Qixian Zhou, Ke Gong, Xiaodan Liang, Jianheng Tang, and Liang Lin. 2019. End-to-end knowledge-routed relational dialogue system for automatic diagnosis. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 7346–7353.

Digital Library

[30]

Mengting Xu, Yanrong Cao, Guozi Sun, and Huakang Li. 2019. CRQA: Credibility retrieval for medical question answer service. In 2019 IEEE International Conference on Real-time Computing and Robotics (RCAR). IEEE, 347–350.

[31]

Man Yuan, Yuan Xin Ouyang, and Zhang Xiong. 2013. A text categorization method using extended vector space model by frequent term sets. Journal of Information Science and Engineering 29, 1 (2013), 99–114.

[32]

Chenwei Zhang, Nan Du, Wei Fan, Yaliang Li, Chun-Ta Lu, and S. Yu Philip. 2017. Bringing semantic structures to user intent detection in online medical queries. In 2017 IEEE International Conference on Big Data (Big Data). IEEE, 1019–1026.

[33]

Chenwei Zhang, Wei Fan, Nan Du, and Philip S. Yu. 2016. Mining user intentions from medical queries: A neural network based heterogeneous jointly modeling approach. In Proceedings of the 25th International Conference on World Wide Web, WWW 2016, Montreal, Canada, April 11–15, 2016. ACM, 1373–1384.

Digital Library

[34]

Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang, Hui Zong, Zheng Yuan, Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Buzhou Tang, and Qingcai Chen. [n.d.]. CBLUE: A Chinese biomedical language understanding evaluation benchmark. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22–27, 2022. 7888–7915.

[35]

Binggui Zhou, Guanghua Yang, Zheng Shi, and Shaodan Ma. 2021. Natural language processing for smart healthcare. arXiv preprint arXiv:2110.15803 (2021).

[36]

Wei Zhu, Yuan Ni, Xiaoling Wang, and Guotong Xie. 2021. Discovering better model architectures for medical query understanding. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers. 230–237.

Cited By

Huang XHuang LTong GZhou XLuo J(2024)Security Challenges and Reflections on Large ModelsFrontiers in Computing and Intelligent Systems10.54097/xy2amt059:2(1-3)Online publication date: 27-Aug-2024
https://doi.org/10.54097/xy2amt05
Zhang BTu ZWang CSun HChu D(2024)Requirements elicitation and response generation for conversational servicesApplied Intelligence10.1007/s10489-024-05454-654:7(5576-5592)Online publication date: 23-Apr-2024
https://dl.acm.org/doi/10.1007/s10489-024-05454-6

Index Terms

Conco-ERNIE: Complex User Intent Detect Model for Smart Healthcare Cognitive Bot
1. Applied computing
  1. Life and medical sciences
    1. Health care information systems
2. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Discourse, dialogue and pragmatics
      2. Language resources

Recommendations

Smart Healthcare. a General Encyclopaedia for Healthcare Problems
Automatic recommendation of medical departments to outpatients based on text analyses and medical knowledge graph

In many countries, outpatients generally visit a major hospital without a referral from health professionals due to the shortage of family physicians. Not knowing at which medical specialty department to register, outpatients have to wait in long queues ...
E-health interoperability and smart interactions in healthcare
CASCON '09: Proceedings of the 2009 Conference of the Center for Advanced Studies on Collaborative Research

Healthcare delivery is becoming increasingly complex as it shifts from care provided by a single provider and setting to collaborative care provided by multiple providers across multiple settings. For example, patients with chronic illness frequently ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Internet Technology

ACM Transactions on Internet Technology Volume 23, Issue 1

February 2023

564 pages

ISSN:1533-5399

EISSN:1557-6051

DOI:10.1145/3584863

Editor:
Ling Liu
Georgia Institute of Technology, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 February 2023

Online AM: 08 December 2022

Accepted: 27 November 2022

Revised: 20 September 2022

Received: 06 October 2021

Published in TOIT Volume 23, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key R&D Program of China
National Science Foundation of China
Key projects of Shandong Natural Science Foundation
Key Research and Development Program of Shandong Province

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
384
Total Downloads

Downloads (Last 12 months)175
Downloads (Last 6 weeks)9

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huang XHuang LTong GZhou XLuo J(2024)Security Challenges and Reflections on Large ModelsFrontiers in Computing and Intelligent Systems10.54097/xy2amt059:2(1-3)Online publication date: 27-Aug-2024
https://doi.org/10.54097/xy2amt05
Zhang BTu ZWang CSun HChu D(2024)Requirements elicitation and response generation for conversational servicesApplied Intelligence10.1007/s10489-024-05454-654:7(5576-5592)Online publication date: 23-Apr-2024
https://dl.acm.org/doi/10.1007/s10489-024-05454-6

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents