research-article

MSQ-BioBERT: Ambiguity Resolution to Enhance BioBERT Medical Question-Answering

Authors:

Edward T. Dougherty,

Fang JinAuthors Info & Claims

WWW '23: Proceedings of the ACM Web Conference 2023

Pages 4020 - 4028

https://doi.org/10.1145/3543507.3583878

Published: 30 April 2023 Publication History

Abstract

Question answering (QA) is a task in the field of natural language processing (NLP) and information retrieval, which has pivotal applications in areas such as online reading comprehension and web search engines. Currently, Bidirectional Encoder Representations from Transformers (BERT) and its biomedical variation (BioBERT) achieve impressive results on the reading comprehension QA datasets and medical-related QA datasets, and so they are widely used for a variety of passage-based QA tasks. However, their performances rapidly deteriorate when encountering passage and context ambiguities. This issue is prevalent and unavoidable in many fields, notably the web-based medical field. In this paper, we introduced a novel approach called the Multiple Synonymous Questions BioBERT (MSQ-BioBERT), which integrates question augmentation, rather than the typical single question used by traditional BioBERT, to elevate BioBERT’s performance on medical QA tasks. In addition, we constructed an ambiguous medical dataset based on the information from Wikipedia web. Experiments with both this web-based constructed medical dataset and open biomedical datasets demonstrate the significant performance gains of the MSQ-BioBERT approach, showcasing a new method for addressing ambiguity in medical QA tasks.

References

[1]

Zahra Abbasiantaeb and Saeedeh Momtazi. 2021. Text-based question answering from information retrieval and deep neural network perspectives: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 11, 6 (2021), e1412.

[2]

Chris Alberti, Kenton Lee, and Michael Collins. 2019. A bert baseline for the natural questions. arXiv preprint arXiv:1901.08634 (2019).

[3]

Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. " O’Reilly Media, Inc.".

Digital Library

[4]

Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Bengio. 2015. Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349 (2015).

[5]

Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Wojciech Gajewski, Andrea Gesmundo, Neil Houlsby, and Wei Wang. 2017. Ask the right questions: Active question reformulation with reinforcement learning. arXiv preprint arXiv:1705.07830 (2017).

[6]

Raquel Cerdán, Eduardo Vidal-Abarca, Tomas Martinez, Ramiro Gilabert, and Laura Gil. 2009. Impact of question-answering tasks on search processes and reading comprehension. Learning and Instruction 19, 1 (2009), 13–27.

[7]

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).

[8]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[9]

Li Dong, Jonathan Mallinson, Siva Reddy, and Mirella Lapata. 2017. Learning to paraphrase for question answering. arXiv preprint arXiv:1708.06022 (2017).

[10]

Yongping Du, Jingya Yan, Yiliang Zhao, Yuxuan Lu, and Xingnan Jin. 2021. Dual Model Weighting Strategy and Data Augmentation in Biomedical Question Answering. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 659–662.

[11]

Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. arXiv preprint arXiv:1808.09381 (2018).

[12]

Sergey Edunov, Myle Ott, Marc’Aurelio Ranzato, and Michael Auli. 2019. On The Evaluation of Machine Translation Systems Trained With Back-Translation. arXiv preprint arXiv:1908.05204 (2019).

[13]

Alexander Halavais. 2017. Search engine society. John Wiley & Sons.

[14]

Minbyul Jeong, Mujeen Sung, Gangwoo Kim, Donghyeon Kim, Wonjin Yoon, Jaehyo Yoo, and Jaewoo Kang. 2020. Transferability of natural language inference to biomedical question answering. arXiv preprint arXiv:2007.00217 (2020).

[15]

Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2020. What Disease does this Patient Have¿ A Large-scale Open Domain Question Answering Dataset from Medical Exams. arXiv preprint arXiv:2009.13081 (2020).

[16]

Oleksandr Kolomiyets and Marie-Francine Moens. 2011. A survey on question answering technology from an information retrieval perspective. Information Sciences 181, 24 (2011), 5412–5434.

Digital Library

[17]

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics 7 (2019), 453–466.

[18]

Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36, 4 (2020), 1234–1240.

[19]

Vladimir I Levenshtein 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, Vol. 10. Soviet Union, 707–710.

[20]

Edward Ma. 2019. NLP Augmentation. https://github.com/makcedward/nlpaug.

[21]

Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. AmbigQA: Answering ambiguous open-domain questions. arXiv preprint arXiv:2004.10645 (2020).

[22]

Gonzalo Navarro. 2001. A guided tour to approximate string matching. ACM computing surveys (CSUR) 33, 1 (2001), 31–88.

[23]

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016).

[24]

Martin Sundermeyer, Tamer Alkhouli, Joern Wuebker, and Hermann Ney. 2014. Translation modeling with bidirectional recurrent neural networks. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 14–25.

[25]

George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, 2015. An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC bioinformatics 16, 1 (2015), 1–28.

[26]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. arXiv preprint arXiv:1706.03762 (2017).

[27]

Wei Wang, Ming Yan, and Chen Wu. 2018. Multi-granularity hierarchical attention fusion networks for reading comprehension and question answering. arXiv preprint arXiv:1811.11934 (2018).

[28]

Jason Wei and Kai Zou. 2019. EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, 6383–6389. https://www.aclweb.org/anthology/D19-1670

[29]

Andrew Wen, Mohamed Y Elwazir, Sungrim Moon, and Jungwei Fan. 2020. Adapting and evaluating a deep learning language model for clinical why-question answering. JAMIA open 3, 1 (2020), 16–20.

[30]

Wei Yang, Yuqing Xie, Luchen Tan, Kun Xiong, Ming Li, and Jimmy Lin. 2019. Data augmentation for bert fine-tuning in open-domain question answering. arXiv preprint arXiv:1904.06652 (2019).

Cited By

Guo MGuo MSu JChen JYu JWang JDu HSahu PSharma AJin FChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Bayesian Iterative Prediction and Lexical-based Interpretation for Disturbed Chinese Sentence Pair MatchingProceedings of the ACM Web Conference 202410.1145/3589334.3648149(4618-4629)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3648149
Meng XRao JQi SWang LXiao JWang X(2024)Harnessing the Power of Prompt Experts: Efficient Knowledge Distillation for Enhanced Language UnderstandingMachine Learning and Knowledge Discovery in Databases. Research Track and Demo Track10.1007/978-3-031-70371-3_13(218-234)Online publication date: 22-Aug-2024
https://doi.org/10.1007/978-3-031-70371-3_13
Guo MMa YEworuke EKhashei MSong JZhao YJin F(2023)Identifying COVID-19 cases and extracting patient reported symptoms from Reddit using natural language processingScientific Reports10.1038/s41598-023-39986-713:1Online publication date: 22-Aug-2023
https://doi.org/10.1038/s41598-023-39986-7

Index Terms

Recommendations

Medical Question Answering for Clinical Decision Support
CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

The goal of modern Clinical Decision Support (CDS) systems is to provide physicians with information relevant to their management of patient care. When faced with a medical case, a physician asks questions about the diagnosis, the tests, or treatments ...
Flat and Nested Protein Name Recognition Based on BioBERT and Biaffine Decoder
Bioinformatics Research and Applications
Abstract
With the rapid growth in the volume of literature in the biomedical field, it has become increasingly important to extract key information from it in an automated manner. Protein name recognition is of great research importance as an important pre-...
A novel self-attention enriching mechanism for biomedical question answering
Highlights
- Novel self-attention enriching mechanism for biomedical question answering.
- New stat-of-the-art results for factoid biomedical question answering.
- Correlation between transformer attention scores and final prediction is validated.
Abstract
The task of biomedical question answering is a subtask of the more general question answering task, that is concerned only with biomedical questions. The current state-of-the-art models in this task like BioBERT, and BioM-ELECTRA are all based on ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '23: Proceedings of the ACM Web Conference 2023

April 2023

4293 pages

ISBN:9781450394161

DOI:10.1145/3543507

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Artifacts Available / v1.1

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '23

Sponsor:

SIGWEB

WWW '23: The ACM Web Conference 2023

April 30 - May 4, 2023

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
168
Total Downloads

Downloads (Last 12 months)102
Downloads (Last 6 weeks)3

Reflects downloads up to 13 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Guo MGuo MSu JChen JYu JWang JDu HSahu PSharma AJin FChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Bayesian Iterative Prediction and Lexical-based Interpretation for Disturbed Chinese Sentence Pair MatchingProceedings of the ACM Web Conference 202410.1145/3589334.3648149(4618-4629)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3648149
Meng XRao JQi SWang LXiao JWang X(2024)Harnessing the Power of Prompt Experts: Efficient Knowledge Distillation for Enhanced Language UnderstandingMachine Learning and Knowledge Discovery in Databases. Research Track and Demo Track10.1007/978-3-031-70371-3_13(218-234)Online publication date: 22-Aug-2024
https://doi.org/10.1007/978-3-031-70371-3_13
Guo MMa YEworuke EKhashei MSong JZhao YJin F(2023)Identifying COVID-19 cases and extracting patient reported symptoms from Reddit using natural language processingScientific Reports10.1038/s41598-023-39986-713:1Online publication date: 22-Aug-2023
https://doi.org/10.1038/s41598-023-39986-7

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents