BERT-Based Models with Attention Mechanism and Lambda Layer for Biomedical Named Entity Recognition

Biomedical named entity recognition (NER) is a crucial subtask in the field of information extraction within natural language processing (NLP). Its primary objective is to identify and classify entities in biomedical text, playing a pivotal role in applications such as medical information retrieval and biomedical knowledge discovery. In this paper, we propose several enhanced versions of BERT-BiLSTM-CRF and BERT-IDCNN-CRF by incorporating an attention mechanism or lambda layer to improve entity recognition accuracy. Specifically, we utilize the attention mechanism to enable the model to learn interrelationships among all words in the input sequence. Additionally, we employ the lambda layer to enhance the model's capacity for capturing semantic relationships between words and considering word order. This integration results in superior accuracy in entity recognition. We evaluate our proposed methods using the i2b2 2010 dataset and six additional biomedical datasets from the Biomedical Language Understanding and Reasoning Benchmark (BLURB), including JNLPBA, BC2GM, BC5CDR, AnatEM, BioNLP-CG, and NCBI-disease. Experimental results demonstrate that our proposed methods achieve higher accuracy than the original methods, indicating superior capabilities in medical knowledge extraction for our models.