Cited By
View all- Gou ZLi Y(2023)Integrating BERT Embeddings and BiLSTM for Emotion Analysis of DialogueComputational Intelligence and Neuroscience10.1155/2023/66184522023Online publication date: 1-Jan-2023
BMCSA: : Multi-feature spatial convolution semantic matching model based on BERT
Pages 4083 - 4093
Abstract
This paper proposes a multi-feature spatial convolutional semantic matching model (BMCSA) based on BERT by enriching different feature spatial information of semantic features. BMCSA employs the BERT model to extract the semantic features of the text, then uses the two-dimensional convolutional network to extract different feature spatial information, and finally combines the Attention mechanism to capture the global feature spatial information. We use two different semantic matching data sets and a text inference data set to verify the effectiveness of the proposed model. Experimental results prove that BMCSA is better than the baseline model.
References
[1]
Zhang X., Sun X. and Wang H., Duplicate question identification by integrating framenet with neural networks, 32(1), 2018.
[2]
Chen J., Chen Q., Liu X., Yang H., Lu D. and Tang B., The bq corpus:A large-scale domainspecific chinese corpus for sentence semantic equivalence identification. (2018), pp. 4946–4951.
[3]
Li Y., McLean D., Bandar Z.A., James O’shea D. and Crockett K., Sentence similarity based on semantic nets and corpus statistics, IEEE Transactions on Knowledge and Data Engineering 18(8) (2006), 1138–1150.
[4]
Samuel Bowman R., Angeli G., Potts C. and Christopher Manning D., A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326, 2015.
[5]
Cai L.-Q., Wei M., Zhou S.-T. and Yan X., Intelligent question answering in restricted domains using deep learning and question pair matching, Ieee Access 8 (2020), 32922–32934.
[6]
Devlin J., Chang M.-W., Lee K. and Toutanova K., Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
[7]
Kalchbrenner N., Grefenstette E. and Blunsom P., A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188, 2014.
[8]
Socher R., Huang E., Pennin J., Manning C.D. and Ng A., Dynamic pooling and unfolding recursive autoencoders for paraphrase detection, Advances in Neural Information Processing Systems 24, 2011.
[9]
Kim Y., Convolutional neural networks for sentence classification. Eprint Arxiv, 2014.
[10]
Elman J.L., Finding structure in time, Cognitive Science 14(2) (1990), 179–211.
[11]
Chen Q., Hu Q., Huang J.X. and He L., Ca-rnn: using context-aligned recurrent neural networks for modeling sentence similarity, 32(1), 2018.
[12]
Ankur Parikh P., Täckström O., Das D. and Uszkoreit J., A decomposable attention model for natural language inference. arXiv preprint arXiv:1606.01933, 2016.
[13]
Lai S., Xu L., Liu K. and Zhao J., Recurrent convolutional neural networks for text classification. 2015.
[14]
Chopra S., Hadsell R. and LeCun Y., Learning a similarity metric discriminatively, with application to face verification, 1 (2005), 539–546.
[15]
Kim S., Kang I. and Kwak N., Semantic sentence matching with densely-connected recurrent and coattentive information, 33(01) (2019), 6586–6593.
[16]
Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser Ł. and Polosukhin I., Attention is all you need. (2017), pp. 5998–6008.
[17]
Lan Z., Chen M., Goodman S., Gimpel K., Sharma P. and Soricut R., Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942, 2019.
[18]
Liu Y., Ott M., Goyal N., Du J., Joshi M., Chen D., Levy O., Lewis M., Zettlemoyer L. and Stoyanov V., Roberta:A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.
[19]
Zhang P., Huang X., Wang Y., Jiang C., He S. and Wang H., Semantic similarity computing model based on multi model fine-grained nonlinear fusion, IEEE Access 9 (2021), 8433–8443.
[20]
Yu R., Lu W., Li Y., Yu J., Zhang G. and Zhang X., Sentence semantic matching with hierarchical cnn based on dimension-augmented representation. (2021), pp. 1–8.
[21]
Shen Y., He X., Gao J., Deng L. and Mesnil G., A latent semantic model with convolutionalpooling structure for information retrieval. (2014), pp. 101–110.
[22]
Huang P.-S., He X., Gao J., Deng L., Acero A. and Heck L., Learning deep structured semantic models for web search using clickthrough data. (2013), pp. 2333–2338.
[23]
Hu B., Lu Z., Li H. and Chen Q., Convolutional neural network architectures for matching natural language sentences. (2014), pp. 2042–2050.
[24]
Yin W. and Schütze H., Convolutional neural network for paraphrase identification. (2015), pp. 901–911.
[25]
Yin W., Schütze H., Xiang B. and Zhou B., Abcnn: Attention-based convolutional neural network for modeling sentence pairs, Transactions of the Association for Computational Linguistics 4 (2016), 259–272.
[26]
Gu J.-C., Ling Z.-H. and Liu Q., Utterance-toutterance interactive matching network for multi-turn response selection in retrieval-based chatbots, IEEE/ACM Transactions on Audio, Speech, and Language Processing 28 (2019), 369–379.
[27]
Yang D., Ke X., Yu Q. and Yang B., Enhanced lstm: a text matching aggregation model, 9 (2020), 659–663.
[28]
Zhao S., Huang Y., Su C., Li Y. and Wang F., Interactive attention networks for semantic text matching. (2020), pp. 861–870.
[29]
Liu M., Zhang Y., Xu J. and Chen Y., Deep bi-directional interaction network for sentence matching, Applied Intelligence (2021), pp. 1–25.
[30]
Zhang K., Lv G., Wu L., Chen E., Liu Q. and Wang M., Ladra-net: Locally aware dynamic reread attention net for sentence semantic matching, IEEE Transactions on Neural Networks and Learning Systems (2021), pp. 1–14.
[31]
Tan M., dos Santos C., Xiang B. and Zhou B., Lstm-based deep learning models for non-factoid answer selection. arXiv preprint arXiv:1511.04108, 2015.
[32]
Chen J., Zhu X., Sang J. and Gong L., Sentence pair similarity modeling based on weighted interaction of multisemantic embedding matrix. (2020), pp. 1118–1123.
[33]
Neculoiu P., Versteegh M. and Rotaru M., Learning text similarity with siamese recurrent networks. (2016), pp. 148–157.
[34]
Wang Z., Hamza W. and Florian R., Bilateral multiperspective matching for natural language sentences. arXiv preprint arXiv:1702.03814, 2017.
[35]
Huang Z., Xu W. and Yu K., Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991, 2015.
[36]
Yuan Z. and Jun S., Siamese network cooperating with multi-head attention for semantic sentence matching. (2020), pp. 215–218.
[37]
Cho K., Van Merriënboer B., Gulcehre C., Bahdanau D., Bougares F., Schwenk H. and Bengio Y., Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXivpreprint arXiv:1406.1078, 2014.
[38]
Reimers N. and Gurevych I., Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019.
[39]
Van Nguyena K., Duy Nguyena N., Nguyen-Thuan Doa P., Gia-Tuan Nguyena A. and Luu-Thuy Nguyena N., Vireader: A wikipedia-based vietnamese reading comprehension system using transfer learning, Journal of Intelligent & Fuzzy Systems 1 (2021), 1–5.
[40]
Lan Z., Chen M., Goodman S., Gimpel K., Sharma P. and Soricut R., Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942, 2019.
[41]
Mikolov T., Chen K., Corrado G. and Dean J., Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.
[42]
Pennington J., Socher R. and Christopher Manning D., Manning, Glove: Global vectors for word representation. (2014), pp. 1532–1543.
[43]
A. financial, ant financial artificial competition.
[44]
Yang Y., Zhang Y., Tar C. and Baldridge J., Paws-x: A cross-lingual adversarial dataset for paraphrase identification. arXiv preprint arXiv:1908.11828, 2019.
[45]
Hu H., Richardson K., Xu L., Li L., Kuebler S. and Moss L., Ocnli: Original chinese natural language inference. In Findings of EMNLP, 2020.
[47]
Papineni K., Roukos S., Ward T. and Zhu W.-J., Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, (2002), pp. 311–318.
Index Terms
- BMCSA: Multi-feature spatial convolution semantic matching model based on BERT
Index terms have been assigned to the content through auto-classification.
Recommendations
Semantic Matching Algorithm for Legal Issues Based on BERT and Graph Convolution with Multi-granularity Features
ICCPR '23: Proceedings of the 2023 12th International Conference on Computing and Pattern RecognitionIn recent years, pre-training models represented by BERT have shown amazing text semantic representation capabilities supported by large-scale corpus, and now have become one of the mainstream solutions for semantic matching tasks in legal Q&A systems. ...
A hybrid approach of Weighted Fine-Tuned BERT extraction with deep Siamese Bi – LSTM model for semantic text similarity identification
AbstractThe conventional semantic text-similarity methods requires high amount of trained labeled data and also human interventions. Generally, it neglects the contextual-information and word-orders information resulted in data sparseness problem and ...
Biotic Stress Management in Soil-Less Agriculture Systems: A Deep Learning Approach for Identification of Leaf Miner Pest Infestation
AbstractLeaf miner pests pose a serious threat to the productivity, profitability, and sustainability of soil-less tomato cultivation systems. Early and accurate identification of leaf miner infestation is crucial for timely pest control measures. This ...
Comments
Information & Contributors
Information
Published In
© 2022 – IOS Press. All rights reserved.
Publisher
IOS Press
Netherlands
Publication History
Published: 01 January 2022
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025
Other Metrics
Citations
Cited By
View all- Gou ZLi Y(2023)Integrating BERT Embeddings and BiLSTM for Emotion Analysis of DialogueComputational Intelligence and Neuroscience10.1155/2023/66184522023Online publication date: 1-Jan-2023