Abstract
In recent years, some effective textual matching models are proposed for solving NLP-related tasks. However, these models have the following issues: they cannot extract semantic information at different levels from the words of text pairs; they cannot integrate the low-level information to fine-tune the high-level information. To address these, this paper proposes a novel deep learning neural network, namely deep hierarchical semantic model (DHSM), for text matching, which consists of multiple semantic processing layers, a pooling layer, and a prediction layer. Specifically, each semantic processing layer consists of three parts: encoding part, interaction part and fusion part; and it can well represent the semantic information and enable the information interaction at different levels through attention mechanism. Moreover, the pooling layer uses pooling method to extract key information of the text pairs, based on which the prediction layer determines the relationship between text pairs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, Q., Zhu, X., Ling, Z., Wei, S., Jiang, H., Inkpen, D.: Enhanced lstm for natural language inference. arXiv preprint arXiv:1609.06038 (2016)
Chen, Z., et al.: Information retrieval: a view from the Chinese IR community. Front. Comp. Sci. 15(1), 1–15 (2021)
Dolan, B., Brockett, C.: Automatically constructing a corpus of sentential paraphrases. In: Third International Workshop on Paraphrasing (IWP2005) (2005)
El-Alfy, E.S.M., Abdel-Aal, R.E., Al-Khatib, W.G., Alvi, F.: Boosting paraphrase detection through textual similarity metrics with abductive networks. Appl. Soft Comput. 26, 444–453 (2015)
Ferreira, R., Cavalcanti, G.D., Freitas, F., Lins, R.D., Simske, S.J., Riss, M.: Combining sentence similarities measures to identify paraphrases. Comput. Speech Lang. 47, 59–73 (2018)
He, B., Huang, J.X., Zhou, X.: Modeling term proximity for probabilistic information retrieval models. Inf. Sci. 181(14), 3017–3031 (2011)
Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for matching natural language sentences. Adv. Neural Inf. Process. Syst. 27 (2014)
Jimenez, S., Duenas, G., Baquero, J., Gelbukh, A.F., Bátiz, A.J.D., Mendizábal, A.: UNAL-NLP: combining soft cardinality features for semantic textual similarity, relatedness and entailment. In: SemEval@ COLING, pp. 732–742 (2014)
Khot, T., Sabharwal, A., Clark, P.: SciTail: a textual entailment dataset from science question answering. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Lai, A., Hockenmaier, J.: Illinois-LH: a denotational and distributional approach to semantics. In: SemEval@ COLING, pp. 329–334 (2014)
Li, P., Yu, H., Zhang, W., Xu, G., Sun, X.: SA-NLI: a supervised attention based framework for natural language inference. Neurocomputing 407, 72–82 (2020)
Liu, P., Qiu, X., Chen, X., Wu, S., Huang, X.J.: Multi-timescale long short-term memory neural network for modelling sentences and documents. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2326–2335 (2015)
Liu, P., Qiu, X., Huang, X.: Dynamic compositional neural networks over tree structure. arXiv preprint arXiv:1705.04153 (2017)
Liu, Y., Sun, C., Lin, L., Wang, X.: Learning natural language inference using bidirectional LSTM model and inner-attention. arXiv preprint arXiv:1605.09090 (2016)
Liu, Y., Tang, A., Sun, Z., Tang, W., Cai, F., Wang, C.: An integrated retrieval framework for similar questions: word-semantic embedded label clustering-LDA with question life cycle. Inf. Sci. 537, 227–245 (2020)
Marchesin, S., Purpura, A., Silvello, G.: Focal elements of neural information retrieval models. An outlook through a reproducibility study. Inf. Process. Manag. 57(6), 102109 (2020)
Marco, M., Luisa, B., Raffaella, B., Stefano, M., Roberto, Z., et al.: Semeval-2014 task 1: evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In: Proceedings of the SemEval, pp. 1–8 (2014)
Marelli, M., Menini, S., Baroni, M., Bentivogli, L., Bernardi, R., Zamparelli, R.: A sick cure for the evaluation of compositional distributional semantic models. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2014), pp. 216–223 (2014)
Parikh, A.P., Täckström, O., Das, D., Uszkoreit, J.: A decomposable attention model for natural language inference. arXiv preprint arXiv:1606.01933 (2016)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Rus, V., McCarthy, P.M., Lintean, M.C., McNamara, D.S., Graesser, A.C.: Paraphrase identification with lexico-syntactic graph subsumption. In: FLAIRS Conference, pp. 201–206 (2008)
Shen, G., Deng, Z.H., Huang, T., Chen, X.: Learning to compose over tree structures via POS tags for sentence representation. Expert Syst. Appl. 141, 112917 (2020)
Sun, C., Huang, L., Qiu, X.: Utilizing bert for aspect-based sentiment analysis via constructing auxiliary sentence. arXiv preprint arXiv:1903.09588 (2019)
Xu, S., Shijia, E., Xiang, Y.: Enhanced attentive convolutional neural networks for sentence pair modeling. Expert Syst. Appl. 151, 113384 (2020)
Zhao, H., Lu, Z., Poupart, P.: Self-adaptive hierarchical sentence model. In: Twenty-fourth International Joint Conference on Artificial Intelligence (2015)
Zhao, J., Zhu, T., Lan, M.: ECNU: one stone two birds: ensemble of heterogenous measures for semantic relatedness and textual entailment. In: SemEval@ COLING, pp. 271–277 (2014)
Zhou, G., Zhao, J., He, T., Wu, W.: An empirical study of topic-sensitive probabilistic model for expert finding in question answer communities. Knowl. Based Syst. 66, 136–145 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hu, C., Gongye, X., Zhang, X. (2023). Deep Hierarchical Semantic Model for Text Matching. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Lecture Notes in Computer Science, vol 13625. Springer, Cham. https://doi.org/10.1007/978-3-031-30111-7_34
Download citation
DOI: https://doi.org/10.1007/978-3-031-30111-7_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30110-0
Online ISBN: 978-3-031-30111-7
eBook Packages: Computer ScienceComputer Science (R0)