Abstract
Document-level relation extraction aims at extracting relations between entities in a document. In contrast to sentence-level correspondences, document-level relation extraction requires reasoning over multiple sentences to extract complex relational triples. Recent work has found that adding additional evidence extraction tasks and using the extracted evidence sentences to help predict can improve the performance of document-level relation extraction tasks, however, these approaches still face the problem of inadequate modeling of the interactions between entity pairs. In this paper, based on the review of human cognitive processes, we propose a hybrid network HIMAC applied to the entity pair feature matrix, in which the multi-head attention sub-module can fuse global entity-pair information on a specific inference path, while the convolution sub-module is able to obtain local information of adjacent entity pairs. Then we incorporate the contextual interaction information learned by the entity pairs into the relation prediction and evidence extraction tasks. Finally, the extracted evidence sentences are used to further correct the relation extraction results. We conduct extensive experiments on two document-level relation extraction benchmark datasets (DocRED/Re-DocRED), and the experimental results demonstrate that our method achieves state-of-the-art performance (62.84/75.89 F1). Experiments demonstrate the effectiveness of the proposed method.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The data that support the findings of this study are openly available in DocRED at https://github.com/thunlp/DocRED.
References
Zeng D, Liu K, Lai S, Zhou G, Zhao J. Relation classification via convolutional deep neural network. Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers; 2014 p. 2335-44.
Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, et al. Attention-based bidirectional long short-term memory networks for relation classification; 2016 p. 207-12.
Zhang Y, Qi P, Manning C. Graph convolution over pruned dependency trees improves relation extraction. Assoc Comput Linguist; 2018 p. 2205-15.
Wei Z, Su J, Wang Y, Tian Y, Chang Y. A novel cascade binary tagging framework for relational triple extraction. Assoc Comput Linguist; 2020 p. 1476-88.
Wang H, Qin K, Lu G, Luo G, Liu G. Direction-sensitive relation extraction using Bi-SDP attention model. Knowl-Based Syst. 2020;21(198): 105928.
Tang R, Chen Y, Huang R, Qin Y. Enhancing interaction representation for joint entity and relation extraction. Cogn Syst Res. 2023;1(82): 101153.
Cheng Q, Liu J, Qu X, Zhao J, Liang J, Wang Z, et al. HacRED: a large-scale relation extraction dataset toward hard cases in practical applications; 2021 p. 2819-31.
Wang H, Qin K, Zakari RY, Lu G, Yin J. Deep neural network-based relation extraction: an overview. Neural Comput Appl. 2022;34(6):4781–801.
Nayak T, Majumder N, Goyal P, Poria S. Deep neural approaches to relation triplets extraction: a comprehensive survey. Cogn Comput. 2021;13(5):1215–32.
Yao Y, Ye D, Li P, Han X, Lin Y, Liu Z, et al. DocRED: a large-scale document-level relation extraction dataset. ACLWeb. Florence, Italy: Assoc Comput Linguist; 2019. p. 764-77.
Nan G, Guo Z, Sekulić I, Lu W. Reasoning with latent structure refinement for document-level relation extraction. Assoc Comp Linguist; 2020 p. 1546-57.
Wang D, Hu W, Cao E. Global-to-local neural networks for document-level relation extraction. Assoc Comput Linguist; 2020 p. 3711-21.
Peng N, Poon H, Quirk C, Toutanova K, Yih W. Cross-sentence N-ary relation extraction with graph LSTMs. Trans Assoc Comput Linguist. 2017;5(5):101–15.
Christopoulou F, Miwa M. Ananiadou S. Connecting the dots: document-level neural relation extraction with edge-oriented graphs; 2019. p. 4925–36.
Guo Z, Zhang Y, Lu W. Attention guided graph convolutional networks for relation extraction. 2019 p. 241-51.
Wang H, Focke C, Sylvester R, Mishra NK, Wang WY. Fine-tune Bert for DocRED with two-step process. arXiv:1706.02677 [cs.CL]. 2019 Sep 26;
Tang H, Cao Y, Zhang Z, Cao J, Fang F, Wang S, et al. HIN: hierarchical inference network for document-level relation extraction. Advances in Knowledge Discovery and Data Mining. 2020;197-209.
Zhou W, Huang K, Ma T, Huang J. Document-level relation extraction with adaptive thresholding and localized context pooling. Proceedings of the AAAI Conference on Artificial Intelligence. 2021;35(16):14612–20.
Zhang N, Chen X, Xie X, Deng S, Tan C, Chen M, et al. Document-level relation extraction as semantic segmentation. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence; 2021.
Xie Y, Shen J, Li S, Mao Y, Han J. Eider: empowering document-level relation extraction with efficient evidence extraction and inference-stage fusion. Findings of the Association for Computational Linguistics: ACL. 2022;2022(1):257–68.
Tan Q, Li X, Bing L, Hwee Tou Ng, Aljunied SM. Revisiting DocRED - addressing the false negative problem in relation extraction. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022;8472-87.
Sahu SK, Christopoulou F, Miwa M, Ananiadou S. Inter-sentence relation extraction with document-level graph convolutional neural network. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019 Jul;4309-16.
Quirk C, Poon H. Distant supervision for relation extraction beyond the sentence boundary. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. 2017 Apr;1171-82.
Jia R, Wong C, Poon H. Document-level N-ary relation extraction with multiscale representation learning. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019 Jun;1:3693-704.
Zeng S, Wu Y, Chang B. SIRE: separate intra-and inter-sentential reasoning for document-level relation extraction. Findings of the Association for Computational Linguistics: ACL-IJCNLP. 2021;2021:524–34.
Zeng S, Xu R, Chang B, Li L. Double graph based reasoning for document-level relation extraction. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020 Nov;1630-40.
Xu W, Chen K, Zhao T. Document-level relation extraction with reconstruction. Proceedings of the AAAI Conference on Artificial Intelligence. 2021;35(16):14167–75.
Xu B, Wang Q, Lyu Y, Zhu Y, Mao Z. Entity structure within and throughout: modeling mention dependencies for document-level relation extraction. Proceedings of the AAAI Conference on Artificial Intelligence. 2021;35(16):14149–57.
Tan Q, He R, Bing L, Hwee Tou Ng. Document-level relation extraction with adaptive focal loss and knowledge distillation. Findings of the Association for Computational Linguistics: ACL 2022. 2022 May;1672-81.
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Adv Neural Inf Process Syst. 2017;30.
Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention - MICCAI. 2015;2015(9351):234–41.
Pan X, Ge C, Lu R, Song S, Chen G, Huang Z, et al. On the integration of self-attention and convolution. https://openaccess.thecvf.com/. 2022. p. 815-25.
Wolpert DH. Stacked generalization. Neural Netw. 1992;5(2):241–59.
Devlin J, Chang MW, Lee K, Google K, Language A. BERT: pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT. 2019;2019:4171–86.
Loshchilov I, Hutter F. Decoupled weight decay regularization. International Conference on Learning Representations.; 2019.
Goyal P, Dollár P, Girshick R, Noordhuis P, Wesolowski L, Kyrola A, et al. Accurate, large minibatch SGD: training ImageNet in 1 Hour. arXiv:1706.02677 [cs]. 2018 Apr 30;
Li B, Ye W, Sheng Z, Xie R, Xi X, Zhang S. Graph enhanced dual attention network for document-level relation extraction. Proceedings of the 28th International Conference on Computational Linguistics; 2020 Dec p. 1551-60.
Xu W, Chen K, Zhao T. Discriminative reasoning for document-level relation extraction. Findings of the Association for Computational Linguistics: ACL-IJCNLP. 2021;2021:1653–63.
Huang K, Qi P, Wang G, Ma T, Huang J. Entity and evidence guided document-level relation extraction. Rogers A, Calixto I, Vulić I, Saphra N, Kassner N, Camburu OM, et al., editors. Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021). 2021 Aug 1;307-15.
Ye D, Lin Y, Du J, Liu Z, Li P, Sun M, et al. Coreferential reasoning learning for language representation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020 Nov;7170-86.
Jiang F, Niu J, Mo S, Fan S. Key mention pairs guided docu-ment-level relation extraction. Calzolari N, Huang CR, Kim H, Pustejovsky J, Wanner L, Choi KS, et al., editors. Proceedings of the 29th International Conference on Computational Linguistics. 2022 Oct;1904-14.
Funding
This work was supported by the National Natural Science Foundation of China (No. U19A2059) and the 2022 Chengdu Textile College Scientific Research Foundation (No. X22032161).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Ethical Approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Conflict of Interest
Guiduo Duan has received research grants from the National Natural Science Foundation of China (No. U19A2059). Tianxi Huang has received research grants from the 2022 Chengdu Textile College Scientific Research Foundation(No. X22032 161). Feiyu Zhang and Ruiming Hu declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, F., Hu, R., Duan, G. et al. Enhancing Document-Level Relation Extraction with Attention-Convolutional Hybrid Networks and Evidence Extraction. Cogn Comput 16, 1113–1124 (2024). https://doi.org/10.1007/s12559-024-10269-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12559-024-10269-1