Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Phrase-level attention network for few-shot inverse relation classification in knowledge graph

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Relation classification aims to recognize semantic relation between two given entities mentioned in the given text. Existing models have performed well on the inverse relation classification with large-scale datasets, but their performance drops significantly for few-shot learning. In this paper, we propose a Phrase-level Attention Network, function words adaptively enhanced attention framework (FAEA+), to attend class-related function words by the designed hybrid attention for few-shot inverse relation classification in Knowledge Graph. Then, an instance-aware prototype network is present to adaptively capture relation information associated with query instances and eliminate intra-class redundancy due to function words introduced. We theoretically prove that the introduction of function words will increase intra-class differences, and the designed instance-aware prototype network is competent for reducing redundancy. Experimental results show that FAEA+ significantly improved over strong baselines on two few-shot relation classification datasets. Moreover, our model has a distinct advantage in solving inverse relations, which outperforms state-of-the-art results by 16.82% under a 1-shot setting in FewRel1.0.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The FewRel 1.0 and FewRel 2.0 datesets we used are available in https://thunlp.github.io/fewrel.html.

References

  1. Dong, H., Wang, W., Coenen, F., Huang, K.: Knowledge base enrichment by relation learning from social tagging data. Inf. Sci. 526, 203–220 (2020). https://doi.org/10.1016/j.ins.2020.04.002

    Article  MathSciNet  Google Scholar 

  2. Lee, Y.: Improving end-to-end task-oriented dialog system with a simple auxiliary task. In: Findings of the Association for Computational Linguistics: EMNLP, pp. 1296–1303 (2021). https://doi.org/10.18653/v1/2021.findings-emnlp.112

  3. Bhattacharya, I., Getoor, L.: Collective entity resolution in relational data. ACM Transactions on Knowledge Discovery from Data 1(1), 5 (2007)

    Article  Google Scholar 

  4. Sridhar, D., Fakhraei, S., Getoor, L.: A probabilistic approach for collective similarity-based drug-drug interaction prediction. Bioinform. 32(20), 3175–3182 (2016)

    Article  Google Scholar 

  5. Han, X., Zhu, H., Yu, P., Wang, Z., Yao, Y., Liu, Z., Sun, M.: Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4803–4809 (2018). https://doi.org/10.18653/v1/d18-1514

  6. Liu, Y., Hu, J., Wan, X., Chang, T.: A simple yet effective relation information guided approach for few-shot relation extraction. In: Findings of the Association for Computational Linguistics, pp. 757–763 (2022). https://doi.org/10.18653/v1/2022.findings-acl.62

  7. Gao, T., Han, X., Liu, Z., Sun, M.: Hybrid attention-based prototypical networks for noisy few-shot relation classification. In: The Thirty-Third AAAI Conference on Artificial Intelligence, pp. 6407–6414 (2019). https://doi.org/10.1609/aaai.v33i01.33016407

  8. Snell, J., Swersky, K., Zemel, R.S.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, pp. 4077–4087 (2017)

  9. Gao, T., Han, X., Zhu, H., Liu, Z., Li, P., Sun, M., Zhou, J.: Fewrel 2.0: Towards more challenging few-shot relation classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 6249–6254 (2019). https://doi.org/10.18653/v1/D19-1649

  10. Sun, S., Sun, Q., Zhou, K., Lv, T.: Hierarchical attention prototypical networks for few-shot text classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 476–485 (2019). https://doi.org/10.18653/v1/D19-1045

  11. Yang, S., Zhang, Y., Niu, G., Zhao, Q., Pu, S.: Entity concept-enhanced few-shot relation extraction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 987–991 (2021). https://doi.org/10.18653/v1/2021.acl-short.124

  12. Bao, Y., Wu, M., Chang, S., Barzilay, R.: Few-shot text classification with distributional signatures. In: 8th International Conference on Learning Representations (2020)

  13. Dou, C., Wu, S., Zhang, X., Feng, Z., Wang, K.: Function-words adaptively enhanced attention networks for few-shot inverse relation classification. In: Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, pp. 2937–2943 (2022). https://doi.org/10.24963/ijcai.2022/407

  14. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186 (2019). https://doi.org/10.18653/v1/n19-1423

  15. Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning, pp. 1126–1135 (2017)

  16. Bo, D., Wang, X., Shi, C., Shen, H.: Beyond low-frequency information in graph convolutional networks. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, pp. 3950–3957 (2021)

  17. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: 7th International Conference on Learning Representations (2019)

  18. Rasooli, M.S., Tetreault, J.R.: Yara parser: A fast and accurate dependency parser. Computer Science 194(1), 93–106 (2015)

    Google Scholar 

  19. Peng, H., Gao, T., Han, X., Lin, Y., Li, P., Liu, Z., Sun, M., Zhou, J.: Learning from context or names? an empirical study on neural relation extraction. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 3661–3672 (2020). https://doi.org/10.18653/v1/2020.emnlp-main.298

  20. Ye, Z., Ling, Z.: Multi-level matching and aggregation network for few-shot relation classification. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 2872–2881 (2019). https://doi.org/10.18653/v1/p19-1277

  21. Satorras, V.G., Estrach, J.B.: Few-shot learning with graph neural networks. In: 6th International Conference on Learning Representations (2018)

  22. Qu, M., Gao, T., Xhonneux, L.A.C., Tang, J.: Few-shot relation extraction via bayesian meta-learning on relation graphs. In: Proceedings of the 37th International Conference on Machine Learning, pp. 7867–7876 (2020)

  23. Wen, W., Liu, Y., Ouyang, C., Lin, Q., Chung, T.L.: Enhanced prototypical network for few-shot relation extraction. Inf. Process. Manag. 58(4), 102596 (2021). https://doi.org/10.1016/j.ipm.2021.102596

    Article  Google Scholar 

  24. Wang, Y., Verspoor, K., Baldwin, T.: Learning from unlabelled data for clinical semantic textual similarity. In: Proceedings of the 3rd Clinical Natural Language Processing Workshop, ClinicalNLP@EMNLP 2020, pp. 227–233 (2020). https://doi.org/10.18653/v1/2020.clinicalnlp-1.25

  25. Han, J., Cheng, B., Lu, W.: Exploring task difficulty for few-shot relation extraction. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 2605–2616 (2021). https://doi.org/10.18653/v1/2021.emnlp-main.204

  26. Liu, Y., Hu, J., Wan, X., Chang, T.: Learn from relation information: Towards prototype representation rectification for few-shot relation extraction. In: Findings of the Association for Computational Linguistics: NAACL, pp. 1822–1831 (2022). https://doi.org/10.18653/v1/2022.findings-naacl.139

  27. Soares, L.B., FitzGerald, N., Ling, J., Kwiatkowski, T.: Matching the blanks: Distributional similarity for relation learning. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 2895–2905 (2019). https://doi.org/10.18653/v1/p19-1279

  28. Xie, Y., Xu, H., Li, J., Yang, C., Gao, K.: Heterogeneous graph neural networks for noisy few-shot relation classification. Knowl. Based Syst. 194, 105548 (2020). https://doi.org/10.1016/j.knosys.2020.105548

    Article  Google Scholar 

  29. Li, Y., Ma, Z., Gao, L., Wu, Y., Xie, F., Ren, X.: Enhance prototypical networks with hybrid attention and confusing loss function for few-shot relation classification. Neurocomputing 493, 362–372 (2022). https://doi.org/10.1016/j.neucom.2022.04.067

    Article  Google Scholar 

  30. Gregory, K., Richard, Z., Ruslan, S.: Siamese neural networks for one-shot image recognition. In: Proceedings of the 3rd International Conference on Learning Representation (2015)

  31. Li, Z., Ouyang, F., Zhou, C., He, Y., Shen, L.: Few-shot relation classification research based on prototypical network and causal intervention. IEEE Access 10, 36995–37002 (2022). https://doi.org/10.1109/ACCESS.2022.3164688

    Article  Google Scholar 

  32. Yang, K., Zheng, N., Dai, X., He, L., Huang, S., Chen, J.: Enhance prototypical network with text descriptions for few-shot relation classification. In: The 29th ACM International Conference on Information and Knowledge Management, pp. 2273–2276 (2020). https://doi.org/10.1145/3340531.3412153

  33. Zhang, J., Zhu, J., Yang, Y., Shi, W., Zhang, C., Wang, H.: Knowledge-enhanced domain adaptation in few-shot relation classification. In: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2183–2191 (2021). https://doi.org/10.1145/3447548.3467438

  34. Tang, Y., Li, Z., Cao, C., Fang, F., Cao, Y., Liu, Y., Fu, J.: Knowledge-based diverse feature transformation for few-shot relation classification. In: Knowledge Science, Engineering and Management - 14th International Conference. Lecture Notes in Computer Science, vol. 12815, pp. 101–114 (2021). https://doi.org/10.1007/978-3-030-82136-4_9

  35. Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: 5th International Conference on Learning Representations (2017)

  36. Dong, B., Yao, Y., Xie, R., Gao, T., Han, X., Liu, Z., Lin, F., Lin, L., Sun, M.: Meta-information guided meta-learning for few-shot relation classification. In: Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, pp. 1594–1605 (2020). https://doi.org/10.18653/v1/2020.coling-main.140

  37. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Networks 20(1), 61–80 (2009). https://doi.org/10.1109/TNN.2008.2005605

    Article  Google Scholar 

  38. Yang, L., Li, L., Zhang, Z., Zhou, X., Zhou, E., Liu, Y.: DPGN: distribution propagation graph network for few-shot learning. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13387–13396 (2020). https://doi.org/10.1109/CVPR42600.2020.01340

  39. Guo, Z., Zhang, Y., Lu, W.: Attention guided graph convolutional networks for relation extraction. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, pp. 241–251 (2019). https://doi.org/10.18653/v1/p19-1024

  40. Sun, K., Zhang, R., Mao, Y., Mensah, S., Liu, X.: Relation extraction with convolutional network over learnable syntax-transport graph. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, pp. 8928–8935 (2020)

  41. Tian, Y., Chen, G., Song, Y., Wan, X.: Dependency-driven relation extraction with attentive graph convolutional networks. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 4458–4471 (2021). https://doi.org/10.18653/v1/2021.acl-long.344

  42. Wang, Y., Zhang, H.: Introducing graph neural networks for few-shot relation prediction in knowledge graph completion task. In: Knowledge Science, Engineering and Management - 14th International Conference. Lecture Notes in Computer Science, vol. 12815, pp. 294–306 (2021). https://doi.org/10.1007/978-3-030-82136-4_24

  43. Li, Z., Sun, Y., Zhu, J., Tang, S., Zhang, C., Ma, H.: Improve relation extraction with dual attention-guided graph convolutional networks. Neural Comput. Appl. 33(6), 1773–1784 (2021). https://doi.org/10.1007/s00521-020-05087-z

    Article  Google Scholar 

  44. Zhao, H., Xie, J., Wang, H.: Graph convolutional network based on multi-head pooling for short text classification. IEEE Access 10, 11947–11956 (2022). https://doi.org/10.1109/ACCESS.2022.3146303

    Article  Google Scholar 

  45. Yang, S., Cai, B., Cai, T., Song, X., Jiang, J., Li, B., Li, J.: Robust cross-network node classification via constrained graph mutual information. Knowl. Based Syst. 257, 109852 (2022). https://doi.org/10.1016/j.knosys.2022.109852

    Article  Google Scholar 

  46. Fang, U., Li, J., Akhtar, N., Li, M., Jia, Y.: Gomic: Multi-view image clustering via self-supervised contrastive heterogeneous graph co-learning. World Wide Web, 1–17 (2022)

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (NSFC) (61972455).

Author information

Authors and Affiliations

  1. Tianjin University, Tianji, China

    Shaojuan Wu, Dazhuang Wang, Jitong Li, Xiaowang Zhang & Zhiyong Feng

  2. CNPC Economics and Technology Research Institute, Beijing, China

    Chunliu Dou

  3. Griffith University, Brisbane, Australia

    Kewen Wang

  4. Debre Berhan University, Debre Berhan, Ethiopia

    Sofonias Yitagesu

Authors
  1. Shaojuan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunliu Dou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dazhuang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jitong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaowang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhiyong Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kewen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sofonias Yitagesu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Shaojuan Wu: Conceptualization, Methodology, Wrote the abstract, introduction and theoretical Analysis. Chunliu Dou: Data curation, Methodology, Validation and Wrote original draft preparation. Dazhuang Wang: Software, Validation, Wrote the experiment and related work. Jitong Li: Software, Validation and prepared some figures. Xiaowang Zhang: Supervision, Project administration. Zhiyong Feng: Supervision, Funding acquisition. Kewen Wang: Analysis logic, Reviewed the manuscript. Sofonias Yitagesu: Reviewed the manuscript.

Corresponding author

Correspondence to Xiaowang Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Knowledge-Graph-Enabled Methods and Applications for the Future Web Guest Editors: Xin Wang, Jeff Pan, Qingpeng Zhang, Yuan-Fang Li.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, S., Dou, C., Wang, D. et al. Phrase-level attention network for few-shot inverse relation classification in knowledge graph. World Wide Web 26, 3001–3026 (2023). https://doi.org/10.1007/s11280-023-01142-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-023-01142-6

Keywords

Search

Navigation