Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Enhanced semantic representation learning for implicit discourse relation classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Implicit discourse relation classification is one of the most challenging tasks in discourse parsing. Without connectives as linguistic clues, classifying discourse relations usually requires understanding text semantics at the word level, sentence level, and sentence span level. In this paper, we mainly proposed a graph-based model for relation classification. A semantic graph is firstly built to describe the syntactic dependencies and sentence interaction. Then, based on the learning principle of graph neural networks, a bidirectional gated recurrent unit (Bi-GRU) was introduced to work with graph attention network (GAT), which allows the expanded GAT to capture syntactic dependencies of long-distance nodes and selectively mine semantic features from multi-hop neighborhood nodes. In addition, we utilized the hierarchical self-organization ability of hyperbolic spaces to classify multi-level discourse relations, improving the accuracy of fine-grained discourse relation classification. Experimental results on Penn Discourse Treebank 2.0 (PDTB 2.0) demonstrated that our model could achieve improvements without any external knowledge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Andrews B, Hopper C (2010) The Ricci flow in Riemannian geometry: a complete proof of the differentiable 1/4-pinching sphere theorem. Springer, Berlin

    MATH  Google Scholar 

  2. Bai H, Zhao H (2018) Deep enhanced representation for implicit discourse relation recognition. In: Proceedings of the International Conference on Computational Linguistics, Santa Fe, New Mexico, USA, pp 571–583

  3. Bécigneul G, Ganea OE (2018) Riemannian adaptive optimization methods. arXiv:181000760

  4. Chen J, Zhang Q, Liu P, Qiu X, Huang XJ (2016) Implicit discourse relation detection via a deep architecture with gated relevance network. In: Proceedings of the annual meeting of the association for computational linguistics, pp 1726–1735

  5. Cohan A, Dernoncourt F, Kim DS, Bui T, Kim S, Chang W, Goharian N (2018) A discourse-aware attention model for abstractive summarization of long documents. In: Proceedings of the conference of the north american chapter of the association for computational linguistics, New Orleans, Louisiana, pp 615–621

  6. Dai Z, Huang R (2018) Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in a paragraph. In: Proceedings of the conference of the north american chapter of the association for computational linguistics. New Orleans, Louisiana, pp 141–151

  7. Dai Z, Huang R (2019) A regularization approach for incorporating event knowledge and coreference relations into neural discourse parsing. In: Proceedings of the conference on empirical methods in natural language processing and the international joint conference on natural language processing, pp 2967–2978

  8. Devlin J, Chang MW, Lee K, Toutanova K (2019) Pre-training Of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics. Minneapolis, Minnesota, pp 4171–4186

  9. Ganea OE, Bécigneul G, Hofmann T (2018) Hyperbolic neural networks. arXiv:180509112

  10. Girdhar R, Ramanan D (2017) Attentional pooling for action recognition. In: NIPS

  11. Guo F, He R, Jin D, Dang J, Wang L, Li X (2018) Implicit discourse relation recognition using neural tensor network with interactive attention and sparse learning. In: Proceedings of the international conference on computational linguistics, pp 547–558

  12. Hamann M (2018) On the tree-likeness of hyperbolic spaces. In: Mathematical proceedings of the cambridge philosophical society, vol 164. Cambridge University Press, Cambridge, pp 345–361

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  14. Jansen P, Surdeanu M, Clark P (2014) Discourse complements lexical semantics for non-factoid answer reranking. In: Proceedings of the annual meeting of the association for computational linguistics, pp 977–986

  15. Ji Y, Eisenstein J (2015) One vector is not enough: Entity-augmented distributed semantics for discourse relations. Transactions of the Association for Computational Linguistics 3:329– 344

    Article  Google Scholar 

  16. Kishimoto Y, Murawaki Y, Kurohashi S (2020) Adapting bert to implicit discourse relation classification with a focus on discourse connectives. In: Proceedings of The language resources and evaluation conference, pp 1152–1158

  17. Lan M, Wang J, Wu Y, Niu ZY, Wang H (2017) Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In: Proceedings of the conference on empirical methods in natural language processing, pp 1299–1308

  18. Lei W, Wang X, Liu M, Ilievski I, He X, Kan MY (2017) Swim: A simple word interaction model for implicit discourse relation recognition. In: IJCAI, pp 4026–4032

  19. Li JJ, Carpuat M, Nenkova A (2014) Assessing the discourse factors that influence the quality of machine translation. In: Proceedings of the annual meeting of the association for computational linguistics, pp 283–288

  20. Lin Z, Kan MY, Ng HT (2009) Recognizing implicit discourse relations in the penn discourse treebank. In: Proceedings of the conference on empirical methods in natural language processing, pp 343–351

  21. Liu X, Ou J, Song Y, Jiang X (2020a) On the importance of word and sentence representation learning in implicit discourse relation classification. In: Proceedings of the international joint conference on artificial intelligence, Yokohama, Japan

  22. Liu X, You X, Zhang X, Wu J, Lv P (2020b) Tensor graph convolutional networks for text classification. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 8409–8416

  23. Liu Y, Li S (2016) Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In: Proceedings of the conference on empirical methods in natural language processing. Austin, Texas, pp 1224–1233

  24. Liu Y, Li S, Zhang X, Sui Z (2016) Implicit discourse relation classification via multi-task neural networks. In: Proceedings of the AAAI conference on artificial intelligence, vol 30, pp 2750–2756

  25. López F, Heinzerling B, Strube M (2019) Fine-grained entity typing in hyperbolic space. arXiv:190602505

  26. Nickel M, Kiela D (2017) Poincaré embeddings for learning hierarchical representations. In: Advances in neural information processing systems, vol 30

  27. Park J, Cardie C (2012) Improving implicit discourse relation recognition through feature set optimization. In: Proceedings of the 13th annual meeting of the special interest group on discourse and dialogue. Seoul, South Korea, pp 108–112

  28. Pitler E, Nenkova A (2009) Using syntax to disambiguate explicit discourse connectives in text. In: Proceedings of the ACL-IJCNLP, pp 13–16

  29. Pitler E, Louis A, Nenkova A (2009) Automatic sense prediction for implicit discourse relations in text. In: ACL-IJCAI, pp 683–691

  30. Prasad R, Dinesh N, Lee A, Miltsakaki E, Robaldo L, Joshi AK, Webber BL (2008) The penn discourse treebank 2.0. In: LREC, pp 2961–1968

  31. Qin L, Zhang Z, Zhao H (2016a) Implicit discourse relation recognition with context-aware character-enhanced embeddings. In: Proceedings of the international conference on computational linguistics, pp 1914–1924

  32. Qin L, Zhang Z, Zhao H (2016b) A stacking gated neural architecture for implicit discourse relation classification. In: Proceedings of the conference on empirical methods in natural language processing, pp 2263–2270

  33. Qin L, Zhang Z, Zhao H, Hu Z, Xing E (2017) Adversarial connective-exploiting networks for implicit discourse relation classification. In: Proceedings of the annual meeting of the association for computational linguistics. Vancouver, Canada, pp 1006–1017

  34. Rönnqvist S, Schenk N, Chiarcos C (2017) A recurrent neural model with attention for the recognition of chinese implicit discourse relations. In: Proceedings of the annual meeting of the association for computational linguistics. Vancouver, Canada, pp 256–262

  35. Rutherford A, Xue N (2014) Discovering implicit discourse relations through brown cluster pair representation and coreference patterns. In: Proceedings of the conference of the european chapter of the association for computational linguistics, pp 645–654

  36. Rutherford A, Demberg V, Xue N (2017) A systematic study of neural discourse models for implicit discourse relation. In: Proceedings of the conference of the european chapter of the association for computational linguistics, pp 281–291

  37. Sarkar R (2011) Low distortion delaunay embedding of trees in hyperbolic plane. In: International symposium on graph drawing. Springer, pp 355–366

  38. Shi W, Demberg V (2019) Next sentence prediction helps implicit discourse relation classification within and across domains. In: Proceedings of the conference on empirical methods in natural language processing and the international joint conference on natural language processing, pp 5794–5800

  39. Varia S, Hidey C, Chakrabarty T (2019) Discourse relation prediction: Revisiting word pairs with convolutional networks. In: Proceedings of the annual SIGdial meeting on discourse and dialogue, pp 442–452

  40. Velickovic P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations

  41. Waswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A, Kaiser L, Polosukhin I (2017) Attention is all you need. In: NIPS

  42. Wilson B, Leimeister M (2018) Gradient descent in hyperbolic space. arXiv:180508207

  43. Xu Y, Hong Y, Ruan H, Yao J, Zhang M, Zhou G (2018) Using active learning to expand training data for implicit discourse relation recognition. In: Proceedings of the conference on empirical methods in natural language processing, pp 725–731

Download references

Acknowledgements

This work is supported by National Science and technology innovation 2030 major projects (2020AAA0109700), National Natural Science Foundation of China (62076167), and Beijing Municipal Education Commission-Beijing Natural Fund Joint Funding Project (KZ201910028039).

Author information

Authors and Affiliations

  1. Information Engineering College, Capital Normal University, 100048, Beijing, China

    Yuhao Ma & Jie Liu

  2. Department of computer teaching and research, University of Chinese Academy of Social Sciences, Beijing, China

    Jian Zhu

Authors
  1. Yuhao Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, Y., Zhu, J. & Liu, J. Enhanced semantic representation learning for implicit discourse relation classification. Appl Intell 52, 7700–7712 (2022). https://doi.org/10.1007/s10489-021-02785-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02785-6

Keywords

Search

Navigation