Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Multivariate graph neural networks on enhancing syntactic and semantic for aspect-based sentiment analysis

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Aspect-based sentiment analysis (ABSA) aims to predict sentiment orientations towards textual aspects by extracting insights from user comments. While pretrained large language models (LLMs) demonstrate proficiency in sentiment analysis, incorporating syntactic and semantic features into ABSA remains a challenge. Additionally, employing LLMs for sentiment analysis often requires significant computational resources, rendering them impractical for use by individuals or small-scale entities. To address this, we propose the semiotic signal integration network (SSIN), which effectively combines syntactic and semantic features. The core syncretic information network leverages isomorphism and syntax to enhance knowledge acquisition. The semantically guided syntactic attention module further enables integrated semiotic representations via sophisticated attention mechanisms. Experiments on the publicly available SemEval dataset show that SSIN performs better than existing state-of-the-art ABSA baselines and LLMs such as Llama and Alpaca with high accuracy and macro-F1 scores. Moreover, our model demonstrates exceptional interpretability and the ability to discern both positive and negative sentiments, which is vitally important for real-world applications such as social media monitoring, health care, and customer service. Code is available at https://github.com/AmbitYuki/SSIN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

Code is available at https://github.com/AmbitYuki/SSIN. The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. https://github.com/HIT-SCIR

References

  1. Alqaryouti O et al (2024) Aspect-based sentiment analysis using smart government review data. Appl Comput Inform 20(1/2):142–161

    Article  Google Scholar 

  2. Bensoltane R, Zaki T (2023) Aspect-based sentiment analysis: an overview in the use of arabic language. Artif Intell Rev 56(3):2325–2363

    Article  Google Scholar 

  3. Gu T et al (2023) Integrating external knowledge into aspect-based sentiment analysis using graph neural network. Knowl-Based Syst 259(110):025

    Google Scholar 

  4. Al-Smadi M et al (2023) Gated recurrent unit with multilingual universal sentence encoder for arabic aspect-based sentiment analysis. Knowl-Based Syst 261:107540

    Article  Google Scholar 

  5. Xiao L et al (2023) Cross-modal fine-grained alignment and fusion network for multimodal aspect-based sentiment analysis. Inform Process Manag 60(6):103508

    Article  Google Scholar 

  6. Lin T, Joe I (2023) An adaptive masked attention mechanism to act on the local text in a global context for aspect-based sentiment analysis. IEEE Access

  7. Huang B, et al.(2019) Syntax-aware aspect level sentiment classification with graph attention networks. In: EMNLP. pp 5469–5477

  8. Sun K, Zhang R, Mensah S, et al. (2019) Aspect-level sentiment analysis via convolution over dependency tree. In: EMNLP-IJCNLP. pp 5679–5688

  9. Zhang C, et al. (2019) Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: EMNLP. pp 4568–4578

  10. Tang H, Ji D, Li C, et al. (2020) Dependency graph enhanced dual-transformer structure for aspect-based sentiment classification. In: ACL. pp 6578–6588

  11. Wang K, Shen W, Yang Y, et al. (2020) Relational graph attention network for aspect-based sentiment analysis. In: ACL. pp 3229–3238

  12. Chen C, et al. (2020) Inducing target-specific latent structures for aspect sentiment classification. In: EMNLP. pp 5596–5607

  13. Li R, Zhang Y, Teng Z, et al. (2021) Dual graph convolutional networks for aspect-based sentiment analysis. In: ACL. pp 1–12

  14. Tan X, et al. (2023) Self-criticism: aligning large language models with their understanding of helpfulness, honesty, and harmlessness. In: Proceedings of the 2023 conference on empirical methods in natural language processing: industry track

  15. Ishaq A, Asghar S, Gillani SA (2020) Aspect-based sentiment analysis using a hybridized approach based on cnn and ga. IEEE Access 8:135499–135512

    Article  Google Scholar 

  16. Mohammadi A, Shaverizade A (2021) Ensemble deep learning for aspect-based sentiment analysis. Int J Nonlin Anal Appl 12(Special Issue):29–38

    Google Scholar 

  17. Xu G et al (2022) Aspect-level sentiment classification based on attention-bilstm model and transfer learning. Knowl-Based Syst 245:108586

    Article  Google Scholar 

  18. Chandio BA et al (2022) Attention-based ru-bilstm sentiment analysis model for roman urdu. Appl Sci 12(7):3641

    Article  MathSciNet  Google Scholar 

  19. Aydin CR, Güngör T (2020) Combination of recursive and recurrent neural networks for aspect-based sentiment analysis using inter-aspect relations. IEEE Access 8:77820–77832

    Article  Google Scholar 

  20. Shafqat W, Byun YC (2021) Incorporating similarity measures to optimize graph convolutional neural networks for product recommendation. Appl Sci 11(4):1366

    Article  Google Scholar 

  21. Li N, Chow CY, Zhang JD (2020) Seml: a semi-supervised multi-task learning framework for aspect-based sentiment analysis. IEEE Access 8:189287–189297

    Article  Google Scholar 

  22. Phan MH, Ogunbona PO (2020) Modelling context and syntactical features for aspect-based sentiment analysis. In: Proceedings of the 58th annual meeting of the association for computational linguistics

  23. Jayanto R, Kusumaningrum R, Wibowo A (2022) Aspect-based sentiment analysis for hotel reviews using an improved model of long short-term memory. Int J Adv Intell Inform 8(3)

  24. Zhang B et al (2020) Knowledge guided capsule attention network for aspect-based sentiment analysis. IEEE/ACM Trans Audio Speech Lang Process 28:2538–2551

    Article  Google Scholar 

  25. Chen Y, Zhuang T, Guo K (2021) Memory network with hierarchical multi-head attention for aspect-based sentiment analysis. Appl Intell 51:4287–4304

    Article  Google Scholar 

  26. Liang Y et al (2021) A dependency syntactic knowledge augmented interactive architecture for end-to-end aspect-based sentiment analysis. Neurocomputing 454:291–302

    Article  Google Scholar 

  27. Wu L et al (2023) Graph neural networks for natural language processing: a survey. Found Trends Mach Learn 16(2):119–328

    Article  Google Scholar 

  28. Liu B, Wu L (2022) Graph neural networks in natural language processing. In: Graph neural networks: foundations, frontiers, and applications. p 463–481

  29. Zhou J, et al. (2020) Hierarchy-aware global model for hierarchical text classification. In: Proceedings of the 58th annual meeting of the association for computational linguistics

  30. Pan S et al (2024) Unifying large language models and knowledge graphs: a roadmap. IEEE Trans Knowl Data Eng

  31. Qiu X et al (2024) An attentive copula-based spatio-temporal graph model for multivariate time-series forecasting. Appl Soft Comput 154(111):324

    Google Scholar 

  32. Li X et al (2023) A survey of graph neural network based recommendation in social networks. Neurocomputing 549(126):441

    Google Scholar 

  33. Kapanipathi P, et al. (2020) Infusing knowledge into the textual entailment task using graph convolutional networks. In: Proceedings of the AAAI conference on artificial intelligence

  34. Fei H et al (2021) Enriching contextualized language model from knowledge graph for biomedical information extraction. Brief Bioinform 22(3):bbaa110

    Article  Google Scholar 

  35. Zhou J et al (2020) Sk-gcn: Modeling syntax and knowledge via graph convolutional network for aspect-level sentiment classification. Knowl-Based Syst 205:106292

    Article  Google Scholar 

  36. Zhang M et al (2021) Dependency-based syntax-aware word representations. Artif Intell 292:103427

    Article  MathSciNet  Google Scholar 

  37. Yang K, Deng J (2020) Strongly incremental constituency parsing with graph neural networks. Adv Neural Inf Process Syst 33:21687–21698

    Google Scholar 

  38. Wang F et al (2022) Klgcn: knowledge graph-aware light graph convolutional network for recommender systems. Expert Syst Appl 195(116):513

    Google Scholar 

  39. Zhao M et al (2023) Multi-task learning with graph attention networks for multi-domain task-oriented dialogue systems. Knowl-Based Syst 259:110069

    Article  Google Scholar 

  40. Gao W, Huang H (2021) A gating context-aware text classification model with bert and graph convolutional networks. J Intell Fuzz Sys 40(3):4331–4343

    Article  Google Scholar 

  41. Qiu X et al (2024) Gk bertdta: a graph representation learning and semantic embedding-based framework for drug-target affinity prediction. Comput Biol Med 173:108376

    Article  Google Scholar 

  42. Lu Y et al (2020) Cnn-g: Convolutional neural network combined with graph for image segmentation with theoretical analysis. IEEE Trans Cognit Dev Sys 13(3):631–644

    Article  MathSciNet  Google Scholar 

  43. Plizzari C, Cannici M, Matteucci M (2021) Skeleton-based action recognition via spatial and temporal transformer networks. Comput Vis Image Underst 208:103219

    Article  Google Scholar 

  44. Peng Y, Choi B, Xu J (2021) Graph learning for combinatorial optimization: a survey of state-of-the-art. Data Sci Eng 6(2):119–141

    Article  Google Scholar 

  45. Liu Q et al (2021) Domain-specific meta-embedding with latent semantic structures. Inf Sci 555:410–423

    Article  MathSciNet  Google Scholar 

  46. Duan J, et al. (2020) A study of pre-trained language models in natural language processing. In: 2020 IEEE International Conference on Smart Cloud (SmartCloud). IEEE

  47. Min B et al (2023) Recent advances in natural language processing via large pre-trained language models: a survey. ACM Comput Surv 56(2):1–40

    Article  Google Scholar 

  48. Hartvigsen T, et al. (2024) Aging with grace: lifelong model editing with discrete key-value adaptors. In: Advances in neural information processing systems

  49. Sun W, et al. (2023) Towards efficient and effective transformers for sequential recommendation. In: International conference on database systems for advanced applications

  50. Rethmeier N, Augenstein I (2023) A primer on contrastive pretraining in language processing: methods, lessons learned, and perspectives. ACM Comput Surv 55(10):1–17

    Article  Google Scholar 

  51. Lin C et al (2020) Does bert need domain adaptation for clinical negation detection? J Am Med Inform Assoc 27(4):584–591

    Article  Google Scholar 

  52. Dogra V, Singh A et al (2021) Analyzing distilbert for sentiment classification of banking financial news. Intell Comput Innov Data Sci 2021:501–510

    Article  Google Scholar 

  53. Yao S, et al. (2022) Reprbert: Distilling bert to an efficient representation-based relevance model for e-commerce. In: Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining

  54. Wang X et al (2021) Kepler: a unified model for knowledge embedding and pre-trained language representation. Trans Assoc Comput Linguistic 9:176–194

    Article  Google Scholar 

  55. Zhu R, Tu X, Huang JX (2021) Utilizing bert for biomedical and clinical text mining. In: Data analytics in biomedical engineering and healthcare. pp 73–103

  56. Alzubi JA et al (2023) Cobert: Covid-19 question answering system using bert. Arab J Sci Eng 48(8):11003–11013

    Article  Google Scholar 

  57. Floridi L, Chiriatti M (2020) Gpt-3: its nature, scope, limits, and consequences. Mind Mach 30:681–694

    Article  Google Scholar 

  58. Zhang Z, Zhou Z, Wang Y (2022) Ssegcn: syntactic and semantic enhanced graph convolutional network for aspect-based sentiment analysis. In: NAACL

  59. Marelli M, Menini S, Baroni M, et al. (2014) SemEval-2014 task 1: evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In: SemEval 2014

  60. Chen P, Sun Z, Bing L, et al. (2017) Recurrent attention network on memory for aspect sentiment analysis. In: EMNLP. pp 452–461

  61. Li X, Bing L, Lam W, et al. (2018) Transformation networks for target-oriented sentiment classification. In: ACL. pp 946–956

  62. Fan F, Feng Y, Zhao D (2018) Multi-grained attention network for aspect-level sentiment classification. In: EMNLP. pp 3433–3442

  63. Zhang M, Qian T (2020) Convolution over hierarchical syntactic and lexical graphs for aspect level sentiment analysis. In: EMNLP. pp 3540–3549

  64. Liang B, Yin R, Gui L, et al. (2020) Jointly learning aspect-focused and inter-aspect relations with graph convolutional networks for aspect sentiment analysis. In: COLING. pp 150–161

  65. Touvron H, et al. (2023) Llama: open and efficient foundation language models. arXiv:2302.13971

  66. Wang Y, et al. (2022) Self-instruct: aligning language models with self-generated instructions. arXiv:2212.10560

  67. Masalkhi M et al (2024) A side-by-side evaluation of llama 2 by meta with chatgpt and its application in ophthalmology. Eye 1–4

  68. Huang Y, et al. (2024) C-eval: a multi-level multi-discipline chinese evaluation suite for foundation models. In: Advances in neural information processing systems

  69. Huang B, Ou Y, Carley KM (2018) Aspect level sentiment classification with attention-over-attention neural networks. In: SBP. Springer, pp 197–206

  70. Xu L, Bing L, Lu W, et al. (2020) Aspect sentiment classification with aspect-specific opinion spans. In: EMNLP. pp 3561–3567

  71. Song Y, Wang J, Jiang T, et al. (2019) Targeted sentiment classification with attentional encoder network. In: ICANN. Springer, pp 93–103

  72. Han B, Chen Z, Qian Y (2022) Local information modeling with self-attention for speaker verification. In: ICASSP. IEEE

  73. Yang T et al (2019) Aspect-based sentiment analysis with new target representation and dependency attention. IEEE Trans Affect Comput 13(2):640–650

    Article  Google Scholar 

  74. Tao Z et al (2023) Multi-head attention graph convolutional network model: End-to-end entity and relation joint extraction based on multi-head attention graph convolutional network. CAAI Trans Intell Technol 8(2):468–477

    Article  Google Scholar 

Download references

Funding

This work is supported by the National Natural Science Foundation of China (Grant No.62102241) and “Science and Technology Innovation Action Plan” Natural Science Foundation of Shanghai (Grant No.23ZR1425400).

Author information

Author notes

  1. Haoyu Wang, Xihe Qiu and Xiaoyu Tan contributed equally to this work.

Authors and Affiliations

  1. School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, No.333 Longteng Road, 201620, Shanghai, Shanghai, China

    Haoyu Wang & Xihe Qiu

  2. INF Technology (Shanghai) Co., Ltd., No.88 Shangke Road, 201203, Shanghai, Shanghai, China

    Xiaoyu Tan

  3. Department of Mechanical Engineering, National University of Singapore, 9 Engineering Drive 1, 117575, Singapore, Singapore

    Xiaoyu Tan

Authors
  1. Haoyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xihe Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyu Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xihe Qiu.

Ethics declarations

Conflicts of Interest

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Qiu, X. & Tan, X. Multivariate graph neural networks on enhancing syntactic and semantic for aspect-based sentiment analysis. Appl Intell 54, 11672–11689 (2024). https://doi.org/10.1007/s10489-024-05802-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05802-6

Keywords

Search

Navigation