research-article

Dynamic multichannel fusion mechanism based on a graph attention network and BERT for aspect-based sentiment classification

Authors:

Shinan SongAuthors Info & Claims

Applied Intelligence, Volume 53, Issue 6

Pages 6800 - 6813

https://doi.org/10.1007/s10489-022-03851-3

Published: 11 July 2022 Publication History

Abstract

Aspect-based sentiment classification aims to predict the sentiment polarity on given aspect terms in a sentence. Recent works have incorporated syntactic information by developing graph neural networks (GNNs) over dependency trees to better establish connections between aspect items and their related context. However, the advancement is restricted because the dependency tree derived by the external parser is not entirely accurate, especially for high complexity and arbitrary expression datasets. To address this constraint, we propose a dynamic multichannel fusion mechanism based on the Graph AttenTion network and BERT (DMF-GAT-BERT), which regards the complementarity of semantic and syntactic information captured by GAT and BERT, respectively. Specifically, to alleviate the damage of incorrect dependency tree information to the model, we propose a two-layer dynamic fusion mechanism to adaptively adjust the fusion weight of semantic and syntax-related information channels. In addition, to capture accurate syntactic features, we propose an attentive layer ensemble (ALE) to integrate the contextual features learned by GAT in different layers. We conducted experiments on four datasets with different complexity, the Laptop, Restaurant, Twitter, and MAMS datasets, and achieved 80.38%, 86.10%, 76.22%, and 83.86% accuracy, respectively, outperforming robust baseline approaches.

References

[1]

Liu B Sentiment analysis and opinion mining Synth Lect Hum Lang Technol 2012 5 1 1-167

[2]

Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S (2014) Semeval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the international workshop on semantic evaluations, pp 27–35

[3]

Jin W, Ho H H, Srihari R K (2009) A novel lexicalized hmm-based learning framework for web opinion mining. In: Proceedings of the 26th annual international conference on machine learning, pp 465–472

[4]

Wagner J, Arora P, Cortes S, Barman U, Bogdanova D, Foster J, Tounsi L (2014) Dcu: aspect-based polarity classification for semeval task 4. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 223–229

[5]

Nazir A, Rao Y, Wu L, Sun L (2020) Issues and challenges of aspect-based sentiment analysis: a comprehensive survey. IEEE Trans Affect Comput. In press

[6]

Tang D, Qin B, Feng X, Liu T (2010) Effective lstms for target-dependent sentiment classification. In: COLING, pp 3298–3307

[7]

Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. In: Proceedings of the 56th annual meeting of the association for computational linguistics, vol 2018, pp 2514–2523

[8]

Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. In: EMNLP, pp 214–224

[9]

Liu H, Chatterjee I, Zhou M, Lu XS, and Abusorrah A Aspect-based sentiment analysis: a survey of deep learning methods IEEE Trans Comput Soc Syst 2020 7 6 1358-1375

[10]

TN Kipf M W (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th international conference on learning representations

[11]

Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In: Proceedings of the 6th international conference on learning representations

[12]

Zhang C, Li Q, Song D (2019) Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: EMNLP-IJCNLP, pp 4560–4570

[13]

Tian Y, Chen G, Song Y (2021) Aspect-based sentiment analysis with type-aware graph convolutional networks and layer ensemble. In: Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 2910–2922

[14]

Huang L, Sun X, Li S, Zhang L, Wang H (2020) Syntax-aware graph attention network for aspect-level sentiment classification. In: Proceedings of the 28th international conference on computational linguistics, pp 799–810

[15]

Wang K, Shen W, Yang Y, Quan X, Wang R (2020) Relational graph attention network for aspect-based sentiment analysis. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 3229–3238

[16]

Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, pp 4171–4186

[17]

Wang A, Singh A, Michael J, Hill F, Levy O, Bowman S R (2019) Glue: a multi-task benchmark and analysis platform for natural language understanding. In: Proceedings of the 7th international conference on learning representations

[18]

Xu H, Zhengyan Z, Ning D, Yuxian G, Xiao L, Yuqi H, Jiezhong Q, Liang Z, Wentao H, Minlie H et al (2021) Pre-trained models: past, present and future. AI Open

[19]

Li R, Chen H, Feng F, Ma Z, Wang X, Hovy E (2021) Dual graph convolutional networks for aspect-based sentiment analysis. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing, pp 6319–6329

[20]

Jawahar G, Sagot B, Seddah D (2019) What does bert learn about the structure of language?. In: ACL 2019-57th annual meeting of the association for computational linguistics, pp 3651–3657

[21]

Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606–615

[22]

Tay Y, Tuan L A, Hui S C (2018) Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis. In: Proceedings of the AAAI conference on artificial intelligence, vol 32

[23]

Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. In: Proceedings of the 26th international joint conference on artificial intelligence, pp 4068–4074

[24]

Fan F, Feng Y, Zhao D (2018) Multi-grained attention network for aspect-level sentiment classification. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 3433–3442

[25]

Song Y, Wang J, Jiang T, Liu Z, Rao Y (2019) Attentional encoder network for targeted sentiment classification. arXiv:1902.09314

[26]

Huang B, Carley K M (2019) Parameterized convolutional neural networks for aspect level sentiment classification. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 1091–1096

[27]

Zhang X and Gao T Multi-head attention model for aspect level sentiment analysis J Intell Fuzzy Syst 2020 38 1 89-96

[28]

Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. Computer Science

[29]

Pennington J, Socher R, Manning C D (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

[30]

Pota M, Ventura M, Fujita H, and Esposito M Multilingual evaluation of pre-processing for bert-based sentiment analysis of tweets Expert Syst Appl 2021 181 115119

[31]

Xu H, Liu B, Shu L, Yu P S (2019) Bert post-training for review reading comprehension and aspect-based sentiment analysis. In: Proceedings of NAACL-HLT, pp 2324–2335

[32]

Sun C, Huang L, Qiu X (2019) Utilizing bert for aspect-based sentiment analysis via constructing auxiliary sentence. In: Proceedings of NAACL-HLT, pp 380–385

[33]

Gao Z, Feng A, Song X, and Wu X Target-dependent sentiment classification with bert IEEE Access 2019 7 154290-154299

[34]

Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics, pp 49–54

[35]

Nguyen T H, Shirai K (2015) Phrasernn: phrase recursive neural network for aspect-based sentiment analysis. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 2509–2514

[36]

Phan M H, Ogunbona P O (2020) Modelling context and syntactical features for aspect-based sentiment analysis. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 3211–3220

[37]

Liang Y, Meng F, Zhang J, Chen Y, Xu J, and Zhou J A dependency syntactic knowledge augmented interactive architecture for end-to-end aspect-based sentiment analysis Neurocomputing 2021 454 291-302

[38]

Xu K, Zhao H, and Liu T Aspect-specific heterogeneous graph convolutional network for aspect-based sentiment classification IEEE Access 2020 8 139346-139355

[39]

Huang B, Carley K M (2019) Syntax-aware aspect level sentiment classification with graph attention networks. In: Proceedings of EMNLP-IJCNLP, pp 5472–5480

[40]

Liang B, Su H, Gui L, Cambria E, Xu R (2021) Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks. Knowl-Based Syst, 107643

[41]

Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics, pp 49–54

[42]

Jiang Q, Chen L, Xu R, Ao X, Yang M (2019) A challenge dataset and effective models for aspect-based sentiment analysis. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 6280–6285

[43]

Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp 249–256

[44]

Kingma D P, Ba J L (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

[45]

Dozat T, Manning C D (2016) Deep biaffine attention for neural dependency parsing. In: Proceedings of the 5th international conference on learning representations

[46]

Liang Y, Meng F, Zhang J, Xu J, Chen Y, Zhou J (2019) A novel aspect-guided deep transition model for aspect based sentiment analysis. In: Proceedings of EMNLP-IJCNLP, pp 5568–5579

Cited By

He J(2025)A multichannel fusion learning model with syntax for Chinese-oriented aspect-level sentiment classificationThe Journal of Supercomputing10.1007/s11227-024-06674-w81:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11227-024-06674-w
Kikuchi MAnda SOzono T(2024)Enhancing False-Sentence Pairs of BERT-Pair for Low-Frequency Aspect Category DetectionAdvances and Trends in Artificial Intelligence. Theory and Applications10.1007/978-981-97-4677-4_12(135-146)Online publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1007/978-981-97-4677-4_12

Index Terms

Dynamic multichannel fusion mechanism based on a graph attention network and BERT for aspect-based sentiment classification
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees
      2. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Hybrid sentiment classification on twitter aspect-based sentiment analysis

Social media sites and applications, including Facebook, YouTube, Twitter and blogs, have become major social media attractions today. The huge amount of information from this medium has become an attractive resource for organisations to monitor the ...
Aspect-based Sentiment Classification with Dual Cooperative Graph Attention Networks
MLNLP '22: Proceedings of the 2022 5th International Conference on Machine Learning and Natural Language Processing

Aspect-based sentiment analysis is a fine-grained sentiment classification task. The task analysis determines the corresponding sentiment polarity based on the aspect words. Recent approaches perform aspect-based sentiment analysis outstandingly by ...
Filter channel network based on contextual position weight for aspect-based sentiment classification
Abstract
In recent years, sentiment analysis in the field of natural language processing has garnered increasing attention from researchers. Aspect-level sentiment classification is a fine-grained task aiming to discern the sentiment polarity of specific ...

Comments

Information & Contributors

Information

Published In

cover image Applied Intelligence

Applied Intelligence Volume 53, Issue 6

Mar 2023

1224 pages

ISSN:0924-669X

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 11 July 2022

Accepted: 03 June 2022

Author Tags

Qualifiers

Research-article

Funding Sources

Key Program of Science and Technology Research during the 13th Five-Year Plan Period, the Educational Department of Jilin Province of China
Youth Growth Science and Technology Plan Project of Jilin Provincial Department of Science and Technology

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

He J(2025)A multichannel fusion learning model with syntax for Chinese-oriented aspect-level sentiment classificationThe Journal of Supercomputing10.1007/s11227-024-06674-w81:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11227-024-06674-w
Kikuchi MAnda SOzono T(2024)Enhancing False-Sentence Pairs of BERT-Pair for Low-Frequency Aspect Category DetectionAdvances and Trends in Artificial Intelligence. Theory and Applications10.1007/978-981-97-4677-4_12(135-146)Online publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1007/978-981-97-4677-4_12

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents