Abstract
The goal of Aspect-level Sentiment Classification (ASC) is to identify the sentiment polarity towards a specific aspect of a given sentence. Mainstream methods design complicated models and require a large scale of annotated training samples, and even perform finetuning based on pre-trained language models. Therefore, those supervised methods may be impractical in real-world scenarios due to the limited availability of labeled training corpora, a.k.a. low-resource settings. To this end, we propose an aspect-specific prompt learning approach (AS-Prompt) that fully utilizes the pre-trained knowledge and aspect-related information to deal with ASC tasks, enabling pre-trained models with huge parameters to achieve considerable results under the few-shot settings. Specifically, we transfer the sentiment classification task into Masked Language Modeling (MLM) by designing appropriate prompts and searching for the ideal expression of prompts in continuous space. Meanwhile, we integrate the prompts into the input sentence, thus adapting the model to the classification task under the guidance of sentiment labels. Experimental results on SemEval-2014 Task 4 show our proposed method achieves noticeably improvement compared with the original BERT models and discrete prompt methods. In addition, we test the performance of the model’s transfer on different datasets and demonstrate the superiority of prompt learning when adapting to a new domain, especially under a low-resource setting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Beigi, O.M., Moattar, M.H.: Automatic construction of domain-specific sentiment lexicon for unsupervised domain adaptation and sentiment classification. Knowl. Based Syst. 213, 106423 (2021)
Ben-David, E., Oved, N., Reichart, R.: PADA: a prompt-based autoregressive approach for adaptation to unseen domains. arXiv preprint arXiv:2102.12206 (2021)
Brown, T.B., et al.: Language models are few-shot learners. In: Proceedings of NeurIPS, pp. 1877–1901 (2020)
Cao, Z., Zhou, Y., Yang, A., Peng, S.: Deep transfer learning mechanism for fine-grained cross-domain sentiment classification. Connect. Sci. 33(4), 911–928 (2021)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL, pp. 4171–4186 (2019)
Gao, T., Fisch, A., Chen, D.: Making pre-trained language models better few-shot learners. In: Proceedings of ACL, pp. 3816–3830 (2021)
Hambardzumyan, K., Khachatrian, H., May, J.: WARP: word-level adversarial reprogramming. In: Proceedings of ACL, pp. 4921–4933 (2021)
Heinzerling, B., Inui, K.: Language models as knowledge bases: on entity representations, storage capacity, and paraphrased queries. In: Proceedings of EACL, pp. 1772–1791 (2021)
Howard, J., Ruder, S.: Universal language model fine-tuning for text classification. In: Gurevych, I., Miyao, Y. (eds.) Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, 15–20 July 2018, Volume 1: Long Papers, pp. 328–339. Association for Computational Linguistics (2018)
Hu, M., Liu, B.: Mining and summarizing customer reviews. In: Proceedings of ACM SIGKDD, pp. 168–177 (2004)
Jiang, Z., Xu, F.F., Araki, J., Neubig, G.: How can we know what language models know. Trans. Assoc. Comput. Linguist. 8, 423–438 (2020)
Li, L., Liu, Y., Zhou, A.: Hierarchical attention based position-aware network for aspect-level sentiment analysis. In: Proceedings of CoNLL, pp. 181–189 (2018)
Li, X.L., Liang, P.: Prefix-tuning: optimizing continuous prompts for generation. In: Proceedings of ACL, pp. 4582–4597 (2021)
Li, Z., Qin, Y., Liu, Z., Wang, W.: Powering comparative classification with sentiment analysis via domain adaptive knowledge transfer. arXiv preprint arXiv:2109.03819 (2021)
Liu, X., et al.: GPT understands, too. arXiv preprint arXiv:2103.10385 (2021)
Logan, R.L., IV., Balažević, I., Wallace, E., Petroni, F., Singh, S., Riedel, S.: Cutting down on prompts and parameters: simple few-shot learning with language models. arXiv preprint arXiv:2106.13353 (2021)
Ni, J., Li, J., McAuley, J.J.: Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In: Proceedings of EMNLP, pp. 188–197 (2019)
Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., Manandhar, S.: Semeval-2014 task 4: aspect based sentiment analysis. In: Proceedings of COLING, pp. 27–35 (2014)
Rietzler, A., Stabinger, S., Opitz, P., Engl, S.: Adapt or get left behind: domain adaptation through BERT language model finetuning for aspect-target sentiment classification. In: Proceedings of LREC, pp. 4933–4941 (2020)
Schick, T., Schütze, H.: Exploiting cloze-questions for few-shot text classification and natural language inference. In: Proceedings of EACL, pp. 255–269 (2021)
Schick, T., Schütze, H.: It’s not just size that matters: small language models are also few-shot learners. In: Proceedings of NAACL, pp. 2339–2352 (2021)
Seoh, R., Birle, I., Tak, M., Chang, H., Pinette, B., Hough, A.: Open aspect target sentiment classification with natural language prompts. In: Proceedings of EMNLP, pp. 6311–6322 (2021)
Shin, T., Razeghi, Y., IV., R.L.L., Wallace, E., Singh, S.: AutoPrompt: eliciting knowledge from language models with automatically generated prompts. In: Proceedings of EMNLP, pp. 4222–4235 (2020)
Sun, C., Huang, L., Qiu, X.: Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence. In: Proceedings of NAACL, pp. 380–385 (2019)
Tang, H., Ji, D., Li, C., Zhou, Q.: Dependency graph enhanced dual-transformer structure for aspect-based sentiment classification. In: Proceedings of ACL, pp. 6578–6588 (2020)
Wolf, T., et al.: HuggingFace’s transformers: state-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019)
Xu, H., Liu, B., Shu, L., Yu, P.S.: BERT post-training for review reading comprehension and aspect-based sentiment analysis. In: Proceedings of NAACL, pp. 2324–2335 (2019)
Yan, H., Dai, J., Ji, T., Qiu, X., Zhang, Z.: A unified generative framework for aspect-based sentiment analysis. In: Proceedings of ACL, pp. 2416–2429 (2021)
Yin, W., Hay, J., Roth, D.: Benchmarking zero-shot text classification: datasets, evaluation and entailment approach. In: Proceedings of EMNLP, pp. 3912–3921 (2019)
Zhao, C., Wang, S., Li, D., Liu, X., Yang, X., Liu, J.: Cross-domain sentiment classification via parameter transferring and attention sharing mechanism. Inf. Sci. 578, 281–296 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, G., Lin, F., Chen, W., Dong, D., Liu, B. (2023). Prompt-Based Learning for Aspect-Level Sentiment Classification. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Lecture Notes in Computer Science, vol 13625. Springer, Cham. https://doi.org/10.1007/978-3-031-30111-7_43
Download citation
DOI: https://doi.org/10.1007/978-3-031-30111-7_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30110-0
Online ISBN: 978-3-031-30111-7
eBook Packages: Computer ScienceComputer Science (R0)