research-article

MUTUAL: Multi-Domain Sentiment Classification via Uncertainty Sampling

Authors:

Katerina Katsarou,

Kostas StefanidisAuthors Info & Claims

SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing

Pages 331 - 339

https://doi.org/10.1145/3555776.3577765

Published: 07 June 2023 Publication History

Abstract

Multi-domain sentiment classification trains a classifier using multiple domains and then tests the classifier on one of the domains. Importantly, no domain is assumed to have sufficient labeled data; instead, the goal is leveraging information between domains, making multi-domain sentiment classification a very realistic scenario. Typically, labeled data is costly because humans must classify it manually. In this context, we propose the MUTUAL approach that learns general and domain-specific sentence embeddings that are also context-aware due to the attention mechanism. In this work, we propose using a stacked BiLSTM-based Autoencoder with an attention mechanism to generate the two above-mentioned types of sentence embeddings. Then, using the Jensen-Shannon (JS) distance, the general sentence embeddings of the four most similar domains to the target domain are selected. The selected general sentence embeddings and the domain-specific embeddings are concatenated and fed into a dense layer for training. Evaluation results on public datasets with 16 different domains demonstrate the efficiency of our model. In addition, we propose an active learning algorithm that first applies the elliptic envelope for outlier removal to a pool of unlabeled data that the MUTUAL model then classifies. Next, the most uncertain data points are selected to be labeled based on the least confidence metric. The experiments show higher accuracy for querying 38% of the original data than random sampling.

References

[1]

Dana Angluin. 1988. Queries and concept learning. Machine learning 2, 4 (1988), 319--342.

[2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate.

[3]

Eric B Baum and Kenneth Lang. 1992. Query learning can work poorly when a human oracle is used. In International joint conference on neural networks, Vol. 8. 8.

[4]

Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomás Mikolov. 2016. Enriching Word Vectors with Subword Information. CoRR abs/1607.04606 (2016). arXiv:1607.04606 http://arxiv.org/abs/1607.04606

[5]

Yitao Cai and Xiaojun Wan. 2019. Multi-Domain Sentiment Classification Based on Domain-Aware Embedding and Attention. In IJCAI. 4904--4910.

[6]

Rich Caruana. 1997. Multitask learning. Machine learning 28, 1 (1997), 41--75.

[7]

David Cohn, Les Atlas, and Richard Ladner. 1994. Improving generalization with active learning. Machine learning 15, 2 (1994), 201--221.

[8]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT. 4171--4186.

[9]

John R. Firth. 1957. A synopsis of linguistic theory 1930-55. Studies in Linguistic Analysis (special volume of the Philological Society) (1957), 1--32.

[10]

Nils Haldenwang, Katrin Ihler, Julian Kniephoff, and Oliver Vornberger. 2017. A comparative study of uncertainty based active learning strategies for general purpose twitter sentiment analysis with deep neural networks. In International Conference of the German Society for Computational Linguistics and Language Technology. Springer, 208--215.

[11]

Katerina Katsarou, Nabil Douss, and Kostas Stefanidis. 2023. REFORMIST: Hierarchical Attention Networks for Multi-Domain Sentiment Classification with Active Learning (SAC '23). Association for Computing Machinery, New York, NY, USA.

[12]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization.

[13]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).

[14]

David D Lewis and William A Gale. 1994. A sequential algorithm for training text classifiers. In SIGIR. 3--12.

[15]

Lianghao Li, Xiaoming Jin, Sinno Jialin Pan, and Jian-Tao Sun. 2012. Multi-domain active learning for text classification. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. 1086--1094.

Digital Library

[16]

Cheng-Yuan Liou, Wei-Chen Cheng, Jiun-Wei Liou, and Daw-Ran Liou. 2014. Autoencoder for words. Neurocomputing 139 (2014), 84--96.

Digital Library

[17]

Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. 2008. Isolation forest. In ICDM. 413--422.

Digital Library

[18]

Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016. Deep Multi-Task Learning with Shared Memory for Text Classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, 118--127.

[19]

Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016. Recurrent Neural Network for Text Classification with Multi-Task Learning. In IJCAI. 2873--2879.

[20]

Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2017. Adversarial Multi-task Learning for Text Classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vancouver, Canada, 1--10.

[21]

Qi Liu, Yue Zhang, and Jiangming Liu. 2018. Learning domain representation for multi-domain sentiment classification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 541--550.

[22]

Peter J Rousseeuw and Katrien Van Driessen. 1999. A fast algorithm for the minimum covariance determinant estimator. Technometrics 41, 3 (1999), 212--223.

[23]

Nicholas Roy and Andrew McCallum. 2001. Toward Optimal Active Learning through Sampling Estimation of Error Reduction. In ICML. Morgan Kaufmann Publishers Inc., 441--448.

[24]

M. Schuster and K.K. Paliwal. 1997. Bidirectional Recurrent Neural Networks. Trans. Sig. Proc. 45, 11 (Nov. 1997), 2673--2681.

Digital Library

[25]

Burr Settles. 2009. Active learning literature survey. Technical Report. University of Wisconsin-Madison Department of Computer Sciences.

[26]

Burr Settles and Mark Craven. 2008. An Analysis of Active Learning Strategies for Sequence Labeling Tasks. In EMNLP. 1070--1079.

[27]

H Sebastian Seung, Manfred Opper, and Haim Sompolinsky. 1992. Query by committee. In Proceedings of the fifth annual workshop on Computational learning theory. 287--294.

Digital Library

[28]

Xuefeng Su, Ru Li, and Xiaoli Li. 2020. Multi-domain transfer learning for text classification. In CCF International Conference on Natural Language Processing and Chinese Computing. Springer, 457--469.

Digital Library

[29]

Renjie Zheng, Junkun Chen, and Xipeng Qiu. 2018. Same Representation, Different Attentions: Shareable Sentence Representation Learning from Multiple Tasks. In IJCAI. 4616--4622.

[30]

Xiaojin Zhu, John Lafferty, and Zoubin Ghahramani. 2003. Combining active learning and semi-supervised learning using gaussian fields and harmonic functions. In ICML 2003 workshop on the continuum from labeled to unlabeled data in machine learning and data mining, Vol. 3.

Cited By

Kruthika CNamitha Devi NMadlur VAsha KAruna T(2023)Hybrid Optimization Based BERT Model for Drug Detection in NLP2023 IEEE 3rd Mysore Sub Section International Conference (MysuruCon)10.1109/MysuruCon59703.2023.10396892(1-7)Online publication date: 1-Dec-2023
https://doi.org/10.1109/MysuruCon59703.2023.10396892

Index Terms

MUTUAL: Multi-Domain Sentiment Classification via Uncertainty Sampling
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Information extraction
  2. Machine learning
    1. Learning settings
      1. Active learning settings
    2. Machine learning approaches
      1. Neural networks

Recommendations

REFORMIST: Hierarchical Attention Networks for Multi-Domain Sentiment Classification with Active Learning
SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing

In multi-domain sentiment classification, the classifier is trained on the source domain that includes multiple domains and is tested on the target domain. It is essential to highlight that the domain in the target domain is one of the domains in the ...
Multi-domain Sentiment Classification on Self-constructed Indonesian Dataset
Natural Language Processing and Chinese Computing
Abstract
Domain-dependence limits the application of a well-trained sentiment classifier based on one domain data in other different domains. To solve this problem, multi-domain sentiment classification has received great attention recently. It aims to ...
Collaborative Multi-domain Sentiment Classification
ICDM '15: Proceedings of the 2015 IEEE International Conference on Data Mining (ICDM)

Sentiment classification is a hot research topic in both industrial and academic fields. The mainstream sentiment classification methods are based on machine learning and treat sentiment classification as a text classification problem. However, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing

March 2023

1932 pages

ISBN:9781450395175

DOI:10.1145/3555776

Conference Chairs:
Jiman Hong
Soongsil University, South Korea
,
Maart Lanperne
Tallinn University, Estonia
,
Program Chairs:
Juw Won Park
University of Louisville, USA
,
Tomas Cerny
Baylor University, USA
,
Publication Chair:
Hossain Shahriar
Kennesaw State University, USA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 June 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SAC '23

Sponsor:

SIGAPP

SAC '23: 38th ACM/SIGAPP Symposium on Applied Computing

March 27 - 31, 2023

Tallinn, Estonia

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
34
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)3

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kruthika CNamitha Devi NMadlur VAsha KAruna T(2023)Hybrid Optimization Based BERT Model for Drug Detection in NLP2023 IEEE 3rd Mysore Sub Section International Conference (MysuruCon)10.1109/MysuruCon59703.2023.10396892(1-7)Online publication date: 1-Dec-2023
https://doi.org/10.1109/MysuruCon59703.2023.10396892

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents