research-article

Active Learning With Complementary Sampling for Instructing Class-Biased Multi-Label Text Emotion Classification

Authors:

Fuji RenAuthors Info & Claims

IEEE Transactions on Affective Computing, Volume 14, Issue 1

Pages 523 - 536

https://doi.org/10.1109/TAFFC.2020.3038401

Published: 01 January 2023 Publication History

Abstract

High-quality corpora have been very scarce for the text emotion research. Existing corpora with multi-label emotion annotations have been either too small or too class-biased to properly support a supervised emotion learning. In this article, we propose a novel active learning method for efficiently instructing the human annotations for a less-biased and high-quality multi-label emotion corpus. Specifically, to compensate annotation for the minority-class examples, we propose a complementary sampling strategy based on unlabeled resources by measuring a probabilistic distance between the expected emotion label distribution in a temporary corpus and an uniform distribution. Qualitative evaluations are also given to the unlabeled examples, in which we evaluate the model uncertainties for multi-label emotion predictions, their syntactic representativeness for the other unlabeled examples, and their diverseness to the labeled examples, for a high-quality sampling. Through active learning, a supervised emotion classifier gets progressively improved by learning from these new examples. Experiment results suggest that by following these sampling strategies we can develop a corpus of high-quality examples with significantly relieved bias for emotion classes. Compared to the learning procedures based on traditional active learning algorithms, our learning procedure indicates the most efficient learning curve and estimates the best multi-label emotion predictions.

References

[1]

A. Yadollahi, A. G. Shahraki, and O. R. Zaiane, “Current state of text sentiment analysis from opinion to emotion mining,” ACM Comput. Surv., vol. 50, no. 2, 2017, Art. no.

[2]

E. Cambria, “Affective computing and sentiment analysis,” IEEE Intell. Syst., vol. 31, no. 2, pp. 102–107, Mar./Apr. 2016.

Digital Library

[3]

S. M. Mohammad, “Sentiment analysis: Detecting valence, emotions, and other affectual states from text,” in Emotion Measurement. Amsterdam, The Netherlands: Elsevier, 2016, pp. 201–237.

[4]

F. Ren and Y. Wu, “Predicting user-topic opinions in twitter with social and topical context,” IEEE Trans. Affect. Comput., vol. 4, no. 4, pp. 412–424, Oct.-Dec. 2013.

[5]

H. Liu, H. Lieberman, and T. Selker, “A model of textual affect sensing using real-world knowledge,” in Proc. 8th Int. Conf. Intell. User Interfaces, 2003, pp. 125–132.

[6]

N. Colneriĉ and J. Demsar, “Emotion recognition on twitter: Comparative study and training a unison model,” IEEE Trans. Affect. Comput., vol. 11, no. 3, pp. 433–446, Third Quarter 2020.

[7]

X. Li, Y. Rao, H. Xie, R. Y. K. Lau, J. Yin, and F. L. Wang, “Bootstrapping social emotion classification with semantically rich hybrid neural networks,” IEEE Trans. Affect. Comput., vol. 8, no. 4, pp. 428–442, Fourth Quarter 2017.

[8]

X. Li, Q. Peng, Z. Sun, L. Chai, and Y. Wang, “Predicting social emotions from readersâĂŻ perspective,” IEEE Trans. Affect. Comput., vol. 10, no. 2, pp. 255–264, Second Quarter 2019.

[9]

U. Krcadinac, P. Pasquier, J. Jovanovic, and V. Devedzic, “Synesketch: An open source library for sentence-based emotion recognition,” IEEE Trans. Affect. Comput., vol. 4, no. 3, pp. 312–325, Third Quarter 2013.

Digital Library

[10]

P. Ekman, “Facial expression and emotion,” Amer. Psychol., vol. 48, no. 4, 1993, Art. no.

[11]

C. Quan and F. Ren, “A blog emotion corpus for emotional expression analysis in chinese,” Comput. Speech Lang., vol. 24, no. 4, pp. 726–749, 2010.

Digital Library

[12]

D.-A. Phan, Y. Matsumoto, and H. Shindo, “Autoencoder for semisupervised multiple emotion detection of conversation transcripts,” IEEE Trans. Affect. Comput., to be published.

[13]

X. Kang, F. Ren, and Y. Wu, “Exploring latent semantic information for textual emotion recognition in blog articles,” IEEE/CAA J. Automatica Sinica, vol. 5, no. 1, pp. 204–216, Jan. 2018.

[14]

V. Kuperman, Z. Estes, M. Brysbaert, and A. B. Warriner, “Emotion and language: Valence and arousal affect word recognition.” J. Exp. Psychol.: General, vol. 143, no. 3, 2014, Art. no.

[15]

D. A. Havas, A. M. Glenberg, and M. Rinck, “Emotion simulation during language comprehension,” Psychonomic Bull. Rev., vol. 14, no. 3, pp. 436–441, 2007.

[16]

C. A. Lutz, “Language and the politics of emotion,” Editions De La Maison Des Sci. De Lhomme, vol. 180, no. 3, p. 210, 1990.

[17]

L.-C. Yu, J.-L. Wu, P.-C. Chang, and H.-S. Chu, “Using a contextual entropy model to expand emotion words and their intensity for the sentiment classification of stock market news,” Knowl.-Based Syst., vol. 41, pp. 89–97, 2013.

[18]

F. Ren and C. Quan, “Linguistic-based emotion analysis and recognition for measuring consumer satisfaction: An application of affective computing,” Inf. Technol. Manage., vol. 13, no. 4, pp. 321–332, 2012.

[19]

J. Bollen, H. Mao, and X. Zeng, “Twitter mood predicts the stock market,” J. Comput. Sci., vol. 2, no. 1, pp. 1–8, 2011.

[20]

M. D. Conover, J. Ratkiewicz, M. Francisco, B. Gonçalves, F. Menczer, and A. Flammini, “Political polarization on twitter,” in Proc. 5th Int. AAAI Conf. Weblogs Social Media, 2011, pp. 89–96.

[21]

M. D. Conover, B. Gonçalves, J. Ratkiewicz, A. Flammini, and F. Menczer, “Predicting the political alignment of twitter users,” in Proc. IEEE 3rd Int. Conf. Privacy Secur. Risk Trust, IEEE 3rd Int. Conf. Soc. Comput., 2011, pp. 192–199.

[22]

D. Gayo-Avello, “”I wanted to predict elections with twitter and all i got was this lousy paper”–a balanced survey on election prediction using twitter data,” 2012,.

[23]

V. Lampos, D. Preo ţiuc-Pietro, and T. Cohn, “A user-centric model of voting intention from social media,” in Proc. 51st Annu. Meeting Assoc. Comput. Linguistics, 2013, pp. 993–1003.

[24]

L. Yang, D. Jiang, and H. Sahli, “Integrating deep and shallow models for multi-modal depression analysis–hybrid architectures,” IEEE Trans. Affect. Comput., to be published.

[25]

J. C. Eichstaedtet al., “Psychological language on twitter predicts county-level heart disease mortality,” Psychol. Sci., vol. 26, no. 2, pp. 159–169, 2015.

[26]

F. Ren, X. Kang, and C. Quan, “Examining accumulated emotional traits in suicide blogs with an emotion topic model,” IEEE J. Biomed. Health Inform., vol. 20, no. 5, pp. 1384–1396, Sep. 2016.

[27]

H. A. Schwartzet al., “Characterizing geographic variation in well-being using tweets,” in Proc. 7th Int. AAAI Conf. Weblogs Soc. Media, 2013, pp. 583–591.

[28]

L. Bylsma, B. Morris, and J. Rottenberg, “A meta-analysis of emotional reactivity in major depressive disorder,” Clin. Psychol. Rev., vol. 28, no. 4, pp. 676–691, 2008.

[29]

X. Kang, Y. Wu, and F. Ren, “Progressively improving supervised emotion classification through active learning,” in Proc. Int. Conf. Multi-Disciplinary Trends Artif. Intell., 2018, pp. 49–57.

[30]

X. Kang, F. Ren, and Y. Wu, “Semi-supervised learning of author-specific emotions in micro-blogs,” IEEJ Trans. Elect. Electron. Eng., vol. 11, no. 6, pp. 768–775, 2016.

[31]

F. Ren and K. Matsumoto, “Semi-automatic creation of youth slang corpus and its application to affective computing,” IEEE Trans. Affect. Comput., vol. 7, no. 2, pp. 176–189, Second Quarter 2016.

Digital Library

[32]

F. Ren and X. Kang, “Employing hierarchical Bayesian networks in simple and complex emotion topic analysis,” Comput. Speech Lang., vol. 27, no. 4, pp. 943–968, 2013.

Digital Library

[33]

C. Katsimerou, J. Albeda, A. Huldtgren, I. Heynderickx, and J. A. Redi, “Crowdsourcing empathetic intelligence: The case of the annotation of emma database for emotion and mood recognition,” ACM Trans. Intell. Syst. Technol., vol. 7, no. 4, pp. 1–27, 2016.

Digital Library

[34]

A. Metallinou and S. Narayanan, “Annotation and processing of continuous emotional attributes: Challenges and opportunities,” in Proc. 10th IEEE Int. Conf. Workshops Autom. Face Gesture Recognit., 2013, pp. 1–8.

[35]

E. A. Kolog, C. S. Montero, and E. Sutinen, “Annotation agreement of emotions in text: The influence of counsellors’ emotional state on their emotion perception,” in Proc. IEEE 16th Int. Conf. Advanced Learn. Technol., 2016, pp. 357–359.

[36]

S. Aman and S. Szpakowicz, “Identifying expressions of emotion in text,” in Proc. Int. Conf. Text Speech Dialogue, 2007, pp. 196–205.

[37]

E. Kim and R. Klinger, “Who feels what and why? annotation of a literature corpus with semantic roles of emotions,” in Proc. 27th Int. Conf. Comput. Linguistics, 2018, pp. 1345–1359.

[38]

S. M. Mohammad and P. D. Turney, “Crowdsourcing a word–emotion association lexicon,” Comput. Intell., vol. 29, no. 3, pp. 436–465, 2013.

[39]

S. M. Mohammad, “# emotional tweets,” in Proc. 1st Joint Conf. Lexical Comput. Semantics. Proc. Main Conf. Shared Task, Proc. 6th Int. Workshop Semantic Eval., 2012, pp. 246–255.

[40]

H. Schuff, J. Barnes, J. Mohme, S. Padó, and R. Klinger, “Annotation, modelling and analysis of fine-grained emotions on a stance and sentiment detection corpus,” in Proc. 8th Workshop Comput. Approaches Subjectivity Sentiment Social Media Anal., 2017, pp. 13–23.

[41]

Y. Li, H. Su, X. Shen, W. Li, Z. Cao, and S. Niu, “Dailydialog: A manually labelled multi-turn dialogue dataset,” 2017,.

[42]

D. Preoţiuc-Pietroet al., “Modelling valence and arousal in facebook posts,” in Proc. 7th Workshop Comput. Approaches Subjectivity Sentiment Social Media Anal., 2016, pp. 9–15.

[43]

C. O. Alm, D. Roth, and R. Sproat, “Emotions from text: machine learning for text-based emotion prediction,” in Proc. Conf. Human Lang. Technol. Empir. Methods Natural Lang. Process., 2005, pp. 579–586.

[44]

B. Settles, “Active learning literature survey,” Dept. Comput. Sci., Univ. Wisconsin-Madison, Madison, WI, 2009.

[45]

D. D. Lewis and W. A. Gale, “A sequential algorithm for training text classifiers,” in Proc. 17th Annu. Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 1994, pp. 3–12.

[46]

H. S. Seung, M. Opper, and H. Sompolinsky, “Query by committee,” in Proc. 5th Annu. Workshop Comput. Learn. Theory, 1992, pp. 287–294.

[47]

B. Settles, M. Craven, and S. Ray, “Multiple-instance active learning,” in Proc. Advances Neural Inf. Process. Syst., 2007, pp. 1289–1296.

[48]

D. A. Cohn, Z. Ghahramani, and M. I. Jordan, “Active learning with statistical models,” J. Artif. Intell. Res., vol. 4, no. 1, pp. 129–145, 1996.

Digital Library

[49]

N. Roy and A. McCallum, “Toward optimal active learning through sampling estimation of error reduction,” in Proc. 18th Int. Conf. Mach. Learn., 2001, pp. 441–448.

[50]

B. Settles and M. Craven, “An analysis of active learning strategies for sequence labeling tasks,” in Proc. Conf. Empir. Methods Natural Lang. Process., 2008, pp. 1070–1079.

[51]

J. Shao, Q. Wang, and Y. Lin, “Skyblocking for entity resolution,” Inf. Syst., vol. 85, pp. 30–43, 2019.

Digital Library

[52]

B. Kwoleket al., “Breast cancer classification on histopathological images affected by data imbalance using active learning and deep convolutional neural network,” in Proc. Int. Conf. Artif. Neural Netw., 2019, pp. 299–312.

[53]

A. Sharma and R. Rani, “Be-dtiâĂŻ: Ensemble framework for drug target interaction prediction using dimensionality reduction and active learning,” Comput. Methods Programs Biomed., vol. 165, pp. 151–162, 2018.

[54]

D. Himaja, T. M. Padmaja, and P. R. Krishna, “Oversample based large scale support vector machine for online class imbalance problem,” in Proc. Int. Conf. Big Data Analytics, 2018, pp. 348–362.

[55]

J. Błaszczyński and J. Stefanowski, “Actively balanced bagging for imbalanced data,” in Proc. Int. Symp. Methodologies Intell. Syst., 2017, pp. 271–281.

[56]

M. Zięba and J. M. Tomczak, “Boosted svm with active learning strategy for imbalanced data,” Soft Comput., vol. 19, no. 12, pp. 3357–3368, 2015.

Digital Library

[57]

J. Zhang, X. Wu, and V. S. Shengs, “Active learning with imbalanced multiple noisy labeling,” IEEE Trans. Cybern., vol. 45, no. 5, pp. 1095–1107, May 2015.

[58]

X. Zhang, J. Cheng, C. Xu, H. Lu, and S. Ma, “Multi-view multi-label active learning for image classification,” in Proc. IEEE Int. Conf. Multimedia Expo, 2009, pp. 258–261.

[59]

O. Reyes, C. Morell, and S. Ventura, “Effective active learning strategy for multi-label learning,” Neurocomputing, vol. 273, pp. 494–508, 2018.

Digital Library

[60]

K. Brinker, “On active learning in multi-label classification,” in From Data and Information Analysis to Knowledge Engineering. Berlin, Germany: Springer, 2006, pp. 206–213.

[61]

B. Yang, J.-T. Sun, T. Wang, and Z. Chen, “Effective multi-label active learning for text classification,” in Proc. 15th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2009, pp. 917–926.

[62]

C.-A. Brust, C. Käding, and J. Denzler, “Active learning for deep object detection,” 2018,.

[63]

S. Khanchi, A. Vahdat, M. I. Heywood, and A. N. Zincir-Heywood, “On botnet detection with genetic programming under streaming data label budgets and class imbalance,” Swarm Evol. Computation, vol. 39, pp. 123–140, 2018.

[64]

H. Zhang, W. Liu, J. Shan, and Q. Liu, “Online active learning paired ensemble for concept drift and class imbalance,” IEEE Access, vol. 6, pp. 73 815–73 828, 2018.

[65]

S. Khanchi, M. I. Heywood, and A. N. Zincir-Heywood, “Properties of a GP active learning framework for streaming data with class imbalance,” in Proc. Genetic Evol. Comput. Conf., 2017, pp. 945–952.

[66]

S. Khanchi, M. I. Heywood, and N. Zincir-Heywood, “On the impact of class imbalance in GP streaming classification with label budgets,” in Proc. Eur. Conf. Genetic Program., 2016, pp. 35–50.

[67]

X. Zhu, J. Lafferty, and Z. Ghahramani, “Combining active learning and semi-supervised learning using gaussian fields and harmonic functions,” in Proc. 20th Int. Conf. Mach. Learn. Workshop, 2003, pp. 58–65.

[68]

S. C. H. Hoi, R. Jin, J. Zhu, and M. R. Lyu, “Batch mode active learning and its application to medical image classification,” in Proc. 23rd Int. Conf. Mach. Learn., 2006, pp. 417–424.

[69]

A. Fujii, T. Tokunaga, K. Inui, and H. Tanaka, “Selective sampling for example-based word sense disambiguation,” Comput. Linguistics, vol. 24, no. 4, pp. 573–597, 1998.

Digital Library

[70]

B. Krawczyk, “Learning from imbalanced data: open challenges and future directions,” Prog. Artif. Intell., vol. 5, no. 4, pp. 221–232, 2016.

[71]

P. O. Branco, L. Torgo, and R. P. Ribeiro, “A survey of predictive modeling on imbalanced domains,” ACM Comput. Surv., vol. 49, no. 2, 2016, Art. no.

[72]

N. Japkowicz and S. Stephen, “The class imbalance problem: A systematic study,” Intell. Data Anal., vol. 6, no. 5, pp. 429–449, 2002.

[73]

S. Doyle, J. Monaco, M. Feldman, J. Tomaszewski, and A. Madabhushi, “An active learning based classification strategy for the minority class problem: Application to histopathology annotation,” BMC Bioinf., vol. 12, no. 1, 2011, Art. no.

[74]

S. Ertekin, J. Huang, L. Bottou, and L. Giles, “Learning on the border: active learning in imbalanced data classification,” in Proc. 16th ACM Conf. Conf. Inf. Knowl. Manage., 2007, pp. 127–136.

[75]

T. M. Hospedales, S. Gong, and T. Xiang, “Finding rare classes: Active learning with generative and discriminative models,” IEEE Trans. Knowl. Data Eng., vol. 25, no. 2, pp. 374–386, Feb. 2013.

Digital Library

[76]

S.-J. Huang, S. Chen, and Z.-H. Zhou, “Multi-label active learning: Query type matters,” in Proc. 24th Int. Conf. Artif. Intelli., 2015, pp. 946–952.

[77]

S. Li, S. Ju, G. Zhou, and X. Li, “Active learning for imbalanced sentiment classification,” in Proc. Joint Conf. Empir. Methods Natural Lang. Process. Comput. Natural Lang. Learn., 2012, pp. 139–148.

[78]

X. Li and Y. Guo, “Active learning with multi-label SVM classification,” in Proc. 23rd Int. Joint Conf. Artif. Intell., 2013, pp. 1479–1485.

[79]

H. T. Nguyen and A. Smeulders, “Active learning using pre-clustering,” in Proc. 21st Int. Conf. Mach. Learn., 2004, Art. no.

[80]

M. Ptaszynski, R. Rzepka, K. Araki, and Y. Momouchi, “Automatically annotating a five-billion-word corpus of japanese blogs for sentiment and affect analysis,” Comput. Speech Lang., vol. 28, no. 1, pp. 38–55, 2014.

Digital Library

[81]

T. Reitmaier and B. Sick, “Let us know your decision: Pool-based active training of a generative classifier with the selection strategy 4DS,” Inf. Sci., vol. 230, pp. 106–131, 2013.

Digital Library

[82]

S. Tong and D. Koller, “Support vector machine active learning with applications to text classification,” J. Mach. Learn. Res., vol. 2, pp. 45–66, 2001.

Digital Library

[83]

C. Riquelme, R. Johari, and B. Zhang, “Online active linear regression via thresholding,” in Proc. 31st AAAI Conf. Artif. Intell., 2017, pp. 2506–2512.

[84]

S. Sabato and R. Munos, “Active regression by stratification,” in Proc. Advances Neural Inf. Process. Syst., 2014, pp. 469–477.

[85]

R. Willett, R. Nowak, and R. M. Castro, “Faster rates in regression via active learning,” in Proc. Advances Neural Inf. Process. Syst., 2006, pp. 179–186.

[86]

E. Lughofer and M. Pratama, “Online active learning in data stream regression using uncertainty sampling based on evolving generalized fuzzy models,” IEEE Trans. Fuzzy Syst., vol. 26, no. 1, pp. 292–309, Feb. 2018.

Digital Library

[87]

D. Angluin, “Queries and concept learning,” Mach. Learn., vol. 2, no. 4, pp. 319–342, 1988.

[88]

R. Aluret al., “Syntax-guided synthesis,” in Proc. Formal Methods Comput.-Aided Design, 2013, pp. 1–8.

[89]

R. Hwa, “Sample selection for statistical parsing,” Comput. Linguistics, vol. 30, no. 3, pp. 253–276, 2004.

Digital Library

[90]

M. Sharma and M. Bilgic, “Evidence-based uncertainty sampling for active learning,” Data Mining Knowl. Discovery, vol. 31, no. 1, pp. 164–202, 2017.

Digital Library

[91]

P. Melville and R. J. Mooney, “Diverse ensembles for active learning,” in Proc. 21st Int. Conf. Mach. Learn., 2004, Art. no.

[92]

H. Wang, Y. Jin, and J. Doherty, “Committee-based active learning for surrogate-assisted particle swarm optimization of expensive problems,” IEEE Trans. Cybern., vol. 47, no. 9, pp. 2664–2677, Sep. 2017.

[93]

W. Cai, Y. Zhang, and J. Zhou, “Maximizing expected model change for active learning in regression,” in Proc. IEEE 13th Int. Conf. Data Mining, 2013, pp. 51–60.

[94]

W. Cai, M. Zhang, and Y. Zhang, “Batch mode active learning for regression with expected model change,” IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 7, pp. 1668–1681, Jul. 2017.

[95]

T. Zhang and F. Oles, “A probability analysis on the value of unlabeled data for classification problems,” in Proc. 17th Int. Conf. Mach. Learn., 2000, vol. 20, pp. 1191–1198.

[96]

Z. Xu, R. Akella, and Y. Zhang, “Incorporating diversity and density in active learning for relevance feedback,” in Proc. Eur. Conf. Inf. Retrieval, 2007, pp. 246–257.

[97]

M.-L. Zhang and Z.-H. Zhou, “A review on multi-label learning algorithms,” IEEE Trans. Knowl. Data Eng., vol. 26, no. 8, pp. 1819–1837, Aug. 2014.

[98]

O. Reyes and S. Ventura, “Evolutionary strategy to perform batch-mode active learning on multi-label data,” ACM Trans. Intell. Syst. Technol., vol. 9, no. 4, 2018, Art. no.

[99]

B. Grünbaumet al., “Partitions of mass-distributions and of convex bodies by hyperplanes,” Pacific J. Math., vol. 10, no. 4, pp. 1257–1261, 1960.

[100]

J. B. Walther, “Interpersonal effects in computer-mediated interaction: A relational perspective,” Commun. Res., vol. 19, no. 1, pp. 52–90, 1992.

[101]

J. B. Walther, T. Loh, and L. Granka, “Let me count the ways: The interchange of verbal and nonverbal cues in computer-mediated and face-to-face affinity,” J. Lang. Soc. Psychol., vol. 24, no. 1, pp. 36–65, 2005.

[102]

J. T. Hancock, C. Landrigan, and C. Silver, “Expressing emotion in text-based communication,” in Proc. SIGCHI Conf. Human Factors Comput. Syst., 2007, pp. 929–932.

[103]

E. Fox, Emotion Science Cognitive and Neuroscientific Approaches to Understanding Human Emotions. London, U.K.: Palgrave Macmillan, 2008.

[104]

S. V. Toller, “Emotion: Theory, research, and experience” Biol. Psychol., vol. 23, no. 3, pp. 320–322, 1986.

[105]

P. Shaver, J. Schwartz, D. Kirson, and C. O'connor, “Emotion knowledge: Further exploration of a prototype approach,” J. Pers. Soc. Psychol., vol. 52, no. 6, 1987, Art. no.

[106]

W. Shi, H. Wang, and S. He, “Sentiment analysis of chinese microblogging based on sentiment ontology: A case study of 7.23 Wenzhou train collision,” Connection Sci., vol. 25, no. 4, pp. 161–178, 2013.

Digital Library

[107]

B. Gunter, N. Koteyko, and D. Atanasova, “Sentiment analysis: A market-relevant and reliable measure of public feeling?,” Int. J. Market Res., vol. 56, no. 2, pp. 231–247, 2014.

[108]

X. Kang, F. Ren, and Y. Wu, “Bottom up: Exploring word emotions for chinese sentence chief sentiment classification,” in Proc. Int. Conf. Natural Lang. Process. Knowl. Eng., 2010, pp. 1–5.

[109]

Y. Wang and A. Pal, “Detecting emotions in social media: A constrained optimization approach,” in Proc. 24th Int. Joint Conf. Artif. Intell., 2015, pp. 996–1002.

[110]

A. Bandhakavi, N. Wiratunga, D. Padmanabhan, and S. Massie, “Lexicon based feature extraction for emotion text classification,” Pattern Recognit. Lett., vol. 93, pp. 133–142, 2017.

[111]

C. Yang, K. Lin, and H. Chen, “Building emotion lexicon from weblog corpora,” in Proc. 45th Annu. Meeting ACL Interactive Poster Demonstration Sessions, 2007, pp. 133–136.

[112]

R. Tokuhisa, K. Inui, and Y. Matsumoto, “Emotion classification using massive examples extracted from the web,” in Proc. 22nd Int. Conf. Comput. Linguistics, 2008, pp. 881–888.

[113]

I. Maks and P. Vossen, “A verb lexicon model for deep sentiment analysis and opinion mining applications,” in Proc. 2nd Workshop Comput. Approaches Subjectivity Sentiment Anal., 2011, Art. no.

[114]

S. Mohammad and T. Yang, “Tracking sentiment in mail: How genders differ on emotional axes,” in Proc. 2nd Workshop Comput. Approaches Subjectivity Sentiment Anal., 2011, Art. no.

[115]

M. Li, Q. Lu, Y. Long, and L. Gui, “Inferring affective meanings of words from word embedding,” IEEE Trans. Affect. Comput., vol. 8, no. 4, pp. 443–456, Fourth Quarter 2017.

[116]

Y. Wu, K. Kita, F. Ren, K. Matsumoto, and X. Kang, “Modification relations based emotional keywords annotation using conditional random fields,” in Proc. 4th Int. Conf. Intell. Netw. Intell. Syst., 2011, pp. 81–84.

[117]

Y. Wu, K. Kita, F. Ren, K. Matsumoto, and X. Kang, “Exploring emotional words for chinese document chief emotion analysis,” in Proc. 25th Pacific Asia Conf. Lang., 2011, pp. 597–606.

[118]

D. Das and S. Bandyopadhyay, “Word to sentence level emotion tagging for bengali blogs,” in Proc. ACL-IJCNLP 2009 Conf. Short Papers, 2009, pp. 149–152.

[119]

Y. Rao, “Contextual sentiment topic model for adaptive social emotion classification,” IEEE Intell. Syst., vol. 31, no. 1, pp. 41–47, Jan./Feb. 2016.

Digital Library

[120]

Y. Rao, H. Xie, J. Li, F. Jin, F. L. Wang, and Q. Li, “Social emotion classification of short text via topic-level maximum entropy model,” Inf. Manage., vol. 53, no. 8, pp. 978–986, 2016.

Digital Library

[121]

A. Summa, B. Resch, and M. Strube, “Microblog emotion classification by computing similarity in text, time, and space,” in Proc. Workshop Comput. Model. People's Opinions Pers. Emotions Soc. Media, 2016, pp. 153–162.

Cited By

Upadhyay SChien WSu BLee C(2024)Learning With Rater-Expanded Label Space to Improve Speech Emotion RecognitionIEEE Transactions on Affective Computing10.1109/TAFFC.2024.336042815:3(1539-1552)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TAFFC.2024.3360428
Liu FYang PShu YYan FZhang GLiu Y(2024)Emotion Dictionary Learning With Modality Attentions for Mixed Emotion ExplorationIEEE Transactions on Affective Computing10.1109/TAFFC.2023.333452015:3(1289-1302)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TAFFC.2023.3334520
Khan PRanjan PKumar S(2023)AT2GRU: A Human Emotion Recognition Model With Mitigated Device HeterogeneityIEEE Transactions on Affective Computing10.1109/TAFFC.2021.311412314:2(1520-1532)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TAFFC.2021.3114123
Show More Cited By

Recommendations

Effective multi-label active learning for text classification
KDD '09: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining

Labeling text data is quite time-consuming but essential for automatic text classification. Especially, manually creating multiple labels for each document may become impractical when a very large amount of data is needed for training multi-label text ...
Weak Labeled Multi-Label Active Learning for Image Classification
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

In order to achieve better classification performance with even fewer labeled images, active learning is suitable for these situations. Several active learning methods have been proposed for multi-label image classification, but all of them assume that ...
Semi-supervised multi-label classification using incomplete label information
Highlights
- An inductive semi-supervised method called Smile is proposed for multi-label classification using incomplete label information.
Abstract
Classifying multi-label instances using incompletely labeled instances is one of the fundamental tasks in multi-label learning. Most existing methods regard this task as supervised weak-label learning problem and assume sufficient ...

Comments

Information & Contributors

Information

Published In

1949-3045 © 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Computer Society Press

Washington, DC, United States

Publication History

Published: 01 January 2023

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Upadhyay SChien WSu BLee C(2024)Learning With Rater-Expanded Label Space to Improve Speech Emotion RecognitionIEEE Transactions on Affective Computing10.1109/TAFFC.2024.336042815:3(1539-1552)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TAFFC.2024.3360428
Liu FYang PShu YYan FZhang GLiu Y(2024)Emotion Dictionary Learning With Modality Attentions for Mixed Emotion ExplorationIEEE Transactions on Affective Computing10.1109/TAFFC.2023.333452015:3(1289-1302)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TAFFC.2023.3334520
Khan PRanjan PKumar S(2023)AT2GRU: A Human Emotion Recognition Model With Mitigated Device HeterogeneityIEEE Transactions on Affective Computing10.1109/TAFFC.2021.311412314:2(1520-1532)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TAFFC.2021.3114123
Gu YDuan JYu HYang XGao S(2023)PLVI-CE: a multi-label active learning algorithm with simultaneously considering uncertainty and diversityApplied Intelligence10.1007/s10489-023-05008-253:22(27844-27864)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1007/s10489-023-05008-2

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents