Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Learning Word Representations for Sentiment Analysis

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Word embedding has been proven to be a useful model for various natural language processing tasks. Traditional word embedding methods merely take into account word distributions independently from any specific tasks. Hence, the resulting representations could be sub-optimal for a given task. In the context of sentiment analysis, there are various types of prior knowledge available, e.g., sentiment labels of documents from available datasets or polarity values of words from sentiment lexicons. We incorporate such prior sentiment information at both word level and document level in order to investigate the influence each word has on the sentiment label of both target word and context words. By evaluating the performance of sentiment analysis in each category, we find the best way of incorporating prior sentiment information. Experimental results on real-world datasets demonstrate that the word representations learnt by DLJT2 can significantly improve the sentiment analysis performance. We prove that incorporating prior sentiment knowledge into the embedding process has the potential to learn better representations for sentiment analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. For example, we extract the synonym of word ‘like’ from page http://www.urbandictionary.com/define.php?term=like

  2. For example, we extract the synonym of word ‘like’ from the page of http://dict.youdao.com/search?q=like

References

  1. Bansal M, Gimpel K, Livescu K. Tailoring continuous word representations for dependency parsing. In: ACL (2). 2014. p. 809–815.

  2. Bengio Y, Schwenk H, Senécal J S, Morin F, Gauvain JL. A neural probabilistic language model. J Mach Learn Res 2003;3(6):1137–1155.

    Google Scholar 

  3. Cambria E, Das D, Bandyopadhyay S, Feraco A. A practical guide to sentiment analysis. Switzerland: Springer, Cham; 2017.

    Book  Google Scholar 

  4. Cambria E, Poria S, Bajpai R, Björn S. SenticNet 4: A Semantic resource for sentiment analysis based on conceptual primitives. In: COLING; 2016. p. 2666–2677.

  5. Chaturvedi I, Ragusa E, Gastaldo P, Zunino R, Cambria E. 2017. Bayesian network based extreme learning machine for subjectivity detection. J Franklin Inst. doi:1016/j.jfranklin.2017.06.007

  6. Collobert R, Weston J. A unified architecture for natural language processing: deep neural networks with multitask learning. In: International Conference, Helsinki, Finland, June; 2008. p. 160– 167.

  7. Duchi J, Hazan E, Singer Y. Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 2011;12(Jul):2121–2159.

    Google Scholar 

  8. Harris ZS. Distributional structure. Synthese Language Library 1954;10(2-3):146–162.

    Google Scholar 

  9. Hogenboom A, Bal D, Frasincar F, Bal M, de Jong F, Kaymak U. Exploiting emoticons in sentiment analysis Inproceedings of the 28th Annual ACM Symposium on Applied Computing; 2013. p. 703–710. ACM.

  10. Hu X, Tang J, Gao H, Liu H. Unsupervised sentiment analysis with emotional signals. In: Proceedings of the 22nd international conference on World Wide Web; 2013. p. 607–618, ACM.

  11. Huang EH, Socher R, Manning CD, Ng AY. Improving word representations via global context and multiple word prototypes. In: Meeting of the Association for Computational Linguistics: Long Papers; 2012. p. 873–882.

  12. Kim Y. Convolutional neural networks for sentence classification. In: EMNLP. 2014.

  13. Lin C -C, Ammar W, Dyer C, Levin LS. Unsupervised POS Induction with word embeddings. In: NAACL HLT 2015, The 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; 2015. p. 1311–1316.

  14. Liu Y, Liu Z, Chua TS, Sun M. 2015. Topical word embeddings. In: Twenty-Ninth AAAI Conference on Artificial Intelligence.

  15. Ma Y, Cambria E, Gao S. Label embedding for zero-shot fine-grained named entity typing. In COLING; 2016. p. 171–180, Osaka.

  16. Majumder N, Poria S, Gelbukh A, Cambria E. Deep learning-based document modeling for personality detection from text. IEEE Intelligent Systems 2017;32(2):74–79.

    Article  Google Scholar 

  17. Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. CoRR, arXiv:1301.3781; 2013.

  18. Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 2013;26:3111–3119.

    Google Scholar 

  19. Mnih A, Hinton G. Three new graphical models for statistical language modelling. In: International Conference on Machine Learning; 2007, p. 641–648.

  20. Mohammad SM, Turney PD. Crowdsourcing a word-emotion association lexicon. Comput Intell 2013;29(3): 436–465.

    Article  Google Scholar 

  21. Pang B, Lee L. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of the ACL.

  22. Bo P, Lee L. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In: Proceedings of the ACL.

  23. Pennington J, Socher R, Manning C. 2014. Glove: Global vectors for word representation. In: proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).

  24. Poria S, Cambria E, Bajpai R, Hussain A. A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion 2017;37:98–125.

    Article  Google Scholar 

  25. Poria S, Cambria E, Gelbukh A. Aspect extraction for opinion mining with a deep convolutional neural network. Knowl-Based Syst 2016;108:42–49.

    Article  Google Scholar 

  26. Poria S, Cambria E, Hazarika D, Vij P. A deeper look into sarcastic tweets using deep convolutional neural networks. In: COLING; 2016. p. 1601–1612.

  27. Poria S, Chaturvedi I, Cambria E, Hussain A. Convolutional MKL based multimodal emotion recognition and sentiment analysis. In: ICDM; 2016. p. 439–448, Barcelona.

  28. Rajagopal D, Cambria E, Olsher D, Kwok K. A graph-based approach to commonsense concept extraction and semantic similarity detection. In: WWW; 2013. p. 565–570, Rio De Janeiro.

  29. Socher R, Perelygin A, Wu JY, Chuang J, Manning CD, Ng AY, Potts C. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In: proceedings of the conference on empirical methods in natural language processing (EMNLP), volume 1631, p. 1642, Citeseer.

  30. Tang D, Wei F, Qin B, Yang N, Liu T, Zhou M. Sentiment embeddings with applications to sentiment analysis. Knowledge and Data Engineering, IEEE Transactions on 2016;28(2):496–509.

    Article  Google Scholar 

  31. Tang J, Qu M, Mei Q. Pte: Predictive text embedding through large-scale heterogeneous text networks. In: proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2015. p. 1165–1174, ACM.

  32. Wang S, Tang J, Aggarwal C, Liu H. Linked document embedding for classification. In: CIKM. ACM; 2016.

  33. Wang Y, Wang S, Tang J, Liu H, Li B. Unsupervised sentiment analysis for social media images. In: Proceedings of the Twenty Fourth International Joint Conference on Artificial Intelligence, IJCAI, 2015, Buenos Aires, Argentina; 2015. p. 2378– 2379.

  34. Wilson T, Wiebe J, Hoffmann P. Recognizing contextual polarity in phrase-level sentiment analysis. International Journal of Computer Applications 2005;7(5):347–354.

    Google Scholar 

  35. Zhong X, Sun A, Cambria E. 2017. Time expression analysis and recognition using syntactic token types and general heuristic rules. In: ACL.

  36. Zhou C, Sun C, Liu Z, Lau FCM. 2015. Category enhanced word embedding. Computer Science.

  37. Zou WY, Socher R, Cer DM, Manning CD. Bilingual word embeddings for phrase-based machine translation. In: EMNLP; 2013. p. 1393–1398.

Download references

Author information

Authors and Affiliations

  1. School of Automation, NorthWestern Polytechnical University, Xián, ShanXi, 710072, People’s Republic of China

    Yang Li, Quan Pan & Tao Yang

  2. Department of Computer Science and Engineering, Arizona State University, Tempe, AZ, 85281, USA

    Suhang Wang

  3. Computer Science and Engineering, Michigan State University, East Lansing, MI, 48824, USA

    Jiliang Tang

  4. School of Computer Science and Engineering, Nanyang Technological University, 50 Nanyang Ave, Singapore, 639798, Singapore

    Erik Cambria

Authors
  1. Yang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Quan Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Suhang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiliang Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Erik Cambria
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erik Cambria.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Informed Consent

Informed consent was not required as no human or animals were involved.

Human and Animal Rights

This article does not contain any studies with human or animal subjects performed by any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Pan, Q., Yang, T. et al. Learning Word Representations for Sentiment Analysis. Cogn Comput 9, 843–851 (2017). https://doi.org/10.1007/s12559-017-9492-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-017-9492-2

Keywords

Search

Navigation