Improving the Accuracy of Pre-trained Word Embeddings for

Sentiment Analysis

Seyed Mahdi Rezaeinia a,b, Ali Ghodsi a, Rouhollah Rahmani b

a
Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Canada
b
Network Science and Technology Department, University of Tehran, Tehran, Iran

Abstract
Sentiment analysis is one of the well-known tasks and fast growing research areas in natural language
processing (NLP) and text classifications. This technique has become an essential part of a wide range of
applications including politics, business, advertising and marketing. There are various techniques for
sentiment analysis, but recently word embeddings methods have been widely used in sentiment
classification tasks. Word2Vec and GloVe are currently among the most accurate and usable word
embedding methods which can convert words into meaningful vectors. However, these methods ignore
sentiment information of texts and need a huge corpus of texts for training and generating exact vectors
which are used as inputs of deep learning models. As a result, because of the small size of some corpuses,
researcher often have to use pre-trained word embeddings which were trained on other large text corpus
such as Google News with about 100 billion words. The increasing accuracy of pre-trained word
embeddings has a great impact on sentiment analysis research. In this paper we propose a novel method,
Improved Word Vectors (IWV), which increases the accuracy of pre-trained word embeddings in sentiment
analysis. Our method is based on Part-of-Speech (POS) tagging techniques, lexicon-based approaches
and Word2Vec/GloVe methods. We tested the accuracy of our method via different deep learning models
and sentiment datasets. Our experiment results show that Improved Word Vectors (IWV) are very effective
for sentiment analysis.

1. Introduction
Sentiment analysis is a practical technique that allows businesses, researchers, governments, politicians
and organizations to know about people’s sentiments, which play an important role in decision making
processes. Word Embedding is one of the most useful deep learning methods used for constructing vector
representations of words and documents. These methods have received a lot of attention in text and
sentiment analysis because of their abilities to capture the syntactic and semantic relations among words.
The two successful deep learning methods of word embeddings are Word2Vec [1,2] and Global Vectors
(GloVe) [3]. Many researchers have used these two methods in their sentiment analysis research [4,5,6,7].

Although very effective, these methods have several limits and need to be improved. The Word2Vec and
GloVe need very large corpuses for training and presenting an acceptable vector for each word [8,6]. For
example, Google has used about 100 billion words for training Word2Vec algorithms and has re-released
pre-trained word vectors with 300 dimensions. Because of the small size of some datasets, investigators
have to use pre-trained word vectors such as Word2Vec and GloVe, which may not be the best fit for their
data [9,10,11,12,13,14,15]. Another problem is that the word vector calculations of the two methods that
are used to represent a document do not consider the context of the document [16]. For example, the word
vector for “beetle” as a car is equal to its word vector as an animal. In addition, both models ignore the
relationships between terms that do not literally co-occur [16]. Also, Cerisara et al. [17] have found that the
standard Word2Vec word embedding techniques don't bring valuable information for dialogue act
recognition in three different languages. Another important problem of these word embedding techniques
is ignoring the sentiment information of the given text [6,7,8]. The side effect of this problem is that those
words with opposite polarity are mapped into close vectors and it is a disaster for sentiment analysis [4].

In this research, we propose a novel method to improve the accuracy of pre-trained Word2Vec/Glove
vectors in sentiment analysis tasks. The proposed method was tested by different sentiment datasets and
various deep learning models from published papers. The results show that the method increases the
accuracy of pre-trained word embeddings vectors for sentiment analysis. The organization of this paper is
as follows: Section 2 describes the related works and literature review for this research. Section 3 presents
our proposed method and algorithm, and additionally describes the proposed deep learning model for
testing the method. Section 4 reports our experiments, showing results along with evaluations and
discussions. Section 5 is the conclusion and future works.

2. Related Works

2.1. Lexicon-based method

Sentiment classification techniques are mainly divided into lexicon-based methods and machine learning
methods such as Deep Learning [18,19]. The lexicon-based sentiment analysis approach is typically based
on lists of words and phrases with positive and negative connotations [20,21,22]. This approach needs a
dictionary of negative and positive sentiment values assigned to words. These methods are simple,
scalable, and computationally efficient. As a result, they are mostly used to solve general sentiment analysis
problems [18]. However, lexicon-based methods depend on human effort in human-labeled documents [19]
and sometimes suffer from low coverage [8]. Also, they depend on finding the sentiment lexicon which is
applied to analysis the text [18].

Because of the accuracy improvement of text classification, the lexicon-based approaches have combined
with machine learning methods recently. Several authors found that the machine learning methods were
more accurate than the lexicon methods [19,23]. Mudinas et al. [24] increased the accuracy of sentiment
analysis by combining lexicon-based and Support Vector Machine (SVM) methods. Zhang et al. [25]
successfully combined lexicon-based approach and binary classifier for sentiment classification of Twitter
data. Basari et al. [26] have combined the Particle Swarm Optimization (PSO) technique and SVM method
for sentiment analysis of movie reviews. In all of these cases, the machine learning techniques improved
the accuracy of text classifications.

2.2. Deep learning method

Recently, deep learning methods have played an increasing role in natural language processing. Most of
the deep learning tasks in NLP has been oriented towards methods which using word vector
representations [6]. Continuous vector representations of words algorithms such as Word2Vec and GloVe
are deep learning techniques which can convert words into meaningful vectors. The vector representations
of words are very useful in text classification, clustering and information retrieval. Word embeddings
techniques have some advantages compare to bag-of-words representation. For instance, words close in
meaning are near together in the word embedding space. Also, word embeddings have lower
dimensionality than the bag-of-words [2].

The accuracy of the Word2vec and Glove depends on text corpus size. Meaning, the accuracy increases
with the growth of text corpus. Tang et al. [4] proposed learning continuous word representations for
sentiment analysis on Twitter which is a large social networks dataset. Severyn and Moschitti [27] used
Word2Vec method to learn the word embeddings on 50M tweets and applied generated pre-trained vectors
as inputs of a deep learning model. Recently, Lauren et al. [28] have proposed a discriminant document
embeddings method which has used skip-gram for generating word embeddings of clinical texts. Fu et al.
[5] applied Word2Vec for word embeddings of English Wikipedia dataset and Chinese Wikipedia dataset.
The word embeddings used as inputs of recursive autoencoder for sentiment analysis approach. Ren et al.
[7] proposed a new word representation method for Twitter sentiment classification. They used Word2Vec
to generate word embeddings of some datasets in their method. Qin et al. [29] trained Word2Vec algorithm
by English Wikipedia corpus which has 408 million words. They used those pre-trained vectors as inputs
of convolutional neural networks for data-driven tasks.
Nevertheless, as already mentioned, these word embedding algorithms need a huge corpus of texts for
training [8] and most of them ignore the sentiment information of text [4,6]. Because of the limitations and
restrictions in some corpuses, researchers prefer to use pre-trained word embeddings vectors as inputs of
machine learning models. Kim [9] has used pre-trained Word2Vec vectors as inputs to convolutional neural
networks and has increased the accuracy of sentiment classification. Also, Camacho-Collados et al. [11]
used pre-trained Word2Vec vectors for the representation of concepts. Zhang and Wallace [10] have
applied pre-trained GloVe and Word2Vec vectors in their deep learning model and have improved the
accuracy of sentence and sentiment classification. Caliskan et al. [12] used pre-trained GloVe word
embeddings vectors for increasing the accuracy of their proposed method. Wang et al. [13] applied pre-
trained GloVe vectors as inputs of attention-based LSTMs model for aspect-level sentiment analysis. Liu
et al. [15] have used pre-trained Word2Vec as a word embedding representation for recommending Idioms
in essay writing.

As a result, increasing the accuracy of pre-trained word embedding is very important and plays a vital role
in sentiment classification methods. Zhang and Wallace [10] combined pre-trained Word2Vec and GloVe
vectors in their deep learning model, but the accuracies were decreased. Some results are shown in
table 1.

Table1: The results of the combination of pre-trained Word2Vec and Glove on three datasets [10]

Dataset Reduced accuracy rate (%)

Movie reviews (MR) -0.22 %
Stanford sentiment treebank (SST) -1.1%
TREC -0.17%

According to table1, combination of pre-trained Word2Vec and Glove decreased the accuracy of text and
sentiment classification on some datasets. Also, Kamkarhaghighi and Makrehchi [16] have proposed an
algorithm for increasing the accuracy of pre-trained Word2Vec and Glove. Their algorithm was tested on
two datasets and the accuracy of Word2Vec was decreased on one dataset by the proposed algorithm. In
the following section we present in detail our proposed model, its algorithm and the proposed deep learning
model for checking our method.
3. Proposed method

In our proposed method, Improved Word Vector (IWV) we have increased the accuracy of word embedding
vectors based on the combination of natural language processing techniques, lexicon-based approaches
and Word2Vec/GloVe methods which have high accuracies. The main architecture of the proposed method
has been shown in figure 1.

Figure1: The main architecture of the proposed method (Improved word vector)

3.1. Word2Vec and GloVe

Word2Vec and GloVe are two successful word embedding algorithms which have high accuracy. Word2Vec
is based on continuous Bag-of-Words (CBOW) and Skip-gram architectures which can provide high quality
word embedding vectors. CBOW predicts a word given its context and Skip-gram can predict the context
given a word. The generated vectors of words which appear in common contexts in the corpus are located
close to each other in the vector space. GloVe word embedding is a global log-bilinear regression model
and is based on co-occurrence and factorization of matrix in order to get vectors. In this study, we used
300-dimension word2vec embeddings trained on Google News and 300-dimension GloVe word
embeddings.
3.2. POS2Vec (P2V)

Part-of-speech (POS) tagging is an important and fundamental step in Natural Language Processing which
is the process of assigning to each word of a text the proper POS tag. The Part-of-speech gives large
amount of information about a word and its neighbors, syntactic categories of words (nouns, verbs,
adjectives, adverbs, etc.) and similarities and dissimilarities between them. We converted each generated
POS tag to a constant vector and concatenated with Word2Vec/GloVe vectors. We assigned 50-
dimensional vectors to each word. As a result, Word2Vec/Glove vectors will have syntactic information of
words.

3.3. Lexicon2Vec (L2V)

The sentiment and emotion lexicons are lists of phrases and words which have polarity scores and can be
used to analyze texts. Each lexicon contains of words and their values which are the sentiment scores for
that words. There are various sentiment and emotion lexicons that can be used, but choosing the proper
lexicon is very important. We selected six lexicons as our resources and assigned 6-dimensional vectors
to each word.

• National Research Council Canada (NRC) Emoticon Affirmative Context Lexicon and NRC
Emoticon Negated Context Lexicon [30,31,32].

• NRC Hashtag Affirmative Context Sentiment Lexicon and NRC Hashtag Negated Context
Sentiment Lexicon [30,31,32].

• NRC Emoticon Lexicon [30,31,32].

• NRC Hashtag Sentiment Lexicon [30,31,32].

• SemEval-2015 English Twitter Sentiment Lexicon [33,31].

• Amazon Laptop Sentiment Lexicon [34].

Finding the proper combination of lexicons is not easy and plays a vital role in sentiment analysis. We
tested various lexicons and selected six of them as one of the best combination.

Algorithm 1 shows the process of our proposed method. It gets a sentence and returns improved word
vectors of the sentence. In the first step, assign a constant vector to each POS tag. In the second step,
each word vector of the input sentence is extracted from Word2Vec or GloVe datasets and if a word doesn’t
exist in the datasets its vector will generate randomly. In the third step, POS tag of each word is determined
and assign a constant vector to each one. In the next step, sentiment scores of each word are extracted
from all lexicons and will normalize them. If a word doesn’t exist in any lexicons, its score will be zero. The
generated vectors from each step will be concatenated with other vectors from previous steps.
 Algorithm 1 : Improved Word Vector (IWV) generation
Inputs:
S = {W1, W2,……., Wn} , Input sentence S contains n words
PT = {T1, T2,……, Tm}, All POS tags
W2VD: Google Word2Vec Dataset
GloVeD: Glove Dataset
h: Number of lexicons
LEX1, LEX2,……., LEXh All lexicons
Output:
IMV: Improved word vectors of sentence S

1. for j=1 to m do
2. VTj GenerateVector ( Tj )
3. Tj < Tj , VTj >
4. end for

5. for each Wi in S do
6. If Wi exist in W2VD then extract VecWi
7. MVi VecWi
8. elseif Wi exist in GloVeD then extract VecWi
9. MVi VecWi
10. else
11. MVi RandomGenVec (VecWi)
12. endif

13. POS ExtractPOS ( Wi )

14. for k=1 to m do
15. If POS=Tk then ADD VTk into MVi
16. end if
17. end for

18. for k=1 to h do

19. If Wi in LEXk then
20. Sik FindVector ( Wi )
21. Sik Normalize ( Sik ) Between -0.995 and +0.995
22. else
23. Sik 0
24. end if
25. ADD Sik into MVi
26. end for
27. ADD MVi into IMV
28. Return IMV
29. end for
We proposed a deep learning model (we called it Model 1) for evaluating our generated vectors on well-
known datasets. The model consists of three convolutional neural networks (CNN), a pooling, and a fully
connected layer and the inputs of the model are the improved word vectors (IWV). The model is shown in
figure 2.

Figure 2: The proposed model (Model1) for evaluating our proposed approach

In addition, for more confidence about the accuracy of our method, we checked the proposed method by
other three deep learning models from published papers. The second deep learning model (Model2) was
introduced by Kim [9]. The third model (Model3) was applied by Ouyang et al. [35] and the fourth one
(Model4) was proposed by Deriu et al. [36].

4. Experiments

In this section, we describe the datasets and experimental evaluations to show the effectiveness of our
proposed method.

4.1. Datasets

Datasets that were used in our research are as follows:

MR: Movie reviews dataset with equal positive and negative sentences and each review contains a
sentence [37].

CR: Customer reviews of products classified into positive and negative reviews [22].
 SST: The Stanford sentiment treebank introduced by Socher et al. [38] contains train/development/test
texts. We used only train and test sentences and binary labels (positive, negative) for our model.

4.2. Results

We have tested our approach on four different deep learning models and various sentiment datasets which
have different features. Our implementations were GPU-based and have trained our models on four
GeForce GTX Titan X GPUs. We have used Tensorflow for implementing and training all the deep learning
models in our research. All reports are based on the average of accuracies calculated over multiple runs of
10-fold cross-validation(CV), however SST has predefined train and test sets. In 10-fold CV, the dataset is
randomly partitioned into 10 folds. Among them, nine folds are used for training and the remaining one fold
is used for testing. We compared the results of the 356-dimension IWV with 300-dimension Word2Vec and
300-dimension GloVe.

As already mentioned before, six sentiment lexicons were used to extract and generate the lexicon vectors.
We only used unigram scores for our research. The distribution of the lexicons used in our research is listed
in table 2.

Lexicon Positive Negative Neutral Total Scores Ranges

NRC Emoticon Affirmative Context Lexicon 28025 27121 0 55146 -5.844 to +4.495
and NRC Emoticon Negated Context Lexicon
NRC Hashtag Affirmative Context Sentiment 19502 24447 0 43949 -10.025 to +10.661
Lexicon and NRC Hashtag Negated Context
Sentiment Lexicon
NRC Emoticon Lexicon 38312 24156 0 62468 -4.999 to +5.0
NRC Hashtag Sentiment Lexicon 32048 22081 0 54129 -6.925 to +7.526
SemEval-2015 English Twitter Sentiment 776 726 13 1515 -0.984 to +0.984
Lexicon
Amazon Laptop Sentiment Lexicon 14651 11926 0 26577 -5.27 to +3.702

Table 2: Statistics of the lexicons which were used in the research

In our proposed deep learning model the filter size of CNN1, CNN2 and CNN3, respectively are 3,4 and 5
with 100 feature maps each. The flatten layer has 95 nodes and the activation function of the layers is
Rectified Linear Unit (ReLU). The results are shown in table 3.
 Table 3: The accuracy (%) comparisons between our method (IWV) and other methods based on the deep learning
model 1

Method Dim MR CR SST

Word2Vec 300 79.3 81.8 82.0

Glove 300 79.2 81.3 81.0

IWV 356 79.8 82.5 83.7

As shown in table3, the accuracy of our combined vector is higher than the existing pre-trained vectors on
three sentiment datasets. In other word, the proposed method increased the accuracy of sentiment analysis
in our proposed deep learning model. The results show that the accuracy of SST, CR and MR were
increased by 1.7% ,0.7% and 0.5% respectively. Levy et al. [39] showed that Word2Vec performs better
than GloVe in various tasks. Also, we found that the Word2Vec is generally more accurate than the GloVe,
so all words are searched firstly on the Word2Vec dataset and then are searched on GloVe by the proposed
algorithm.

In order to test the IWV more fully, we checked our approach by other three deep learning models on MR
and CR datasets which are balanced and unbalanced. The results have shown in table 4.

Table 4: The accuracy (%) comparisons between our method and other methods based on models 2,3 and 4. Each
cell reports the average accuracies calculated over multiple runs of 10-fold CV.

Model Dataset Word2Vec(300) GloVe(300) IWV(356)

Model 2 MR 79.4 78.7 79.8
Kim [9] CR 82.8 82.9 83.2
Model 3 MR 78.0 77.3 78.2
Ouyang et al. [35] CR 80.0 79.5 81.1
Model 4 MR 79.4 78.4 79.6
Deriu et al. [36] CR 81.6 80.1 81.8

According to table 4, our proposed method is more accurate than other methods in models 2, 3 and 4. As
a result, the IWV was compared to other methods six times and all accuracies were improved.
 The MR dataset
80
79.5
79
Accuracy (%) 78.5
78
77.5
77
Model 1 Model 2 Model 3 Model 4

Glove Word2Vec IWV

Figure 3: The accuracy (%) comparisons of three methods on four deep learning models for MR dataset

Figure 3 shows the accuracy of Glove, Word2Vec and IWV vectors on four deep learning models for MR
which is a balanced dataset. It contains a total of 10662 reviews are divided into equal positive and negative
reviews. As can be seen, IWV method has the highest accuracy and the Glove has the lowest accuracy
among three methods. More generally, the IWV method has increased the accuracy of sentiment analysis
in MR dataset between 0.2% and 0.5%.

The CR dataset
83.5
83
82.5
82
Accuracy (%)

81.5
81
80.5
80
79.5
79
Model 1 Model 2 Model 3 Model 4

Glove Word2Vec IWV

Figure 4: The accuracy (%) comparisons of three methods on four deep learning models for CR dataset
Figure 4 indicates that the IWV method generally has performed better than other per-trained word
embeddings for sentiment analysis of CR dataset. The CR is an unbalanced dataset which contains 2397
positive and 1406 negative reviews. As can be seen, pre-trained Word2vec embedding is almost more
accurate than pre-trained Glove embedding, however it is reverse in the model 2. The IWV provides
absolute accuracy improvements of 0.7%, 0.4%, 1.1% and 0.2% for model 1, model 2, model 3 and model
4, respectively.

5. Conclusion
In this paper, we proposed a new method to improve the accuracy of well-known pre-trained word
embeddings for sentiment analysis. Our method has improved the accuracy of pre-trained word
embeddings based on the combination of three approaches such as lexicon-based approach, POS tagging
approach and Word2Vec/GloVe approach. In order to ensure about the accuracy of our proposed method,
we have tested it nine times with different deep learning models and sentiment datasets. The experimental
results indicated that our method has increased the accuracy of sentiment classification tasks in all models
and datasets. Briefly, the main advantages of the proposed method are:

• Because of the accuracy of pre-trained Word2Vec/Glove, adding any vector to them decreased the
accuracy according to previous researches, but our proposed method has increased the accuracy
of pre-trained vectors in sentiment analysis for the first time.

• One of the best combination of lexicons was introduced in our research. This combination
increased the accuracy of all tested deep learning models and datasets.

• Any improvements in pre-trained word embeddings/POS tagging/Lexicons in the future, will

increase the accuracy of our method.

As a result, our proposed method can be the basis for all sentiment analysis techniques which are used
deep learning approaches.
References

[1] T. Mikolov, I. Sutskever, K. Chen, G. Corrado, J. Dean, Distributed representations of words and phrases
and their compositionality, In: Proceedings of the 26th International Conference on Neural Information
Processing Systems. 2 (2013) 3111-3119.
[2] T. Mikolov, K. Chen, G. Corrado, J. Dean, Efficient estimation of word representations in vector space,
ICLR Workshop. (2013).
[3] J. Pennington, R. Socher, C. Manning, GloVe: Global Vectors for Word Representation, In: Proceedings
of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). (2014) 1532–
1543.
[4] D. Tang, F. Wei, N. Yang, M. Zhou, T. Liu, b. Qin, Learning Sentiment-Specific Word Embedding for
Twitter Sentiment Classification, In: Proceedings of the 52nd Annual Meeting of the Association for
Computational Linguistics. 1 (2014) 1555–1565.
[5] X. Fu, W. Liu, Y. Xu, L. Cui, Combine HowNet lexicon to train phrase recursive autoencoder for
sentence-level sentiment analysis, Neurocomputing. 241 (2017) 18-27.
[6] O. Araque, I. Corcuera-Platas, J. Sánchez-Rada, A. Iglesias, Enhancing deep learning sentiment
analysis with ensemble techniques in social applications, Expert Systems with Applications. 77 (2017) 236-
246.
[7] Y. Ren, R. Wang, D. Ji, A topic-enhanced word embedding for Twitter sentiment classification,
Information Sciences. 369 (2016) 188-198.
[8] M. Giatsoglou, M. Vozalis, K. Diamantaras, A. Vakali, G. Sarigiannidis, K. Chatzisavvas, Sentiment
analysis leveraging emotions and word embeddings, Expert Systems with Applications. 69 (2017) 214-224.
[9] Y. Kim, Convolutional Neural Networks for Sentence Classification, In: Proceedings of the 2014
Conference on Empirical Methods in Natural Language Processing (EMNLP). (2014) 1746-1751.
[10] Y. Zhang, B. Wallace, A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural
Networks for Sentence Classification, arXiv, 1510.03820v4. (2015)
[11] J. Camacho-Collados, M. Pilehvar, R. Navigli, Nasari: Integrating explicit knowledge and corpus
statistics for a multilingual representation of concepts and entities, Artificial Intelligence. 24 (2016) 36-64.
[12] A. Caliskan, J.J. Bryson, A. Narayanan, Semantics derived automatically from language corpora
contain human-like biases, Science. 356 (2017) 183-186.
[13] Y. Wang, M. Huang, L. Zhao, X. Zhu, Attention-based LSTM for Aspect-level Sentiment Classification,
In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. (2106)
606-615.
[14] M. Iyyer, V. Manjunatha, J. Boyd-Graber, H. Daume, Deep Unordered Composition Rivals Syntactic
Methods for Text Classification, In: Proceedings of the 53rd Annual Meeting of the Association for
Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 1
(2015) 1681-1691.
[15] Y. Liu, B. Liu, L. Shan, X. Wang, Modelling Context with Neural Networks for Recommending Idioms
in Essay Writing, Neurocomputing. (2017) doi:10.1016/j.neucom.2017.11.005
[16] M. Kamkarhaghighi, M. Makrehchi, Content Tree Word Embedding for document representation,
Expert Systems with Applications. 90 (2017) 241-249.
[17] C. Cerisara, P. Kral, L. Lenc, On the effects of using word2vec representations in neural networks for
dialogue act recognition, Computer Speech & Language. 47 (2018) 175-193.
[18] W. Medhat, H. Hassan, H. Korashy, Sentiment analysis algorithms and applications: A survey. Ain
Shams Engineering Journal. 5 (2014) 1093-1113.
[19] Z. Hailong, W. Gan, B. Jiang, Machine Learning and Lexicon based Methods for Sentiment
Classification: A Survey. In: Proceedings of the 2014 11th Web Information System and Application
Conference. (2014) 262-265.
[20] M. Taboada, J. Brooke, M. Tofiloski, K. Voll, M. Stede, Lexicon-based methods for sentiment
analysis, Computational Linguistics. 37 (2011) 267-307.
[21] X. Ding, B. Liu, P. Yu, A holistic lexicon-based approach to opinion mining, In: Proceedings of the 2008
international conference on web search and data mining. (2008) 231-240.
[22] M. Hu, B. Liu, Mining and summarizing customer reviews. In: Proceedings of the tenth ACM SIGKDD
international conference on Knowledge discovery and data mining. (2004) 168-177.
[23] K. Ravi, V. Ravi, A survey on opinion mining and sentiment analysis: Tasks, approaches and
applications, Knowledge-Based Systems. 89 (2015) 14-46.
[24] A. Mudinas, D. Zhang, M. Levene, Combining Lexicon and Learning based Approaches for Concept-
Level Sentiment Analysis, In: Proceedings of the First International Workshop on Issues of Sentiment
Discovery and Opinion Mining. 5 (2012).
[25] L. Zhang, R. Ghosh, M. Dekhil, M. Hsu, B. Liu, Combining Lexicon-based and Learning-based Methods
for Twitter Sentiment Analysis, Hewlett -Packard Development Company. (2011)

[26] A.S.H. Basari, B. Hussin, I.G.P. Ananta, J. Zeniarja, Opinion mining of movie review using hybrid
method of support vector machine and particle swarm optimization, Proc. Eng. 53 (2013) 453-462.
[27] A. Severyn, A. Moschitti, Twitter Sentiment Analysis with Deep Convolutional Neural Networks, In:
Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information
Retrieval. (2015) 959-962.
[28] P. Lauren, G. Qu, F. Zhang, A. Lendasse, Discriminant document embeddings with an extreme learning
machine for classifying clinical narratives, Neurocomputing. (2017) 1-10.
[29] P. Qin, W. Xu, J. Guo, An empirical convolutional neural network approach for semantic relation
classification, Neurocomputing.190 (2016) 1-9.
[30] S. Mohammad, S. Kiritchenko, X. Zhu, NRC-Canada: Building the State-of-the-Art in Sentiment
Analysis of Tweets, In: Proceedings of the seventh International Workshop on Semantic Evaluation
Exercises. (2013) 321-327.
[31] S. Kiritchenko, X. Zhu, S. Mohammad, Sentiment Analysis of Short Informal Texts, Journal of Artificial
Intelligence Research. 50 (2014) 723-762.
[32] X. Zhu, S. Kiritchenko, S. Mohammad, NRC-Canada-2014: Recent Improvements in Sentiment
Analysis of Tweets, In: Proceedings of the eight international workshop on Semantic Evaluation Exercises.
(2014)
[33] S. Rosenthal, P. Nakov, S. Kiritchenko, S. Mohammad, A. Ritter, V. Stoyanov, SemEval-2015 Task
10: Sentiment Analysis in Twitter, In: Proceedings of the ninth international workshop on Semantic
Evaluation Exercises. (2015) 451-463.
[34] S. Kiritchenko, X. Zhu, C. Cherry, S. Mohammad, NRC-Canada-2014: Detecting Aspects and
Sentiment in Customer Reviews. In: Proceedings of the eight international workshop on Semantic
Evaluation Exercises. (2014)
[35] X. Ouyang, P. Zhou, C. Li, L. Liu, Sentiment Analysis Using Convolutional Neural Network, IEEE
International Conference on Computer and Information Technology; Ubiquitous Computing and
Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing
(CIT/IUCC/DASC/PICOM). (2015)
[36] J. Deriu, A. Lucchi, V. Luca, T. Hofmann, M. Jaggi, Leveraging Large Amounts of Weakly Supervised
Data for Multi-Language Sentiment Classification, International World Wide Web Conference Committee
(IW3C2). (2017) 1045-1052.

[37] B. Pang, L. Lee, Seeing stars: Exploiting class relationships for sentiment categorization with respect
to rating scales, In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics.
(2005) 115-124.
[38] R. Socher, A. Perelygin, J.Wu, J. Chuang, C. Manning, A. Ng, C. Potts, Recursive Deep Models for
Semantic Compositionality Over a Sentiment Treebank, In: Proceedings of EMNLP. (2013)
[39] O. Levy, Y. Goldberg, I. Dagan, Improving distributional similarity with lessons learned from word
embeddings, Transactions of the Association for Computational Linguistics. 3 (2015) 211-225.