On smoothing and scaling language model for sentiment based information retrieval

Abstract

Sentiment analysis or opinion mining refers to the discovery of sentiment information within textual documents, tweets, or review posts. This field has emerged with the social media outgrowth which becomes of great interest for several applications such as marketing, tourism, and business. In this work, we approach Twitter sentiment analysis through a novel framework that addresses simultaneously the problems of text representation such as sparseness and high-dimensionality. We propose an information retrieval probabilistic model based on a new distribution namely the Smoothed Scaled Dirichlet distribution. We present a likelihood learning method for estimating the parameters of the distribution and we propose a feature generation from the information retrieval system. We apply the proposed approach Smoothed Scaled Relevance Model on four Twitter sentiment datasets: STD, STS-Gold, SemEval14, and SentiStrength. We evaluate the performance of the offered solution with a comparison against the baseline models and the related-works.

A Calculation of first order derivatives

$$\begin{aligned} \frac{\partial \mathcal {L}}{\partial \alpha _{v}}&=\sum _{i=1}^N \Big ( \log \beta _{v} -(\log (\alpha _{v}) +1) \nonumber \\&\quad + \log (w_{iv}^s) + \log \alpha _+ +1 - \log \left( \sum _{v=1}^D \beta _{v} w_{iv}^s \right) \end{aligned}$$

(20)

$$\begin{aligned} \frac{\partial \mathcal {L}}{\partial \alpha _{v}}&= 0\nonumber \\&\iff \end{aligned}$$

(21)

$$\begin{aligned} \sum _{i=1}^N \log \alpha _{v}&=\sum _{i=1}^N \Big ( \log \beta _{v} + \log (w_{iv}^s) - \log \Bigg (\sum _{v=1}^D \beta _{v} w_{iv}^s \Bigg ) \end{aligned}$$

(22)

$$\begin{aligned}&\hat{\alpha }_{v} = \sum _{i=1}^N \beta _{v} \frac{w^s_{iv}}{\sum _{v=1}^V \beta _{v} w^s_{iv}} \end{aligned}$$

(23)

$$\begin{aligned} \frac{\partial \mathcal {L}}{\partial \beta _{v}}&=\sum _{i=1}^N \Bigg ( \alpha _{v} \frac{1}{\beta _{v}} - \alpha _+ \frac{w_{vi}^s}{\sum _{v=1}^D \beta _{v} w_{iv}^s} \Bigg ) \end{aligned}$$

(24)

$$\begin{aligned} \frac{\partial \mathcal {L}}{\partial \beta _{v}}&= 0\nonumber \\&\iff \end{aligned}$$

(25)

$$\begin{aligned} \sum _{i=1}^N \frac{1}{\beta _{v}}&=\sum _{i=1}^N \Bigg ( \frac{\alpha _+}{\alpha _{v}} \frac{w_{vi}^s}{\sum _{v=1}^D \beta _{v} w_{iv}^s} \Bigg ) \end{aligned}$$

(26)

$$\begin{aligned}&\hat{\beta }_{v} = \sum _{i=1}^N \frac{\alpha _{v}}{\alpha _+} \frac{\sum _{v=1}^V \beta _{v} w^s_{iv}}{w^s_{iv}} \end{aligned}$$

(27)