International Journal of Information Management 48 (2019) 263–279

Contents lists available at ScienceDirect

International Journal of Information Management

journal homepage: www.elsevier.com/locate/ijinfomgt

Social media analytics: Extracting and visualizing Hilton hotel ratings T

and reviews from TripAdvisor
⁎
Yung-Chun Changa,b, Chih-Hao Kuc, , Chun-Hung Chend
a
Graduate Institute of Data Science, Taipei Medical University, Taipei, Taiwan
b
Clinical Big Data Research Center, Taipei Medical University Hospital, Taipei, Taiwan
c
College of Management, Lawrence Technological University, Southfield, USA
d
Institute of Information Science, Academia Sinica, Taipei, Taiwan

A R T I C L E I N F O A B S T R A C T

Keywords: Analyzing and extracting insights from user-generated data has become a topic of interest among businesses and
Sentiment analysis research groups because such data contains valuable information, e.g., consumers’ opinions, ratings, and re-
Hospitality commendations of products and services. However, the true value of social media data is rarely discovered due
Natural language processing to overloaded information. Existing literature in analyzing online hotel reviews mainly focuses on a single data
Social media analytics
resource, lexicon, and analysis method and rarely provides marketing insights and decision-making information
Visual analytics
to improve business’ service and quality of products. We propose an integrated framework which includes a data
Google trends
TripAdvisor crawler, data preprocessing, sentiment-sensitive tree construction, convolution tree kernel classification, aspect
extraction and category detection, and visual analytics to gain insights into hotel ratings and reviews. The
empirical findings show that our proposed approach outperforms baseline algorithms as well as well-known
sentiment classification methods, and achieves high precision (0.95) and recall (0.96). The visual analytics
results reveal that Business travelers tend to give lower ratings, while Couples tend to give higher ratings. In
general, users tend to rate lowest in July and highest in December. The Business travelers more frequently use
negative keywords, such as “rude,” “terrible,” “horrible,” “broken,” and “dirty,” to express their dissatisfied
emotions toward their hotel stays in July.

1. Introduction a manual analysis or survey is costly, and as a result, only data from
limited customers can be reached and leveraged. Therefore, an auto-
Nowadays, the ubiquitous Internet serves as a dominant channel for mated analysis of reviews can be fruitful for both business providers
information diffusion. Popular social media, such as Twitter and and their customers.
Facebook, as well as online commerce company Amazon, enable users Recent studies related to hotel review analysis mainly focused on an
to express their opinions toward a variety of products and services. This econometric approach (Jin, Karen, Ali, & Yong, 2017; Xie, Zhang, &
form of word-of-mouth (WOM) communication introduces a new and Zhang, 2014), a survey approach (Filieri, Alguezaui, & McLeay, 2015),
important source of information for business intelligence and marketing a statistical and modeling approach (Ya-Han & Kuanchin, 2016), and an
(Zhang, Li, & Chen, 2012). Specifically, not only are online user-gen- opinion mining approach (Eivind, Burnett, & Nørvåg, 2012; Liu, 2012).
erated contents (UGCs), such as hotel reviews, essential to other po- However, most of these studies either use overall ratings, hotel stars, or
tential customers, retailers, and product manufacturers, but also to the number of hotel reviews, as well as responses (Jin et al., 2017).
businesses owners as they reveal customers’ opinions towards their Another approach is to choose a few, selected cities (Ya-Han &
products and services (Giachanou & Crestani, 2016). Such information Kuanchin, 2016) for hotel review analysis due to the significant number
may have a significant impact on product sales such as online hotel of reviews available. A fine-grained text analysis such as an aspect-level
bookings (Zhao, Ye, & Zhu, 2016). Further, owners can employ such sentiment analysis together with rating analysis and information vi-
information to enhance the quality of their services and products, and sualizations can provide additional insights to the hotel review analysis.
even devise new marketing strategies (Rhee, Yang, & Kim, 2016). Un- Geetha, Singha, and Sinha (2017) point out that sentiment analysis
fortunately, the rapid growth of the amount of UGCs impedes us from has been mostly performed on Twitter and Amazon reviews to measure
obtaining a comprehensive view of these opinions. That is, carrying out customer sentiments, while a research study specialized in sentiment

⁎
Corresponding author at: College of Management, Lawrence Technological University, 21000 W 10 Mile Rd, Southfield, MI 48075, USA.
E-mail address: cku@ltu.edu (C.-H. Ku).

https://doi.org/10.1016/j.ijinfomgt.2017.11.001
Received 29 April 2017; Received in revised form 22 September 2017; Accepted 4 November 2017
Available online 14 November 2017
0268-4012/ © 2017 Elsevier Ltd. All rights reserved.
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

analysis of hotel reviews and its relations with customer ratings is still crawling tools. Opinion mining, sentiment analysis, topic modeling, and
lacking. Furthermore, most studies use sentiment analysis and search trend analysis are often used to uncover hidden gems in these data.
data separately (Hutchin, 2016). To the best of our knowledge, limited Different techniques such as network analysis, text and data mining, as
research pays attention to developing a heuristic framework which well as natural language processing (NLP) are frequently used in this
includes data collection, data preprocessing, aspect extraction, and data stage (Fan, Wallace, Rich, & Zhang, 2006). To present or make sense of
analysis, and then ultimately makes sense of the data with visualiza- large volumes of information, visualizations and visual analytics can
tions for marketing. To bridge this knowledge gap, we developed an support data exploration, discovery, and complex reasoning (Ribarsky,
integrated framework for conducting sentiment and aspect analysis, Xiaoyu Wang, & Dou, 2014). Statistical analysis, data mining, and
along with an experimental study. We then used this framework to machine learning are often used together with interactive visualizations
conduct an empirical study to collect and analyze hotel ratings and (Ribarsky et al., 2014). Recently, there is an increasing interest in social
reviews from TripAdvisor data and analyze Google Trends data with media analytics using crawlers, NLP, and machine learning models to
novel sentiment analysis and visual analytics. track trending topics for conducting sentiment analysis as well as opi-
This study contributes to both research on and practice in multi- nion mining about products and services (Fan & Gordon, 2014; Xiang
disciplinary domains such as tourism, information systems, and com- et al., 2017).
puter science. We present a novel aspect-sentiment analysis method
which concentrates on aspect detection for sentiment analysis. To de- 2.2. Internet search results (Google trends)
tect sentiment with aspects from online reviews effectively, we cast
aspect-sentiment analysis into a two-stage classification task. First, raw Search engines enable users to retrieve relevant information by
text reviews are transformed into a sentiment-sensitive tree structure typing keywords. Such search queries not only lead to a huge amount of
where syntactic, semantic, and contextual information are preserved. UGC but also provide valuable information about users’ interests and
The convolution tree kernel is then used to identify sentiment in online intentions. For example, the search volume and frequency data have
reviews. Once the sentiment of reviews is recognized, the aspects of been studied to measure perceived pollution and its relation to inbound
sentiment are further detected. We propose an aspect detection method tourism in China (Xu & Reed, 2017), to measure investors’ attention to
of sentiment in online reviews that employs six features covering syn- predict stock prices (Da, Engelberg, & Gao, 2011), to predict housing
tactic and contextual clues. market trends and outperform experts’ predictions from the National
Our experimental results validate that the proposed two-stage Association of Realtors for future U.S. home sales (Wu & Brynjolfsson,
method can successfully recognize sentiment and detecting aspects. In 2015), and to measure public concern over swine flu (Fenichel,
addition, the proposed sentiment sensitive tree structure effectively Kuminoff, & Chowell, 2013). Among these studies, Google Trends data
employs syntactic structures, sentiment semantics, and the content of is the most frequently used because of the popularity of the Google
these text segments. Meanwhile, the developed features for aspect de- search engine.
tection can capture the relation between sentiment and aspects. As a Google Trends1 enables users to explore ‘trending stories,’ a col-
result, the proposed method outperforms baseline models as well as lection of searching queries detected by Google’s algorithms, by cate-
LibShortText (Yu, Ho, Juan, & Lin, 2013) and LDA-SVM (Blei, Ng, & gories and locations. Google Trends are based on Google search results
Jordan, 2003), which are widely used to identify aspect-sentiment and present the search index of a particular keyword relative to the
analysis for online reviews. Finally, hidden patterns and relations from total search volume over a period by different categories and locations
the collected data can be discovered through visual analytics techni- (Bangwayo-Skeete & Skeete, 2015). Consumers have been increasingly
ques, such as timeline analysis and word clouds. using Google for travel plans such as searching destination hotels and
The rest of this paper is organized as follows. First, we provide a flight information (Bangwayo-Skeete & Skeete, 2015). Recently, Google
review of contemporary studies in social media analytics, internet Trends data has been studied to conduct tourism research. Li, Pan, Law,
search results, hospitality and tourism, and sentiment, as well as aspect- and Huang (2017) proposed a framework with search trend data to
based sentiment analysis. Next, we develop a scraping program to forecast tourism demand, and it achieved better performance than
collect Hilton hotel reviews and ratings from TripAdvisor. We then econometric models. Similarly, Bangwayo-Skeete and Skeete (2015)
propose a framework to conduct sentiment and aspects analysis with and Yang, Pan, and Song (2014) use Google Trends data to forecast
empirical experiments. Visual analytics techniques are applied to vi- tourism demand. To better understand consumer purchasing decisions
sualize TripAdvisor and Google Trends data to explore hidden in- for hotel bookings, Zhao et al. (2016) investigated the relationship
formation and relationships, which can be used for marketing or deci- between consumers’ hotel searching behavior and hotel booking data
sion-making purposes. Finally, we draw conclusions and posit future on Expedia.com. They found that the higher volume of search queries
research streams. for a hotel was associated with a higher room booking rate.
Fig. 1 shows a sample of query results for a recent five-year period,
2. Literature review 2012–2017, across three popular hotel brands, Hilton, Marriott, and
IHG, on Google Trends. Among them, the brand Hilton has been con-
2.1. Social media analytics sistently searched most frequently during this period, and the Hilton
and Marriott brands demonstrate similar user search patterns, such as
Social media websites enable users to generate and distribute in- peaks and downtime. The numbers of search interests are relative to the
formation and experiences by interactive Web 2.0 technologies and highest value in the U.S. during the sampled five-year period. For ex-
applications, e.g., Facebook, Twitter, TripAdvisor, etc. This has led to ample, a value of 100 represents the peak in June 2012.
an explosion of online UGCs, e.g., tweets, opinions, and reviews.
However, such massive amounts of unstructured and semi-structured
2.3. Hospitality, tourism and hotel reviews
data also give rise to challenges and opportunities for data analysis.
Social media analytics involves techniques to collect, extract, ana-
The hospitality and tourism industry is an ideal application for so-
lyze, and present user-generated data to support decision making, in-
cial media analytics (Lu & Stepchenkova, 2015) because consumers’
sight discovery, or other business-related operations (Holsapple, Hsiao,
opinions, recommendations, ratings, and behavioral data contain rich
& Pakath, 2014). Similarly, Fan and Gordon (2014) specify three im-
information for data analysis. Hotel review websites such as
portant stages: capture, understand, and present for social media ana-
lytics. During the capture stage, data are collected from various sources
1
through news feeds, application program interfaces (APIs), or data Google Trends, https://trends.google.com/trends/.

264
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 1. A sample of query results for Hilton, Marriott, and IHG in Google Trends.

Fig. 2. A Sample of Hotel Review and Ratings.

TripAdvisor include rich WOM information which may affect other involves different aspects of this facility. For example, TripAdvisor
consumers’ behaviors (Herrero, San Martín, & Hernández, 2015). For currently provides six aspects for users to review: sleep quality, loca-
example, Herrero et al. (2015) examine users’ perceptions regarding tion, room, service, value, and cleanliness. This type of online review
hotel reviews and show that users’ decision-making process is based on can be used by marketing researchers for gathering the public percep-
the perceived value of the given information, credibility of the source, tion of this location through sentiment analysis and topic categorization
and similarity between users and review generators. Additionally, (Huang, Peng, Li, & Lee, 2013).
consumers’ intention to book a room could be affected by Wi-Fi price
and reviews (Eriksson & Fagerstrøm, 2017). Customer views and ratings 2.4. Sentiment analysis
provide great business values to hotels (Xie et al., 2014) and positively
influence hotel bookings (Torres, Singh, & Robertson-Ring, 2015). The Sentiment analysis involves the process of extracting and classifying
hotel reviews include quantitative values (e.g., numerical or rating opinions expressed in digital documents or Web content, and then the
values) and qualitative evaluations (e.g., users’ opinions). Fig. 2 shows resulting data are analyzed against well-developed lexicons, a list of
three major components of a hotel review from TripAdvisor: overall words or phrases classified into positive, negative, and neutral
ratings, reviews, and aspect ratings. A traveler’s experience at an ac- (Aydoğan & Akcayol, 2016; Geetha et al., 2017). Recently, sentiment
commodation is measured by ratings on a scale of 1 (terrible) to 5 analysis has attracted increasing attention in academia (Zhang, Guo, &
(excellent), represented by green circles in Fig. 2. An overall rating Goes, 2013). For example, companies can analyze online comments

265
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 3. A Sample Hotel Review.

which reflect people’s sentiments or perceptions on movie reviews and In recent years, some research has been conducted based on Latent
foretell public interests (Singh, Piryani, Uddin, & Waila, 2013), con- Dirichlet Allocation (LDA) (Blei et al., 2003) to develop topic modelling
sumer reviews on products (Lipizzi, Iandoli, & Ramirez Marquez, 2015; methods for aspect-based sentiment analysis tasks. These approaches
Shelke, Deshpande, & Thakare, 2017), and purchase decision on mar- utilize variations of LDA to uncover latent topics in a dataset, expecting
keting (Lak & Turetken, 2014; Popescu & Etzioni, 2005). For tourism that these topics will correspond to ratable aspects of the entity under
research, Hutchin (2016) predicted future visitors to the UK and Ca- review. For instance, Brody and Elhadad (2010) develop a local topic
nada through sentiment analysis of collected hotel reviews and com- model for extracting aspects and determining sentiment in the review
ments from TripAdvisor and Facebook, respectively. To explore cus- text. This method works at the sentence level and uses a small number
tomer opinions on hotel services, Wu et al. (2010) developed an of topics to detect the aspects automatically. Zhao, Jiang, Yan, and Li
interactive visualization system that enables users to compare opinions (2010) present a maximum entropy hybrid model that can collabora-
of different groups. tively recognize aspect and opinion words. The hybrid model in-
In general, sentiment analysis contains three fundamental chal- corporates a discriminative maximum entropy model into a generative
lenges: aspect detection (Bhattacharjee & Petzold, 2016), opinion word topic model, and it leverages syntactic features to enhance the perfor-
detection, and sentiment orientation identification (Paltoglou & mance for recognizing aspects and opinion words.
Thelwall, 2012). An aspect (also called an opinion target) is an im- Existing methods primarily treat aspect-based sentiment analysis as
portant indicator for clarifying the topic of an opinion. Without this supervised learning problems. Given a training dataset containing a
information, determining whether the opinion is expressed on a specific collection of manually-tagged instances of aspect-sentiments, computer
service/amenity or an entire accommodation review lacks clarity. Fig. 3 scientists often use a supervised learning method to train a classifier for
displays a typical hotel review that contains the overall rating of a sentiment classification and further recognize aspects. Jakob and
variety of facets of the hotel, including check-in/out, location, and Gurevych (2010) view aspect extractions as a sequence labeling pro-
service. However, it would be difficult for decision makers to determine blem and employ conditional random fields (CRFs) (Lafferty,
the reviewer’s assessment of individual aspects. Depending on the McCallum, & Pereira, 2001) to recognize aspect expressions. They
nature of the traveler, the emphasis may be different. For instance, a combine dependency information of a text expression with syntactic
family may focus on the safety of the public area while a solo traveler features, such as part-of-speech (POS) tags, to detect and extract opi-
may prefer a high-quality bed. Knowing the distinctive aspect of each nion aspects. To identify the important aspects, Yu, Zha, Wang, and
review is imperative given that the overall score may be similar across Chua (2011) develop an algorithm for aspect ranking based on the
consumers’ reviews. Consequently, an increasing amount of work is aspect frequency simultaneously with the influence of consumers’ opi-
aimed at aspect-based sentiment analysis. nions towards each aspect of their overall opinions. The experiment
results indicate that the performance of sentiment classification tasks
can be improved through this approach on aspects recognition, and it
2.5. Aspect-based sentiment analysis can be further applied to the aspect ranking results. Liu, Xu, and Zhao
(2015) formulate the opinion aspect extraction as a graph-based co-
Various degrees of granularities of sentiment analysis can be con- ranking task. Their proposed method extracts candidates of the opinion
ducted depending on the scope of the data being a document, sentence, aspect and word through constructing a heterogeneous graph to model
entity or at aspect-level (Liu, 2012). Unfortunately, in a document- or semantic relations and opinion relations. Finally, each candidate is
sentence-level analysis, fine-grained opinions cannot be easily ob- given a confidence score which is then ranked by a co-ranking algo-
tained. However, an aspect-level analysis can be used to resolve this rithm. A threshold is used to filter out those candidates with lower
issue. For example, the sentence “…very convenient to O'Hare and decent scores, and the remaining candidates are considered to be the results.
service, but outside of that it's way overpriced for the rooms and level of Our method differs from existing aspect-sentiment analysis ap-
comfort…” is positive about the service, but negative about the room. proaches for two reasons. First, the proposed method can represent
That is, an opinion consists of a sentiment (positive or negative) and a syntactic, content, and semantic information in text segments of the
target of opinion. Both aspect extraction and sentiment analysis are review through the proposed sentiment sensitive tree structure.
required to improve the quality of services or products. NLP techniques Therefore, the method can effectively utilize the structured information
are often used together with aspect-based sentiment analysis. and constituent dependencies knowledge derived from sentences for
Different types of aspect-based sentiment analysis have been pro- sentiment classification tasks. Second, most approaches analyze the text
posed, including rule-based, topic model-based, and supervised of reviews for aspect detection. Our method further considers the
learning. For instance, Hu and Liu (2004) develop an association rule- properties of reviews (e.g., a customer's profile and travel type) to en-
based method to extract aspects from product reviews. To mine the hance the aspect detection performance.
associated rules, they extract frequent itemsets from noun phrases of
product reviews based on the Apriori algorithm (Han, Kamber, & Pei,
2011). However, this method extracts product features and opinion 3. Methodology
words, but neglects to connect product features at an aspect-level,
which renders it difficult to group the extracted terms into distinct We propose a method that automatically recognizes the sentiment
aspect categories. of online hotel reviews. Aspect categories are detected from the review

266
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 4. The system architecture.

texts to help readers comprehend these reviews. Fig. 4 shows the system
architecture of the proposed aspect-based sentiment analysis method, Table 1
The statistics of data corpus.
which is comprised of five key components that are described in the
following corresponding sections: data collection and processing (section # of positive reviews 500,316
3.1), sentiment sensitive tree (SST) construction (section 3.2), convolution # of negative reviews 133,961
tree kernel classification (section 3.3), aspect category detection (section # of aspects on Value 538,898
# of aspects on Location 524,583
3.4), and data visualization (section 3.5). We first gather hotel reviews
# of aspects on Sleep Quality 483,969
from TripAdvisor and further process data cleaning to ensure data # of aspects on Rooms 526,889
quality. Text syntactic information (e.g., parse tree) is useful to resolve # of aspects on Cleanliness 540,187
the relationships between entities; therefore, we propose the SST # of aspects on Service 629,871
structure, which depicts a path along the syntactic parse tree that # of aspects on Check-in/Front Desk 15,188
# of aspects on Business Service 10,289
consists of sentiment expressions. Also, the semantics in the content is
assessed to decorate the SST with polarity. We adopt the convolution
tree kernel (Collins & Duffy, 2002) to measure the similarity between most frequently searched keyword among all hotel brands. TripAdvisor
review texts regarding their SSTs. The tree kernel is incorporated into a was selected because it is a well-known travel company and provides an
support vector machine (SVM) (Joachims, 1998) to learn a classifier for enormous amount of user-generated content. By selecting a highly
each structural type, which detects sentiment in hotel reviews. The trafficked company like TripAdvisor, the likelihood of collecting rich
aspects behind the sentiment are then detected based on the properties and sufficient data for our analysis greatly increases.
(e.g., travel type, customer’s profile, and location) of reviews. Next, the In addition to the overall rating, reviewers also provided eight as-
data visualization provides an integrated, multi-feature analysis service pect ratings in each review: Value, Location, Sleep Quality, Rooms,
for data exploration and decision making. Specific implementation of Cleanliness, Service, Check-in/Front Desk, and Business Service. The rat-
sub-components in this model is introduced in the following sections. ings ranged from 1 to 5 stars, which can serve as ground-truth for
quantitative evaluations of our method. Customer profiles and proper-
3.1. Data collection and preprocessing ties of the hotels, such as Location, Highlight, and Amenities, were also
collected. Simple preprocessing was then performed on the review data:
The Hilton hotel was selected in this study for two reasons. First, it 1) define sentiment of reviews (i.e. a 1–3 rating star for negative and
is the most well-known and valuable hotel brand, with a value of US 4–5 rating star for positive); 2) remove reviews without any aspect
$7.8 billion in 2016, based on Brand Finance data.2 Secondly, we used label; 3) convert words into lower cases; 4) remove punctuations, stop
Google Trends to compare a variety of hotel brands, e.g., Hilton, Mar- words3 and words appearing less than 5 times in the corpus; and 5)
riott, Hyatt, Sheraton, Holiday Inn, etc., and found that “Hilton” is the stem each word to its root with Porter Stemmer (Porter, 1980). After

2 3
The World’s Most Valuable Hotel Brands, http://ftnnews.com/accommodation/ http://jmlr.csail.mit.edu/papers/volume5/lewis04a/a11-smart-stop-list/english.
30048-the-world-s-most-valuable-hotel-brands.html. stop.

267
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 5. The CT and SST decoration operation of the example segment.

Fig. 6. The extension operation of SST of the example hotel review.

these preprocessing steps, the dataset consists of 634,277 reviews and m = N(S) − k as the number of reviews containing S without word w,
5,456,307 sentences from 749 Hilton hotels in the U.S.; the details are and n = N(¬S) − l which means those with neither S nor w. A max-
illustrated in Table 1. The number of aspects on Check-in/Front Desk imum likelihood estimation is performed to obtain probabilities p(w), p
and Business Service is typically low. We will explore the reason in the (w|S), and p(w|^S). Finally, Eq. (1) can be applied to determine LLR
visual analytics section. values of every word in the corpus, where those with a higher LLR value
are believed to be strongly related to a certain sentiment. We then sort
3.2. SST construction LLR values of all words in the training data, and the top 300 candidates
are kept as a polarity keyword list. For each SST that holds a polarity
Next, we represent hotel reviews with the SST structure, which is a word, we attach a positive or negative sentiment tag (denoted as PST
constituent tree (CT) of review titles enhanced with three operations: and NST) as a child under the root to integrate sentimental semantic
decoration, extension, and pruning. Jiang, Zhang, Fu, Niu, and Yang information into the SST structure (as shown in Fig. 5(b)).
(2010) and Gamon (2004) had shown that CT of review titles is effec-
p (w S ) k (1 − p (w S ))mp (w ¬ S )l (1 − p (w ¬ S ))n ⎤
tive in sentiment classification because it can model the syntactic LLR (w, S ) = 2 log ⎡
structure of text that affects the sentiment classification performance. ⎢
⎣ p (w ) k + l (1 − p (w ))m + n ⎥
⎦
We exemplify the operators through the examples in Fig. 5 to further (1)
facilitate comprehension. Fig. 5(a) presents the syntactic parse tree of
the example review title (c.f. Fig. 3), while Fig. 5(b) demonstrates how To the best of our knowledge, the majority of existing tree kernel-
SST polishes CT with three operators that will be explained in the fol- based approaches working on the sentence level verified the usefulness
lowing paragraphs. of constituent tree structure of review title on sentiment classification;
The first operation we propose is SST Decoration. Intuitive in- however, the information within CT may not be sufficient for the cur-
dicators of sentiment may include verbs and adjectives, yet not all de- rent task. The review title “Business trip,” for instance, is too short to
liver clear sentimental expression. Therefore, we intend to find only signify any sentiment information. To include indicative sentiment
those that are strongly associated with a polarity of sentiment in an SST context, the extension operator extracts sentiment expressions from the
to improve the sentiment classification performance (denoted as po- review content. The constituent tree structure of the extracted sentence
larity words hereafter.) To achieve this, a log-likelihood ratio (LLR) is concatenated into the SST if it contains a sentiment lexicon.4 As
(Manning, Raghavan, & Schütze, 2008) is utilized to compile a list of shown in Fig. 6, the extended SST includes richer context information
polarity words representing each sentiment. The adopted definition of than the original after incorporating the context from three sentences.
LLR is given in Eq. (1), which calculates the log-likelihood of the hy- In this study, we adopt SVM for sentiment classification. SVM per-
pothesis that the presence of w in set S is beyond chance. Let w denote a forms classification on a vector space by estimating a boundary be-
word and S a type of sentiment. N(S) and N(¬S) are the number of tween data of different classes with maximum margin (Manning et al.,
reviews that do and do not contain sentiment S, respectively. N(w^S), 2008). The model itself is error-tolerant to a certain degree;
which is shortened as k, is the number of reviews containing w and S
simultaneously, while N(w^¬S) is the number of reviews including w 4
Here, we adopt a well-known sentiment lexicon compiled by Wilson, Wiebe, and
but not S, denoted as l. To further simplify the formula, we also define Hoffmann (2005).

268
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 7. The pruning operation of SST of the example segment.

nevertheless, we would like to improve the quality of the input data as structural similarity as follows:
much as possible. We noticed that SSTs may be comprised of redundant
K CTK (T1 T2) = φ (T1) φ (T2)
elements that can potentially undermine the effectiveness of sentiment
classification. Therefore, we performed pruning on the tree. The = ∑ #subtreek (T1)⋅#subtreek (T2)
k
pruning operator condensed SSTs through the following procedures.
First, even though the problem of insufficient sentiment information ⎛ ⎞⎛ ⎞
can be alleviated by including extra sentences from review content,
= ∑ ⎜ ∑ Isubtreek (n1) ⎟ ⎜ ∑ Isubtreek (n2) ⎟
k ⎝ n1∈ N1 ⎠ ⎝ n2 ∈ N2 ⎠
many irrelevant contexts are likely to be included in sentiment analysis.
To isolate contexts associated with the sentiment of review, we present = ∑ Δ (n1, n2)
n1∈ N1, n2 ∈ N2 (3)
sentiment contexts for each extended sentence as a sub-tree attached to
the lowest common parent of the sentiment lexicon. For instance, the where N1 and N2 are the sets of nodes in trees T1 and T2, respectively, and
sentiment context “resort amazing” is extracted from the extended Isubtreek (n) is a function of node n whose output is 1 if a subtreek exists under
sentence “The staff was all friendly and helpful and the resort itself was n, and zero otherwise. Consequently, the kernel KCTK assesses structural
amazing,” by the sub-tree rooted at “NP” node, which is the lowest similarity between two sentiment sensitive trees SST1 and SST2 by com-
common parent of nodes “resort” and “amazing.” Next, since frequent puting the amount of shared sub-trees, as in the following formula:
words are generally too common to be useful for sentiment dis-
crimination, the pruning operator removes punctuations and stop K CTK (SST1, SST2) = ∑ Δ (n1, n2),
n1∈ N1, n2 ∈ N2 (4)
words5; any word with less than five times of occurrence in the corpus
is also removed along with its corresponding elements in SST. where N1 and N2 are the sets of nodes in SST1 and SST2, respectively.
Second, an SST might contain duplicate nodes without siblings and Furthermore, Δ (n1, n2) denotes the common sub-trees under nodes n1 and
carry the same tag as their parents. These nodes may obstruct the n2. The recursive algorithm is:
computation of similarity in the tree kernel, since the similarity is de-
termined by the part where two trees overlap. To overcome this issue, (1) if n1 and n2 have different direct children nodes, Δ (n1, n2) = 0;
duplicate nodes are removed by the pruning operator to produce a ti- (2) if both n1 and n2 are pre-terminals (POS tags), Δ (n1, n2) = 1 × λ;
dier SST, as shown in Fig. 7. (3) otherwise, evaluate Δ (n1, n2) recursively as:
# ch (n1)
3.3. Convolution tree kernel classification Δ (n1, n2) = λ ∏ (1 + Δ (ch (n1, i),ch (n2, i)))
i=1 (5)
SVM can employ various kernels as the internal mechanism for com- where #ch(n1) is the number of children of node n1, ch(n,i) is the ith
puting the similarity (i.e., dot product) among pairs of samples projected child of node n, and λ a decaying parameter in the range of [0,1] for the
onto a different vector space. It is straightforward to adopt the convolution purpose of smoothing the output value among trees of variable sizes.
tree kernel (CTK) (Collins & Duffy, 2002) to resolve the similarity between Collins & Duffy (2002) stated that the time complexity of this kernel is
sentences under our representation scheme. CTK can model multi-level O ( N1 ⋅ N2 ) , which is dependents on the number of nodes in the two
structural information and is suitable for types of data such as parse trees. A input trees.
syntactic parse tree T can be denoted by a vector of integers:
ϕ (T ) = (#subtree1 (T ), ..., #subtreen (T ), ..., #subtreem (T )), (2) 3.4. Aspect category detection
th
where #subtreen(T) is the number of occurrences of the n type of sub-trees
Aspect category detection is a task that aims to detect aspect cate-
(subtreen) in T. However, the number of different sub-trees in T is ex-
gories mentioned in a piece of text. Typically, this can be considered as
ponential to the size of T. Thus, a direct calculation of the feature vector
a multi-label classification problem. Compared with the specific terms,
ϕ (T ) is not viable. To circumvent this problem, the CTK evaluates the
aspect categories are more generic and sometimes can only be im-
plicated by the surface terms. For instance, the review in Fig. 3 contains
5
http://jmlr.csail.mit.edu/papers/volume5/lewis04a/a11-smart-stop-list/english. three latent aspect categories, Check-in/out, Value, and Service. To
stop. resolve the differences between surface and latent aspects in text, we

269
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

domain (Sacha et al., 2014). It provides varying visual analyses such as

timeline analysis, location-based analysis, and dashboard analysis.
Additionally, it provides trendline, forecast, and cluster models for data
analytics. However, the collected raw data cannot be visualized di-
rectly. We first process the raw data to exclude data records with a
wrong year and incomplete locations. The raw data includes multi-va-
lues such as “|Value-2|Location-5|Sleep Quality-4|Rooms-3|Cleanliness-
1|Service-3|.” Therefore, we split and classify each aspect and extract
the numerical value for each aspect. Finally, we also transform each
hotel address into the corresponding geographical location. The data
visualization and exploration are then reported in Section 5: Visual
Analytics, Results and Discussion.

4. Experiments

4.1. Experimental settings

Fig. 8. The DAV model represents each hotel review Rk as aspect keyword vectors AKi We conducted an extensive evaluation of the performance of the
that are present in the review, weighted by scaling LLR scores λi. proposed system and other baseline methods. First, we implemented a
support vector regression (SVR) model that uses averaged word em-
present the Distributed Aspect Vectors (DAV) based on embeddings to beddings (with 300 dimensions) to represent each hotel review, de-
model the aspect-sentiment of hotel reviews. Compared with the Bag- noted as SVRWE. Next, we verified the effect of our keyword extraction
Of-Words (BOW) model, this distributed model of hotel reviews can approach by including a sentiment keyword-based model learned by
incorporate wider context information that covers the entire review SVM (denoted as SVMKW). Moreover, a probabilistic graphical model
into the representation. Moreover, semantic relations of various surface that exploits the LDA (Blei et al., 2003) as review representation was
words can also be captured due to the nature of a vector space model. employed to train an SVM to classify the hotel reviews, denoted as
These characteristics cannot be easily accomplished in a BOW-based SVMLDA. Included next was a logistic regression model from Lib-
approach, since it would consume a significant amount of storage for a ShortText (Yu et al., 2013) which was trained by review title, denoted
sizable n-gram dictionary. In addition, Chang, Chen, Hsieh, Chen, and as LST. We also included the Naïve Bayes method, denoted as NB
Hsu (2015) experimented with using keywords to classify emotions and (McCallum & Nigam, 1998), as the baseline. The implementation de-
demonstrated their effectiveness, which encouraged us to employ this tails of these methods are provided as follows. We employed the
approach in the aspect category detection task. Stanford parser7 to generate the output of parse trees and POS tagging.
At the outset, we use the Continuous Bag-of-Words (CBOW) method (Le The dictionary required by NB and SVMLDA was created by excluding
& Mikolov, 2014) to acquire vector representations of words in the corpus. words in the English stop word list mentioned in Section 3.2.1. As for
We then utilize LLR to extract keywords associated with the aspect cate- unobserved words, we used Laplace smoothing in NB, and an LDA
gories. Subsequently; a hotel review is jointly represented by aspect key- toolkit8 was employed in SVMLDA. The convolution kernel of an SST was
word embeddings. More specifically; weight λi for an aspect keyword; AKi; formed by the Moschitti’s tree kernel toolkit (Moschitti, 2004). The
is determined by its LLR value; and a hotel review; Rk; is represented by the distributed aspect keyword vector was integrated with vectors of review
weighted average of those embeddings; as depicted in Fig. 8. As illustrated properties for LibSVM9 to detect aspect categories. We adopted a 10-
in Fig. 9; if an aspect keyword is not present in a piece of review; we first fold cross-validation scheme. Evaluation metrics include the precision,
calculate the mean vector of all words and cosine similarity between the recall, F1-measure, and accuracy. Precision and recall are used to
mean and every keyword vector. Then; the keywords 1with a higher si- measure the positive predictive values and sensitivity, respectively, and
milarity are selected to represent this review. In essence; each hotel review F1-measure is a balanced mixture of precision and recall values.
is projected onto a point in the keyword vector space where various types
of classifiers can be employed. In addition to aspect keyword vectors; the 4.2. Performance evaluation and comparison
properties of hotel reviews are also represented through feature vectors
which are appended to the aspect keyword vectors; including customer’s The performances of sentiment classification systems are listed in
profile (i.e. TripCollective Progress and Badges); travel type; and time of Table 2. First, our approach outperforms all of the compared methods.
year. This is not surprising because the compared methods and feature sets
simply use syntactic features and cannot successfully uncover deep
semantics of reviewers’ sentiments in hotel reviews. In contrast, our
3.5. Visual analytics method incorporates semantic and context-dependent features, thus
achieving the best performance. As for other methods, the NB, which is
Visual analytics provides an interactive visual interface to reveal a statistical method with keywords as features, performs less than ideal.
hidden structures and details (Fan & Gordon, 2014). It combines ma- It shows that inter-word relations cannot be modelled by this method,
chine and human strength to process and explore large data and pro- since it only considers surface word weightings. The F1-measure of the
vides decision-support information (Sacha et al., 2014). The power of NB classifier is 0.77. SVRWE, on the other hand, overcomes the above
visual analytics comes from the machine’s capability to store and ana- weakness by utilizing a vector space regression model, which results in
lyze big data efficiently together with humans’ perceptive skills, cog- a 5% increase in F1-measure. Moreover, SVMKW greatly outperforms the
nitive reasoning, and domain knowledge. This paper aims to use visual methods above and achieves substantial proficiency in classifying the
analytics techniques to explore the extracted data, e.g., hotel ratings, sentiments with a 0.848 F1-measure. This indicates that keyword
aspect-sentiment, and types of travelers using Tableau. Tableau6 is a
representative of applications from the information visualization
7
http://nlp.stanford.edu/software/lex-parser.shtml.
8
http://nlp.stanford.edu/software/tmt/tmt-0.4/.
6 9
Tableau, https://www.tableau.com/. https://www.csie.ntu.edu.tw/∼cjlin/libsvm/.

270
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 9. DAV calculates the weighted average of nearest aspect keyword vectors when a review contains no keywords.

Table 2 identify aspects. These representative features can be optimally utilized

The sentiment classification results of the compared methods. by SVM or other vector-based classifiers. In addition, the proposed
method requires no feature engineering or human supervision, as the
System Precision Recall F1-measure Accuracy
weights of keywords are calculated automatically. We further con-
NB 0.846 0.711 0.772 0.711 ducted deeper analysis of the detection results and found that a large
SVRWE 0.843 0.768 0.788 0.768 number of reviews convey aspects on Value, Location, Sleep Quality,
SVMKW 0.852 0.844 0.848 0.844 Rooms, Cleanliness, and Service. We, therefore, can achieve high per-
SVMLDA 0.893 0.897 0.895 0.879
LST 0.895 0.899 0.897 0.889
formances on all of these aspects. However, a relatively lower number
Our method 0.954 0.967 0.935 0.893 of hotel reviews on certain aspects, Check-in/Front Desk and Business
Service, led to a lower precision and recall rate and less-than-ideal
system performance.
information ranked by LLR alone can be sufficiently effective in de- In sum, the proposed model effectively extracts the syntactic, con-
ciding the sentiment of a review. Notably, LST obtained an outstanding textual, and semantic information in hotel reviews. Moreover, irrele-
0.9 F1-measure since it further examines the content of the title through vant syntactic elements are eliminated, essentially reducing noise, so
an n-gram model. Hence, the LST method outperforms other baselines. that the informative context of those sentences that convey sentiments
To detect aspect of hotel reviews, word vectors are learned using can be efficiently learned. Therefore, our method outperforms the
CBOW (Le & Mikolov, 2014) with a hierarchical soft-max algorithm at compared approaches and achieves an encouraging sentiment classifi-
the outset of the training stage. The dimensionality is set to 300. In the cation performance. Moreover, the distributed aspect vector is effective
meantime, for every word in the corpus, we calculate its LLR values. for aspect determination of hotel reviews since it incorporates aspect
According to Chang et al. (2015), 100 keywords can be useful for ca- keywords vectors with property vectors. Hence, our method obtains
tegorization of texts. Thus, we retain only this amount of words after high performance in detecting aspects of hotel reviews.
sorting by LLR as keywords for each sentiment. Next, we transform each
hotel review into the weighted sum of keyword vectors, where the
5. Visual analytics, results and discussion
weights are determined by its scaled LLR values. We adopt LibSVM
(Chang & Lin, 2011)with the radial basis function (RBF) kernel and
5.1. Google trends exploration
multi-label output format for the classification of aspects. More speci-
fically, the multi-label mode operates by building a binary classification
The data collected from TripAdvisor included overall ratings, aspect
model for each label of an instance, and each instance may carry
ratings, and Hilton hotel reviews in the U.S. We first explore the search
multiple labels. Table 3 shows the precision, recall, F1-measure, and
interest in Hilton and TripAdvisor by interest over time, interest by sub-
accuracy for detecting the aspects mentioned in over six hundred
region (states, metros, and cities), and related topics as well as queries.
thousand hotel reviews.
The query results are then exported to .csv files for further analysis in
The results in Table 3 show that DAV can successfully detect aspects
Tableau. The query results are accumulated from January 2004 to
contained in hotel reviews. It indicates that aspect keywords and
March 2017.
properties in reviews are valuable clues when identifying aspects
A timeline analysis and dashboard (see Fig. 10) are used to compare
therein. It also demonstrates that dense vector representation, such as
the average ratings (top panel), the number of reviews (middle panel),
embeddings, can capture latent associations between aspects and key-
and the relative value of “Hilton” in Google Trends (bottom panel) from
words. Furthermore, using LLR as a weighting mechanism can posi-
January 2004 to March 2017. A red-blue diverging color and date an-
tively affect the discriminating capability of the classifier to correctly
notations are used to facilitate data exploration and comparison. A red
and blue color denotes lower and higher scores, respectively. The rating
scores on TripAdvisor range from 1 (a lower rating) to 5 (a higher
Table 3
The performance of DAV on aspect classification.
rating), and the maximum search interest (index) is 100 on Google
Trends. The average ratings and the number of hotel reviews have in-
Aspect Precision Recall F1-measure Accuracy creased, while there is a decreasing trend of searching “Hilton” since
January 2004. The peak time of searching “Hilton” usually occurs in
Value 0.850 0.997 0.918 0.848
Location 0.830 0.996 0.904 0.828
June or July, and the valley of the curve occurs in December of each
Sleep Quality 0.772 0.978 0.863 0.762 year. Additionally, there is an increasing trend of hotel reviews gen-
Rooms 0.832 0.996 0.907 0.829 erated in July and a decreasing trend in December and January of each
Cleanliness 0.851 0.998 0.919 0.849 year. A couple of interesting observations are that the average rating
Service 0.993 0.921 0.956 0.993
score in July is usually the lowest and the highest rating score often
Check-in/Front Desk 0.490 0.465 0.478 0.976
Business Service 0.469 0.432 0.450 0.983 occurs in December and January. This indicates the more users search
hotel information using Google, the more hotel reviews will be

271
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 10. Timeline analysis on average review ratings

(top), the number of reviews (middle), and Google
Trends indexes (bottom).

generated in TripAdvisor later; however, a lower rating score can be “Hilton hotels,” followed by the District of Columbia, Maryland, and
expected. Florida. For a city-level result (see Fig. 11), Honolulu was the most
To better understand the search interest of a “Hilton hotel” by popular city to search “Hilton hotel,” followed by Washington, Orlando,
various locations during the sample period in Google Trends, we ex- Nashville, and San Diego.
amine the results by sub-region (state) and city level. The searching We examine an equally interesting question; what keywords (topics)
results reveal that Hawaii was the most popular location to search do users usually search together with a “Hilton hotel” query? The results

272
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 11. Search interest of “Hilton Hotel” by cities in Google Trends.

Fig. 12. Search interest of Hilton, Marriott, Hyatt, and Holiday Inn in Google Trends.

Fig. 13. Timeline analysis on aspects of Hilton hotels.

273
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 14. Geographical analysis of travel types and rating scores.

Fig. 15. Geographical analysis of specific hotels on aspect ratings.

show that users also search for Hilton’s major competitors, such as results. Referring back to Table 1, we notice that the number of aspects
Marriott, Hyatt, and Holiday Inn. A comparative analysis reveals that on Check-in/Front Desk and Business Service are especially low, but the
Hilton is still the most popular brand during the sample period (see reason is unknown. Several observations from this timeline analysis are
Fig. 12), which is consistent with the brand value report conducted by identified. First, the aspect ratings Business Service and Check-in/Front
Brand Finance.10 Other related topics include Airbnb, Expedia, Tri- Desk were removed in 2013. More precisely, only sparse ratings were
pAdvisor, Trivago, Google Flights, Yelp, etc. This real-time trend may identified on both ratings after ten years. Secondly, the Sleep Quality
be useful for exploring future opportunities and generating marketing rating has been introduced since 2007, and we notice a few low ratings
value. For example, if many users are inclined to search TripAdvisor in 2007, 2008, and 2009. After 2009, no notable change in rating scores
and Yelp together, a hotel manager may consider establishing a part- occurred.
nership with Yelp. Today, social media monitoring is important for To gain deeper insights from the data, we developed an interactive
brand management (Rout & Bontcheva, 2015). Further, the search vo- geographical visualization (see Fig. 14). Each pie chart in the map re-
lume of TripAdvisor on Google Trends can be a good indicator of hotel fers to one or more hotel locations. If two Hilton hotels are very close,
managers’ response strategies (Lee, Xie, & Besharat, 2016). the result will be combined in a single pie chart. A larger pie chart
indicates a higher number of hotel reviews, while the five distinct colors
5.2. Visual analytics using tableau and word cloud represent five different travel types: Business, Couples, Families,
Friends, and Solo travelers. We then filter time and review ratings to
A timeline analysis is conducted for all average ratings of eight as- focus on the important locations. Overall, Orlando has the most hotel
pects and review star ratings. We filtered the noise, such as check-in views, followed by California, New York, and Texas. When a single city
time earlier than 2014, because of limited ratings (less than ten ratings is selected, the Hilton hotel in Oahu, HI, received the most reviews
in a month) and wrong review years, such as 1899. After data filtering, (9087) between January 2004 and March 2017, followed by the Hilton
the total number of data records is 633,866. Fig. 13 shows the overall hotel in Chicago, IL (6458 reviews), New York City (6210 reviews), and
Waikoloa, HI (5288 reviews).
The pie charts on the map enable users to explore types of travelers
10
The World’s Most Valuable Hotel Brands, http://ftnnews.com/accommodation/
that frequent a hotel and provides an overview of the average ratings
30048-the-world-s-most-valuable-hotel-brands.html.

274
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 16. Timeline analysis of the number of reviews for each travel type.

Fig. 17. Heatmap analysis of type of travelers and aspects.

for each aspect, e.g., Location, Sleep Quality, Room, Service, opportunities for each hotel.
Cleanliness, Check-in/Front Desk, Business Service (Internet), Average To know the total number of hotel reviews for each travel type every
Star Rating, and Value. For example, Fig. 15 (left) reveals that Families year, we generated a line chart (see Fig. 16). The majority of reviews
and Couples are two major types of travelers to Honolulu, and the came from Business, Families, and Couples. The total number of re-
overall rating value is relatively lower compared to some of the aspect views has increased dramatically since 2010. More specifically, Busi-
ratings. Timothy and Sung-Byung (2015) investigated and compared six ness and Families leave hotel reviews most frequently. For example, the
hotel aspects: value, location, sleep quality, rooms, cleanliness, and total number of reviews generated by Families was 48,254 in 2016,
service and found that guests place different importance on each aspect. followed by Business (45,386) and Couples (36,000). Therefore, the
This further indicates that overall rating cannot be directly used to overall rating scores may be strongly influenced by these three types of
enhance the quality of service. Therefore, it is important to know the travelers. To positively influence overall rating scores, knowing the
ratings on each aspect. Fig. 15 (right) indicates that more than half of aspect rating for each travel type is also very important.
travelers are Business travelers at a Hilton hotel in Texas and Family The relationship between each type of travelers and aspects is ex-
travelers at a Hilton hotel in Orlando. This is not surprising because plored next. Since the aspects of Check-in/Front Desk and Business
Orlando is a well-known city for theme parks, which is a popular des- Service are no longer available, so we exclude both aspects from this
tination for family trips (Ya-Han & Kuanchin, 2016). Compared to ho- analysis to avoid extreme values. A table with the red-blue-white di-
tels in Orlando, the hotels in Texas received a relatively lower rating verging palette (see Fig. 17) was used to indicate higher (blue) and
(less than 4 out of 5) on the Average Star rating, Value, and Location. lower (red) values. Overall, travelers perceive a relatively lower value
By performing similar analyses, hotel managers have the opportunity to in Value and a higher value in Location. For each type of traveler, for
identify the types of travelers in each hotel and examine the ratings for example, business people pay more attention to Room and Sleep
each aspect in order to enhance services and explore marketing Quality. Similar findings were reported by Timothy and Sung-Byung

275
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 18. Timeline analysis of the average review star ratings and each travel type.

(2015); they found that the Room was the most important aspect when mostly associated with the facilities of a room. For instance, two evi-
hotels received lower ratings from guests. Also, Torres et al. (2015) dent words “A/C” and “carpet” may relate to negative sentiments, e.g.,
found that TripAdvisor’s overall ratings and the number of reviews are “broken” and “dirty” towards the air conditioner and carpet. A deeper
related to the average value of hotel transactions and the popularity of a text analysis indicates a considerable number of business travelers’
hotel. However, this study did not take different travel types into ac- complaints are related to the noise of air conditioner and dirty carpets.
count. When we look at the average review ratings for the sample Levy, Duan, and Boo (2013) developed a complaint framework and
period (see Fig. 18), Business travelers are inclined to rate lowest, fol- found that front desk service, room cleanliness, bathroom issues, and
lowed by Families and Couples, while Solo travelers generally tend to guestroom noise were frequent complaints. Their findings are similar to
rate relatively higher. our word cloud visualization. Our word cloud further reveals the fre-
Finally, a time series analysis (see Fig. 19) using line charts was quency of those complaints. It is worth noting that the overcrowding
conducted to explore high and low rating values based on travelers’ frequently caused by tourists in July may lead to longer check-in lines,
check-in month. We found a similar pattern for each aspect. A higher which may explain the diminished front-desk effectiveness and re-
rating was identified in January, May, September, and December, while sponsiveness that often causes business travelers’ discontent. Conse-
a lower rating value was found in July and October. Travelers usually quently, the co-occurrence of terms, e.g. desk and rude, may indicate
leave a higher rating on Location and Cleanliness, followed by Room, the negative sentiment towards a service. Fig. 20 also reveals a high
Service, and Sleep Quality. Value was clearly the lowest rated aspect, correlation between the overall polarity of a hotel review and important
which may surprise some marketers. aspect-specific keywords. It has been suggested that hotel managers
A more comprehensive analysis of Fig. 19 reveals that a greater should embrace social networks, monitor UGC, and manage online re-
portion of positive reviews was posted by couples on the Location as- putation actively (Levy et al., 2013; O’Connor, 2010). Decision makers
pect in December. In contrast, most negative reviews were made by can refer to this analysis to support and augment their marketing
Business travelers on the Room aspect in July. To gain a deeper un- strategies and activities. For instance, a hotel can establish a partner-
derstanding of reviews, we use word clouds (shown in Fig. 20) to vi- ship with stores to provide an exclusive discount to attract more couple
sualize the aspect-sentiment keywords, in which colors correspond to travelers. Moreover, hotel managers should strive to resolve noise
aspect-sentiment categories and sizes are scaled by LLR. In doing so, problems stemming from air conditioners to minimize business custo-
specific aspect-sentiment can be clearly associated with their descrip- mers’ complaints.
tions. Results indicate that green (positive) words are in accordance
with features, e.g., “shopping,” “breakfast,” and “restaurants.” This
indicates that couples pay much more attention to the convenience of 6. Conclusion and future research
nearby shopping locations during their stay. Moreover, couples inclined
to give positive ratings for restaurants in a hotel, especially those that This paper sheds light on a more inclusive analysis and visualization
serve breakfast. On the contrary, negative-related keywords (in red) are of reviews from TripAdvisor and data on Google Trends. This study
reveals the intricacies of users’ ratings, sentiments, and types of

276
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Fig. 19. Timeline analysis of high and low ratings for each aspect.

travelers at different times and locations. It also indicates that a sole framework to collect and process data, extract and classify aspect-level
source of data, lexicon, and method is not sufficient to discover the information, and finally visualize TripAdvisor and Google Trends data.
genuine value of hotel ratings and reviews. Furthermore, existing re- The proposed aspect-level approach, together with text syntactic in-
search is mostly limited to document-level and sentence-level analysis. formation and the CTK, can be justified by its high precision and recall.
Our major contribution to this study includes a proposed, integrated Our visual analytics provides multiple perspectives by using timeline

Fig. 20. An aspect-sentiment keyword cloud. Colors and their corre-

sponding sentiment: Green—Positive sentiment of Couple travel type on
the aspect of Location in December. Red—Negative sentiment of Business
travel type on the aspect of Room in July.

277
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

and location-based analyses. The experimental results demonstrate that Herrero, Á., San Martín, H., & Hernández, J. M. (2015). How online search behavior is
our proposed method outperforms the baseline models. The analyses influenced by user-generated content on review websites and hotel interactive
websites. International Journal of Contemporary Hospitality Management, 27,
results can be used to improve the service of each hotel and discover 1573–1597.
marketing opportunities. Holsapple, C., Hsiao, S.-H., & Pakath, R. (2014). Business social media analytics:
This study is not without limitations, and additional research Definition, benefits, and challenges. AMCIS 2014 proceedings.
Hu, M., & Liu, B. (2004). Mining and summarizing customer reviews. KDD ‘04 proceedings
streams are identified. First, multiple data resources such as of the 10th ACM SIGKDD international conference on knowledge discovery and data
Expedia.com, Hotels.com, and Google Reviews can be used in the fu- mining (pp. 168–177).
ture, even though no clear, direct evidence suggests that differences of Huang, S., Peng, W., Li, J., & Lee, D. (2013). Sentiment and topic analysis on social media:
A multi-task multi-label classification approach. WebSci ‘13 proceedings of the 5th
reviews exist across those websites. Secondly, the use of mobile devices annual ACM web science conference (pp. 172–181).
for leaving reviews can be incorporated into studies to determine Hutchin, E. (2016). How Can Google Trends and Sentiment Analysis of Tripadvisor and
whether device type affects reviews and whether “in the moment” Facebook Predict Visitor Numbers to the United Kingdom and Canada from 2010 to 2015.
Dublin: National College of Ireland [Unpublished Masters].
marketing campaigns can overcome reviews and ratings. Mobile de-
Jakob, N., & Gurevych, I. (2010). Extracting opinion targets in a single- and cross-domain
vices are increasingly being used for marketing.11 Red Roof Inn, for setting with conditional random fields. EMNLP ‘10 proceedings of the 2010 conference
instance, turns flight cancellations into marketing campaigns for new on empirical methods in natural language processing (pp. 1035–1045).
customers12 using a mobile search strategy. Finally, growing concerns Jiang, P., Zhang, C., Fu, F., Niu, Z., & Yang, Q. (2010). An approach based on tree kernels
for opinion mining of online product reviews. 2010 IEEE international conference on
have surfaced about fake and paid online reviews (Filieri et al., 2015); data mining (pp. 256–265).
therefore, the ratings and reviews may have been manipulated. Future Jin, L. Y., Karen, X., Ali, B., & Yong, T. (2017). Management responses to online WOM:
research may benefit from investigating further reviewer profiles, pre- Helpful or detrimental? SSRN Electronic Journal, 44.
Joachims, T. (1998). Text categorization with support vector machines: Learning with
vious reviews, and writing style. many relevant features. ECML ‘98 proceedings of the 10th european conference on
machine learning (pp. 137–142).
References Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001). Conditional random fields:
Probabilistic models for segmenting and labeling sequence data. ICML ‘01 proceedings
of the eighteenth international conference on machine learning (pp. 282–289).
Aydoğan, E., & Akcayol, M. A. (2016). A comprehensive survey for sentiment analysis Lak, P., & Turetken, O. (2014). Star ratings versus sentiment analysis ?A comparison of
tasks using machine learning techniques. 2016 international symposium on innovations explicit and implicit measures of opinions. 2014 47th hawaii international conference
in intelligent systems and applications (INISTA), 1–7. on system sciences (pp. 796–805).
F.Bangwayo-Skeete, P., & W.Skeete, R. (2015). Can google data improve the forecasting Le, Q., & Mikolov, T. (2014). Distributed representations of sentences and documents.
performance of tourist arrivals? Mixed-data sampling approach. Tourism Management, ICML'14 proceedings of the 31 st international conference on international conference on
46, 454–464. machine learning: 32, (pp. 1188–1196).
Bhattacharjee, K., & Petzold, L. (2016). What drives consumer choices? Mining aspects Lee, Y., Xie, K., & Besharat, A. (2016). Management response to online WOM: Helpful or
and opinions on large scale review data using distributed representation of words. detrimental? Amcis 2016 (pp. 1–10).
2016 IEEE 16th international conference on data mining workshops (ICDMW), 908–915. Levy, S. E., Duan, W., & Boo, S. (2013). An analysis of one-star online reviews and re-
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. The Journal of sponses in the Washington, D.C., lodging market. Cornell Hospitality Quarterly, 54,
Machine Learning Research, 3, 993–1022. 49–63.
Brody, S., & Elhadad, N. (2010). An unsupervised aspect-sentiment model for online re- Li, X., Pan, B., Law, R., & Huang, X. (2017). Forecasting tourism demand with composite
views. HLT ‘10 human language technologies: The 2010 annual conference of the north search index. Tourism Management, 59, 57–66.
american chapter of the association for computational linguistics (pp. 804–812). Lipizzi, C., Iandoli, L., & Ramirez Marquez, J. E. (2015). Extracting and evaluating con-
Chang, C. C., & Lin, C. J. (2011). LIBSVM: A library for support vector machines. ACM versational patterns in social media: A socio-semantic analysis of customers’ reactions
Transactions on Intelligent Systems and Technology, 2 27:21-27:27. to the launch of new products using twitter streams. International Journal of
Chang, Y. C., Chen, C. C., Hsieh, Y.-L., Chen, C. C., & Hsu, W. L. (2015). Linguistic Information Management, 35, 490–503.
template extraction for recognizing reader-Emotion and emotional resonance writing Liu, K., Xu, L., & Zhao, J. (2015). Co-extracting opinion targets and opinion words from
assistance. Proceedings of the 53rd annual meeting of the association for computational online reviews based on the word alignment model. IEEE Transactions on Knowledge
linguistics and the 7th international joint conference on natural language processing (pp. and Data Engineering, 27, 636–650.
775–780). Liu, B. (2012). Sentiment analysis and opinion mining. San Rafael: Morgan & Claypool
Collins, M., & Duffy, N. (2002). Convolution kernels for natural language. In T. G. Publishers.
Dietterich, S. Becker, & Z. Ghahramani (Vol. Eds.), Advances in neural information Lu, W., & Stepchenkova, S. (2015). User-generated content as a research mode in tourism
processing systems: vol. 14, (pp. 625–632). MIT Press. and hospitality applications: Topics, methods, and software. Journal of Hospitality
Da, Z., Engelberg, J., & Gao, P. (2011). In search of attention. The Journal of Finance, 66, Marketing & Management, 24, 119–154.
1461–1499. Manning, C. D., Raghavan, P., & Schütze, H. (2008). Introduction to information retrieval
Eivind, B., Burnett, T. H., & Nørvåg, K. (2012). A study of opinion mining and visualization (1st ed.). New York: Cambridge University Press.
of hotel reviews. 229–238: ACM. McCallum, A., & Nigam, K. (1998). A comparison of event models for naive bayes text
Eriksson, N., & Fagerstrøm, A. (2017). The relative impact of wi-fi service on young classification. Learning for text categorization: Papers from the 1998 AAAI workshop (pp.
consumers’ hotel booking online. Journal of Hospitality & Tourism Research 41–48).
1096348017696844. Moschitti, A. (2004). A study on convolution kernels for shallow semantic parsing. ACL
Fan, W., & Gordon, M. D. (2014). The power of social media analytics. Communications of the ‘04 proceedings of the 42nd annual meeting on association for computational linguistics
ACM, Vol. 57, New York, NY: ACM74–81. (pp. 8). .
Fan, W., Wallace, L., Rich, S., & Zhang, Z. (2006). Tapping the power of text mining. O’Connor, P. (2010). Managing a hotel's image on TripAdvisor. Journal of Hospitality
Communications of the ACM, Vol. 49, New York, NY: ACM76–82. Marketing & Management, 19, 754–772.
Fenichel, E. P., Kuminoff, N. V., & Chowell, G. (2013). Skip the trip: Air travelers' be- Paltoglou, G., & Thelwall, M. (2012). Twitter, MySpace, Digg: Unsupervised sentiment
havioral responses to pandemic influenza. PUBLIC LIBRARY OF SCIENCE, 8, e58249. analysis in social media. ACM Transactions on Intelligent Systems and Technology, 3
Filieri, R., Alguezaui, S., & McLeay, F. (2015). Why do travelers trust TripAdvisor? 66:61-66:19.
Antecedents of trust towards consumer-generated media and its influence on re- Popescu, A.-M., & Etzioni, O. (2005). Extracting product features and opinions from re-
commendation adoption and word of mouth. Tourism Management, 51, 174–185. views. HLT ‘05 proceedings of the conference on human language technology and em-
Gamon, M. (2004). Sentiment classification on customer feedback data: Noisy data, large pirical methods in natural language processing (pp. 339–346).
feature vectors, and the role of linguistic analysis. COLING ‘04 proceedings of the 20th Porter, M. F. (1980). An algorithm for suffix stripping. Program, 14, 130–137.
international conference on computational linguistics. Rhee, H. T., Yang, S.-B., & Kim, K. (2016). Exploring the comparative salience of res-
Geetha, M., Singha, P., & Sinha, S. (2017). Relationship between customer sentiment and taurant attributes: A conjoint analysis approach. International Journal of Information
online customer ratings for hotels − an empirical analysis. Tourism Management, 61, Management, 36, 1360–1370.
43–54. Ribarsky, W., Xiaoyu Wang, D., & Dou, W. (2014). Social media analytics for competitive
Giachanou, A., & Crestani, F. (2016). Like it or not: A survey of twitter sentiment analysis advantage. Computers & Graphics, 38, 328–331.
methods. ACM Computing Surveys, 49 28:21-28:41. Rout, D., & Bontcheva, K. (2015). A human-annotated dataset for evaluating tweet
Han, J., Kamber, M., & Pei, J. (2011). Data mining: Concepts and techniques (3rd ed.). ranking algorithms. HT ‘15 proceedings of the 26th ACM conference on hypertext &
Morgan Kaufmann. social media (pp. 95–99).
Sacha, D., Stoffel, A., Stoffel, F., Kwon, B. C., Ellis, G., & Keim, D. A. (2014). Knowledge
generation model for visual analytics. IEEE Transactions on Visualization and Computer
Graphics, 20, 1604–1613.
11
Mobile marketing strategies for hotels, http://www.traveltripper.com/blog/four- Shelke, N., Deshpande, S., & Thakare, V. (2017). Domain independent approach for as-
important-mobile-marketing-strategies-for-hotels/. pect oriented sentiment analysis for product reviews. Proceedings of the 5th interna-
12
Red Roof Inn Case, http://www.mmaglobal.com/case-study-hub/case_studies/ tional conference on frontiers in intelligent computing: Theory and applications (pp. 651–
view/31739. 659).

278
Y.-C. Chang et al. International Journal of Information Management 48 (2019) 263–279

Singh, V. K., Piryani, R., Uddin, A., & Waila, P. (2013). Sentiment analysis of movie Management Perspectives, 21, 109–112.
reviews: A new feature-Based heuristic for aspect-Level sentiment classification. 2013 Ya-Han, H., & Kuanchin, C. (2016). Predicting hotel review helpfulness: The impact of
international mutli-conference on automation, computing, communication, control and review visibility, and interaction between hotel stars and review ratings. International
compressed sensing (iMac4s) (pp. 712–717). Journal of Information Management, 36, 929–944.
Timothy, R. H., & Sung-Byung, Y. (2015). Does hotel attribute importance differ by hotel? Yang, Y., Pan, B., & Song, H. (2014). Predicting hotel demand using destination mar-
Focusing on hotel star-classifications and customers’ overall ratings. Computers in keting organization’s web traffic data. Journal of Travel Research, 53, 433–447.
Human Behavior, 50, 576–587. Yu, J., Zha, Z.-J., Wang, M., & Chua, T.-S. (2011). Aspect ranking: Identifying important
Torres, E. N., Singh, D., & Robertson-Ring, A. (2015). Consumer reviews and the creation product aspects from online consumer reviews. HLT ‘11 proceedings of the 49th annual
of booking transaction value: Lessons from the hotel industry. International Journal of meeting of the association for computational linguistics: Human language technologies: 1,
Hospitality Management, 50, 77–83. (pp. 1496–1505).
Wilson, T., Wiebe, J., & Hoffmann, P. (2005). Recognizing contextual polarity in phrase- Yu, H. F., Ho, C. H., Juan, Y. C., & Lin, C. J. (2013). LibShortText: A Library for Short-text
level sentiment analysis. HLT ‘05 proceedings of the conference on human language Classification and Analysis LibShortText: A Library for Short-text Classification and
technology and empirical methods in natural language processing (pp. 347–354). Analysis. In 1–5.
Wu, L., & Brynjolfsson, E. (2015). The future of prediction: How google searches fore- Zhang, Z., Li, X., & Chen, Y. (2012). Deciphering word-of-mouth in social media: Text-
shadow housing prices and sales. SSRN Electronic Journal, 89–118. based metrics of consumer reviews. ACM Transactions on Management Information
Wu, Y., Wei, F., Liu, S., Au, N., Cui, W., Zhou, H., et al. (2010). OpinionSeer: Interactive Systems (TMIS), 3 5:1-5:23.
visualization of hotel customer feedback. IEEE Transactions on Visualization and Zhang, Z., Guo, C., & Goes, P. (2013). Product comparison networks for competitive
Computer Graphics, 16, 1109–1118. analysis of online word-of-Mouth. ACM Transactions on Management Information
Xiang, Z., Du, Q., Ma, Y., & Fan, W. (2017). A comparative analysis of major online review Systems (TMIS), 3 20:21-20:22.
platforms: Implications for social media analytics in hospitality and tourism. Tourism Zhao, W. X., Jiang, J., Yan, H., & Li, X. (2010). Jointly modeling aspects and opinions
Management, 58, 51–65. with a maxEnt-LDA hybrid. EMNLP ‘10 proceedings of the 2010 conference on empirical
Xie, K. L., Zhang, Z., & Zhang, Z. (2014). The business value of online consumer reviews methods in natural language processing (pp. 56–65).
and management response to hotel performance. International Journal of Hospitality Zhao, D., Ye, Q., & Zhu, K. (2016). The influence of consumers' information search be-
Management, 43, 1–12. havior on purchase decisions for experience goods: Empirical evidence from hotel
Xu, X., & Reed, M. (2017). Perceived pollution and inbound tourism in China. Tourism industry. WHICEB 2016 proceedings (pp. 297–302).

279