research-article

Free access

A Feature-Oriented Sentiment Rating for Mobile App Reviews

Authors:

Washington Luiz,

Rafael Alencar,

Fernando Mourão,

Dárlinton Carvalho,

Marcos Andre Gonçalves,

Leonardo RochaAuthors Info & Claims

WWW '18: Proceedings of the 2018 World Wide Web Conference

Pages 1909 - 1918

https://doi.org/10.1145/3178876.3186168

Published: 10 April 2018 Publication History

All formats PDF

Abstract

In this paper, we propose a general framework that allows developers to filter, summarize and analyze user reviews written about applications on App Stores. Our framework extracts automatically relevant features from reviews of apps (e.g., information about functionalities, bugs, requirements, etc) and analyzes the sentiment associated with each of them. Our framework has three main building blocks, namely, (i) topic modeling, (ii) sentiment analysis and (iii) summarization interface. The topic modeling block aims at finding semantic topics from textual comments, extracting the target features based on the most relevant words of each discovered topic. The sentiment analysis block detects the sentiment associated with each discovered feature. The summarization interface provides to developers an intuitive visualization of the features (i.e., topics) and their associated sentiment, providing richer information than a 'star rating' strategy. Our evaluation shows that the topic modeling block is able to organize information provided by users in subcategories that facilitate the understanding of which features more positively/negatively impact the overall evaluation of the application. Regarding user satisfaction, we can observe that, in spite of the star rating being a good measure of evaluation, the Sentiment Analysis technique is more accurate in capturing the sentiment transmitted by the user by means of a comment.

References

[1]

A. I. Anam and M. Yeasin. Accessibility in smartphone applications: What do we learn from reviews? In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS '13, pages 35:1--35:2, New York, NY, USA, 2013. ACM.

Digital Library

[2]

W. H. Auden. The Complete Works of W. H. Auden: Prose, volume 2. Princeton University Press, April 2002.

[3]

A. Bhattacharyya. On a measure of divergence between two statistical populations defined by their probability distributions. Bulletin of Cal. Math. Soc., 35(1):99--109, 1943.

[4]

P. V. Bicalho, T. de Oliveira Cunha, F. H. J. Mourão, G. L. Pappa, and W. M. Jr. Generating cohesive semantic topics from latent factors. In BRACIS, pages 271--276. IEEE Computer Society, 2014.

Digital Library

[5]

D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. Journal of machine Learning research, 3(Jan):993--1022, 2003.

Digital Library

[6]

J. Bobadilla, F. Ortega, A. Hernando, and A. Gutiérrez. Recommender systems survey. Knowledge-Based Systems, 46:109--132, 2013.

Digital Library

[7]

Y. Chen, M. Rege, M. Dong, and J. Hua. Non-negative matrix factorization for semi-supervised data clustering. Knowledge and Information Systems, 17(3):355-- 379, 2008.

[8]

Z. Chen and B. Liu. Topic modeling using topics from many domains, lifelong learning and big data. In Proceedings of the 31st International Conference on International Conference on Machine Learning - Volume 32, ICML'14, pages II-- 703--II--711. JMLR.org, 2014.

Digital Library

[9]

X. Cheng, X. Yan, Y. Lan, and J. Guo. Btm: Topic modeling over short texts. IEEE Transactions on Knowledge and Data Engineering, 26(12):2928--2941, Dec 2014.

[10]

J. Choo, C. Lee, C. K. Reddy, and H. Park. Utopian: User-driven topic modeling based on interactive nonnegative matrix factorization. IEEE Transactions on Visualization and Computer Graphics, 19(12):1992--2001, Dec 2013.

Digital Library

[11]

K. Dave, S. Lawrence, and D. M. Pennock. Mining the peanut gallery: opinion extraction and semantic classification of product reviews. In ACM WWW, 2003.

Digital Library

[12]

S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman. Indexing by latent semantic analysis. Journal of the American society for information science, 41(6):391, 1990.

[13]

E. C. Dragut, C. Yu, P. Sistla, and W. Meng. Construction of a sentimental word dictionary. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management, CIKM '10, pages 1761--1764, New York, NY, USA, 2010. ACM.

Digital Library

[14]

A. Esuli and F. Sebastiani. Determining the semantic orientation of terms through gloss classification. In Proceedings of the 14th ACM International Conference on Information and Knowledge Management, CIKM '05, pages 617--624, New York, NY, USA, 2005. ACM.

Digital Library

[15]

G. H. Golub and C. Reinsch. Singular value decomposition and least squares solutions. Numerische mathematik, 14(5):403--420, 1970.

Digital Library

[16]

E. Guzman and W. Maalej. How do users like this feature? a fine grained sentiment analysis of app reviews. In T. Gorschek and R. R. Lutz, editors, RE - 22nd International Requirements Engineering Conference, pages 153--162. IEEE Computer Society, 2014.

[17]

W. L. Hamilton, K. Clark, J. Leskovec, and D. Jurafsky. Inducing domain-specific sentiment lexicons from unlabeled corpora. CoRR, abs/1606.02820, 2016.

[18]

S. Hedegaard and J. G. Simonsen. Extracting usability and user experience information from online user reviews. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '13, pages 2089--2098, New York, NY, USA, 2013. ACM.

Digital Library

[19]

T. Hofmann. Probabilistic latent semantic indexing. In Proceedings of the 22Nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '99, pages 50--57, New York, NY, USA, 1999. ACM.

Digital Library

[20]

X. Hu, J. Tang, H. Gao, and H. Liu. Unsupervised sentiment analysis with emotional signals. In Proc. of WWW '13, pages 607--618, Republic and Canton of Geneva, Switzerland, 2013.

Digital Library

[21]

C. J. Hutto and E. Gilbert. VADER: A parsimonious rule-based model for sentiment analysis of social media text. In ICWSM'14, 2014.

[22]

H. Korhonen, J. Arrasvuori, and K. Väänänen-Vainio-Mattila. Let users tell the story: Evaluating user experience with experience reports. In CHI '10 Extended Abstracts on Human Factors in Computing Systems, CHI EA '10, pages 4051--4056, New York, NY, USA, 2010. ACM.

Digital Library

[23]

D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755):788--791, 1999.

[24]

C.-J. Lin. Projected gradient methods for nonnegative matrix factorization. Neural Comput., 19(10):2756--2779, 2007.

Digital Library

[25]

Y. Lu, M. Castellanos, U. Dayal, and C. Zhai. Automatic construction of a contextaware sentiment lexicon: an optimization approach. In Proc. of WWW '11, pages 347--356, Hyderabad, India, 2011.

Digital Library

[26]

G. H. Mealy. A method for synthesizing sequential circuits. Bell System Technical Journal, 34(5):1045--1079, 1955.

[27]

G. A. Miller. Wordnet: A lexical database for english. Commun. ACM, 38(11):39--41, 1995.

Digital Library

[28]

K. A. Neuendorf. The Content Analysis Guidebook. Sage Publications, 2002.

[29]

D. Pagano and W. Maalej. User feedback in the appstore: An empirical study. In Requirements Engineering Conference (RE), 2013 21st IEEE International, pages 125--134. IEEE, 2013.

[30]

A. Pak and P. Paroubek. Twitter as a corpus for sentiment analysis and opinion mining. In LREC, 2010.

[31]

B. Pang and L. Lee. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1--2):1--135, Jan. 2008.

Digital Library

[32]

B. Pang, L. Lee, and S. Vaithyanathan. Thumbs up? Sentiment classification using machine learning techniques. In EMNLP, pages 79--86, 2012.

Digital Library

[33]

L. Polanyi and A. Zaenen. Contextual valence shifters. Computing attitude and affect in text: Theory and applications, pages 1--10, 2006.

[34]

L. Rocha, F. Mourão, T. Silveira, R. Chaves, G. Sá, F. Teixeira, R. Vieira, and R. Ferreira. Saci: Sentiment analysis by collective inspection on social media content. Web Semantics: Science, Services and Agents on the World Wide Web, 34:27--39, 2015.

Digital Library

[35]

P. Rodrigues, I. S. Silva, G. A. R. Barbosa, F. R. d. S. Coutinho, and F. Mourão. Beyond the stars: Towards a novel sentiment rating to evaluate applications in web stores of mobile apps. In Proceedings of the 26th International Conference on World Wide Web Companion, WWW '17 Companion, pages 109--117, Republic and Canton of Geneva, Switzerland, 2017. International World Wide Web Conferences Steering Committee.

Digital Library

[36]

S. Rothe, S. Ebert, and H. Schütze. Ultradense word embeddings by orthogonal transformation. CoRR, abs/1602.07572, 2016.

[37]

G. Salton and C. Buckley. Term-weighting approaches in automatic text retrieval. Inf. Process. Manage., 1988.

Digital Library

[38]

G. Sá, T. Silveira, R. Chaves, F. Teixeira, F. Mourão, and L. Rocha. Legi: Contextaware lexicon consolidation by graph inspection. In ACM SAC, 2014.

[39]

M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and M. Stede. Lexicon-based methods for sentiment analysis. Comput. Linguist., 37(2):267--307, June 2011.

Digital Library

[40]

M. Thelwall, K. Buckley, and G. Paltoglou. Sentiment strength detection for the social web. Journal of the Association for Information Science and Technology, 63(1):163--173, 2012.

Digital Library

[41]

L. Velikovich, S. Blair-Goldensohn, K. Hannan, and R. McDonald. The viability of web-derived polarity lexicons. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, HLT '10, pages 777--785, Stroudsburg, PA, USA, 2010. Association for Computational Linguistics.

Digital Library

[42]

T. Wilson, J. Wiebe, and P. Hoffmann. Recognizing contextual polarity in phraselevel sentiment analysis. In EMLP, pages 347--354, USA, 2005.

Digital Library

Cited By

Pasin ACunha WGonçalves MFerro NOosterhuis HBast HXiong C(2024)A Quantum Annealing Instance Selection Approach for Efficient and Effective Transformer Fine-TuningProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672515(205-214)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672515
Siautama KRudy RSyahchari D(2024)A Comparison Study on Logistic Services Between Indonesia, Malaysia, and Singapore for Support E-Commerce2024 2nd International Conference on Cyber Resilience (ICCR)10.1109/ICCR61006.2024.10532819(1-5)Online publication date: 26-Feb-2024
https://doi.org/10.1109/ICCR61006.2024.10532819
Wang XZhang TTan YShang WLi Y(2024)How to effectively mine app reviews concerning software ecosystem? A survey of review characteristicsJournal of Systems and Software10.1016/j.jss.2024.112040213(112040)Online publication date: Jul-2024
https://doi.org/10.1016/j.jss.2024.112040
Show More Cited By

Index Terms

A Feature-Oriented Sentiment Rating for Mobile App Reviews
1. Information systems
  1. Information retrieval
    1. Document representation
      1. Document collection models
    2. Retrieval models and ranking
      1. Language models

Recommendations

Topic sentiment change analysis
MLDM'11: Proceedings of the 7th international conference on Machine learning and data mining in pattern recognition

Public opinions on a topic may change over time. Topic Sentiment change analysis is a new research problem consisting of two main components: (a) mining opinions on a certain topic, and (b) detect significant changes of sentiment of the opinions on the ...
Aspect based fine-grained sentiment analysis for online reviews
Abstract
Fine-grained sentiment analysis for online reviews plays more and more important role in many applications. The key techniques here are how to efficiently extract multi-grained aspects, identify associated opinions, and classify ...
Joint sentiment/topic model for sentiment analysis
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Sentiment analysis or opinion mining aims to use automated tools to detect subjective information such as opinions, attitudes, and feelings expressed in text. This paper proposes a novel probabilistic modeling framework based on Latent Dirichlet ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '18: Proceedings of the 2018 World Wide Web Conference

April 2018

2000 pages

ISBN:9781450356398

General Chairs:
Pierre-Antoine Champin
Universitè Claude Bernard Lyon 1, France
,
Fabien Gandon
Inria, Université Côte d'Azur, CNRS, I3S, France
,
Lionel Médini
Université Claude Bernard Lyon 1, France
,
Program Chairs:
Mounia Lalmas
Spotify, UK
,
Panagiotis G. Ipeirotis
New York University, USA

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

IW3C2: International World Wide Web Conference Committee

In-Cooperation

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

International World Wide Web Conferences Steering Committee

Republic and Canton of Geneva, Switzerland

Publication History

Published: 10 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

MasWeb
CNPq
InWeb
CAPES
Finep
Fapemig

Conference

WWW '18

Sponsor:

IW3C2

WWW '18: The Web Conference 2018

April 23 - 27, 2018

Lyon, France

Acceptance Rates

WWW '18 Paper Acceptance Rate 170 of 1,155 submissions, 15%;

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
1,654
Total Downloads

Downloads (Last 12 months)311
Downloads (Last 6 weeks)55

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pasin ACunha WGonçalves MFerro NOosterhuis HBast HXiong C(2024)A Quantum Annealing Instance Selection Approach for Efficient and Effective Transformer Fine-TuningProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672515(205-214)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672515
Siautama KRudy RSyahchari D(2024)A Comparison Study on Logistic Services Between Indonesia, Malaysia, and Singapore for Support E-Commerce2024 2nd International Conference on Cyber Resilience (ICCR)10.1109/ICCR61006.2024.10532819(1-5)Online publication date: 26-Feb-2024
https://doi.org/10.1109/ICCR61006.2024.10532819
Wang XZhang TTan YShang WLi Y(2024)How to effectively mine app reviews concerning software ecosystem? A survey of review characteristicsJournal of Systems and Software10.1016/j.jss.2024.112040213(112040)Online publication date: Jul-2024
https://doi.org/10.1016/j.jss.2024.112040
Yahya AGharbi AYafooz WAl-Dhaqm A(2023)A Novel Hybrid Deep Learning Model for Detecting and Classifying Non-Functional Requirements of Mobile Apps IssuesElectronics10.3390/electronics1205125812:5(1258)Online publication date: 6-Mar-2023
https://doi.org/10.3390/electronics12051258
Cunha WViegas FFrança CRosa TRocha LGonçalves M(2023)A Comparative Survey of Instance Selection Methods applied to Non-Neural and Transformer-Based Text ClassificationACM Computing Surveys10.1145/358200055:13s(1-52)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3582000
Sun RXue MTyson GWang SCamtepe SNepal S(2023)Not Seen, Not Heard in the Digital World! Measuring Privacy Practices in Children’s AppsProceedings of the ACM Web Conference 202310.1145/3543507.3583327(2166-2177)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583327
Cunha WFrança CFonseca GRocha LGonçalves MChen HDuh WHuang HKato MMothe JPoblete B(2023)An Effective, Efficient, and Scalable Confidence-based Instance Selection Framework for Transformer-Based Text ClassificationProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591638(665-674)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591638
Wan HAn ZWang BXiong T(2023)Sentiment Analysis for Requirements Elicitation from App Reviews: A Systematic Mapping Study2023 30th Asia-Pacific Software Engineering Conference (APSEC)10.1109/APSEC60848.2023.00030(201-210)Online publication date: 4-Dec-2023
https://doi.org/10.1109/APSEC60848.2023.00030
Islam MDatta RIqbal A(2023)Actual rating calculation of the zoom cloud meetings app using user reviews on google play store with sentiment annotation of BERT and hybridization of RNN and LSTMExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.119919223:COnline publication date: 10-May-2023
https://dl.acm.org/doi/10.1016/j.eswa.2023.119919
Alturayeif NAljamaan HHassine J(2023)An automated approach to aspect-based sentiment analysis of apps reviews using machine and deep learningAutomated Software Engineering10.1007/s10515-023-00397-730:2Online publication date: 9-Sep-2023
https://dl.acm.org/doi/10.1007/s10515-023-00397-7
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents