research-article

Free access

Sentiment summarization: evaluating and learning user preferences

Authors:

Sasha Blair-Goldensohn,

Ryan McDonaldAuthors Info & Claims

EACL '09: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics

Pages 514 - 522

Published: 30 March 2009 Publication History

Abstract

We present the results of a large-scale, end-to-end human evaluation of various sentiment summarization models. The evaluation shows that users have a strong preference for summarizers that model sentiment over non-sentiment baselines, but have no broad overall preference between any of the sentiment-based models. However, an analysis of the human judgments suggests that there are identifiable situations where one summarizer is generally preferred over the others. We exploit this fact to build a new summarizer by training a ranking SVM model over the set of human preference judgments that were collected during the evaluation, which results in a 30% relative reduction in error over the previous best summarizer.

References

[1]

D. Ariely, G. Loewenstein, and D. Prelec. 2008. Coherent arbitrariness: Stable demand curves without stable preferences. The Quarterly Journal of Economics, 118:73105.

[2]

S. Blair-Goldensohn, K. Hannan, R. McDonald, T. Neylon, G. A. Reis, and J. Reynar. 2008. Building a sentiment summarizer for local service reviews. In WWW Workshop on NLP in the Information Explosion Era.

[3]

S. R. K. Branavan, H. Chen, J. Eisenstein, and R. Barzilay. 2008. Learning document-level semantic properties from free-text annotations. In Proceedings of the Annual Conference of the Association for Computational Linguistics (ACL).

[4]

G. Carenini and J. Cheung. 2008. Extractive vs. nlgbased abstractive summarization of evaluative text: The effect of corpus controversiality. In International Conference on Natural Language Generation (INLG).

Digital Library

[5]

G. Carenini, R. T. Ng, and E. Zwart. 2005. Extracting knowledge from evaluative text. In Proceedings of the International Conference on Knowledge Capture.

Digital Library

[6]

G. Carenini, R. Ng, and A. Pauls. 2006. Multi-document summarization of evaluative text. In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (EACL).

[7]

Y. Choi. C. Cardie, E. Riloff, and S. Patwardhan. 2005. Identifying sources of opinions with conditional random fields and extraction patterns. In Proceedings the Joint Conference on Human Language Technology and Empirical Methods in Natural Language Processing (HLT-EMNLP).

Digital Library

[8]

E. Filatova and V. Hatzivassiloglou. 2004. A formal model for information selection in multi-sentence text extraction. In Proceedings of the International Conference on Computational Linguistics (COLING).

Digital Library

[9]

M. Gamon, A. Aue, S. Corston-Oliver, and E. Ringger. 2005. Pulse: Mining customer opinions from free text. In Proceedings of the 6th International Symposium on Intelligent Data Analysis (IDA).

Digital Library

[10]

J. Goldstein, V. Mittal, J. Carbonell, and M. Kantrowitz. 2000. Multi-document summarization by sentence extraction. In Proceedings of the ANLP/NAACL Workshop on Automatic Summarization.

Digital Library

[11]

M. Hu and B. Liu. 2004a. Mining and summarizing customer reviews. In Proceedings of the International Conference on Knowledge Discovery and Data Mining (KDD).

Digital Library

[12]

M. Hu and B. Liu. 2004b. Mining opinion features in customer reviews. In Proceedings of National Conference on Artificial Intelligence (AAAI).

Digital Library

[13]

N. Jindal and B. Liu. 2006. Mining comprative sentences and relations. In Proceedings of 21st National Conference on Artificial Intelligence (AAAI).

Digital Library

[14]

T. Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the ACM Conference on Knowledge Discovery and Data Mining (KDD).

Digital Library

[15]

S. M. Kim and E. Hovy. 2004. Determining the sentiment of opinions. In Proceedings of Conference on Computational Linguistics (COLING).

Digital Library

[16]

C. Y. Lin and E. Hovy. 2003. Automatic evaluation of summaries using n-gram cooccurrence statistics. In Proceedings of the Conference on Human Language Technologies and the North American Chapter of the Association for Computational Linguistics (HLT-NAACL).

Digital Library

[17]

R. McDonald. 2007. A Study of Global Inference Algorithms in Multi-document Summarization. In Proceedings of the European Conference on Information Retrieval (ECIR).

Digital Library

[18]

K. McKeown, R. J. Passonneau, D. K. Elson, A. Nenkova, and J. Hirschberg. 2005. Do Summaries Help? A Task-Based Evaluation of Multi-Document Summarization. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval.

Digital Library

[19]

A. M. Popescu and O. Etzioni. 2005. Extracting product features and opinions from reviews. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).

Digital Library

[20]

V. Stoyanov and C. Cardie. 2008. Topic identification for fine-grained opinion analysis. In Proceedings of the Conference on Computational Linguistics (COLING).

Digital Library

[21]

I. Titov and R. McDonald. 2008a. A joint model of text and aspect ratings. In Proceedings of the Annual Conference of the Association for Computational Linguistics (ACL).

[22]

I. Titov and R. McDonald. 2008b. Modeling online reviews with multi-grain topic models. In Proceedings of the Annual World Wide Web Conference (WWW).

Digital Library

[23]

L. Zhuang, F. Jing, and X. Y. Zhu. 2006. Movie review mining and summarization. In Proceedings of the International Conference on Information and Knowledge Management (CIKM).

Digital Library

Cited By

Ma YLi Q(2019)A weakly-supervised extractive framework for sentiment-preserving document summarizationWorld Wide Web10.1007/s11280-018-0591-022:4(1401-1425)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1007/s11280-018-0591-0
Zheng XHe WLi L(2019)Distributed representations based collaborative filtering with reviewsApplied Intelligence10.1007/s10489-018-01406-z49:7(2623-2640)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1007/s10489-018-01406-z
Ma SSun XLin JRen X(2018)A hierarchical end-to-end model for jointly improving text summarization and sentiment classificationProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304222.3304361(4251-4257)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304222.3304361
Show More Cited By

Index Terms

Sentiment summarization: evaluating and learning user preferences
1. Computing methodologies

Recommendations

Sentiment Lossless Summarization
Abstract
The aim of automatic text summarization (ATS) is to extract representative texts from documents and keep major points of the extracted texts consistent with the original documents. However, most existing studies ignore sentimental ...
Sentiment diversification for short review summarization
WI '17: Proceedings of the International Conference on Web Intelligence

With the abundance of reviews published on the Web about a given product, consumers are looking for ways to view major opinions that can be presented in a quick and succinct way. Reviews contain many different opinions, making the ability to show a ...
Aspect-based sentence segmentation for sentiment summarization
TSA '09: Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion

Aspect-based sentiment summarization systems generally use sentences associated with relevant aspects extracted from the reviews as the basis for summarization. However, in real reviews, a single sentence often exhibits several aspects for opinions. ...

Comments

Information & Contributors

Information

Published In

cover image DL Hosted proceedings

EACL '09: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics

March 2009

905 pages

General Chair:
Alex Lascarides
University of Edinburgh (UK)
,
Program Chairs:
Claire Gardent
CNRS/LORIA Nancy (France)
,
Joakim Nivre
Uppsala University and Vaxjo University (Sweden)

Publisher

Association for Computational Linguistics

United States

Publication History

Published: 30 March 2009

Qualifiers

Research-article

Acceptance Rates

EACL '09 Paper Acceptance Rate 100 of 360 submissions, 28%;

Overall Acceptance Rate 100 of 360 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
928
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)8

Reflects downloads up to 10 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ma YLi Q(2019)A weakly-supervised extractive framework for sentiment-preserving document summarizationWorld Wide Web10.1007/s11280-018-0591-022:4(1401-1425)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1007/s11280-018-0591-0
Zheng XHe WLi L(2019)Distributed representations based collaborative filtering with reviewsApplied Intelligence10.1007/s10489-018-01406-z49:7(2623-2640)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1007/s10489-018-01406-z
Ma SSun XLin JRen X(2018)A hierarchical end-to-end model for jointly improving text summarization and sentiment classificationProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304222.3304361(4251-4257)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304222.3304361
Marstawi ASharef NAris TMustapha A(2017)Ontology-based Aspect Extraction for an Improved Sentiment Analysis in Summarization of Product ReviewsProceedings of the 8th International Conference on Computer Modeling and Simulation10.1145/3036331.3036362(100-104)Online publication date: 20-Jan-2017
https://dl.acm.org/doi/10.1145/3036331.3036362
Gisdakis SGiannetsos TPapadimitratos PCheng XLiang Z(2016)Android privacy C(R)acheProceedings of the 1st ACM Workshop on Privacy-Aware Mobile Computing10.1145/2940343.2940346(1-10)Online publication date: 5-Jul-2016
https://dl.acm.org/doi/10.1145/2940343.2940346
Vural ACambazoglu BKaragoz P(2014)Sentiment-Focused Web CrawlingACM Transactions on the Web10.1145/26448218:4(1-21)Online publication date: 6-Nov-2014
https://dl.acm.org/doi/10.1145/2644821
Arapakis ILalmas MCambazoglu BMarcos MJose J(2014)User engagement in online NewsJournal of the Association for Information Science and Technology10.1002/asi.2309665:10(1988-2005)Online publication date: 1-Oct-2014
https://dl.acm.org/doi/10.1002/asi.23096
McAuley JLeskovec JYang QKing ILi QPu PKarypis G(2013)Hidden factors and hidden topicsProceedings of the 7th ACM conference on Recommender systems10.1145/2507157.2507163(165-172)Online publication date: 12-Oct-2013
https://dl.acm.org/doi/10.1145/2507157.2507163
Zhu LGao SPan SLi HDeng DShahabi CRokne JFaloutsos C(2013)Graph-based informative-sentence selection for opinion summarizationProceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining10.1145/2492517.2492651(408-412)Online publication date: 25-Aug-2013
https://dl.acm.org/doi/10.1145/2492517.2492651
Potthast MStein BLoose FBecker S(2012)Information Retrieval in the CommentsphereACM Transactions on Intelligent Systems and Technology10.1145/2337542.23375533:4(1-21)Online publication date: 1-Sep-2012
https://dl.acm.org/doi/10.1145/2337542.2337553
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents