research-article

Public Access

3 Stars on Yelp, 4 Stars on Google Maps: A Cross-Platform Examination of Restaurant Ratings

Authors:

Brent HechtAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 4, Issue CSCW3

Article No.: 254, Pages 1 - 25

https://doi.org/10.1145/3432953

Published: 05 January 2021 Publication History

Abstract

Even though a restaurant may receive different ratings across review platforms, people often see only one rating during a local search (e.g. 'best burgers near me'). In this paper, we examine the differences in ratings between two commonly used review platforms-Google Maps and Yelp. We found that restaurant ratings on Google Maps are, on average, 0.7 stars higher than those on Yelp, with the increase being driven in large part by higher ratings for chain restaurants on Google Maps. We also found extensive diversity i¬¬n top-ranked restaurants by geographic region across platforms. For example, for a given metropolitan area, there exists little overlap in its top ten lists of restaurants on Google Maps and Yelp. Our results problematize the use of a single review platform in local search and have implications for end users of ratings and local search technologies. We outline concrete design recommendations to improve communication of restaurant evaluation and discuss the potential causes for the divergence we observed.

References

[1]

Michael Anderson and Jeremy Magruder. 2012. Learning from the Crowd: Regression Discontinuity Estimates of the Effects of an Online Review Database. Econ J 122, 563 (September 2012), 957--989.

[2]

Saeideh Bakhshi, Partha Kanuparthy, and Eric Gilbert. 2014. Demographics, Weather and Online Reviews: A Study of Restaurant Recommendations. In Proceedings of the 23rd International Conference on World Wide Web (WWW '14), ACM, New York, NY, USA, 443--454.

Digital Library

[3]

Saeideh Bakhshi, Partha Kanuparthy, and David A. Shamma. 2015. Understanding Online Reviews: Funny, Cool or Useful? In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15), ACM, New York, NY, USA, 1270--1276.

Digital Library

[4]

Taryn Bipat, Susan R. Fussell, Brent Hecht, Charles Kiene, David W. McDonald, and Mark Zachry. 2018. Navigating the Challenges of Multi-Site Research. In Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '18), ACM, New York, NY, USA, 409--415.

Digital Library

[5]

Mike Blumenthal. 2014. Why does Google show a 4.8 rating when I have all 5-star reviews? Local University. Retrieved January 31, 2020 from https://localu.org/2014/12/22/google-show-4--8-rating-5-star-reviews/

[6]

US Census Bureau. New Census Data Show Differences Between Urban and Rural Populations. The United States Census Bureau. Retrieved October 15, 2019 from https://www.census.gov/newsroom/press-releases/2016/cb16--210.html

[7]

Motahhare Eslami, Kristen Vaccaro, Karrie Karahalios, and Kevin Hamilton. 2017. 'Be Careful; Things Can Be Worse than They Appear?: Understanding Biased Algorithms and Users? Behavior Around Them in Rating Platforms. In Eleventh International AAAI Conference on Web and Social Media. Retrieved April 18, 2019 from https://aaai.org/ocs/index.php/ICWSM/ICWSM17/paper/view/15697

[8]

Motahhare Eslami, Kristen Vaccaro, Min Kyung Lee, Amit Elazari Bar On, Eric Gilbert, and Karrie Karahalios. 2019. User Attitudes Towards Algorithmic Opacity and Transparency in Online Reviewing Platforms. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19), ACM, New York, NY, USA, 494:1--494:14.

Digital Library

[9]

Andrew Hall, Jacob Thebault-Spieker, Shilad Sen, Brent Hecht, and Loren Terveen. 2018. Exploring the Relationship Between ?Informal Standards? and Contributor Practice in OpenStreetMap. In Proceedings of the 14th International Symposium on Open Collaboration (OpenSym '18), ACM, New York, NY, USA, 10:1--10:11.

Digital Library

[10]

I. Johnson, J. Henderson, C. Perry, J. Schöning, and B. Hecht. 2017. Beautiful?but at What Cost?: An Examination of Externalities in Geographic Vehicle Routing. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 2 (June 2017), 15:1--15:21.

Digital Library

[11]

Dan Jurafsky, Victor Chahuneau, Bryan R. Routledge, and Noah A. Smith. 2014. Narrative framing of consumer sentiment in online restaurant reviews. First Monday 19, 4 (March 2014). Retrieved September 26, 2018 from http://firstmonday.org/ojs/index.php/fm/article/view/4944

[12]

Ruogu Kang, Stephanie Brown, and Sara Kiesler. 2013. Why Do People Seek Anonymity on the Internet?: Informing Policy and Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), ACM, New York, NY, USA, 2657--2666.

Digital Library

[13]

Senior PR Coordinator Kathleen Liu and February 1 Friday. 2019. Consumer Alerts: No Tolerance for Review Manipulation. Yelp. Retrieved September 18, 2019 from https://blog.yelp.com/2019/02/consumer-alerts-no-tolerance-for-review-manipulation

[14]

Balázs Kovács, Glenn R. Carroll, and David W. Lehman. 2013. Authenticity and Consumer Value Ratings: Empirical Tests from the Restaurant Domain. Organization Science 25, 2 (July 2013), 458--478.

Digital Library

[15]

Stuart E. Levy, Wenjing Duan, and Soyoung Boo. 2013. An Analysis of One-Star Online Reviews and Responses in the Washington, D.C., Lodging Market. Cornell Hospitality Quarterly 54, 1 (February 2013), 49--63.

[16]

Michael Luca. 2015. User-Generated Content and Social Media. In Handbook of Media Economics, Simon P. Anderson, Joel Waldfogel and David Strömberg (eds.). North-Holland, 563--592.

[17]

Michael Luca. 2016. Reviews, Reputation, and Revenue: The Case of Yelp.Com. Social Science Research Network, Rochester, NY. Retrieved November 25, 2018 from https://papers.ssrn.com/abstract=1928601

[18]

Michael Luca and Georgios Zervas. 2016. Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud. Management Science 62, 12 (December 2016), 3412--3427.

Digital Library

[19]

Farhad Manjoo. 2018. Stumbles? What Stumbles? Big Tech Is as Strong as Ever. The New York Times. Retrieved April 4, 2019 from https://www.nytimes.com/2018/08/01/technology/big-tech-earnings-stumbles.html

[20]

Mary McGlohon, Natalie Glance, and Zach Reiter. 2010. Star Quality: Aggregating Reviews to Rank Products and Merchants. In Fourth International AAAI Conference on Weblogs and Social Media. Retrieved August 13, 2019 from https://www.aaai.org/ocs/index.php/ICWSM/ICWSM10/paper/view/1507

[21]

Connor McMahon, Isaac L Johnson, and Brent Hecht. 2017. The Substantial Interdependence of Wikipedia and Google: A Case Study on the Relationship Between Peer Production Communities and Information Technologies. In ICWSM, 142--151.

[22]

Arjun Mukherjee, Vivek Venkataraman, Bing Liu, and Natalie Glance. 2013. What Yelp Fake Review Filter Might Be Doing? In Seventh International AAAI Conference on Weblogs and Social Media. Retrieved October 15, 2019 from https://www.aaai.org/ocs/index.php/ICWSM/ICWSM13/paper/view/6006

[23]

Xi Niu, Xiangyu Fan, and Tao Zhang. 2019. Understanding Faceted Search from Data Science and Human Factor Perspectives. ACM Trans. Inf. Syst. 37, 2 (January 2019), 14:1--14:27.

Digital Library

[24]

Paul Phillips, Krystin Zigan, Maria Manuela Santos Silva, and Roland Schegg. 2015. The interactive effects of online reviews on the determinants of Swiss hotel performance: A neural network analysis. Tourism Management 50, (October 2015), 130--141.

[25]

Lee Rainie, Kristen Purcell, Amy Mitchell, and Tom Rosenstiel. 2011. Where people get information about restaurants and other local businesses. Pew Research Center: Internet, Science & Tech. Retrieved August 27, 2019 from https://www.pewinternet.org/2011/12/14/where-people-get-information-about-restaurants-and-other-local-businesses/

[26]

Kenneth Rogoff. 2019. Big tech has too much monopoly power -- it's right to take it on. the Guardian (April 2019). Retrieved from https://www.theguardian.com/technology/2019/apr/02/big-tech-monopoly-power-elizabeth-warren-technology

[27]

Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cédric Langbort. 2014. Auditing Algorithms?: Research Methods for Detecting Discrimination on Internet Platforms.

[28]

Jerry O. Talton III, Krishna Dusad, Konstantinos Koiliaris, and Ranjitha S. Kumar. 2019. How Do People Sort by Ratings? In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19), ACM, New York, NY, USA, 305:1--305:10.

Digital Library

[29]

Chenhao Tan, Ed H. Chi, David Huffaker, Gueorgi Kossinets, and Alexander J. Smola. 2013. Instant foodie: predicting expert ratings from grassroots. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge management (CIKM '13), ACM, New York, NY, USA, 1127--1136.

Digital Library

[30]

Hongning Wang, Yue Lu, and ChengXiang Zhai. 2011. Latent Aspect Rating Analysis Without Aspect Keyword Supervision. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '11), ACM, New York, NY, USA, 618--626.

Digital Library

[31]

Zhongmin Wang. 2010. Anonymity, Social Image, and the Competition for Volunteers: A Case Study of the Online Market for Reviews. The B.E. Journal of Economic Analysis & Policy 10, 1 (January 2010).

[32]

Zheng Xiang, Qianzhou Du, Yufeng Ma, and Weiguo Fan. 2017. A comparative analysis of major online review platforms: Implications for social media analytics in hospitality and tourism. Tourism Management 58, (February 2017), 51--65.

[33]

Inc Yelp and March 18 Thursday. 2010. Yelp's Recommendation Software Explained. Yelp. Retrieved September 18, 2019 from https://blog.yelp.com/2010/03/yelp-review-filter-explained

[34]

Georgios Zervas, Davide Proserpio, and John Byers. 2015. A First Look at Online Reputation on Airbnb, Where Every Stay is Above Average. Social Science Research Network, Rochester, NY. Retrieved September 7, 2019 from https://papers.ssrn.com/abstract=2554500

[35]

Ye Zhang and Shu Tian Cole. 2016. Dimensions of lodging guest satisfaction among guests with mobility challenges: A mixed-method analysis of web-based texts. Tourism Management 53, (April 2016), 13--27.

[36]

2017. Report: Consumers prefer independent restaurants over chains. Nation's Restaurant News. Retrieved April 23, 2019 from https://www.nrn.com/consumer-trends/report-consumers-prefer-independent-restaurants-over-chains

[37]

2019. GoogleChrome/puppeteer. GoogleChrome. Retrieved September 16, 2019 from https://github.com/GoogleChrome/puppeteer

[38]

2019. Franchising. Wikipedia. Retrieved October 14, 2019 from https://en.wikipedia.org/w/index.php?title=Franchising&oldid=919113784

[39]

QSR & restaurant holiday strategy. Think with Google. Retrieved January 16, 2020 from https://www.thinkwithgoogle.com/marketing-resources/qsr-restaurant-strategy/

[40]

Google Maps: 1 Billion Monthly Users -- GPS Business News. Retrieved September 12, 2019 from https://gpsbusinessnews.com/Google-Maps-1-Billion-Monthly-Users_a4964.html

[41]

Factsheet. Yelp. Retrieved September 16, 2019 from https://www.yelp.com/factsheet

[42]

? Top U.S. mapping apps by reach 2018 | Statista. Retrieved October 14, 2019 from https://www.statista.com/statistics/865419/most-popular-us-mapping-apps-ranked-by-reach/

[43]

U.S. online review usage for local business research 2017. Statista. Retrieved September 13, 2019 from https://www.statista.com/statistics/695736/online-review-site-usage-local-business-research-usa/

[44]

Write reviews and add ratings of places - iPhone & iPad - Google Maps Help. Retrieved September 11, 2019 from https://support.google.com/maps/answer/6230175?co=GENIE.Platform%3DAndroid&hl=en&oco=1

[45]

review filter. Yelp. Retrieved January 30, 2020 from https://blog.yelp.com/search/review+filter

[46]

Daily Access Limiting - Yelp Fusion. Retrieved March 31, 2020 from https://www.yelp.com/developers/documentation/v3/rate_limiting

[47]

Pricing & Plans | Google Maps Platform. Google Cloud. Retrieved March 31, 2020 from https://cloud.google.com/maps-platform/pricing

[48]

Yelp Dataset. Retrieved September 16, 2019 from https://www.yelp.com/dataset

[49]

Mintel 2018. Retrieved June 5, 2019 from https://store.mintel.com/us-fast-casual-restaurants-market-report

[50]

Restaurants in the U.S. Statista. Retrieved June 4, 2019 from http://www.statista.com/study/11721/restaurants-in-the-us-statista-dossier/

[51]

How Not To Sort By Average Rating -- Evan Miller. Retrieved September 18, 2019 from http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

[52]

Google Trends. Google Trends. Retrieved January 31, 2020 from https://trends.google.com/trends/explore?q=restaurants%20near%20me&geo=US

[53]

Local Guides. Retrieved September 8, 2020 from https://maps.google.com/localguides/howto

Cited By

Kirilenko AStepchenkova SGromoll RJo Y(2024)Comprehensive examination of online reviews divergence over time and platform typesInternational Journal of Hospitality Management10.1016/j.ijhm.2023.103647117(103647)Online publication date: Feb-2024
https://doi.org/10.1016/j.ijhm.2023.103647
Li HVincent NChancellor SHecht B(2023)The Dimensions of Data Labor: A Road Map for Researchers, Activists, and Policymakers to Empower Data ProducersProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594070(1151-1161)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594070
Nurgalieva LFrik ADoherty G(2023)A Narrative Review of Factors Affecting the Implementation of Privacy and Security Practices in Software DevelopmentACM Computing Surveys10.1145/358995155:14s(1-27)Online publication date: 4-Apr-2023
https://dl.acm.org/doi/10.1145/3589951
Show More Cited By

Index Terms

3 Stars on Yelp, 4 Stars on Google Maps: A Cross-Platform Examination of Restaurant Ratings
1. Human-centered computing
  1. Collaborative and social computing
    1. Empirical studies in collaborative and social computing

Recommendations

Beyond the stars: exploiting free-text user reviews to improve the accuracy of movie recommendations
TSA '09: Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion

In this paper we show that the extraction of opinions from free-text reviews can improve the accuracy of movie recommendations. We present three approaches to extract movie aspects as opinion targets and use them as features for the collaborative ...
Remembering the stars?: effect of time on preference retrieval from memory
RecSys '12: Proceedings of the sixth ACM conference on Recommender systems

Many recommendation systems rely on explicit ratings provided by their users. Often these ratings are provided long after consuming the item, relying heavily on people's representation of the quality of the item in memory. This paper investigates a ...
Fault in your stars: an analysis of Android app reviews
CODS-COMAD '18: Proceedings of the ACM India Joint International Conference on Data Science and Management of Data

Mobile app distribution platforms such as Google Play Store allow users to share their feedback about downloaded apps in the form of a review comment and a corresponding star rating. Typically, the star rating ranges from one to five stars, with one ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 4, Issue CSCW3

CSCW

December 2020

1825 pages

EISSN:2573-0142

DOI:10.1145/3446568

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2021 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 January 2021

Published in PACMHCI Volume 4, Issue CSCW3

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
1,229
Total Downloads

Downloads (Last 12 months)527
Downloads (Last 6 weeks)64

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kirilenko AStepchenkova SGromoll RJo Y(2024)Comprehensive examination of online reviews divergence over time and platform typesInternational Journal of Hospitality Management10.1016/j.ijhm.2023.103647117(103647)Online publication date: Feb-2024
https://doi.org/10.1016/j.ijhm.2023.103647
Li HVincent NChancellor SHecht B(2023)The Dimensions of Data Labor: A Road Map for Researchers, Activists, and Policymakers to Empower Data ProducersProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594070(1151-1161)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594070
Nurgalieva LFrik ADoherty G(2023)A Narrative Review of Factors Affecting the Implementation of Privacy and Security Practices in Software DevelopmentACM Computing Surveys10.1145/358995155:14s(1-27)Online publication date: 4-Apr-2023
https://dl.acm.org/doi/10.1145/3589951
Feizizadeh BOmrazadeh DGhasemi MBageri SLakes TKitzmann RGhanbari ABlaschke T(2023)Urban restaurants and online food delivery during the COVID-19 pandemic: a spatial and socio-demographic analysisInternational Journal of Digital Earth10.1080/17538947.2023.221031316:1(1725-1751)Online publication date: 12-May-2023
https://doi.org/10.1080/17538947.2023.2210313
Tessmann RElbert R(2022)Multi-sided platforms in competitive B2B networks with varying governmental influence – a taxonomy of Port and Cargo Community System business modelsElectronic Markets10.1007/s12525-022-00529-z32:2(829-872)Online publication date: 26-Apr-2022
https://doi.org/10.1007/s12525-022-00529-z
Kim SOh CCho WShin DSuh BLee J(2021)Trkic G00gle: Why and How Users Game Translation AlgorithmsProceedings of the ACM on Human-Computer Interaction10.1145/34760855:CSCW2(1-24)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3476085
Bandy J(2021)Problematic Machine BehaviorProceedings of the ACM on Human-Computer Interaction10.1145/34491485:CSCW1(1-34)Online publication date: 22-Apr-2021
https://dl.acm.org/doi/10.1145/3449148

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents