research-article

Public Access

What I See is What You Don't Get: The Effects of (Not) Seeing Emoji Rendering Differences across Platforms

Authors:

Hannah Miller Hillberg,

Zachary Levonian,

Brent HechtAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 2, Issue CSCW

Article No.: 124, Pages 1 - 24

https://doi.org/10.1145/3274393

Published: 01 November 2018 Publication History

Abstract

Emoji are popular in digital communication, but they are rendered differently on different viewing platforms (e.g., iOS, Android). It is unknown how many people are aware that emoji have multiple renderings, or whether they would change their emoji-bearing messages if they could see how these messages render on recipients' devices. We developed software to expose the multi-rendering nature of emoji and explored whether this increased visibility would affect how people communicate with emoji. Through a survey of 710 Twitter users who recently posted an emoji-bearing tweet, we found that at least 25% of respondents were unaware that the emoji they posted could appear differently to their followers. Additionally, after being shown how one of their tweets rendered across platforms, 20% of respondents reported that they would have edited or not sent the tweet. These statistics reflect millions of potentially regretful tweets shared per day because people cannot see emoji rendering differences across platforms. Our results motivate the development of tools that increase the visibility of emoji rendering differences across platforms, and we contribute our cross-platform emoji rendering software to facilitate this effort.

Supplementary Material

ZIP File (cscw124.zip)

Survey and Emoji Used in Study

Download
1.62 MB

References

[1]

Fathiya Al Rashdi. 2015. Forms and Functions of Emojis in Whatsapp Interaction among Omanis. Georgetown University, Washington, DC, USA.

[2]

Francesco Barbieri, German Kruszewski, Francesco Ronzano, and Horacio Saggion. 2016. How Cosmopolitan Are Emojis?: Exploring Emojis Usage and Meaning over Different Languages with Distributional Semantics. In Proceedings of the 2016 ACM on Multimedia Conference (MM '16), 531--535.

Digital Library

[3]

Francesco Barbieri, Francesco Ronzano, and Horacio Saggion. 2016. What does this Emoji Mean? A Vector Space Skip-Gram Model for Twitter Emojis. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016).

[4]

Rachel Been and Agustin Fonts. 2017. Redesigning Android Emoji. Google Design. Retrieved October 14, 2017 from https://medium.com/google-design/redesigning-android-emoji-cb22e3b51cc6

[5]

Bianca Bosker. 2014. How Emoji Get Lost In Translation. Huffington Post. Retrieved from https://www.huffingtonpost.com/2014/06/27/emoji-meaning_n_5530638.html

[6]

Jeremy Burge. 2018. Apple's Emoji Crackdown. Emojipedia. Retrieved July 8, 2018 from https://blog.emojipedia.org/apples-emoji-crackdown/

[7]

Jeremy Burge. 2018. Slack Overhauls Emoji Support With One Catch. Emojipedia. Retrieved April 19, 2018 from https://blog.emojipedia.org/slack-overhauls-emoji-support/

[8]

Jeremy Burge. 2018. Twitter Switches to Twemoji on Android. Emojipedia. Retrieved September 1, 2018 from https://blog.emojipedia.org/twitter-switches-to-twemoji-on-android/

[9]

Kenneth Burke. 73 Texting Statistics That Answer All Your Questions. Text Request. Retrieved July 8, 2018 from https://www.textrequest.com/blog/texting-statistics-answer-questions/

[10]

Mathias Bynens. 2018. emoji-regex: A regular expression to match all Emoji-only symbols as per the Unicode Standard. Retrieved from https://github.com/mathiasbynens/emoji-regex

[11]

Mathias Bynens. 2018. unicode-tr51: Emoji data extracted from Unicode Technical Report #51. Retrieved from https://github.com/mathiasbynens/unicode-tr51

[12]

Zhenpeng Chen, Xuan Lu, Sheng Shen, Wei Ai, Xuanzhe Liu, and Qiaozhu Mei. 2017. Through a Gender Lens: An Empirical Study of Emoji Usage over Large-Scale Android Users. In WWW '18 Proceedings of the 2018 World Wide Web Conference (WWW '18), 763--772.

Digital Library

[13]

Martin Clancy. 2018. User Agent parsing: how it works and how it can be used. DeviceAtlas. Retrieved September 3, 2018 from https://deviceatlas.com/blog/user-agent-parsing-how-it-works-and-how-it-can-be-used

[14]

Ben Congdon. 2018. python-emojipedia:sunglasses: Emoji data from Emojipedia?:tada: Retrieved from https://github.com/bcongdon/python-emojipedia

[15]

Henriette Cramer, Paloma de Juan, and Joel Tetreault. 2016. Sender-intended Functions of Emojis in US Messaging. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16), 504--509.

Digital Library

[16]

DeviceAtlas. Device Detection. Retrieved July 11, 2018 from https://deviceatlas.com/device-detection

[17]

Thomas Dimson. 2015. Emojineering Part 1: Machine Learning for Emoji Trends. Instagram Engineering Blog. Retrieved from http://instagram-engineering.tumblr.com/post/117889701472/emojineering-part-1-machine-learning-for-emoji

[18]

Ben Eisner, Tim Rocktäschel, Isabelle Augenstein, Matko Bonjak, and Sebastian Riedel. 2016. emoji2vec: Learning Emoji Representations from their Description. In Proceedings of the 4th International Workshop on Natural Language Processing for Social Media at EMNLP 2016.

[19]

Motahhare Eslami, Aimee Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie Karahalios, Kevin Hamilton, and Christian Sandvig. 2015. "I Always Assumed That I Wasn'T Really That Close to {Her}": Reasoning About Invisible Algorithms in News Feeds. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 153--162.

Digital Library

[20]

Eric Goldman. 2017. Surveying the Law of Emojis. Social Science Research Network, Rochester, NY. Retrieved July 31, 2017 from https://papers.ssrn.com/abstract=2961060

[21]

Fredrik Hallsmar and Jonas Palm. 2016. Multi-class Sentiment Classification on Twitter using an Emoji Training Heuristic. KTH Royal Institute of Technology.

[22]

Bo Han, Afshin Rahimi, Leon Derczynski, and Timothy Baldwin. 2016. Twitter Geolocation Prediction Shared Task of the 2016 Workshop on Noisy User-generated Text. Proceedings of the 2nd Workshop on Noisy User-generated Text (WNUT): 213--217.

[23]

Ran R. Hassin, Hillel Aviezer, and Shlomo Bentin. 2013. Inherently Ambiguous: Facial Expressions of Emotions, in Context. Emotion Review 5, 1: 60--65.

[24]

Susan Herring and Ashley Dainas. 2017. "Nice Picture Comment!" Graphicons in Facebook Comment Threads. Hawaii International Conference on System Sciences 2017 (HICSS-50).

[25]

Ankit Kariryaa, Isaac Johnson, Johannes Schöning, and Brent Hecht. 2018. Defining and Predicting the Localness of Volunteered Geographic Information using Ground Truth Data. In Proceedings of the 36th Annual ACM Conference on Human Factors in Computing Systems (CHI '18).

Digital Library

[26]

Ryan Kelly and Leon Watts. 2015. Characterising the inventive appropriation of emoji as relationally meaningful in mediated close personal relationships. Experiences of Technology Appropriation: Unanticipated Users, Usage, Circumstances, and Design.

[27]

Kun-Lin Liu, Wu-Jun Li, and Minyi Guo. 2012. Emoticon Smoothed Language Models for Twitter Sentiment Analysis. In AAAI'12 Proceedings of the 26th AAAI Conference on Artificial Intelligence, 1678--1684.

Digital Library

[28]

Xuan Lu, Wei Ai, Xuanzhe Liu, Qian Li, Ning Wang, Gang Huang, and Qiaozhu Mei. 2016. Learning from the Ubiquitous Language: An Empirical Analysis of Emoji Usage of Smartphone Users. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16), 770--780.

Digital Library

[29]

Jalal Mahmud, Jeffrey Nichols, and Clemens Drews. 2014. Home Location Identification of Twitter Users. ACM TIST 5, 3: 1--21.

Digital Library

[30]

H. B. Mann and D. R. Whitney. 1947. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. The Annals of Mathematical Statistics 18, 1: 50--60.

[31]

Ben Medlock and Gretchen McCulloch. 2016. The Linguistic Secrets Found in Billions of Emoji. Retrieved October 24, 2016 from http://www.slideshare.net/SwiftKey/the-linguistic-secrets-found-in-billions-of-emoji-sxsw-2016-presentation-59956212

[32]

Hannah J. Miller, Daniel Kluver, Jacob Thebault-Spieker, Loren G. Terveen, and Brent J. Hecht. 2017. Understanding Emoji Ambiguity in Context: The Role of Text in Emoji-Related Miscommunication. In Proceedings of the 11th International AAAI Conference on Web and Social Media (ICWSM '17), 152--161.

[33]

Hannah J. Miller, Jacob Thebault-Spieker, Shuo Chang, Isaac Johnson, Loren Terveen, and Brent Hecht. 2016. "Blissfully Happy" or "Ready toFight": Varying Interpretations of Emoji. In Proceedings of the 10th International AAAI Conference on Web and Social Media (ICWSM '16), 259--268.

[34]

Noa Na'aman, Hannah Provenza, and Orion Montoya. 2017. Varying Linguistic Purposes of Emoji in (Twitter) Context. Proceedings of ACL 2017, Student Research Workshop: 136--141.

[35]

Petra Kralj Novak, Jasmina Smailovic, Borut Sluban, and Igor Mozetic. 2015. Sentiment of emojis. PloS one 10, 12.

[36]

Reid Priedhorsky. 2014. Inferring the Origin Locations of Tweets with Quantitative Confidence. CSCW 29.

[37]

Slack. About Us. Slack. Retrieved July 8, 2018 from https://slack.com/about

[38]

Aaron Smith and Monica. 2018. Social Media Use in 2018. Pew Research Center: Internet, Science & Tech. Retrieved July 7, 2018 from http://www.pewinternet.org/2018/03/01/social-media-use-in-2018/

[39]

T. M. F. Smith. 1991. Post-Stratification. Journal of the Royal Statistical Society. Series D (The Statistician) 40, 3: 315--323.

[40]

SwiftKey. 2015. Most-used emoji revealed: Americans love skulls, Brazilians love cats, the French love hearts. SwiftKey Blog. Retrieved from https://blog.swiftkey.com/americans-love-skulls-brazilians-love-cats-swiftkey-emoji-meanings-report/

[41]

Garreth W. Tigwell and David R. Flatla. 2016. Oh That's What You Meant!: Reducing Emoji Misunderstanding. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '16), 859--866.

Digital Library

[42]

Joseph B. Walther and Kyle P. D'Addario. 2001. The impacts of emoticons on message interpretation in computer-mediated communication. Social science computer review 19, 3: 324--347.

[43]

Sanjaya Wijeratne, Lakshika Balasuriya, Amit Sheth, and Derek Doran. 2016. EmojiNet: Building a Machine Readable Sense Inventory for Emoji. In Social Informatics, 527--541.

[44]

Sanjaya Wijeratne, Lakshika Balasuriya, Amit Sheth, and Derek Doran. 2017. EmojiNet: An Open Service and API for Emoji Sense Discovery. In Proceedings of the 11th International AAAI Conference on Web and Social Media (ICWSM '17).

[45]

2015. #ChevyGoesEmoji. media.gm.com. Retrieved January 24, 2018 from https://media.gm.com/content/media/us/en/chevrolet/news.detail.html/content/Pages/news/us/en/2015/jun/0622-cruze-emoji.html

[46]

Oxford's 2015 Word of the Year Is This Emoji. Time. Retrieved January 24, 2018 from http://time.com/4114886/oxford-word-of-the-year-2015-emoji/

[47]

Unicode Consortium. Retrieved January 24, 2018 from http://unicode.org/consortium/consort.html

[48]

Emojipedia. Retrieved January 14, 2017 from http://emojipedia.org/

[49]

EmojiNet. Retrieved April 19, 2018 from http://emojinet.knoesis.org/home.php

[50]

Full Emoji List, v11.0. Retrieved July 10, 2018 from https://unicode.org/emoji/charts/full-emoji-list.html

[51]

"iEmoji.com - "Lookup, "Convert, and Get Emoji!". Retrieved September 3, 2018 from https://www.iemoji.com/

[52]

Microsoft Windows 10 April 2018 Update Emoji List. Retrieved August 16, 2018 from https://emojipedia.org/microsoft/windows-10-april-2018-update/changed/

[53]

10 Heuristics for User Interface Design: Article by Jakob Nielsen. Nielsen Norman Group. Retrieved April 19, 2018 from https://www.nngroup.com/articles/ten-usability-heuristics/

[54]

Global Twitter user distribution by gender 2018 | Statistic. Statista. Retrieved July 7, 2018 from https://www.statista.com/statistics/828092/distribution-of-users-on-twitter-worldwide-gender/

[55]

Millennials Say GIFs, Emojis Communicate Their Thoughts Better Than Words. Time. Retrieved April 19, 2018 from http://time.com/4834112/millennials-gifs-emojis/

[56]

UTS #51: Unicode Emoji. Retrieved January 24, 2018 from http://unicode.org/reports/tr51/

[57]

Twitter Usage Statistics - Internet Live Stats. Retrieved July 11, 2018 from http://www.internetlivestats.com/twitter-statistics/

Cited By

Griggio CGorman BTigwell G(2024)Party Face Congratulations! Exploring Design Ideas to Help Sighted Users with Emoji Accessibility when Messaging with Screen Reader UsersProceedings of the ACM on Human-Computer Interaction10.1145/36410148:CSCW1(1-31)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641014
Smith CMiller Hillberg HLevonian Z(2023)"Thoughts & Prayers" or " ❤️ & 🙏 ": How the Release of New Reactions on CaringBridge Reshapes Supportive Communication in Health CrisesProceedings of the ACM on Human-Computer Interaction10.1145/36100357:CSCW2(1-39)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610035
Lee KLi HWellyanto MTham YMonroy-Hernández ALiu FSmith BVaish R(2023)Exploring Immersive Interpersonal Communication via ARProceedings of the ACM on Human-Computer Interaction10.1145/35794837:CSCW1(1-25)Online publication date: 16-Apr-2023
https://dl.acm.org/doi/10.1145/3579483
Show More Cited By

Index Terms

What I See is What You Don't Get: The Effects of (Not) Seeing Emoji Rendering Differences across Platforms
1. Human-centered computing
  1. Collaborative and social computing
    1. Empirical studies in collaborative and social computing

Recommendations

A case for emojis, more or less: An analysis of word and emoji expressivity in text messaging for high and low alexithymia levels
Abstract
The impact of alexithymia on verbal and non-verbal expressivity was investigated in one of the most prevalent mediums for present-day human interacting, smartphone text messaging. The variety of words and emojis used to express ...
Graphical abstract

Display Omitted
Highlights
- Deficits in verbal and non-verbal expressivity not universal in alexithymia.
- ...
The communicative role of non-face emojis

Emojis have evolved from imitations of facial expressions meant to communicate affect into pictures of objects, food, and places that are not directly linked to affect. While emojis that resemble facial expressions are well-researched, emojis that ...
The Interplay Between Emojis and Linguistic Text
Chinese Lexical Semantics
Abstract
This paper examines the interplay between emojis and the accompanying linguistic text when the two channels disagree with each other. In general, emojis are typically regarded as an explicit way to convey a specific emotion. However, it is ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 2, Issue CSCW

November 2018

4104 pages

EISSN:2573-0142

DOI:10.1145/3290265

Editors:
Karrie Karahalios
University of Illinois & Adobe
,
Andrés Monroy-Hernández
Snap Inc.
,
Airi Lampinen
Stockholm University
,
Geraldine Fitzpatrick
Vienna University of Technology

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2018

Published in PACMHCI Volume 2, Issue CSCW

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
1,423
Total Downloads

Downloads (Last 12 months)243
Downloads (Last 6 weeks)43

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Griggio CGorman BTigwell G(2024)Party Face Congratulations! Exploring Design Ideas to Help Sighted Users with Emoji Accessibility when Messaging with Screen Reader UsersProceedings of the ACM on Human-Computer Interaction10.1145/36410148:CSCW1(1-31)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641014
Smith CMiller Hillberg HLevonian Z(2023)"Thoughts & Prayers" or " ❤️ & 🙏 ": How the Release of New Reactions on CaringBridge Reshapes Supportive Communication in Health CrisesProceedings of the ACM on Human-Computer Interaction10.1145/36100357:CSCW2(1-39)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610035
Lee KLi HWellyanto MTham YMonroy-Hernández ALiu FSmith BVaish R(2023)Exploring Immersive Interpersonal Communication via ARProceedings of the ACM on Human-Computer Interaction10.1145/35794837:CSCW1(1-25)Online publication date: 16-Apr-2023
https://dl.acm.org/doi/10.1145/3579483
Akadal ESöyük M(2022)Interpretation of Emoji-Only Messages by Message Producers and ReceiversInternational Journal of Technology and Human Interaction10.4018/IJTHI.29906718:1(1-17)Online publication date: 22-Mar-2022
https://doi.org/10.4018/IJTHI.299067
Sun NKong R(2022)Internet of Things Information System and Clothing Computer Renderings Digital ArtScientific Programming10.1155/2022/92801032022(1-11)Online publication date: 28-Sep-2022
https://doi.org/10.1155/2022/9280103
Krekhov AEmmerich KFuchs JKrueger J(2022)Interpolating Happiness: Understanding the Intensity Gradations of Face Emojis Across CulturesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517661(1-17)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517661
Wikström VFalcon MMartikainen SPejoska JDurall EBauters MSaarikivi K(2021)Heart Rate Sharing at the WorkplaceMultimodal Technologies and Interaction10.3390/mti51000605:10(60)Online publication date: 8-Oct-2021
https://doi.org/10.3390/mti5100060
Fugate JFranco C(2021)Implications for Emotion: Using Anatomically Based Facial Coding to Compare Emoji Faces Across PlatformsFrontiers in Psychology10.3389/fpsyg.2021.60592812Online publication date: 25-Feb-2021
https://doi.org/10.3389/fpsyg.2021.605928
Robertson AMagdy WGoldwater S(2021)Black or White but Never Neutral: How Readers Perceive Identity from Yellow or Skin-toned EmojiProceedings of the ACM on Human-Computer Interaction10.1145/34760915:CSCW2(1-23)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3476091
Ling CAbuHilal IBlackburn JDe Cristofaro EZannettou SStringhini G(2021)Dissecting the Meme Magic: Understanding Indicators of Virality in Image MemesProceedings of the ACM on Human-Computer Interaction10.1145/34491555:CSCW1(1-24)Online publication date: 22-Apr-2021
https://dl.acm.org/doi/10.1145/3449155
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents