research-article

Complex and Ambiguous: Understanding Sticker Misinterpretations in Instant Messaging

Authors:

Uichin LeeAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 2, Issue CSCW

Article No.: 30, Pages 1 - 22

https://doi.org/10.1145/3274299

Published: 01 November 2018 Publication History

Abstract

Stickers, though similar in appearance to emoji, have distinct characteristics because they often contain animation, diverse gestures, and multiple characters and objects. Stickers can convey richer meaning than emoji, but their complexity and placement constraint may result in miscommunication. In this paper, we aim to understand how people perceive emotion in stickers, as well as how miscommunication related to sticker occurs in actual chat contexts. Toward this goal, we conducted an online survey (n = 87) and in-depth interviews (n = 28) in South Korea. We found emotional and contextual aspects of sticker misinterpretation. In particular, emotion misinterpretation mostly happened due to stickers' ambiguous (multiple) facial/bodily expressions and different perception of dynamism in gestures. In real chat settings, there were also contextual misinterpretations where senders and receivers differently interpret stickers' visual representation/reference, or/and corresponding textual messages. Based on these findings, we provide several practical design implications such as context awareness support in sticker interaction.

References

[1]

James A Russell. 2003. Core Affect and the Psychological Construction of Emotion. Psychological Review, Vol. 110, 1 (2003), 145--172.

[2]

Jr Artemio Ramirez and Kathy Broneck. 2009. `IM me': Instant messaging as relational maintenance and everyday communication. Journal of Social and Personal Relationships, Vol. 26, 2--3 (2009), 291--314.

[3]

Eric PS Baumer, Morgan G Ames, Jenna Burrell, Jed R Brubaker, and Paul Dourish. 2015. Why Study Technology Non-use? First Monday, Vol. 20, 11 (2015). http://firstmonday.org/ojs/index.php/fm/article/view/6310

[4]

Hugh Beyer and Karen Holtzblatt. 1998. Contextual Design: Defining Customer-Centered Systems .Academic Press.

Digital Library

[5]

Dan Bouhnik and Mor Deshen. 2014. WhatsApp Goes to School: Mobile Instant Messaging between Teachers and Students. Journal of Information Technology Education: Research, Vol. 13 (1 2014), 217--231.

[6]

Judee K. Burgoon and Beth A. Le Poire. 1999. Nonverbal Cues and Interpersonal Judgments: Participant and Observer Perceptions of Intimacy, Dominance, Composure, and Formality. Communication Monographs, Vol. 66, 2 (1999), 105--124.

[7]

Karen Church and Rodrigo de Oliveira. 2013. What's Up with Whatsapp?: Comparing Mobile Instant Messaging Behaviors with Traditional SMS. In Proceedings of the 15th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '13). ACM.

Digital Library

[8]

Herbert H. Clark. 1996. Using Language .Cambridge University Press.

[9]

Kakao Corp. 2018. Investor Relations. (2018). https://t1.kakaocdn.net/kakaocorp/operating/ir/results-announcement/3246.pdf

[10]

Henriette Cramer, Paloma de Juan, and Joel Tetreault. 2016. Sender-intended Functions of Emojis in US Messaging. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16). ACM.

Digital Library

[11]

Daantje Derks, Arjan E. R. Bos, and Jasper von Grumbkow. 2008. Emoticons and Online Message Interpretation. Social Science Computer Review, Vol. 26, 3 (2008), 379--388.

Digital Library

[12]

Alan Dix. 1994. Computer Supported Cooperative Work: A Framework .Springer London, 9--26.

[13]

Alan Dix. 2007. Designing for Appropriation. In Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...But Not As We Know It - Volume 2 (BCS-HCI '07). BCS Learning & Development Ltd.

Digital Library

[14]

Paul Dourish. 2004. What We Talk About when We Talk About Context. Personal and Ubiquitous Computing, Vol. 8, 1 (Feb. 2004), 19--30.

Digital Library

[15]

Eli Dresner and Susan Herring. 2010. Functions of the Nonverbal in CMC: Emoticons and Illocutionary Force., Vol. 20, 3 (2010), 249 -- 268.

[16]

Sara H. Hsieh and Timmy H. Tseng. 2017. Playfulness in Mobile Instant Messaging. Comput. Hum. Behav., Vol. 69, C (2017), 405--414.

Digital Library

[17]

Maximilian J. Johnston, Dominic King, Sonal Arora, Nebil Behar, Thanos Athanasiou, Nick Sevdalis, and Ara Darzi. 2015. Smartphones Let Surgeons Know WhatsApp: An Analysis of Communication in Emergency Surgical Teams. The American Journal of Surgery, Vol. 209, 1 (2015), 45 -- 51.

[18]

Shipra Kayan, Susan R. Fussell, and Leslie D. Setlock. 2006. Cultural Differences in the Use of Instant Messaging in Asia and North America. In Proceedings of the 20th Anniversary Conference on Computer Supported Cooperative Work (CSCW '06). ACM.

Digital Library

[19]

Linda K. Kaye, Helen J. Wall, and Stephanie A. Malone. 2016. "Turn that Frown Upside-down": A Contextual Account of Emoticon Usage on Different Virtual Platforms. Computers in Human Behavior, Vol. 60 (2016), 463 -- 467. Issue C.

Digital Library

[20]

Ryan Kelly and Leon Watts. 2015. Characterising the Inventive Appropriation of Emoji as Relationally Meaningful in Mediated Close Personal Relationships. In Experiences of Technology Appropriation: Unanticipated Users, Usage, Circumstances, and Design . http://opus.bath.ac.uk/46780/

[21]

Da-jung Kim and Youn-kyung Lim. 2015. Dwelling Places in KakaoTalk: Understanding the Roles and Meanings of Chatrooms in Mobile Instant Messengers. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15). ACM.

Digital Library

[22]

Joon Young Lee, Nahi Hong, Soomin Kim, Jonghwan Oh, and Joonhwan Lee. 2016. Smiley Face: Why We Use Emoticon Stickers in Mobile Messaging. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '16). ACM.

Digital Library

[23]

Shao-Kang Lo. 2008. The Nonverbal Communication Functions of Emoticons in Computer-Mediated Communication. Cyberpsychology & Behavior, Vol. 11 (11 2008), 595--597.

[24]

Hannah Jean Miller, Daniel Kluver, Jacob Thebault-Spieker, Loren G. Terveen, and Brent J. Hecht. 2017. Understanding Emoji Ambiguity in Context: The Role of Text in Emoji-Related Miscommunication. In Proceedings of the 11th International Conference on Web and Social Media (ICWSM' 17). AAAI Press, 152--161.

[25]

Hannah Jean Miller, Jacob Thebault-Spieker, Shuo Chang, Isaac L. Johnson, Loren G. Terveen, and Brent J. Hecht. 2016. "Blissfully Happy" or "Ready to Fight": Varying Interpretations of Emoji. In Proceedings of the 10th International Conference on Web and Social Media (ICWSM' 16). AAAI Press, 259--268.

[26]

Fred Morstatter, Kai Shu, Suhang Wang, and Huan Liu. 2017. Cross-Platform Emoji Interpretation: Analysis, a Solution, and Applications. CoRR, Vol. abs/1709.04969 (2017). arxiv: 1709.04969 http://arxiv.org/abs/1709.04969

[27]

Midas Nouwens, Carla F. Griggio, and Wendy E. Mackay. 2017. "WhatsApp is for Family; Messenger is for Friends": Communication Places in App Ecosystems. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM.

Digital Library

[28]

Kenton P. O'Hara, Michael Massimi, Richard Harper, Simon Rubens, and Jessica Morris. 2014. Everyday Dwelling with WhatsApp. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14). ACM, 13.

Digital Library

[29]

Catherine Shu. 2015. The Secret Language Of Line Stickers. (10 7 2015). https://techcrunch.com/2015/07/10/creepy-cute-line/

[30]

Satomi Sugiyama. 2015. Kawaii Meiru and Maroyaka Neko: Mobile Emoji for Relationship Maintenance and Aesthetic Expressions among Japanese Teens. First Monday, Vol. 20 (10 2015).

[31]

Channary Tauch and Eiman Kanjo. 2016. The Roles of Emojis in Mobile Phone Notifications. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (UbiComp '16). ACM.

Digital Library

[32]

Kakao Team. 2017. Kakao Celebrates 6-year Anniversary of Kakao Emoticons. (2017). https://blog.kakaocorp.com/?p=1860

[33]

David R Thomas. 2006. A General Inductive Approach for Analyzing Qualitative Evaluation Data. American Journal of Evaluation, Vol. 27, 2 (2006), 237--246.

[34]

Garreth W. Tigwell and David R. Flatla. 2016. Oh That's What You Meant!: Reducing Emoji Misunderstanding. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '16). ACM.

Digital Library

[35]

Joseph B. Walther. 1992. Interpersonal Effects in Computer-Mediated Interaction: A Relational Perspective. Communication Research, Vol. 19, 1 (1992), 52--90.

[36]

Joseph B. Walther and Kyle P. D'Addario. 2001. The Impacts of Emoticons on Message Interpretation in Computer-Mediated Communication. Social Science Computer Review, Vol. 19, 3 (2001), 324--347.

[37]

Sarah Wiseman and Sandy J. J. Gould. 2018. Repurposing Emoji for Personalised Communication: Why Slice of Pizza Emoji means “I Love You”. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM.

Digital Library

[38]

Rui Zhou, Jasmine Hentschel, and Neha Kumar. 2017. Goodbye Text, Hello Emoji: Mobile Communication on WeChat in China. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM.

Digital Library

Cited By

Linares Bernabéu EAlvarado Ortega M(2024)Unveiling humour in digital discourse: the pragmatic functions of humorous stickers in Spanish WhatsApp chat groupsLodz Papers in Pragmatics10.1515/lpp-2024-0008Online publication date: 15-Nov-2024
https://doi.org/10.1515/lpp-2024-0008
Chen JWang WHu YChen JLiu HHu XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)TGCA-PVT: Topic-Guided Context-Aware Pyramid Vision Transformer for Sticker Emotion RecognitionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680781(9709-9718)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680781
Mohamed MKhairuddin MLee M(2024)Exploring Students’ Written Discourse in Resumes: Unveiling Insights for Enhancing Social IntelligenceSHS Web of Conferences10.1051/shsconf/202418204007182(04007)Online publication date: 6-Feb-2024
https://doi.org/10.1051/shsconf/202418204007
Show More Cited By

Index Terms

Complex and Ambiguous: Understanding Sticker Misinterpretations in Instant Messaging
1. Human-centered computing
  1. Human computer interaction (HCI)
  2. Interaction design
2. Information systems
  1. World Wide Web
    1. Web applications
      1. Internet communications tools
        Chat

Recommendations

Culturally-Embedded Visual Literacy: A Study of Impression Management via Emoticon, Emoji, Sticker, and Meme on Social Media in China

Social media is often analyzed as a "front stage" where social media users perform their identities and manage impressions through various forms of user-generated content: text, picture, video, as well as visual expression such as emoticon, emoji, ...
Research on Sticker Cognition for Older Adult Using Instant Messaging
Human-Computer Interaction. Theories, Methods, and Human Issues
Abstract
The growing ubiquity of smartphones has changed how older people live and the advent of instant messaging apps has reshaped their communication methods. Stickers are arguably the most popular feature of LINE, an instant messaging app. In this ...
Perceived Usefulness of Emoticons, Emojis, and Stickers in Text Messaging: Effect of gender and Text-Messaging Dependency

In text-based communication, which lacks nonverbal cues, various techniques for expressing communicative intent are now available. Most prominently used are emoticons, emojis, and stickers. Although previous studies have separately examined emoticons ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 2, Issue CSCW

November 2018

4104 pages

EISSN:2573-0142

DOI:10.1145/3290265

Editors:
Karrie Karahalios
University of Illinois & Adobe
,
Andrés Monroy-Hernández
Snap Inc.
,
Airi Lampinen
Stockholm University
,
Geraldine Fitzpatrick
Vienna University of Technology

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2018

Published in PACMHCI Volume 2, Issue CSCW

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Electronics and Telecommunications Research Institute (ETRI) grant funded by the Korean government
Institute for Information communications Technology Promotion (IITP) grant funded by the Korea government (MSIT)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
740
Total Downloads

Downloads (Last 12 months)123
Downloads (Last 6 weeks)11

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Linares Bernabéu EAlvarado Ortega M(2024)Unveiling humour in digital discourse: the pragmatic functions of humorous stickers in Spanish WhatsApp chat groupsLodz Papers in Pragmatics10.1515/lpp-2024-0008Online publication date: 15-Nov-2024
https://doi.org/10.1515/lpp-2024-0008
Chen JWang WHu YChen JLiu HHu XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)TGCA-PVT: Topic-Guided Context-Aware Pyramid Vision Transformer for Sticker Emotion RecognitionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680781(9709-9718)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680781
Mohamed MKhairuddin MLee M(2024)Exploring Students’ Written Discourse in Resumes: Unveiling Insights for Enhancing Social IntelligenceSHS Web of Conferences10.1051/shsconf/202418204007182(04007)Online publication date: 6-Feb-2024
https://doi.org/10.1051/shsconf/202418204007
Lu BTa LDai HXu XYan WZhang Z(2024)Unfreezing the Freelancers: Investigating the Strategy of Digital Platform-Based Instant Messaging Communication in Increasing Freelancers’ Response in Gig EconomyInformation Systems Frontiers10.1007/s10796-024-10560-4Online publication date: 12-Dec-2024
https://doi.org/10.1007/s10796-024-10560-4
Wang YChang YLi LSheng YSow SFussell S(2024)Emoji Interpretation and Usage in Bilingual CommunicationHCI International 2024 Posters10.1007/978-3-031-61966-3_18(157-167)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61966-3_18
Lee KLi HWellyanto MTham YMonroy-Hernández ALiu FSmith BVaish R(2023)Exploring Immersive Interpersonal Communication via ARProceedings of the ACM on Human-Computer Interaction10.1145/35794837:CSCW1(1-25)Online publication date: 16-Apr-2023
https://dl.acm.org/doi/10.1145/3579483
Yang CYoshida SKuzuoka HNarumi TYamashita N(2023)Affective Profile Pictures: Exploring the Effects of Changing Facial Expressions in Profile Pictures on Text-Based CommunicationProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581061(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581061
Chandra YArdiyansyah S(2023)Analysis of the Use of E-Stickers in Chat Conversations for Higher Education Students2023 International Conference on Information Management and Technology (ICIMTech)10.1109/ICIMTech59029.2023.10277862(413-417)Online publication date: 24-Aug-2023
https://doi.org/10.1109/ICIMTech59029.2023.10277862
Costagliola GDe Rosa MFuccella VTabari P(2023)The Impact of the COVID-19 Pandemic on Human–Computer Interaction Empirical ResearchInteracting with Computers10.1093/iwc/iwad03135:5(578-589)Online publication date: 6-Apr-2023
https://doi.org/10.1093/iwc/iwad031
Akadal ESöyük M(2022)Interpretation of Emoji-Only Messages by Message Producers and ReceiversInternational Journal of Technology and Human Interaction10.4018/IJTHI.29906718:1(1-17)Online publication date: 22-Mar-2022
https://doi.org/10.4018/IJTHI.299067
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents