research-article

Understanding emotions in SNS images from posters' perspectives

Authors:

Sang-Wook KimAuthors Info & Claims

SAC '20: Proceedings of the 35th Annual ACM Symposium on Applied Computing

Pages 450 - 457

https://doi.org/10.1145/3341105.3373923

Published: 30 March 2020 Publication History

Abstract

As the popularity of media-based social networking services (SNS), such as Instagram and Snapchat, has increased significantly, a growing body of research has analyzed SNS images in relation to emotional analysis and classification model development. However, these prior studies were based on relatively small amounts of data, where the emotions of images were labeled from viewers' perspectives, not posters' perspectives. Consequently, we analyze 120K images that reflect poster's emotion. We develop color- and content-based classification models by considering: (1) the dynamics of SNS, in terms of the volume and variety of images shared, and (2) the fact that people express their emotions through colors and objects. We demonstrate the comparable performance of our model with models proposed in prior studies and discuss the applications.

References

[1]

Morgan Ames and Mor Naaman. 2007. Why We Tag: Motivations for Annotation in Mobile and Online Media. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. 971--980.

Digital Library

[2]

Barry J Babin, David M Hardesty, and Tracy A Suter. 2003. Color and Shopping Intentions: The Intervening Effect of Price Fairness and Perceived Affect. Journal of business research 56, 7 (2003), 541--551.

[3]

Matteo Baldoni, Cristina Baroglio, Viviana Patti, and Paolo Rena. 2012. From Tags to Emotions: Ontology-driven Sentiment Analysis in The Social Semantic Web. Intelligenza Artificiale 6, 1 (2012), 41--54.

[4]

David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent Dirichlet Allocation. Journal of machine Learning research 3, Jan (2003), 993--1022.

Digital Library

[5]

Damian Borth, Tao Chen, Rongrong Ji, and Shih-Fu Chang. 2013. Sentibank: Large-scale Ontology and Classifiers for Detecting Sentiment and Emotions in Visual Content. In Proceedings of the ACM International Conference on Multimedia. 459--460.

Digital Library

[6]

Damian Borth, Rongrong Ji, Tao Chen, Thomas Breuel, and Shih-Fu Chang. 2013. Large-scale Visual Sentiment Ontology and Detectors using Adjective Noun Pairs. In Proceedings of the ACM International Conference on Multimedia. 223--232.

Digital Library

[7]

Ming Chen, Lu Zhang, and Jan P Allebach. 2015. Learning Deep Features for Image Emotion Classification. In Proceedings of IEEE International Conference on Image Processing. 4491--4495.

Digital Library

[8]

François Chollet. 2016. Xception: Deep Learning with Depthwise Separable Convolutions. arXiv preprint (2016).

[9]

Mihaly Csikszentmihalyi and Eugene Halton. 1981. The Meaning of Things: Domestic Symbols and The Self. Cambridge University Press.

[10]

Paul Ekman. 1992. An Argument for Basic Emotions. Cognition & emotion 6, 3--4 (1992), 169--200.

[11]

Richard Hanna, Andrew Rohm, and Victoria L Crittenden. 2011. We're All Connected: The Power of The Social Media Ecosystem. Business horizons 54, 3 (2011), 265--273.

[12]

Simon Haykin and Neural Network. 2004. A Comprehensive Foundation. Neural networks 2, 2004 (2004), 41.

[13]

Carroll E Izard. 1991. The Psychology of Emotions. Springer Science & Business Media.

[14]

Carroll E Izard. 2007. Basic Emotions, Natural kinds, Emotion Schemas, and A New Paradigm. Perspectives on psychological science 2, 3 (2007), 260--280.

[15]

Jin Yea Jang, Kyungsik Han, Patrick C Shih, and Dongwon Lee. 2015. Generation Like: Comparative Characteristics in Instagram. In Proceedings of the ACM Conference on Human Factors in Computing Systems. 4039--4042.

Digital Library

[16]

A. Kaplan and M. Haenlein. 2010. Users of The World, Unite! The Challenges and Opportunities of Social Media. Business Horizons 53, 1 (2010), 59--68.

[17]

Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. 2014. Large-scale Video Classification with Convolutional Neural Networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 1725--1732.

Digital Library

[18]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980 (2014).

[19]

Peter J Lang, Margaret M Bradley, and Bruce N Cuthbert. 1997. International Affective Picture System (IAPS): Technical Manual and Affective Ratings. NIMH Center for the Study of Emotion and Attention (1997), 39--58.

[20]

Xin Li, Lei Guo, and Yihong Eric Zhao. 2008. Tag-based Social Interest Discovery. In Proceedings of the International Conference on World Wide Web. 675--684.

Digital Library

[21]

Xin Lu, Reginald B Adams, Jia Li, Michelle G Newman, and James Z Wang. 2017. An Investigation into Three Visual Characteristics of Complex Scenes that Evoke Human Emotion. In Proceedings of the IEEE International Conference on Affective Computing and Intelligent Interaction. 440--447.

[22]

Xin Lu, Poonam Suryanarayan, Reginald B Adams Jr, Jia Li, Michelle G Newman, and James Z Wang. 2012. On Shape and The Computability of Emotions. In Proceedings of the ACM International Conference on Multimedia. 229--238.

Digital Library

[23]

Jana Machajdik and Allan Hanbury. 2010. Affective Image Classification using Features Inspired by Psychology and Art Theory. In Proceedings of the ACM International Conference on Multimedia. 83--92.

Digital Library

[24]

Aqdas Malik, Amandeep Dhir, and Marko Nieminen. 2016. Uses and Gratifications of Digital Photo Sharing on Facebook. Telematics and Informatics 33, 1 (2016), 129--138.

Digital Library

[25]

W Glynn Mangold and David J Faulds. 2009. Social Media: The New Hybrid Element of The Promotion Mix. Business horizons 52, 4 (2009), 357--365.

[26]

KAYA NAz and Helena Epps. 2004. Relationship between Color and Emotion: A Study of College Students. College Student J 38, 3 (2004), 396.

[27]

Oded Nov, Mor Naaman, and Chen Ye. 2008. What Drives Content Tagging: The Case of Photos on Flickr. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. 1097--1100.

Digital Library

[28]

Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-scale Image Recognition. arXiv preprint arXiv:1409.1556 (2014).

[29]

Aaron Smith, Monica Anderson, and Tom Caiazza. 2018. Social Media Use in 2018.

[30]

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the Inception Architecture for Computer Vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2818--2826.

[31]

Joshua B Tenenbaum, Vin De Silva, and John C Langford. 2000. A Global Geometric Framework for Nonlinear Dimensionality Reduction. science 290, 5500 (2000), 2319--2323.

[32]

Kate Vaisutis, Margot Brereton, Toni Robertson, Frank Vetere, Jeannette Durick, Bjorn Nansen, and Laurie Buys. 2014. Invisible Connections: Investigating Older People's Emotions and Social Relations around Objects. In Proceedings of the ACM Conference on Human Factors in Computing Systems. 1937--1940.

Digital Library

[33]

Patricia Valdez and Albert Mehrabian. 1994. Effects of Color on Emotions. Journal of experimental psychology: General 123, 4 (1994), 394.

[34]

José Van Dijck. 2013. The Culture of Connectivity: A Critical History of Social Media. Oxford University Press.

[35]

Anita Whiting and David Williams. 2013. Why People Use Social Media: A Uses and Gratifications Approach. Qualitative Market Research: An International Journal 16, 4 (2013), 362--369.

[36]

Can Xu, Suleyman Cetintas, Kuang-Chih Lee, and Li-Jia Li. 2014. Visual Sentiment Prediction with Deep Convolutional Neural Networks. arXiv preprint arXiv:1411.5731 (2014).

[37]

Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang. 2016. Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark. In Proceedings of the AAAI Conference on Artificial Intelligence. 308--314.

[38]

Sicheng Zhao, Yue Gao, Xiaolei Jiang, Hongxun Yao, Tat-Seng Chua, and Xiaoshuai Sun. 2014. Exploring Principles-of-art Features for Image Emotion Recognition. In Proceedings of the ACM International Conference on Multimedia. 47--56.

Digital Library

Cited By

Zhai YGuo NZhang JZhang HSun SDing Y(2024)The thousand faces of images in AI news: psychological distance, dialectical relationships and sensationalismInformation, Communication & Society10.1080/1369118X.2024.2406811(1-23)Online publication date: 26-Sep-2024
https://doi.org/10.1080/1369118X.2024.2406811

Index Terms

Understanding emotions in SNS images from posters' perspectives
1. Applied computing
  1. Arts and humanities
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Sentiment analysis

Recommendations

SNS flow, SNS self-disclosure and post hoc interpersonal relations change: Focused on Korean Facebook user
Highlights
- The SNS flow experiences of intrinsic motivation have impact on the post hoc interpersonal relations.
- The SNS flow experiences of intrinsic motivation have impact on the SNS self-disclosure.
- The SNS self-disclosure has impact on ...
Abstract
This study empirically investigates how the flow experiences when using SNS have impacted SNS interaction and interpersonal relationships. Using a survey of Facebook users, this study examines how the SNS flow as an intrinsic motivation have an ...
Does social capital affect SNS usage? A look at the roles of subjective well-being and social identity

Examines the effects of social capital on SNS use (qualitative and quantitative use).Found that bridging capital significantly impacts qualitative SNS use.Verified the mediating role of subjective well-being between bonding capital and quantitative SNS ...
Analyzing negative SNS behaviors of elementary and middle school students in Korea

Increase of SNS use increases committing 'contact strangers' to the greatest extent.Grade difference is much more significant than that of gender in negative behavior.We established a regression model to identify factors affecting negative ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '20: Proceedings of the 35th Annual ACM Symposium on Applied Computing

March 2020

2348 pages

ISBN:9781450368667

DOI:10.1145/3341105

Conference Chairs:
Chih-Cheng Hung
Kennesaw State University
,
Tomas Cerny
Baylor University
,
Program Chairs:
Dongwan Shin
New Mexico Tech
,
Alessio Bechini
University of Pisa, Italy

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Research Foundation of Korea

Conference

SAC '20

Sponsor:

SIGAPP

SAC '20: The 35th ACM/SIGAPP Symposium on Applied Computing

March 30 - April 3, 2020

Brno, Czech Republic

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25

Sponsor:
sigapp

The 40th ACM/SIGAPP Symposium on Applied Computing

March 31 - April 4, 2025

Catania , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
323
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)3

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhai YGuo NZhang JZhang HSun SDing Y(2024)The thousand faces of images in AI news: psychological distance, dialectical relationships and sensationalismInformation, Communication & Society10.1080/1369118X.2024.2406811(1-23)Online publication date: 26-Sep-2024
https://doi.org/10.1080/1369118X.2024.2406811

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten