research-article

Open access

It is an online platform and not the real world, I don’t care much: Investigating Twitter Profile Credibility With an Online Machine Learning-Based Tool

Authors:

Ville Paananen,

Sharadhi Alape Suryanarayana,

Miikka Kuutila,

Mika Mäntylä,

Simo HosioAuthors Info & Claims

CHIIR '23: Proceedings of the 2023 Conference on Human Information Interaction and Retrieval

Pages 117 - 127

https://doi.org/10.1145/3576840.3578308

Published: 20 March 2023 Publication History

All formats PDF

Abstract

Social media is now an important source of everyday information. Given the plethora of scandals concerning the rapid spread of misinformation and disinformation on social media, the credibility of the content on these platforms is now a pivotal research area. Much of the existing work on social media credibility focuses on content credibility. In this study, however, we focus on the credibility of the profile as the virtual representation of the content author. We developed a real-time machine-learning-based online tool that assesses the credibility of profiles on Twitter, one of the most common and versatile social media platforms. To investigate user perceptions on credibility-related issues, we used our tool as a stimulus for people to reflect on their profile’s credibility and collected 100 responses. The combination of our quantitative and qualitative analysis reveals that the latest tweets and retweet behavior are two of the most critical factors for profile credibility. It is also observed that people demonstrate a limited interest in their profile credibility but agree that the author’s credibility is of paramount importance. With an open-source tool to assess user credibility on Twitter and a user study to establish its utility, we contribute a timely piece of research on the topic of online credibility.

References

[1]

2022. Python Package treeinterpreter. https://github.com/andosa/treeinterpreter.

[2]

2022. Scikit Learn. https://scikit-learn.org/stable.

[3]

Mohammad-Ali Abbasi and Huan Liu. 2013. Measuring user credibility in social media. In International Conference on Social Computing, Behavioral-Cultural Modeling, and Prediction. Springer, 441–448.

Digital Library

[4]

Hunt Allcott and Matthew Gentzkow. 2017. Social media and fake news in the 2016 election. Journal of economic perspectives 31, 2 (2017), 211–36.

[5]

Majed Alrubaian, Muhammad Al-Qurishi, Mabrook Al-Rakhami, Mohammad Mehedi Hassan, and Atif Alamri. 2017. Reputation-based credibility analysis of Twitter social network users. Concurrency and Computation: Practice and Experience 29, 7(2017), e3873.

[6]

Inc. Amazon.com. 2022. Amazon Mechanical Turk. https://www.mturk.com.

[7]

Michael Barthel, Amy Mitchell, Dorene Asare-Marfo, Courtney Kennedy, and Kirsten Worden. 2020. Measuring news consumption in a digital era. Pew Research Center, Washington, DC. See https://www. journalism. org/2020/12/08/measuring-news-consumption-in-a-digital-era(2020).

[8]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.

[9]

Yudith Cardinale, Irvin Dongo, Germán Robayo, David Cabeza, Ana Aguilera, and Sergio Medina. 2021. T-CREo: A Twitter Credibility Analysis Framework. IEEE Access 9(2021), 32498–32516.

[10]

Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Information credibility on twitter. In Proceedings of the 20th international conference on World wide web. 675–684.

Digital Library

[11]

Nurendra Choudhary, Rajat Singh, Ishita Bindlish, and Manish Shrivastava. 2018. Neural network architecture for credibility assessment of textual claims. arXiv preprint arXiv:1803.10547(2018).

[12]

THM De Silva. 2021. A Network Analysis Based Credibility Ranking Model to Combat Misinformation on Twitter. Ph. D. Dissertation.

[13]

Yecely Aridaí Díaz-Beristain, Guillermo-de-Jesús Hoyos-Rivera, and Nicandro Cruz-Ramírez. 2017. Strategies for Growing User Popularity through Retweet: An Empirical Study. Advances in Multimedia 2017 (2017).

[14]

Irvin Dongo, Yudith Cardinale, Ana Aguilera, Fabiola Martinez, Yuni Quintero, German Robayo, and David Cabeza. 2021. A qualitative and quantitative comparison between Web scraping and API methods for Twitter credibility analysis. International Journal of Web Information Systems (2021).

[15]

Nora A Draper and Joseph Turow. 2019. The corporate cultivation of digital resignation. New media & society 21, 8 (2019), 1824–1839.

[16]

Chad Edwards, Brett Stoll, Natalie Faculak, and Sandi Karman. 2015. Social presence on LinkedIn: Perceived credibility and interpersonal attractiveness based on user profile picture. Online Journal of Communication and Media Technologies 5, 4(2015), 102.

[17]

Claudia Flores-Saviaga and Saiph Savage. 2021. Fighting disaster misinformation in Latin America: the# 19S Mexican earthquake case study. Personal and Ubiquitous Computing 25, 2 (2021), 353–373.

Digital Library

[18]

Ujwal Gadiraju, Ricardo Kawase, Stefan Dietze, and Gianluca Demartini. 2015. Understanding malicious behavior in crowdsourcing platforms: The case of online surveys. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 1631–1640.

Digital Library

[19]

Seyed Mohssen Ghafari, Amin Beheshti, Aditya Joshi, Cecile Paris, Adnan Mahmood, Shahpar Yakhchi, and Mehmet A Orgun. 2020. A survey on trust prediction in online social networks. IEEE Access 8(2020), 144292–144309.

[20]

James J Heckman. 2010. Selection bias and self-selection. In Microeconometrics. Springer, 242–266.

[21]

Matthew Hindman and Vlad Barash. 2018. Disinformation, and influence campaigns on twitter. Knight Foundation: George Washington University (2018).

[22]

E. K. Hmlinen, C. Kiili, M. Marttunen, E. Rikknen, and Pht Leppnen. 2020. Promoting sixth graders’ credibility Evaluation of Web pages: An intervention study. Computers in Human Behavior(2020), 106372.

[23]

Celestine Iwendi, Ali Kashif Bashir, Atharva Peshkar, R Sujatha, Jyotir Moy Chatterjee, Swetha Pasupuleti, Rishita Mishra, Sofia Pillai, and Ohyun Jo. 2020. COVID-19 patient health prediction using boosted random forest algorithm. Frontiers in public health 8 (2020), 357.

[24]

Maurice Jakesch, Megan French, Xiao Ma, Jeffrey T Hancock, and Mor Naaman. 2019. AI-mediated communication: How the perception that profile text was written by AI affects trustworthiness. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[25]

Lars König and Regina Jucks. 2019. Hot topics in science communication: Aggressive language decreases trustworthiness and credibility in scientific debates. Public understanding of science 28, 4 (2019), 401–416.

[26]

Haewoon Kwak, Changhyun Lee, Hosung Park, and Sue Moon. 2010. What is Twitter, a social network or a news media?. In Proceedings of the 19th international conference on World wide web. 591–600.

Digital Library

[27]

Chen Lou and Shupei Yuan. 2019. Influencer marketing: how message value and credibility affect consumer trust of branded content on social media. Journal of Interactive Advertising 19, 1 (2019), 58–73.

[28]

Scott R Maier, Marcus Mayorga, and Paul Slovic. 2017. Personalized news stories affect men as well as women. Newspaper Research Journal 38, 2 (2017), 172–186.

[29]

Scott R Maier, Paul Slovic, and Marcus Mayorga. 2017. Reader reaction to news of mass suffering: Assessing the influence of story form and emotional response. Journalism 18, 8 (2017), 1011–1029.

[30]

Tanushree Mitra and Eric Gilbert. 2015. Credbank: A large-scale social media corpus with associated credibility annotations. In Ninth international AAAI conference on web and social media.

[31]

Stefan Palan and Christian Schitter. 2018. Prolific.ac — A subject pool for online experiments. Journal of Behavioral and Experimental Finance 17 (2018), 22–27.

[32]

Julio CS Reis, André Correia, Fabrício Murai, Adriano Veloso, and Fabrício Benevenuto. 2019. Supervised learning for fake news detection. IEEE Intelligent Systems 34, 2 (2019), 76–81.

Digital Library

[33]

Ellen Rusman, Jan van Bruggen, Peter Sloep, and Martin Valcke. 2013. Profile elements that inform first impressions of trustworthiness in virtual project teams. INTERNATIONAL JOURNAL OF INFORMATION TECHNOLOGY PROJECT MANAGEMENT 3, 1(2013), 15–35.

Digital Library

[34]

Giuseppe Sansonetti, Fabio Gasparetti, Giuseppe D’aniello, and Alessandro Micarelli. 2020. Unreliable users detection in social media: Deep learning techniques for automatic detection. IEEE Access 8(2020), 213154–213167.

[35]

Koichi Sato, Junbo Wang, and Zixue Cheng. 2018. Credibility evaluation of Twitter-based event detection by a mixing analysis of heterogeneous data. IEEE Access 7(2018), 1095–1106.

[36]

Charles C Self. 2014. Credibility. In An integrated approach to communication theory and research. Routledge, 449–470.

[37]

Erwin B Setiawan, Dwi H Widyantoro, and Kridanto Surendro. 2020. Measuring information credibility in social media using combination of user profile and message content dimensions.International Journal of Electrical & Computer Engineering (2088-8708) 10, 4(2020).

[38]

Shafiza Mohd Shariff, Xiuzhen Zhang, and Mark Sanderson. 2014. User perception of information credibility of news on twitter. In European conference on information retrieval. Springer, 513–518.

[39]

Elizabeth Sillence, Pam Briggs, Peter Richard Harris, and Lesley Fishwick. 2007. How do patients evaluate and make use of online health information?Social science & medicine 64, 9 (2007), 1853–1862.

[40]

Shivangi Singhal, Rajiv Ratn Shah, Tanmoy Chakraborty, Ponnurangam Kumaraguru, and Shin’ichi Satoh. 2019. Spotfake: A multi-modal framework for fake news detection. In 2019 IEEE fifth international conference on multimedia big data (BigMM). IEEE, 39–47.

[41]

David Sterrett, Dan Malato, Jennifer Benz, Liz Kantor, Trevor Tompson, Tom Rosenstiel, Jeff Sonderman, and Kevin Loker. 2019. Who shared it?: Deciding what news to trust on social media. Digital journalism 7, 6 (2019), 783–801.

[42]

Onur Varol, Emilio Ferrara, Clayton Davis, Filippo Menczer, and Alessandro Flammini. 2017. Online human-bot interactions: Detection, estimation, and characterization. In Proceedings of the international AAAI conference on web and social media, Vol. 11. 280–289.

[43]

Senuri Wijenayake, Danula Hettiachchi, Simo Johannes Hosio, Vassilis Kostakos, and Jorge Goncalves. 2020. Effect of Conformity on Perceived Trustworthiness of News in Social Media. IEEE Internet Computing(2020).

[44]

Shiyang Xuan, Guanjun Liu, Zhenchuan Li, Lutao Zheng, Shuo Wang, and Changjun Jiang. 2018. Random forest for credit card fraud detection. In 2018 IEEE 15th international conference on networking, sensing and control (ICNSC). IEEE, 1–6.

[45]

Waheeb Yaqub, Otari Kakhidze, Morgan L Brockman, Nasir Memon, and Sameer Patil. 2020. Effects of credibility indicators on social media news sharing intent. In Proceedings of the 2020 chi conference on human factors in computing systems. 1–14.

Digital Library

[46]

Fang Zhou, Jianlin Jin, Xiaojiang Du, Bowen Zhang, and Xucheng Yin. 2017. A calculation method for social network user credibility. In 2017 IEEE International Conference on Communications (ICC). IEEE, 1–6.

Cited By

García-Méndez Sde Arriba-Pérez FGonzález-González JGonzález-Castaño F(2024)Explainable assessment of financial experts’ credibility by classifying social media forecasts and checking the predictions with actual market dataExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.124515255:PAOnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.124515
Li JKuutila MHuusko EKariyakarawana NSavic MAhooie NHosio SMäntylä M(2023)Assessing Credibility Factors of Short-Form Social Media Posts: A Crowdsourced Online ExperimentProceedings of the 15th Biannual Conference of the Italian SIGCHI Chapter10.1145/3605390.3605406(1-14)Online publication date: 20-Sep-2023
https://dl.acm.org/doi/10.1145/3605390.3605406

Index Terms

It is an online platform and not the real world, I don’t care much: Investigating Twitter Profile Credibility With an Online Machine Learning-Based Tool
1. Human-centered computing

Recommendations

Assessing Credibility Factors of Short-Form Social Media Posts: A Crowdsourced Online Experiment
CHItaly '23: Proceedings of the 15th Biannual Conference of the Italian SIGCHI Chapter

People commonly turn to the Internet and social media for their information needs. Most popular social media platforms focus on short-form content that can be consumed rapidly. Given how fast such content spreads online, its trustworthiness and ...
In Twitter we trust(ed): How perceptions of Twitter's helpfulness influence news post credibility perceptions and news engagement
Abstract
This study examines the impacts of news veracity, source credibility, collective social endorsement, and perceptions of Twitter's utility on perceived credibility of and engagement with social media news posts. We conducted a 2 (Veracity: real ...
Highlights
- News veracity and source credibility influence assessments of post credibility.
- Social endorsement does not affect post credibility perceptions or behavioral intent.
- Twitter believers more likely perceive posts as credible and ...
Learning in massive open online courses

Various social media mining approaches were adopted for analyzing MOOC learning.The daily, weekly, and monthly trends of MOOC-related tweets were investigated.Sentiment analysis indicated mixed public opinion toward MOOCs.The influencers of sentimental ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHIIR '23: Proceedings of the 2023 Conference on Human Information Interaction and Retrieval

March 2023

520 pages

ISBN:9798400700354

DOI:10.1145/3576840

Editors:
Jacek Gwizdka
School of Information, The University of Texas at Austin, Texas, USA
,
Soo Young Rieh
School of Information, The University of Texas at Austin, Texas, USA

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 March 2023

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Academy of Finland

Conference

CHIIR '23

Sponsor:

CHIIR '23: ACM SIGIR Conference on Human Information Interaction and Retrieval

March 19 - 23, 2023

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 55 of 163 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
794
Total Downloads

Downloads (Last 12 months)566
Downloads (Last 6 weeks)30

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

García-Méndez Sde Arriba-Pérez FGonzález-González JGonzález-Castaño F(2024)Explainable assessment of financial experts’ credibility by classifying social media forecasts and checking the predictions with actual market dataExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.124515255:PAOnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.124515
Li JKuutila MHuusko EKariyakarawana NSavic MAhooie NHosio SMäntylä M(2023)Assessing Credibility Factors of Short-Form Social Media Posts: A Crowdsourced Online ExperimentProceedings of the 15th Biannual Conference of the Italian SIGCHI Chapter10.1145/3605390.3605406(1-14)Online publication date: 20-Sep-2023
https://dl.acm.org/doi/10.1145/3605390.3605406

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents