Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Search & Verify: : Misinformation and source evaluations in Internet search results

Published: 01 August 2023 Publication History

Abstract

In response to online misinformation, source rating services evaluate thousands of popular websites, mark them as reputable or disreputable, and provide detailed information explaining their evaluations. In this paper, we examine the effects of news source evaluations by experts as well as crowd-sourced information (i.e., Wikipedia) on beliefs in political news that appears in Internet search results. Through two online experiments, we investigate the effects of providing a simple source-evaluation flag or a flag with more detailed supporting information explaining the evaluation. We found that negative flags from expert fact-checkers reduced users' beliefs, and supporting information had no additional effect; in contrast, positive flags required information beyond the simple flag to have maximum effectiveness. Positive and negative crowd-sourced information had similar effects but was weaker than evaluations from experts. Thus, we conclude that providing additional information to support source evaluation flags is important to give users confidence in reputable websites.

Highlights

Negative news source rating icons from expert fact-checkers reduced users' belief in articles.
Additional information supporting negative icons had no effect.
Positive news source rating icons from expert fact-checkers had no effect.
Additional information supporting positive icons increased users' belief in articles.
Positive Wikipedia information about a news article's source increased belief, and negative information decreased belief.
Users accessed additional information or Wikipedia information about half the time.

References

[1]
H. Allcott, M. Gentzkow, Social media and fake news in the 2016 election, J. Econ. Perspect. 31 (2) (2017) 211–236.
[2]
A. Asadullah, A. Kankanhalli, I. Faik, Understanding Users’ intention to Verify content on social media platforms, in: PACIS, 2018.
[3]
A. Baiyere, M. Avital, A.R. Dennis, J. Gibbs, D. Te’eni, Fake news: What is it and why does it matter?, in: The Academy of Management Annual Meeting 2020, 2020.
[4]
M. Barthel, Many Americans Believe Fake News Is Sowing Confusion, Pew Research Center, 2016.
[5]
S. Basuroy, S. Chatterjee, S.A. Ravid, How critical are critical reviews? The box office effects of film critics, star power, and budgets, J. Mark. 67 (4) (2003) 103–117.
[6]
M.K. Bendall, C. Robertson, The crisis of democratic culture?, Intern. J. Media Cult. Polit. 14 (2) (2018) 383–391.
[7]
S. Bradshaw, P.N. Howard, The global organization of social media campaigns, J. Int. Aff. 71 (2018) 23–31.
[8]
S. Bradshaw, P.N. Howard, B. Kollanyi, L.-M. Neudert, Sourcing and automation of political news and information over social media in the UnitedStates, 2016-2018, Polit. Commun. 37 (2) (2020) 173–193.
[9]
P.B. Brandtzaeg, A. Følstad, Trust and distrust in online fact-checking services, Commun. ACM 60 (9) (2017) 65–71.
[10]
N.M. Brashier, G. Pennycook, A.J. Berinsky, D.G. Rand, Timing matters when correcting fake news, Proc. Natl. Acad. Sci. 118 (5) (2021).
[11]
A.R. Brown, Wikipedia as a data source for political scientists: accuracy and completeness of coverage, PS: Polit. Sci. Polit. vol. 44 (2) (2011) 339–343.
[12]
Callahan, M. (2018): Faced With a Daily Barrage of News, College Students Find it Hard to tell What's Real and What's 'Fake News'. News@Northeastern https://news.northeastern.edu/.
[13]
J.A. Chevalier, D. Mayzlin, The effect of word of mouth on sales: online book reviews, J. Mark. Res. 43 (3) (2006) 345–354.
[14]
M. Chung, N. Kim, When I learn the news is false: how fact-checking information stems the spread of fake news via third-person perception, Hum. Commun. Res. 47 (1) (2021) 1–24.
[15]
K. Clayton, S. Blair, J.A. Busam, S. Forstner, J. Glance, G. Green, A. Kawata, A. Kovvuri, J. Martin, E. Morgan, Real solutions for fake news? Measuring the effectiveness of general warnings and fact-check tags in reducing belief in false stories on social media, Polit. Behav. (2019) 1–23.
[16]
M. Del Vicario, A. Bessi, F. Zollo, F. Petroni, A. Scala, G. Caldarelli, H.E. Stanley, W. Quattrociocchi, The spreading of misinformation online, Proc. Natl. Acad. Sci. 113 (3) (2016) 554.
[17]
C. Dellarocas, The digitization of word of mouth: promise and challenges of online feedback mechanisms, Manag. Sci. 49 (10) (2003) 1407–1424.
[18]
B. Deng, M. Chau, The effect of anger and sadness on online news believability, J. Manag. Inf. Syst. 38 (4) (2021) 959–988.
[19]
A. Doyle, Analyzing the laws of mil: a five-step scientific conversation on critical information literacy, Communicat. Inform. Liter. 13 (1) (2019) 8.
[20]
M. Eisenegger, K. Imhof, The true, the good and the beautiful: reputation management in the media society, Public Relat. Res. (2008) 125–146.
[21]
R. Epstein, R.E. Robertson, The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections, Proc. Natl. Acad. Sci. 112 (33) (2015) E4512–E4521.
[22]
J.S.B. Evans, K.E. Stanovich, Dual-process theories of higher cognition: advancing the debate, Perspect. Psychol. Sci. 8 (3) (2013) 223–241.
[23]
F. Faul, E. Erdfelder, A.G. Lang, A. Buchner, G*power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences, Behav. Res. Methods 39 (2007) 175–191.
[24]
K. Figl, S. Kiesling, C. Rank, S. Vakulenko, Fake news flags, cognitive dissonance, and the believability of social media posts, in: International Conference on Information Systems, Munich, 2019, pp. 1–9.
[25]
K. Flynn, Facebook outsources its fake news problem to Wikipedia—And an army of human moderators, in: Mashable, 2017.
[26]
B. Fogg, G. Cuellar, D. Danielson, Motivating, influencing, and persuading users: An introduction to Captology, in: A. Sears, J.A. Jacko (Eds.), Human Computer Interaction Fundamentals, CRC Press, Boca Raton, 2009, pp. 109–122.
[27]
M. Freeze, M. Baumgartner, P. Bruno, J.R. Gunderson, J. Olin, M.Q. Ross, J. Szafran, Fake claims of fake news: political misinformation, warnings, and the tainted truth effect, Polit. Behav. (2020) 1–33.
[28]
M. García Lozano, J. Brynielsson, U. Franke, M. Rosell, E. Tjörnhammar, S. Varga, V. Vlassov, Veracity assessment of online data, Decis. Support. Syst. 129 (2020) 113132.
[29]
R.K. Garrett, D. Carnahan, E.K. Lynch, A turn toward avoidance? Selective exposure to online political information, 2004–2008, Polit. Behav. 35 (1) (2013) 113–134.
[30]
R.K. Garrett, S. Poulsen, Flagging Facebook falsehoods: self-identified humor warnings outperform fact checker and peer warnings, J. Comput.-Mediat. Commun. 24 (5) (2019) 240–258.
[31]
C. Gaziano, K. McGrath, Measuring the concept of credibility, Journal. Q. 63 (3) (1986) 12.
[32]
A.W. Geiger, Key Findings about the Online News Landscape in America, in, Pew Research Center, 2019.
[33]
J. George, N. Gerhart, R. Torres, Uncovering the truth about fake news: a research model grounded in multi-disciplinary literature, J. Manag. Inf. Syst. 38 (4) (2021) 1067–1094.
[34]
L. Graves, Boundaries not drawn: mapping the institutional roots of the global fact-checking movement, Journal. Stud. 19 (5) (2018) 613–631.
[35]
A. Grizzle, J. Singh, Five laws of media and information literacy as harbingers of human rights, Media Inform. Liter.: Reinforc. Human Rights Counter. Radical. Extrem. 25 (2016).
[36]
G. Grossman, M. Humphreys, G. Sacramone-Lutz, Information technology and political engagement: mixed evidence from Uganda, J. Polit. 82 (4) (2020) 1321–1336.
[37]
M. Hameleers, Separating truth from lies: comparing the effects of news media literacy interventions and fact-checkers in response to political misinformation in the US and Netherlands, Inf. Commun. Soc. 25 (1) (2022) 110–126.
[38]
Y.-C. Ho, J. Wu, Y. Tan, Disconfirmation effect on online rating behavior: a structural model, Inf. Syst. Res. 28 (3) (2017) 626–642.
[39]
L. Holman Rector, Comparison of Wikipedia and other encyclopedias for accuracy, breadth, and depth in historical articles, Ref. Serv. Rev. 36 (1) (2008) 7–22.
[40]
C.G. Horner, D. Galletta, J. Crawford, A. Shirsat, Emotions: the unexplored fuel of fake news on social media, J. Manag. Inf. Syst. 38 (4) (2021) 1039–1066.
[41]
N. Hu, L. Liu, J.J. Zhang, Do online reviews affect product sales? The role of reviewer characteristics and temporal effects, Inf. Technol. Manag. 9 (3) (2008) 201–214.
[42]
D. Kahneman, Maps of bounded rationality: psychology for behavioral economics, Am. Econ. Rev. 93 (5) (2003) 1449–1475.
[43]
D. Kahneman, Thinking, Fast and Slow, Macmillan, 2011.
[44]
A. Kim, A.R. Dennis, Says who? The effects of presentation format and source rating on fake news in social media, MIS Q. 43 (3) (2019) 1025–1039.
[45]
A. Kim, P.L. Moravec, A.R. Dennis, Combating fake news on social media with source ratings: the effects of user and expert reputation ratings, J. Manag. Inf. Syst. 36 (3) (2019) 931–968.
[46]
K.K. King, B. Wang, D. Escobari, T. Oraby, Dynamic effects of falsehoods and corrections on social media: a theoretical modeling and empirical evidence, J. Manag. Inf. Syst. 38 (4) (2021) 989–1010.
[47]
M.S. Kleinberg, R.R. Lau, The importance of political Knowledge for effective citizenship: differences between the broadcast and internet generations, Public Opin. Q. 83 (2) (2019) 338–362.
[48]
S. Knobloch-Westerwick, B.K. Johnson, A. Westerwick, Confirmation bias in online searches: impacts of selective exposure before an election on political attitude strength and shifts, J. Comput.-Mediat. Commun. 20 (2) (2015) 171–187.
[49]
A. Koriat, S. Lichtenstein, B. Fischhoff, Reasons for confidence, J. Exp. Psychol. Hum. Learn. Mem. 6 (2) (1980) 107.
[50]
N.K. Lankton, C. Speier, E.V. Wilson, Internet-based knowledge acquisition: task complexity and performance, Decis. Support. Syst. 53 (1) (2012) 55–65.
[51]
S. Levin, Mark Zuckerberg: I regret ridiculing fears over Facebook's effect on election, in: The Guardian, 2017.
[52]
W. Lowrey, The emergence and development of news fact-checking sites: institutional logics and population ecology, Journal. Stud. 18 (3) (2017) 376–394.
[53]
J. Lu, What Is ‘Click Bait’ and why Facebook Wants to Display Less of it, The Washington Post, 2014.
[54]
M.M. Luo, S. Chea, J.-S. Chen, Web-based information service adoption: a comparison of the motivational model and the uses and gratifications theory, Decis. Support. Syst. 51 (1) (2011) 10.
[55]
B.A. Lyons, J.M. Montgomery, A.M. Guess, B. Nyhan, J. Reifler, Overconfidence in news judgments is associated with false news susceptibility, Proc. Natl. Acad. Sci. 118 (23) (2021).
[56]
M. Madden, S. Fox, Riding the Waves of “Web 2.0”, Pew Internet & American Life Project, 2006, pp. 1–6.
[57]
G.M. Masullo, T. Lee, M.J. Riedl, Signaling news outlet credibility in a Google search, Journalism & Mass, Commun. Q. 99 (4) (2021) 1148–1168.
[58]
K.E. Matsa, E. Shearer, News Use across Social Medial Platforms 2018, in, Pew Research Center, 2018.
[59]
M. McGlohon, N.S. Glance, Z. Reiter, Star Quality: Aggregating Reviews to Rank Products and Merchants, in: International Conference on Weblogs and Social Media, 2010, pp. 114–121.
[60]
C. McMahon, I. Johnson, B. Hecht, The substantial interdependence of Wikipedia and Google: A case study on the relationship between peer production communities and information technologies, in: In: 11th International Conference on Web and Social Media (ICWSM 2017) Montreal, 2017.
[61]
P. Mena, Cleaning up social media: the effect of warning labels on likelihood of sharing false news on Facebook, Policy Internet 12 (2) (2020) 165–183.
[62]
M. Mesgari, C. Okoli, M. Mehdi, F.Å. Nielsen, A. Lanamäki, ‘The sum of all human knowledge’: a systematic review of scholarly research on the content of Wikipedia, J. Assoc. Inform. Sci. Technol. 66 (2) (2015) 219–245.
[63]
P.L. Moravec, A. Kim, A.R. Dennis, Appealing to sense and sensibility: system 1 and system 2 interventions for fake news on social media, Inf. Syst. Res. 31 (3) (2020) 987–1006.
[64]
P.L. Moravec, A. Kim, A.R. Dennis, R.K. Minas, Do you Really Know if It’s True? How Asking Users to Rate Stories Affects Belief in Fake News on Social Media, Information Systems Research, 2022.
[65]
P.L. Moravec, R.K. Minas, A.R. Dennis, Fake news on social media: people believe what they want to believe when it makes no sense at all, MIS Q. 43 (4) (2019) 1343–1360.
[66]
K. Munger, P.J. Egan, J. Nagler, J. Ronen, J. Tucker, Political Knowledge and Misinformation in the Era of Social Media: Evidence From the 2015 UK election, Br. J. Polit. Sci. (2020) 1–21.
[67]
J. Murtha, How fake news sites frequently trick big-time journalists, Columb. J. Rev. (2016) https://www.cjr.org/analysis/how_fake_news_sites_frequently_trick_big-time_journalists.php.
[68]
NewsGuard, NewsGuard: Rating Process and Criteria, NewsGuard, 2020.
[69]
K.C. Ng, J. Tang, D. Lee, The effect of platform intervention policies on fake news dissemination and survival: an empirical examination, J. Manag. Inf. Syst. 38 (4) (2021) 898–930.
[70]
R.S. Nickerson, Confirmation bias: a ubiquitous phenomenon in many guises, Rev. Gen. Psychol. 2 (2) (1998) 175.
[71]
R.L. Oliver, Effect of expectation and disconfirmation on postexposure product evaluations: an alternative interpretation, J. Appl. Psychol. 62 (4) (1977) 480–486.
[72]
R.L. Oliver, A cognitive model of the antecedents and consequences of satisfaction decisions, J. Mark. Res. 17 (4) (1980) 460–469.
[73]
C. Peacock, G.M. Masullo, N.J. Stroud, What’s in a label? The effect of news labels on perceived credibility, Journalism 23 (2) (2022) 19.
[74]
A. Pekary, CNN public editor: why web video is more divisive than TV, Columb. J. Rev. (2021) https://www.cjr.org/public_editor/cnn-public-editor-why-web-video-is-more-divisive-than-tv.php.
[75]
G. Pennycook, A. Bear, E.T. Collins, D.G. Rand, The Implied Truth Effect: Attaching Warnings to a Subset of Fake News Stories Increases Perceived Accuracy of Stories without Warnings, Management Science, 2019, Forthcoming.
[76]
G. Pennycook, J. McPhetres, Y. Zhang, J.G. Lu, D.G. Rand, Fighting COVID-19 Misinformation on Social Media: Experimental Evidence for a Scalable Accuracy-Nudge Intervention, 31(7), 2020, pp. 770–780.
[77]
G. Pennycook, D.G. Rand, Fighting misinformation on social media using crowdsourced judgments of news source quality, Proc. Natl. Acad. Sci. 116 (7) (2019) 2521.
[78]
G. Pennycook, D.G. Rand, Lazy, not biased: susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning, Cognition 188 (2019) 39–50.
[79]
E. Porter, T.J. Wood, The global effectiveness of fact-checking: evidence from simultaneous experiments in Argentina, Nigeria, South Africa, and the United Kingdom, Proc. Natl. Acad. Sci. 118 (37) (2021).
[80]
D.A. Reinstein, C.M. Snyder, The influence of expert reviews on consumer demand for experience goods: a case study of movie critics, J. Ind. Econ. 53 (1) (2005) 27–51.
[81]
ReviewMeta (2016): Analysis of 7 million Amazon Reviews: Customers Who Receive Free or Discounted Item Much more Likely to Write Positive Review. reviewmeta.com.
[82]
C.T. Robertson, R.R. Mourão, E. Thorson, Who uses fact-checking sites?, Impact Demogr. Polit. Anteced. Media Use Fact-Check. Site Awaren. Attitud. Behav. 25 (2) (2020) 217–237.
[83]
R.E. Robertson, D. Lazer, C. Wilson, Auditing the personalization and composition of politically-related search engine results pages, in: Proceedings of the 2018 World Wide Web Conference, 2018, pp. 955–965.
[84]
B. Ross, A. Jung, J. Heisel, S. Stieglitz, Fake news on social media: The (in)Effectiveness of Warning Messages, in: Internation Conference on Information Systems, San Francisco, 2018.
[85]
A. Rothschild, E. Luire, E. Mustafaraj, How the interplay of Google and Wikipedia affects perceptions of online news sources, in: Computation + Journalism Symposium, Miami, FL, 2019.
[86]
S. Schultheiß, D. Lewandowski, Misplaced trust? The relationship between trust, ability to identify commercially influenced results and search engine preference, J. Inf. Sci. (2021) 15.
[87]
Schulze, E. (2018): Fake News 'To Get Worse' by 2020 Election Unless Social Media Firms act, DNC Tech Chief Says. www.cnbc.com.
[88]
A. Sebenius, How Disinformation Has Morphed for the 2020 Election, 2020.
[89]
S. Shane, The Fake Americans Russia Created to Influence the Election, The New York Times, 2017.
[90]
J. Shin, K. Thorson, Partisan selective sharing: the biased diffusion of fact-checking messages on social media, J. Commun. 67 (2) (2017) 233–255.
[91]
A. Simpser, The culture of corruption across generations: an empirical study of bribery attitudes and behavior, J. Polit. 82 (4) (2020) 1373–1389.
[92]
C. Steindl, E. Jonas, S. Sittenthaler, E. Traut-Mattausch, J. Greenberg, Understanding psychological reactance: new developments and findings, Z. Psychol. 223 (4) (2015) 10.
[93]
E. Tewell, A decade of critical information literacy: a review of the literature, Communicat. Inform. Liter. 9 (1) (2015) 2.
[94]
R. Torres, N. Gerhart, A. Negahban, Combating Fake News: An Investigation of Information Verification Behaviors on Social Networking Sites, HICSS, 2018.
[95]
R. Torres, N. Gerhart, A. Negahban, Epistemology in the era of fake news: an exploration of information verification behaviors among social networking site users, DataBase 49 (3) (2018) 78–97.
[96]
S.E. Toulmin, The Uses of Argument, Cambridge University Press, Cambridge, UK, 1958.
[97]
S.E. Toulmin, The Uses of Argument, Cambridge University Press, Cambridge, UK, 2012.
[98]
O. Turel, B. Osatuyi, Biased credibility and sharing of fake news on social media: considering peer context and self-objectivity state, J. Manag. Inf. Syst. 38 (4) (2021) 931–958.
[99]
E. Turner, L. Rainie, Most Americans Rely on their Own Research to Make Big Decisions, and that Often Means Online Searches, Pew Research Center, 2020.
[100]
K. Umarova, E. Mustafaraj, How Partisanship and Perceived Political Bias Affect Wikipedia Entries of News Sources, World Wide Web'19, San Francisco, 2019.
[101]
A. Vaish, T. Grossmann, A. Woodward, Not all emotions are created equal: the negativity bias in social-emotional development, Psychol. Bull. 134 (3) (2008) 383.
[102]
N.A. Valentino, V.L. Hutchings, A.J. Banks, A.K. Davis, Is a worried citizen a good citizen? Emotions, political information seeking, and learning via the internet, Polit. Psychol. 29 (2) (2008) 247–273.
[103]
E.K. Vraga, L. Bode, Using expert sources to correct health misinformation in social media, Sci. Commun. 39 (5) (2017) 621–645.
[104]
N. Walter, J. Cohen, R.L. Holbert, Y. Morag, Fact-checking: a Meta-analysis of what works and for whom, Polit. Commun. 37 (3) (2020) 350–375.
[105]
A.L. Wintersieck, Debating the truth: the impact of fact-checking during electoral debates, Am. Politics Res. 45 (2) (2017) 304–331.
[106]
N. Xiao, R. Sharman, H.R. Rao, S. Upadhyaya, Factors influencing online health information search: an empirical analysis of a national cancer-related survey, Decis. Support. Syst. 57 (2014) 417–427.
[107]
O. Yair, R. Sulitzeanu-Kenan, Y. Dotan, Can institutions make voters care about corruption?, J. Polit. 82 (4) (2020) 1430–1442.
[108]
H. Yuan, J. Zheng, Q. Ye, Y. Qian, Y. Zhang, Improving fake news detection with domain-adversarial and graph-attention neural network, Decis. Support. Syst. 151 (2021).
[109]
F. Zhu, X. Zhang, Impact of online consumer reviews on sales: the moderating role of product and consumer characteristics, J. Mark. 74 (2) (2010) 133–148.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Decision Support Systems
Decision Support Systems  Volume 171, Issue C
Aug 2023
104 pages

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 August 2023

Author Tags

  1. Fake news
  2. Misinformation
  3. Fact-checking
  4. Expert evaluation
  5. Crowd-sourced evaluation
  6. Internet search

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 12 Nov 2024

Other Metrics

Citations

View Options

View options

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media