research-article

Open access

Investigating the Influence of Featured Snippets on User Attitudes

Authors:

Sebastian Schwarz,

David ElsweilerAuthors Info & Claims

CHIIR '23: Proceedings of the 2023 Conference on Human Information Interaction and Retrieval

Pages 211 - 220

https://doi.org/10.1145/3576840.3578323

Published: 20 March 2023 Publication History

All formats PDF

Abstract

Featured snippets that attempt to satisfy users’ information needs directly on top of the first search engine results page (SERP) have been shown to strongly impact users’ post-search attitudes and beliefs. In the context of debated but scientifically answerable topics, recent research has demonstrated that users tend to trust featured snippets to such an extent that they may reverse their original beliefs based on what such a snippet suggests; even when erroneous information is featured. This paper examines the effect of featured snippets in more nuanced and complicated search scenarios concerning debated topics that have no ground truth and where diverse arguments in favor and against can legitimately be made. We report on a preregistered, online user study (N = 182) investigating how the stances and logics of evaluation (i.e., underlying reasons behind stances) expressed in featured snippets influence post-task attitudes and explanations of users without strong pre-search attitudes. We found that such users tend to not only change their attitudes on debated topics (e.g., school uniforms) following whatever stance a featured snippet expresses but also incorporate the featured snippet’s logic of evaluation into their argumentation. Our findings imply that the content displayed in featured snippets may have large-scale undesired consequences for individuals, businesses, and society, and urgently call for researchers and practitioners to examine this issue further.

References

[1]

Ahmed Allam, Peter Johannes Schulz, Kent Nakamoto, 2014. The impact of search engine selection and sorting criteria on vaccination beliefs and attitudes: two experiments manipulating Google output. Journal of medical internet research 16, 4 (2014), e2642.

[2]

Leif Azzopardi. 2021. Cognitive Biases in Search: A Review and Reflection of Cognitive Biases in Information Retrieval. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval (Canberra ACT, Australia) (CHIIR ’21). Association for Computing Machinery, New York, NY, USA, 27–37. https://doi.org/10.1145/3406522.3446023

Digital Library

[3]

Christian Baden and Nina Springer. 2014. Com(ple)menting the news on the financial crisis: The contribution of news users’ commentary to the diversity of viewpoints in the public debate. In European Journal of Communication 29. European Journal of Communication, 529–548. https://doi.org/10.1177/0267323114538724

[4]

Marcia J Bates. 1989. The design of browsing and berrypicking techniques for the online search interface. Online review (1989).

[5]

Suresh K Bhavnani. 2002. Domain-specific search strategies for the effective retrieval of healthcare and shopping information. In CHI’02 extended abstracts on human factors in computing systems. 610–611.

[6]

Markus Bink, Steven Zimmerman, and David Elsweiler. 2022. Featured Snippets and Their Influence on Users’ Credibility Judgements. In ACM SIGIR Conference on Human Information Interaction and Retrieval (Regensburg, Germany) (CHIIR ’22). Association for Computing Machinery, New York, NY, USA, 113–122. https://doi.org/10.1145/3498366.3505766

Digital Library

[7]

Stuart K. Card, Peter Pirolli, Mija Van Der Wege, Julie B. Morrison, Robert W. Reeder, Pamela K. Schraedley, and Jenea Boshart. 2001. Information Scent as a Driver of Web Behavior Graphs: Results of a Protocol Analysis Method for Web Usability. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’01). 498–505.

Digital Library

[8]

Noel Carroll. 2014. In search we trust: exploring how search engines are shaping society. International Journal of Knowledge Society Research (IJKSR) 5, 1(2014), 12–27.

Digital Library

[9]

Lydia B. Chilton and Jaime Teevan. 2011. Addressing People’s Information Needs Directly in a Web Search Result Page. In Proceedings of the 20th International Conference on World Wide Web(WWW ’11). 27–36.

[10]

Charles L. A. Clarke, Eugene Agichtein, Susan Dumais, and Ryen W. White. 2007. The Influence of Caption Features on Clickthrough Patterns in Web Search. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Amsterdam, The Netherlands) (SIGIR ’07). Association for Computing Machinery, New York, NY, USA, 135–142. https://doi.org/10.1145/1277741.1277767

Digital Library

[11]

Tim Draws, Oana Inel, Nava Tintarev, Christian Baden, and Benjamin Timmermans. 2022. Comprehensive Viewpoint Representations for a Deeper Understanding of User Interactions With Debated Topics. In CHIIR ’22: ACM SIGIR Conference on Human Information Interaction and Retrieval(CHIIR ’22). ACM SIGIR Conference on Human Information Interaction and Retrieval, 135–145.

Digital Library

[12]

Tim Draws, Karthikeyan Natesan Ramamurthy, Ioana Baldini Soares, Amit Dhurandhar, Inkit Padhi, Benjamin Timmermans, and Nava Tintarev. 2023. Explainable Cross-Topic Stance Detection for Search Results. In CHIIR ’23: ACM SIGIR Conference on Human Information Interaction and Retrieval(CHIIR ’23). ACM SIGIR Conference on Human Information Interaction and Retrieval.

[13]

Tim Draws, Nirmal Roy, Oana Inel, Alisa Rieger, Rishav Hada, Mehmet Orcun Yalcin, Benjamin Timmermans, and Nava Tintarev. 2023. Viewpoint Diversity in Search Results. In European Conference on Information Retrieval. Springer.

[14]

Tim Draws, Nava Tintarev, and Ujwal Gadiraju. 2021. Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics. In ACM SIGKDD Explorations Newsletter(ACM SIGKDD ’21). Association for Computing Machinery, New York, NY, USA, 50–58.

[15]

Tim Draws, Nava Tintarev, Ujwal Gadiraju, Alessandro Bozzon, and Benjamin Timmermans. 2021. This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics. In SIGIR ’21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR ’21). Association for Computing Machinery, New York, NY, USA, 295–305.

Digital Library

[16]

Robert Epstein and Ronald E. Robertson. 2015. The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. In Proceedings of the National Academy of Sciences 112. PNAS, E4512–E4521. https://doi.org/doi.org/10.1073/pnas.1419828112

[17]

Franz Faul, Edgar Erdfelder, Albert-Georg Lang, and Axel Buchner. 2007. G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods 39, 2 (2007), 175–191.

[18]

Ruoyuan Gao and Chirag Shah. 2020. Toward creating a fairer ranking in search engine results. Information Processing & Management 57, 1 (2020), 102138.

Digital Library

[19]

Lisa Gevelber. 2018. It’s all about ‘me’—how people are taking search personally. Technical Report. Technical Report. https://www.thinkwithgoogle.com/marketing-strategies ….

[20]

Amira Ghenai, Mark D. Smucker, and Charles L.A. Clarke. 2020. A Think-Aloud Study to Understand Factors Affecting Online Health Search. In CHIIR ’20: Proceedings of the 2020 Conference on Human Information Interaction and Retrieval. CHIIR, 273–282. https://doi.org/doi.org/10.1145/3343413.3377961

Digital Library

[21]

Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2017. Accurately interpreting clickthrough data as implicit feedback. In Acm Sigir Forum, Vol. 51. Acm New York, NY, USA, 4–11.

[22]

Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, and Geri Gay. 2007. Evaluating the Accuracy of Implicit Feedback from Clicks and Query Reformulations in Web Search. ACM Trans. Inf. Syst. 25, 2 (apr 2007), 7–es. https://doi.org/10.1145/1229179.1229181

Digital Library

[23]

Markus Kattenbeck and David Elsweiler. 2019. Understanding credibility judgements for web search snippets. Aslib Journal of Information Management(2019).

[24]

Joshua Klayman and Young-Won Ha. 1987. Confirmation, disconfirmation, and information in hypothesis testing.Psychological review 94, 2 (1987), 211.

[25]

Klaus Krippendorff. 2018. Content analysis: An introduction to its methodology. Sage publications.

[26]

Dilek Küçük and Fazli Can. 2020. Stance detection: A survey. ACM Computing Surveys (CSUR) 53, 1 (2020), 1–37.

Digital Library

[27]

Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna P Gummadi, and Karrie Karahalios. 2019. Search bias quantification: investigating political bias in social media and web search. Information Retrieval Journal 22, 1 (2019), 188–227.

Digital Library

[28]

Dana Mckay, Stephann Makri, Marisela Gutierrez-Lopez, Andrew MacFarlane, Sondess Missaoui, Colin Porlezza, and Glenda Cooper. 2020. We Are the Change That We Seek: Information Interactions During a Change of Viewpoint. In Proceedings of the 2020 Conference on Human Information Interaction and Retrieval (Vancouver BC, Canada) (CHIIR ’20). Association for Computing Machinery, New York, NY, USA, 173–182. https://doi.org/10.1145/3343413.3377975

Digital Library

[29]

Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. Semeval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). 31–41.

[30]

Raymond S. Nickerson. 1998. Confirmation bias: A ubiquitous phenomenon in many guises. Review of general psychology 2, 2 (1998), 175–220.

[31]

Anna-Marie Ortloff, Steven Zimmerman, David Elsweiler, and Niels Henze. 2021. The Effect of Nudges and Boosts on Browsing Privacy in a Naturalistic Environment. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval. 63–73.

Digital Library

[32]

Jahna Otterbacher, Jo Bates, and Paul Clough. 2017. Competent men and warm women: Gender stereotypes and backlash in image search results. In Proceedings of the 2017 chi conference on human factors in computing systems. 6620–6631.

Digital Library

[33]

Bing Pan, Helene Hembrooke, Thorsten Joachims, Lori Lorigo, Geri Gay, and Laura Granka. 2007. In Google we trust: Users’ decisions on rank, position, and relevance. Journal of computer-mediated communication 12, 3 (2007), 801–823.

[34]

Sandeep Pandey, Kedar Dhamdhere, and Christopher Olston. 2004. WIC: A General-Purpose Algorithm for Monitoring Web Information Sources. In Proceedings of the Thirtieth International Conference on Very Large Data Bases - Volume 30(Toronto, Canada) (VLDB ’04). VLDB Endowment, 360–371.

[35]

Sachin Pathiyan Cherumanal, Damiano Spina, Falk Scholer, and W Bruce Croft. 2021. Evaluating Fairness in Argument Retrieval. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 3363–3367.

Digital Library

[36]

Frances A Pogacar, Amira Ghenai, Mark D Smucker, and Charles LA Clarke. 2017. The positive and negative influence of search results on people’s decisions about the efficacy of medical treatments. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval. 209–216.

Digital Library

[37]

Cornelius Puschmann. 2019. Beyond the bubble: Assessing the diversity of political search results. Digital Journalism 7, 6 (2019), 824–843.

[38]

Alisa Rieger, Tim Draws, Mariët Theune, and Nava Tintarev. 2021. This Item Might Reinforce Your Opinion: Obfuscation and Labeling of Search Results to Mitigate Confirmation Bias. In Proceedings of the 32nd ACM Conference on Hypertext and Social Media. 189–199.

Digital Library

[39]

Julia Schwarz and Meredith Morris. 2011. Augmenting Web Pages and Search Results to Support Credibility Assessment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’11). 1245–1254.

Digital Library

[40]

Milad Shokouhi, Ryen White, and Emine Yilmaz. 2015. Anchoring and Adjustment in Relevance Estimation. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR ’15). 963–966.

Digital Library

[41]

Amos Tversky and Daniel Kahneman. 1974. Judgment under uncertainty: Heuristics and biases. Science 185, 4157 (1974), 1124–1131.

[42]

Ryen White. 2013. Beliefs and biases in web search. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. 3–12.

Digital Library

[43]

Ryen W. White. 2014. Belief Dynamics in Web Search. J. Assoc. Inf. Sci. Technol.(2014), 2165–2178.

[44]

Ryen W White, Susan T Dumais, and Jaime Teevan. 2009. Characterizing the influence of domain expertise on web search behavior. In Proceedings of the second ACM international conference on web search and data mining. 132–141.

Digital Library

[45]

Ryen W White and Eric Horvitz. 2015. Belief dynamics and biases in web search. ACM Transactions on Information Systems (TOIS) 33, 4 (2015), 1–46.

Digital Library

[46]

Ryen W White and Dan Morris. 2007. Investigating the querying and browsing behavior of advanced search engine users. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval. 255–262.

Digital Library

[47]

Kyle Williams, Julia Kiseleva, Aidan C. Crook, Imed Zitouni, Ahmed Hassan Awadallah, and Madian Khabsa. 2016. Detecting Good Abandonment in Mobile Search. In Proceedings of the 25th International Conference on World Wide Web(WWW ’16). 495–505.

Digital Library

[48]

Zhijing Wu, Mark Sanderson, Barla B Cambazoglu, Bruce W Croft, and Falk Scholer. 2020. Providing Direct Answers in Search Results: A Study of User Behavior. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management(CIKM ’22). ACM, 1635 – 1644.

Digital Library

[49]

Zhijing Wu, Mark Sanderson, B. Barla Cambazoglu, W. Bruce Croft, and Falk Scholer. 2020. Providing Direct Answers in Search Results: A Study of User Behavior. In CIKM ’20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management(CIKM ’20). Association for Computing Machinery, New York, NY, USA, 1635–1644.

Digital Library

[50]

Yusuke Yamamoto and Katsumi Tanaka. 2011. Enhancing Credibility Judgment of Web Search Results. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’11). 1235–1244.

Digital Library

Cited By

Pathiyan Cherumanal SScholer FTrippas JSpina D(2024)Towards Investigating Biases in Spoken Conversational SearchCompanion Proceedings of the 26th International Conference on Multimodal Interaction10.1145/3686215.3690156(61-66)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3686215.3690156
Bink MElsweiler D(2024)Balancing Act: Boosting Strategies for Informed Search on Controversial TopicsProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638329(254-265)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638329
Merker JMerker LBondarenko A(2024)The Impact of Web Search Result Quality on Decision-MakingExperimental IR Meets Multilinguality, Multimodality, and Interaction10.1007/978-3-031-71736-9_5(100-112)Online publication date: 14-Sep-2024
https://doi.org/10.1007/978-3-031-71736-9_5
Show More Cited By

Index Terms

Investigating the Influence of Featured Snippets on User Attitudes
1. Information systems
  1. Information retrieval
    1. Users and interactive retrieval
      1. Search interfaces

Recommendations

Featured Snippets and their Influence on Users’ Credibility Judgements
CHIIR '22: Proceedings of the 2022 Conference on Human Information Interaction and Retrieval

Search engines often provide featured snippets, which are boxed and placed above other results with the aim of directly answering user queries. To learn about how users judge the credibility of such results and how they influence search outcomes, a ...
Auditing the Partisanship of Google Search Snippets
WWW '19: The World Wide Web Conference

The text snippets presented in web search results provide users with a slice of page content that they can quickly scan to help inform their click decisions. However, little is known about how these snippets are generated or how they relate to a user's ...
Effect of Snippets on User Experience in Web Search
Interacción '15: Proceedings of the XVI International Conference on Human Computer Interaction

In recent years, the search engine results pages (SERP's) have been augmented with new markup elements that introduce seamlessly additional semantic information. Examples of such elements are the aggregated results disseminated by vertical portals, and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHIIR '23: Proceedings of the 2023 Conference on Human Information Interaction and Retrieval

March 2023

520 pages

ISBN:9798400700354

DOI:10.1145/3576840

Editors:
Jacek Gwizdka
School of Information, The University of Texas at Austin, Texas, USA
,
Soo Young Rieh
School of Information, The University of Texas at Austin, Texas, USA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 March 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

CHIIR '23

Sponsor:

CHIIR '23: ACM SIGIR Conference on Human Information Interaction and Retrieval

March 19 - 23, 2023

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 55 of 163 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
649
Total Downloads

Downloads (Last 12 months)317
Downloads (Last 6 weeks)54

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pathiyan Cherumanal SScholer FTrippas JSpina D(2024)Towards Investigating Biases in Spoken Conversational SearchCompanion Proceedings of the 26th International Conference on Multimodal Interaction10.1145/3686215.3690156(61-66)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3686215.3690156
Bink MElsweiler D(2024)Balancing Act: Boosting Strategies for Informed Search on Controversial TopicsProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638329(254-265)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638329
Merker JMerker LBondarenko A(2024)The Impact of Web Search Result Quality on Decision-MakingExperimental IR Meets Multilinguality, Multimodality, and Interaction10.1007/978-3-031-71736-9_5(100-112)Online publication date: 14-Sep-2024
https://doi.org/10.1007/978-3-031-71736-9_5
Wu ZDraws TCau FBarile FRieger ATintarev N(2023)Explaining Search Result Stances to Opinionated PeopleExplainable Artificial Intelligence10.1007/978-3-031-44067-0_29(573-596)Online publication date: 21-Oct-2023
https://doi.org/10.1007/978-3-031-44067-0_29

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten