research-article

Do Humans Trust Advice More if it Comes from AI?: An Analysis of Human-AI Interactions

Authors:

Kailas Vodrahalli,

Roxana Daneshjou,

Tobias Gerstenberg,

James ZouAuthors Info & Claims

AIES '22: Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society

Pages 763 - 777

https://doi.org/10.1145/3514094.3534150

Published: 27 July 2022 Publication History

Abstract

In decision support applications of AI, the AI algorithm's output is framed as a suggestion to a human user. The user may ignore this advice or take it into consideration to modify their decision. With the increasing prevalence of such human-AI interactions, it is important to understand how users react to AI advice. In this paper, we recruited over 1100 crowdworkers to characterize how humans use AI suggestions relative to equivalent suggestions from a group of peer humans across several experimental settings. We find that participants' beliefs about how human versus AI performance on a given task affects whether they heed the advice. When participants do heed the advice, they use it similarly for human and AI suggestions. Based on these results, we propose a two-stage, "activation-integration" model for human behavior and use it to characterize the factors that affect human-AI interactions.

Supplementary Material

MP4 File (aies077.mp4)

Presentation video for "Do Humans Trust Advice More if it Comes from AI? An Analysis of Human-AI Interactions"

Download
20.96 MB

References

[1]

Dale J Barr. 2013. Random effects structure for testing interactions in linear mixed-effects models. Frontiers in psychology, Vol. 4 (2013), 328.

[2]

Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2014. Fitting linear mixed-effects models using lme4. arXiv preprint arXiv:1406.5823 (2014).

[3]

Navoneel Chakrabarty and Sanket Biswas. 2018. A statistical approach to adult census income level prediction. In 2018 International Conference on Advances in Computing, Communication Control and Networking (ICACCCN). IEEE, 207--212.

[4]

Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. https://archive.ics.uci.edu/ml/datasets/census+income

[5]

Mary T Dzindolet, Linda G Pierce, Hall P Beck, and Lloyd A Dawe. 2002. The perceived utility of human and automated aids in a visual detection task. Human Factors, Vol. 44, 1 (2002), 79--94.

[6]

Robin C Feldman, Ehrik Aldana, and Kara Stein. 2019. Artificial intelligence in the health care space: how we can trust what we cannot know. Stan. L. & Pol'y Rev., Vol. 30 (2019), 399.

[7]

Martin A Fischler and Robert C Bolles. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, Vol. 24, 6 (1981), 381--395.

Digital Library

[8]

Susanne Gaube, Harini Suresh, Martina Raue, Alexander Merritt, Seth J Berkowitz, Eva Lermer, Joseph F Coughlin, John V Guttag, Errol Colak, and Marzyeh Ghassemi. 2021. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ digital medicine, Vol. 4, 1 (2021), 1--8.

[9]

Francesca Gino and Don A Moore. 2007. Effects of task difficulty on use of advice. Journal of Behavioral Decision Making, Vol. 20, 1 (2007), 21--35.

[10]

Alexa Hagerty and Igor Rubinov. 2019. Global AI ethics: a review of the social impacts and ethical implications of artificial intelligence. arXiv preprint arXiv:1907.07892 (2019).

[11]

Nigel Harvey and Ilan Fischer. 1997. Taking advice: Accepting help, improving judgment, and sharing responsibility. Organizational behavior and human decision processes, Vol. 70, 2 (1997), 117--133.

[12]

Nigel Harvey, Clare Harries, and Ilan Fischer. 2000. Using advice and assessing its quality. Organizational behavior and human decision processes, Vol. 81, 2 (2000), 252--273.

[13]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, 770--778.

[14]

Ekaterina Jussupow, Izak Benbasat, and Armin Heinzl. 2020. Why are we averse towards Algorithms? A comprehensive literature Review on Algorithm aversion. In ECIS.

[15]

Mikhail Khodak, Nikunj Saunshi, and Kiran Vodrahalli. 2017. A large self-annotated corpus for sarcasm. arXiv preprint arXiv:1704.05579 (2017).

[16]

Min Kyung Lee. 2018. Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management. Big Data & Society, Vol. 5, 1 (2018), 2053951718756684.

[17]

Poornima Madhavan and Douglas A Wiegmann. 2007. Effects of information source, pedigree, and reliability on operator interaction with decision support systems. Human Factors, Vol. 49, 5 (2007), 773--785.

[18]

Neda Mesbah, Christoph Tauchert, and Peter Buxmann. 2021. Whose Advice Counts More--Man or Machine? An Experimental Investigation of AI-based Advice Utilization. In Proceedings of the 54th Hawaii International Conference on System Sciences. ScholarSpace, 4083.

[19]

Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, Vol. 267 (2019), 1--38.

[20]

Dilek Önkal, Paul Goodwin, Mary Thomson, Sinan Gönül, and Andrew Pollock. 2009. The relative influence of advice from human experts and statistical methods on forecast adjustments. Journal of Behavioral Decision Making, Vol. 22, 4 (2009), 390--409.

[21]

Andrew Prahl and Lyn Van Swol. 2017. Understanding algorithm aversion: When is advice from automation discounted? Journal of Forecasting, Vol. 36, 6 (2017), 691--702.

[22]

Prolific 2022. Prolific Academic. https://www.prolific.co

[23]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why should i trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 1135-1144.

Digital Library

[24]

Veronica Rotemberg, Nicholas Kurtansky, Brigid Betz-Stablein, Liam Caffery, Emmanouil Chousakos, Noel Codella, Marc Combalia, Stephen Dusza, Pascale Guitera, David Gutman, et almbox. 2021. A patient-centric dataset of images and metadata for identifying melanomas using clinical context. Scientific data, Vol. 8, 1 (2021), 1--8.

[25]

Sunita Sah, Don A Moore, and Robert J MacCoun. 2013. Cheap talk and credibility: The consequences of confidence and accuracy on advisor credibility and persuasiveness. Organizational Behavior and Human Decision Processes, Vol. 121, 2 (2013), 246--255.

[26]

Thomas Schultze, Anne-Fernandine Rakotoarisoa, and Stefan Schulz-Hardt. 2015. Effects of distance between initial estimates and advice on advice utilization. Judgment & Decision Making, Vol. 10, 2 (2015).

[27]

Christoph Tauchert, Neda Mesbah, et almbox. 2019. Following the Robot? Investigating Users' Utilization of Advice from Robo-Advisors. In ICIS. AIS.

[28]

Lyn M Van Swol, Jihyun Esther Paik, and Andrew Prahl. 2018. Advice recipients: The psychology of advice utilization. (2018).

[29]

Alex West and Anusha Praturu. 2019. Enhancing the Census Income Prediction Dataset. https://people.ischool.berkeley.edu/ alexwest/w210_census_income_html/. Accessed: 2021-05-15.

[30]

Yao Xie, Melody Chen, David Kao, Ge Gao, and Xiang'Anthony' Chen. 2020. CheXplain: Enabling Physicians to Explore and Understand Data-Driven, AI-Enabled Medical Imaging Analysis. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, 1--13.

Digital Library

[31]

Ilan Yaniv and Dean P Foster. 1997. Precision and accuracy of judgmental estimation. Journal of behavioral decision making, Vol. 10, 1 (1997), 21--32.

Cited By

Gomez CCho SKe SHuang CUnberath M(2025)Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic reviewFrontiers in Computer Science10.3389/fcomp.2024.15210666Online publication date: 6-Jan-2025
https://doi.org/10.3389/fcomp.2024.1521066
Ke PNg K(2025)Human-AI Synergy in Survey Development: Implications from Large Language Models in Business and ResearchACM Transactions on Management Information Systems10.1145/370059716:1(1-39)Online publication date: 8-Feb-2025
https://dl.acm.org/doi/10.1145/3700597
Söllner MArnold TBenlian ABretschneider UKnight COhly SRudkowski LSchreiber GWendt D(2025)ChatGPT and Beyond: Exploring the Responsible Use of Generative AI in the WorkplaceBusiness & Information Systems Engineering10.1007/s12599-025-00932-8Online publication date: 12-Feb-2025
https://doi.org/10.1007/s12599-025-00932-8
Show More Cited By

Index Terms

Do Humans Trust Advice More if it Comes from AI?: An Analysis of Human-AI Interactions
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Problems with “Friendly AI”
Abstract
On virtue ethical grounds, Barbro Fröding and Martin Peterson recently recommended that near-future AIs should be developed as ‘Friendly AI’. AI in social interaction with humans should be programmed such that they mimic aspects of human ...
Human-Computer Interaction and AI: What Practitioners Need to Know to Design and Build Effective AI systems from a Human Perspective
CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

AI and ML are now essential parts of many systems that are currently being built. What should CHI practitioners know about the possibilities and potential drawbacks of building AI systems? Understanding the human side of AI/ML based systems requires ...
Human-Computer Interaction and AI: What practitioners need to know to design and build effective AI system from a human perspective
CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

AI and ML are now essential parts of many systems that are currently being built. What should CHI practitioners know about the possibilities and potential drawbacks of building AI systems? Understanding the human side of AI/ML based systems requires ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AIES '22: Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society

July 2022

939 pages

ISBN:9781450392471

DOI:10.1145/3514094

General Chairs:
Vincent Conitzer
Duke University & University of Oxford
,
John Tasioulas
University of Oxford
,
Program Chairs:
Matthias Scheutz
Tufts University
,
Ryan Calo
University of Washington
,
Martina Mara
Johannes Kepler University Linz
,
Annette Zimmermann
University of York

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

AIES '22

Sponsor:

SIGAI

AIES '22: AAAI/ACM Conference on AI, Ethics, and Society

May 19 - 21, 2021

Oxford, United Kingdom

Acceptance Rates

Overall Acceptance Rate 61 of 162 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
1,710
Total Downloads

Downloads (Last 12 months)782
Downloads (Last 6 weeks)103

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gomez CCho SKe SHuang CUnberath M(2025)Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic reviewFrontiers in Computer Science10.3389/fcomp.2024.15210666Online publication date: 6-Jan-2025
https://doi.org/10.3389/fcomp.2024.1521066
Ke PNg K(2025)Human-AI Synergy in Survey Development: Implications from Large Language Models in Business and ResearchACM Transactions on Management Information Systems10.1145/370059716:1(1-39)Online publication date: 8-Feb-2025
https://dl.acm.org/doi/10.1145/3700597
Söllner MArnold TBenlian ABretschneider UKnight COhly SRudkowski LSchreiber GWendt D(2025)ChatGPT and Beyond: Exploring the Responsible Use of Generative AI in the WorkplaceBusiness & Information Systems Engineering10.1007/s12599-025-00932-8Online publication date: 12-Feb-2025
https://doi.org/10.1007/s12599-025-00932-8
Kim HPark Y(2024)Trust Dynamics in Financial Decision Making: Behavioral Responses to AI and Human Expert Advice Following Structural BreaksBehavioral Sciences10.3390/bs1410096414:10(964)Online publication date: 17-Oct-2024
https://doi.org/10.3390/bs14100964
Baines JDalal RPonce LTsai H(2024)Advice from artificial intelligence: a review and practical implicationsFrontiers in Psychology10.3389/fpsyg.2024.139018215Online publication date: 8-Oct-2024
https://doi.org/10.3389/fpsyg.2024.1390182
Schleibaum SFeng LKraus SMüller JLarson K(2024)ADESSEProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/875(7904-7912)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/875
Shi ZHu BLu MChen ZZhang MYu YZhou CZhong JWu BZhang XWei YZhang L(2024)Assessing the Impact of an Artificial Intelligence-Based Model for Intracranial Aneurysm Detection in CT Angiography on Patient Diagnosis and Outcomes (IDEAL Study)—a protocol for a multicenter, double-blinded randomized controlled trialTrials10.1186/s13063-024-08184-925:1Online publication date: 4-Jun-2024
https://doi.org/10.1186/s13063-024-08184-9
Mannan SHansen PVimal VDavies HDiZio PKrishnaswamy N(2024)Combating Spatial Disorientation in a Dynamic Self-Stabilization Task Using AI AssistantsProceedings of the 12th International Conference on Human-Agent Interaction10.1145/3687272.3688329(113-122)Online publication date: 24-Nov-2024
https://dl.acm.org/doi/10.1145/3687272.3688329
Li WBrooks CJoyner DKim MWang XXia M(2024)Influence on Judgements of Learning Given Perceived AI AnnotationsProceedings of the Eleventh ACM Conference on Learning @ Scale10.1145/3657604.3662044(221-231)Online publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1145/3657604.3662044
Cao SLiu AHuang C(2024)Designing for Appropriate Reliance: The Roles of AI Uncertainty Presentation, Initial User Decision, and User Demographics in AI-Assisted Decision-MakingProceedings of the ACM on Human-Computer Interaction10.1145/36373188:CSCW1(1-32)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637318
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten