A Formal Account of AI Trustworthiness: Connecting Intrinsic and Perceived Trustworthiness
Abstract
This paper proposes a formal account of AI trustworthiness, connecting both intrinsic and perceived trustworthiness in an operational schematization. We argue that trustworthiness extends beyond the inherent capabilities of an AI system to include significant influences from observers' perceptions, such as perceived transparency, agency locus, and human oversight. While the concept of perceived trustworthiness is discussed in the literature, few attempts have been made to connect it with the intrinsic trustworthiness of AI systems. Our analysis introduces a novel schematization to quantify trustworthiness by assessing the discrepancies between expected and observed behaviors and how these affect perceived uncertainty and trust. The paper provides a formalization for measuring trustworthiness, taking into account both perceived and intrinsic characteristics. By detailing the factors that influence trust, this study aims to foster more ethical and widely accepted AI technologies, ensuring they meet both functional and ethical criteria.
References
[1]
Bernotat, J., & Eyssel, F. (2017). A robot at home---How affect, technology commitment, and personality traits influence user experience in an intelligent robotics apartment. 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 641--646.
[2]
Bisconti, P. (2021). How Robots' Unintentional Meta-communication Affects Human-Robot Interactions. A Systemic Approach. Minds and Machines, 31(4), 487--504.
[3]
Colquitt, J. A., Scott, B. A., & LePine, J. A. (2007). Trust, trustworthiness, and trust propensity: A meta-analytic test of their unique relationships with risk taking and job performance. Journal of Applied Psychology, 92(4), 909--927.
[4]
Driskell, J. E., Goodwin, G. F., Salas, E., & O'Shea, P. G. (2006). What makes a good team player? Personality and team effectiveness. Group Dynamics: Theory, Research, and Practice, 10(4), 249--271.
[5]
Esterwood, C., Essenmacher, K., Yang, H., Zeng, F., & Robert, L. P. (2021). Birds of a Feather Flock Together: But do Humans and Robots? A Meta-Analysis of Human and Robot Personality Matching. 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), 343--348.
[6]
Felzmann, H., Fosch-Villaronga, E., Lutz, C., & Tamò-Larrieux, A. (2020). Towards transparency by design for artificial intelligence. Science and Engineering Ethics, 26(6), 3333--3361.
[7]
Graziani, M., Dutkiewicz, L., Calvaresi, D., Amorim, J. P., Yordanova, K., Vered, M., Nair, R., Abreu, P. H., Blanke, T., & Pulignano, V. (2023). A global taxonomy of interpretable AI: unifying the terminology for the technical and social sciences. Artificial Intelligence Review, 56(4), 3473--3504.
[8]
Grimes, G. M., Schuetzler, R. M., & Giboney, J. S. (2021). Mental models and expectation violations in conversational AI interactions. Decision Support Systems, 144, 113515.
[9]
Jeronimus, B. F., Kotov, R., Riese, H., & Ormel, J. (2016). Neuroticism's prospective association with mental disorders halves after adjustment for baseline symptoms and psychiatric history, but the adjusted association hardly decays with time: A meta-analysis on 59 longitudinal/prospective studies with 443 313 participants. Psychological Medicine, 46(14), 2883--2906.
[10]
Jiang, J., Kahai, S., & Yang, M. (2022). Who needs explanation and when? Juggling explainable AI and user epistemic uncertainty. International Journal of Human-Computer Studies, 165, 102839.
[11]
John, O. P., Naumann, L. P., & Soto, C. J. (2008). Paradigm shift to the integrative big five trait taxonomy. Handbook of Personality: Theory and Research, 3(2), 114--158.
[12]
John, O. P., Naumann, L. P., & Soto, C. J. (2008). Paradigm shift to the integrative big five trait taxonomy. Handbook of Personality: Theory and Research, 3(2), 114--158.
[13]
Jones, S. L., & Shah, P. P. (2016). Diagnosing the locus of trust: A temporal perspective for trustor, trustee, and dyadic influences on perceived trustworthiness. Journal of Applied Psychology, 101(3), 392.
[14]
Kätsyri, J., Förger, K., Mäkäräinen, M., & Takala, T. (2015). A review of empirical evidence on different uncanny valley hypotheses: support for perceptual mismatch as one road to the valley of eeriness. Frontiers in psychology, 6, 390
[15]
Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2022). Trustworthy artificial intelligence: a review. ACM Computing Surveys (CSUR), 55(2), 1--38.
[16]
Kizilcec, R. F. (2016). How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 2390--2395.
[17]
Lin, J., Amini, S., Hong, J. I., Sadeh, N., Lindqvist, J., & Zhang, J. (2012). Expectation and purpose: understanding users' mental models of mobile app privacy through crowdsourcing. In Proceedings of the 2012 ACM conference on ubiquitous computing (pp. 501--510).
[18]
Liu, B. (2021). In AI We Trust? Effects of Agency Locus and Transparency on Uncertainty Reduction in Human-AI Interaction. Journal of Computer-Mediated Communication, 26(6), 384--402. Scopus.
[19]
Matsumoto, D., & Juang, L. (2016). Culture and psychology. Cengage Learning.
[20]
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An Integrative Model of Organizational Trust. The Academy of Management Review, 20(3), 709.
[21]
McKnight, D. H., Cummings, L. L., & Chervany, N. L. (1998). Initial Trust Formation in New Organizational Relationships. The Academy of Management Review, 23(3), 473.
[22]
Milliken, F. J. (1987). Three types of perceived uncertainty about the environment: State, effect, and response uncertainty. Academy of Management Review, 12(1), 133--143.
[23]
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of algorithms: Mapping the debate. Big Data & Society, 3(2), 205395171667967.
[24]
Ososky, S., Philips, E., Schuster, D., & Jentsch, F. (2013). A Picture is Worth a Thousand Mental Models: Evaluating Human Understanding of Robot Teammates. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 57(1), 1298--1302.
[25]
Papenmeier, A., Kern, D., Englebienne, G., & Seifert, C. (2022). It's Complicated: The Relationship between User Trust, Model Accuracy and Explanations in AI. ACM Transactions on Computer-Human Interaction, 29(4). Scopus.
[26]
Rhee, J., Parent, D., & Basu, A. (2013). The influence of personality and ability on undergraduate teamwork and team performance. SpringerPlus, 2(1), 16.
[27]
Schlicker, N., & Langer, M. (2021). Towards warranted trust: A model on the relation between actual and perceived system trustworthiness. In Proceedings of Mensch und Computer 2021 (pp. 325--329).
[28]
Stanton, B., & Jensen, T. (2021). Trust and artificial intelligence. Preprint.
[29]
Zhang, J., & Curley, S. P. (2018). Exploring Explanation Effects on Consumers' Trust in Online Recommender Agents. International Journal of Human-Computer Interaction, 34(5), 421--432.
Index Terms
- A Formal Account of AI Trustworthiness: Connecting Intrinsic and Perceived Trustworthiness
Index terms have been assigned to the content through auto-classification.
Recommendations
Perceived Trustworthiness Within the Organization: The Moderating Impact of Communication Frequency on Trustor and Trustee Effects
This paper investigates the antecedents of intraorganizational trust and, more specifically, how the frequency of communication between trustor and trustee moderates the impact of these factors on perceived trustworthiness. Data on 157 dyadic ...
Asymmetrical impact of trustworthiness attributes on trust, perceived value and purchase intention: a conceptual framework for cross-cultural study on consumer perception of online auction
ICEC '03: Proceedings of the 5th international conference on Electronic commerceLack of trust has been frequently cited to be one of the key factors that discourage customers from participating in e-commerce, while cultural differences affect the formation of trust. As such, this cross-cultural study aims at establishing a ...
Comments
Information & Contributors
Information
Published In
October 2024
1756 pages
- Conference Co-chairs:
- Sanmay Das,
- Brian Patrick Green,
- Program Co-chairs:
- Kush Varshney,
- Marianna Ganapini,
- Andrea Renda
Sponsors
Publisher
AAAI Press
Publication History
Published: 07 February 2025
Check for updates
Qualifiers
- Research-article
Conference
AIES '24
Sponsor:
AIES '24: 2024 AAAI/ACM Conference on AI, Ethics, and Society
October 21 - 23, 2024
California, San Jose, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 3Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)3
Reflects downloads up to 09 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in