research-article

Reasoning about Cognitive Trust in Stochastic Multiagent Systems

Authors:

Marta Kwiatkowska,

Maciej OlejnikAuthors Info & Claims

ACM Transactions on Computational Logic (TOCL), Volume 20, Issue 4

Article No.: 21, Pages 1 - 64

https://doi.org/10.1145/3329123

Published: 16 July 2019 Publication History

Abstract

We consider the setting of stochastic multiagent systems modelled as stochastic multiplayer games and formulate an automated verification framework for quantifying and reasoning about agents’ trust. To capture human trust, we work with a cognitive notion of trust defined as a subjective evaluation that agent A makes about agent B’s ability to complete a task, which in turn may lead to a decision by A to rely on B. We propose a probabilistic rational temporal logic PRTL*, which extends the probabilistic computation tree logic PCTL* with reasoning about mental attitudes (beliefs, goals, and intentions) and includes novel operators that can express concepts of social trust such as competence, disposition, and dependence. The logic can express, for example, that “agent A will eventually trust agent B with probability at least p that B will behave in a way that ensures the successful completion of a given task.” We study the complexity of the automated verification problem and, while the general problem is undecidable, we identify restrictions on the logic and the system that result in decidable, or even tractable, subproblems.

References

[1]

Kumar Akash, Wan-Lin Hu, Neera Jain, and Tahira Reid. 2018. A classification model for sensing human trust in machines using EEG and GSR. CoRR abs/1803.09861 (2018).

[2]

S. E. Asch. 1946. Forming impression of personality. J. Abnorm. Soc. Psychol. 40, 3 (1946), 258--290.

[3]

Adnan Aziz, Vigyan Singhal, Felice Balarin, Robert Brayton, and Alberto Sangiovanni-Vincentelli. 1995. It usually works: The temporal logic of stochastic systems. In Computer Aided Verification. Springer, Berlin, 155--165.

Digital Library

[4]

Christel Baier, Joost-Pieter Katoen, and Kim Guldstrand Larsen. 2008. Principles of Model Checking. The MIT Press.

Digital Library

[5]

Joyce Berg, John Dickhaut, and Kevin McCabe. 1995. Trust, reciprocity, and social history. Games Econ. Behav. 10, 1 (1995).

[6]

M. E. Bratman. 1987. Intentions, Plans, and Practical Reason. Harvard University Press, Cambridge, MA.

[7]

Cristiano Castelfranchi and Fabio Paglieri. 2007. The role of beliefs in goal dynamics: Prolegomena to a constructive theory of intentions. Synthese 155, 2 (2007), 237--263.

[8]

Souymodip Chakraborty and Joost-Pieter Katoen. 2016. On the satisfiability of some simple probabilistic logics. In Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS’16). ACM, New York, NY, 56--65.

Digital Library

[9]

Ashok K. Chandra, Dexter C. Kozen, and Larry J. Stockmeyer. 1980. Alternation. J. ACM 28, 1 (1980), 114--133.

Digital Library

[10]

Krishnendu Chatterjee, Martin Chmelik, and Mathieu Tracol. 2016. What is decidable about partially observable Markov decision processes with -regular objectives. J. Comput. Syst. Sci. 82, 5 (2016), 878--911.

Digital Library

[11]

Min Chen, Stefanos Nikolaidis, Harold Soh, David Hsu, and Siddhartha Srinivasa. 2018. Planning with trust for human-robot collaboration. CoRR abs/1801.04099 (2018).

Digital Library

[12]

T. Chen, V. Forejt, M. Kwiatkowska, D. Parker, and A. Simaitis. 2013. Automatic verification of competitive stochastic systems. Form. Methods Syst. Des. 43, 1 (2013).

[13]

E. M. Clarke, O. Grumberg, and D. Peled. 1999. Model Checking. The MIT Press.

Digital Library

[14]

Brian J. Corbitt, Theerasak Thanasankit, and Han Yi. 2003. Trust and e-commerce: A study of consumer perceptions. Electr. Commerce Res. Appl. 2, 3 (2003), 203--215.

[15]

Christian Dehnert, Sebastian Junges, Joost-Pieter Katoen, and Matthias Volk. 2017. A storm is coming: A modern probabilistic model checker. CoRR abs/1702.04311 (2017).

[16]

R. Fagin, J. Halpern, Y. Moses, and M. Vardi. 1995. Reasoning about Knowledge. MIT Press.

Digital Library

[17]

R. Falcone and C. Castelfranchi. 2001. Social trust: A cognitive approach. In Trust and Deception in Virtual Societies. Kluwer, 55--90.

Digital Library

[18]

Nir Friedman and Joseph Y. Halpern. 1997. Modeling belief in dynamic systems, part I: Foundations. Artif. Intell. 95, 2 (1997), 257--316.

Digital Library

[19]

D. Gambetta (Ed.). 1990. Trust. Basil Blackwell, Oxford.

[20]

Judy Goldsmith and Martin Mundhenk. 1998. Complexity issues in Markov decision processes. In Proceedings of the 13th Annual IEEE Conference on Computational Complexity.

Digital Library

[21]

P. M. Gollwitzer. 1993. Goal achievement: The role of intentions. Eur. Rev. Soc. Psychol. 4, 1 (1993), 141--185.

[22]

Joseph Y. Halpern and Mark R. Tuttle. 1993. Knowledge, probability, and adversaries. J. ACM 40, 3 (1993), 917--962.

Digital Library

[23]

Hans Hansson and Bengt Jonsson. 1994. A logic for reasoning about time and reliability. Formal Aspects Comput. 6, 5 (1994), 512--535.

Digital Library

[24]

Russell Hardin. 2002. Trust and Trustworthiness. Russell Sage Foundation.

[25]

Andreas Herzig, Emiliano Lorini, Jomi F Hübner, and Laurent Vercouter. 2010. A logic of trust and reputation. Logic J. IGPL 18, 1 (2010), 214--244.

[26]

Andreas Herzig, Emiliano Lorini, and Frédéric Moisan. 2013. A simple logic of trust based on propositional assignments. The Goals of Cognition: Essays in Honor of Cristiano Castelfranchi, 407--419.

[27]

Robin M. Hogarth and Hillel J. Einhorn. 1992. Order effects in belief updating: The belief-adjustment model. Cogn. Psychol. 24, 1 (1992), 1--55.

[28]

Xiaowei Huang and Marta Kwiatkowska. 2017. Reasoning about cognitive trust in stochastic multiagent systems. In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI’17).

Digital Library

[29]

Audun Jøsang. 2001. A logic for uncertain probabilities. Int. J. Uncert. Fuzz. Knowl.-Based Syst. 9, 3 (2001), 279--311.

Digital Library

[30]

L. Kagal, T. Finin, and A. Joshi. 2001. Trust-based security in pervasive computing environments. Computer 34, 12 (2001), 154--157.

Digital Library

[31]

John G. Kemeny, J. Laurie Snell, and Anthony W. Knapp. 1967. Denumerable Markov Chains. Springer-Verlag.

[32]

Karl Krukow, Mogens Nielsen, and Vladimiro Sassone. 2008. Trust models in ubiquitous computing. Philos. Trans. Roy. Soc. Lond. A 366, 1881 (2008), 3781--3793.

[33]

Benjamin Kuipers. 2016. What is trust and how can my robot get some? In A Talk at Social Trust in Autonomous Robots, a Workshop in Robotics: Science and Systems 2016.

[34]

Benjamin Kuipers. 2018. How can we trust a robot? Commun. ACM 61, 3 (Feb. 2018), 86--95.

Digital Library

[35]

M. Kwiatkowska, G. Norman, and D. Parker. 2011. PRISM 4.0: Verification of probabilistic real-time systems. In Proceedings of the 23rd International Conference on Computer Aided Verification (CAV’11), G. Gopalakrishnan and S. Qadeer (Eds.), Lecture Notes in Computer Science, Vol. 6806. Springer, 585--591.

Digital Library

[36]

Morteza Lahijanian and Marta Kwiatkowska. 2016. Social trust: A major challenge for the future of autonomous systems. In Proceedings of the AAAI Fall Symposium on Cross-Disciplinary Challenges for Autonomous Systems (AAAI Fall Symposium’16). AAAI Press.

[37]

Dave Lee. 2016. US opens investigation into Tesla after fatal crash. British Broadcasting Corporation (BBC) News. Retrieved July 1, 2016 from http://www.bbc.co.uk/news/technology-36680043.

[38]

John D. Lee and Katrina A. See. 2004. Trust in automation: Designing for appropriate reliance. J. Hum. Factors Ergon. Soc. 46, 1 (2004), 50--80.

[39]

Roger C. Mayer, James H. Davis, and F. David Schoorman. 1995. An integrative model of organizational trust. Acad. Manage. Rev. 20, 3 (1995), 709--734.

[40]

John-Jules Ch. Meyer, Jan Broersen, and Andreas Herzig. 2014. BDI logics. In Handbook of Epistemic Logic. College Publications.

[41]

J.-J. Ch Meyer, Wiebe van der Hoek, and Bernd van Linder. 1999. A logical approach to the dynamics of commitments. Artif. Intell. 113, 1 (1999), 1--40.

Digital Library

[42]

Azaria Paz. 1971. Introduction to Probabilistic Automata (Computer Science and Applied Mathematics). Academic Press.

Digital Library

[43]

Judea Pearl. 1994. Probabilistic semantics for nonmonotonic reasoning: A survey. In Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning (KR’94).

Digital Library

[44]

Anand S. Rao and Michael P. Georgeff. 1991. Modeling rational agents within a BDI-architecture. In Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning (KR’91).

Digital Library

[45]

Frank E. Ritter, Josef Nerb, Erno Lehtinen, and Timothy O’Shea. 2007. In Order to Learn: How the Sequence of Topics Influences Learning. Oxford University Press.

[46]

Klaus Schild. 2000. On the relationship between BDI logics and standard logics of concurrency. Autonomous Agents and Multi-Agent Systems 3, 3 (2000), 259--283.

Digital Library

[47]

Tina Setter, Andrea Gasparri, and Magnus Egerstedt. 2016. Trust-based interactions in teams of mobile agents. In Proceedings of the American Control Conference. 6158--6163.

[48]

Nicholas Sweet, Nisar R. Ahmed, Ugur Kuter, and Christopher Miller. 2016. Towards self-confidence in autonomous systems. In Proceedings of the AIAA Infotech@ Aerospace Conference. 1651--1652.

[49]

Wen-Guey Tzeng. 1992. A polynomial-time algorithm for the equivalence of probabilistic automata. SIAM J. Comput. 21, 2 (1992), 216--227.

Digital Library

[50]

Wiebe van der Hoek, Wojciech Jamroga, and Michael Wooldridge. 2007. Towards a theory of intention revision. Synthese 155, 2 (2007), 265--290.

Cited By

Akintunde MYazdanpanah VSalehi Fathabadi ACirstea CDastani MMoreau LDastani MSichman JAlechina NDignum V(2024)Actual Trust in Multiagent SystemsProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663078(2114-2116)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3663078
Yu PDong SSheng SFeng LKwiatkowska M(2024)Trust-Aware Motion Planning for Human-Robot Collaboration under Distribution Temporal Logic Specifications2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610874(12949-12955)Online publication date: 13-May-2024
https://doi.org/10.1109/ICRA57147.2024.10610874
Akintunde MYoung VYazdanpanah VSalehi Fathabadi ALeonard PButler MMoreau L(2023)Verifiably Safe and Trusted Human-AI Systems: A Socio-technical PerspectiveProceedings of the First International Symposium on Trustworthy Autonomous Systems10.1145/3597512.3599719(1-6)Online publication date: 11-Jul-2023
https://dl.acm.org/doi/10.1145/3597512.3599719
Show More Cited By

Index Terms

Reasoning about Cognitive Trust in Stochastic Multiagent Systems
1. Theory of computation
  1. Logic
    1. Modal and temporal logics
    2. Verification by model checking
  2. Models of computation
    1. Probabilistic computation

Recommendations

The effects of personalizaion and familiarity on trust and adoption of recommendation agents

In the context of personalization technologies, such as Web-based product-brokering recommendation agents (RAs) in electronic commerce, existing technology acceptance theories need to be expanded to take into account not only the cognitive beliefs ...
Reasoning about cognitive trust in stochastic multiagent systems
AAAI'17: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence

We consider the setting of stochastic multiagent systems and formulate an automated verification framework for quantifying and reasoning about agents' trust. To capture human trust, we work with a cognitive notion of trust defined as a subjective ...
Should I trust him? The effects of reviewer profile characteristics on eWOM credibility

For online consumer reviews to serve as decision-making aids, users have to first trust the reviewer. However, unlike face-to-face communication, where trust develops overtime, consumers have to rely on personal profile information to establish ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computational Logic

ACM Transactions on Computational Logic Volume 20, Issue 4

October 2019

323 pages

ISSN:1529-3785

EISSN:1557-945X

DOI:10.1145/3347091

Editor:
Orna Kupferman
School of Computer Science and Engineering, Hebrew University, Israel

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 July 2019

Accepted: 01 April 2019

Revised: 01 January 2019

Received: 01 June 2018

Published in TOCL Volume 20, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
367
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)4

Reflects downloads up to 02 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Akintunde MYazdanpanah VSalehi Fathabadi ACirstea CDastani MMoreau LDastani MSichman JAlechina NDignum V(2024)Actual Trust in Multiagent SystemsProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663078(2114-2116)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3663078
Yu PDong SSheng SFeng LKwiatkowska M(2024)Trust-Aware Motion Planning for Human-Robot Collaboration under Distribution Temporal Logic Specifications2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610874(12949-12955)Online publication date: 13-May-2024
https://doi.org/10.1109/ICRA57147.2024.10610874
Akintunde MYoung VYazdanpanah VSalehi Fathabadi ALeonard PButler MMoreau L(2023)Verifiably Safe and Trusted Human-AI Systems: A Socio-technical PerspectiveProceedings of the First International Symposium on Trustworthy Autonomous Systems10.1145/3597512.3599719(1-6)Online publication date: 11-Jul-2023
https://dl.acm.org/doi/10.1145/3597512.3599719
Fu CTurrini AHuang XSong LFeng YZhang L(2023)Model Checking for Probabilistic Multiagent SystemsJournal of Computer Science and Technology10.1007/s11390-022-1218-638:5(1162-1186)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1007/s11390-022-1218-6
Huang XPeng BZhao XAlbrecht SWoolridge M(2022)Dependable learning-enabled multiagent systemsAI Communications10.3233/AIC-22012835:4(407-420)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.3233/AIC-220128
Zhang NWang MDuan ZTian C(2022)Verifying Properties of MapReduce-Based Big Data ProcessingIEEE Transactions on Reliability10.1109/TR.2020.299944171:1(321-338)Online publication date: Mar-2022
https://doi.org/10.1109/TR.2020.2999441
Iqbal DBuhnova B(2022)Model-based Approach for Building Trust in Autonomous Drones through Digital Twins2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53654.2022.9945227(656-662)Online publication date: 9-Oct-2022
https://doi.org/10.1109/SMC53654.2022.9945227
Sreedevi ANitya Harshitha TSugumaran VShankar P(2022)Application of cognitive computing in healthcare, cybersecurity, big data and IoTInformation Processing and Management: an International Journal10.1016/j.ipm.2022.10288859:2Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1016/j.ipm.2022.102888
Drawel NBentahar JLaarej ARjoub G(2022)Formal verification of group and propagated trust in multi-agent systemsAutonomous Agents and Multi-Agent Systems10.1007/s10458-021-09542-636:1Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1007/s10458-021-09542-6
Fisher MCardoso RCollins EDadswell CDennis LDixon CFarrell MFerrando AHuang XJump MKourtis GLisitsa ALuckcuck MLuo SPage VPapacchini FWebster M(2021)An Overview of Verification and Validation Challenges for Inspection RobotsRobotics10.3390/robotics1002006710:2(67)Online publication date: 29-Apr-2021
https://doi.org/10.3390/robotics10020067
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents