Characterizing the Limits of Autonomous Systems
Pages 2165 - 2167
Abstract
This note answers two central question in the intersection of decision-making and causal inference -- when human input is needed and, if so, how it should be incorporated into an AI system. We introduce the counterfactual agent who proactively considers human input in its decision-making process. We prove that a counterfactual agent dominates the standard autonomous agent who does not consider any human input (i.e., the experimental agent) in terms of performance. These results suggest a trade-off between autonomy and optimality -- while the full autonomy is often preferred, using human input could potentially improve the efficiency of the system. We further characterize under what conditions experimental and counterfactual agents can reach the same level of performance, which elicits the settings where full autonomy can be achieved.
References
[1]
Elias Bareinboim, Andrew Forney, and Judea Pearl. 2015. Bandits with unobserved confounders: A causal approach Advances in Neural Information Processing Systems. 1342--1350.
[2]
E. Bareinboim and J. Pearl. 2016. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences Vol. 113 (2016), 7345--7352. Issue 27.
[3]
Anthony Cassandra, Michael L Littman, and Nevin L Zhang. 1997. Incremental pruning: A simple, fast, exact method for partially observable Markov decision processes. In Proceedings of the 13th Conference on Uncertainty in Artificial Intelligence (UAI). Morgan Kaufmann, 54--61.
[4]
Hsien-Te Cheng. 1988. Algorithms for partially observable Markov decision processes. Ph.D. Dissertation. University of British Columbia.
[5]
Nicolás Della Penna, Mark D Reid, and David Balduzzi. 2016. Compliance-Aware Bandits. arXiv preprint arXiv:1602.02852 (2016).
[6]
A. Forney, J. Pearl, and E. Bareinboim. 2017. Counterfactual Data-Fusion for Online Reinforcement Learners Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research), Doina Precup and Yee Whye Teh (Eds.), Vol. Vol. 70. 1156--1164.
[7]
J.Y. Halpern. 1998. Axiomatizing Causal Reasoning. Uncertainty in Artificial Intelligence, bibfieldeditorG.F. Cooper and S. Moral (Eds.). Morgan Kaufmann, San Francisco, CA, 202--210. Also, Journal of Artificial Intelligence Research 12:3, 17--37, 2000.
[8]
Finnian Lattimore, Tor Lattimore, and Mark D Reid. 2016. Causal Bandits: Learning Good Interventions via Causal Inference Advances in Neural Information Processing Systems. 1181--1189.
[9]
Michael L Littman. 1994. The witness algorithm: Solving partially observable Markov decision processes. (1994).
[10]
J. Pearl. 2000. Causality: Models, Reasoning, and Inference. Cambridge University Press, New York.
[11]
Martin L Puterman. 2014. Markov decision processes: discrete stochastic dynamic programming. Wiley & Sons.
[12]
Diederik M Roijers, Peter Vamplew, Shimon Whiteson, Richard Dazeley, et almbox. 2013. A Survey of Multi-Objective Sequential Decision-Making. J. Artif. Intell. Res.(JAIR) Vol. 48 (2013), 67--113.
[13]
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et almbox. 2016. Mastering the game of Go with deep neural networks and tree search. Nature, Vol. 529, 7587 (2016), 484--489.
[14]
Satinder P Singh, Tommi S Jaakkola, and Michael I Jordan. 1994. Learning Without State-Estimation in Partially Observable Markovian Decision Processes. ICML. 284--292.
[15]
Richard D Smallwood and Edward J Sondik. 1973. The optimal control of partially observable Markov processes over a finite horizon. Operations Research, Vol. 21, 5 (1973), 1071--1088.
[16]
Chris Urmson, Joshua Anhalt, Drew Bagnell, Christopher Baker, Robert Bittner, MN Clark, John Dolan, Dave Duggins, Tugrul Galatali, Chris Geyer, et almbox. 2008. Autonomous driving in urban environments: Boss and the urban challenge. Journal of Field Robotics Vol. 25, 8 (2008), 425--466.
[17]
Junzhe Zhang and Elias Bareinboim. 2017. Transfer Learning in Multi-Armed Bandits: A Causal Approach Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI).
[18]
Shlomo Zilberstein. 2015. Building Strong Semi-autonomous Systems. In Proceedings of the Twenty-Ninth AAAI Conference. AAAI Press, 4088--4092.
Index Terms
- Characterizing the Limits of Autonomous Systems
Recommendations
Agent-human coordination with communication costs under uncertainty
AAMAS '12: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3As agents' technology becomes increasing more prevalent, coordination in mixed agent-human environments becomes a key issue. Agent-human coordination is becoming even more important in real life situations, where uncertainty and incomplete information ...
Live generation of interactive non-verbal behaviours
AAMAS '12: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3Psychology, robotic and virtual agents communities commonly claim that to enable natural interaction to take place within dyad of agents, the dyad must be the siege of dynamical coupling: to give to their partners a feeling of natural interaction, ...
Employing Models of Human Social Motor Behavior for Artificial Agent Trainers
AAMAS '20: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent SystemsMany everyday tasks require individuals to work together as a team to achieve a task goal. For many complex or high-stakes multi-agent activities, team members are required to participate in simulated training exercises to develop the task- and team-...
Comments
Information & Contributors
Information
Published In
July 2018
2312 pages
- General Chairs:
- Elisabeth Andre,
- Sven Koenig,
- Program Chairs:
- Mehdi Dastani,
- Gita Sukthankar
Sponsors
In-Cooperation
Publisher
International Foundation for Autonomous Agents and Multiagent Systems
Richland, SC
Publication History
Published: 09 July 2018
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
AAMAS '18
Sponsor:
Acceptance Rates
AAMAS '18 Paper Acceptance Rate 149 of 607 submissions, 25%;
Overall Acceptance Rate 1,155 of 5,036 submissions, 23%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 91Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in