research-article

Public Access

Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations

Authors:

Brent Harrison,

Mark O. RiedlAuthors Info & Claims

AIES '18: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society

Pages 81 - 87

https://doi.org/10.1145/3278721.3278736

Published: 27 December 2018 Publication History

Abstract

We introduce \em AI rationalization, an approach for generating explanations of autonomous system behavior as if a human had performed the behavior. We describe a rationalization technique that uses neural machine translation to translate internal state-action representations of an autonomous agent into natural language. We evaluate our technique in the Frogger game environment, training an autonomous game playing agent to rationalize its action choices using natural language. A natural language training corpus is collected from human players thinking out loud as they play the game. We motivate the use of rationalization as an approach to explanation generation and show the results of two experiments evaluating the effectiveness of rationalization. Results of these evaluations show that neural machine translation is able to accurately generate rationalizations that describe agent behavior, and that rationalizations are more satisfying to humans than other alternative methods of explanation.

References

[1]

Andreas, J.; Dragan, A. D.; and Klein, D. 2017. Translating neuralese. CoRR abs/1704.06960.

[2]

Aronson, J. 1995. A pragmatic view of thematic analysis. The qualitative report 2(1):1--3.

[3]

Core, M.; Lane, H. C.; van Lent, M.; Gomboc, D.; Solomon, S.; and Rosenberg, M. 2006. Building Explainable Artificial Intelligence Systems. In Proceedings of the 18th Innovative Applications of Artificial Intelligence Conference.

Digital Library

[4]

Dorst, K., and Cross, N. 2001. Creativity in the design process: co-evolution of problem--solution. Design studies 22(5):425--437.

[5]

Fonteyn, M. E.; Kuipers, B.; and Grobe, S. J. 1993. A description of think aloud method and protocol analysis. Qualitative Health Research 3(4):430--441.

[6]

Goldman, A. I., et al. 2012. Theory of mind. The Oxford handbook of philosophy of cognitive science 402--424.

[7]

Hollander, M.; Wolfe, D. A.; and Chicken, E. 2013. Nonparametric statistical methods. John Wiley & Sons.

[8]

Krause, J.; Perer, A.; and Ng, K. 2016. Interacting with predictions: Visual inspection of black-box machine learning models. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 5686--5697. ACM.

Digital Library

[9]

Laird, J., and van Lent, M. 2001. Human-level aiâ's killer application: Interactive computer games. AI Magazine 22(2):15--25.

[10]

Litman, L.; Robinson, J.; and Abberbock, T. 2017. Turkprime. com: A versatile crowdsourcing data acquisition platform for the behavioral sciences. Behavior research methods 49(2):433--442.

[11]

Luong, M.-T.; Pham, H.; and Manning, C. D. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 1412--1421. Lisbon, Portugal: Association for Computational Linguistics.

[12]

Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A.; Veness, J.; Bellemare, M.; Graves, A.; Riedmiller, M.; Fidjeland, A.; Ostrovski, G.; Petersen, S.; Beattie, C.; Sadik, A.; Antonoglou, I.; King, H.; Kumaran, D.; Wierstra, D.; Legg, S.; and Hassabis, D. 2015. Human-level control through deep reinforcement learning. Nature 518(7540):529--533.

[13]

Papineni, K.; Roukos, S.; Ward, T.; and Zhu, W.-J. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, 311--318. Association for Computational Linguistics.

Digital Library

[14]

Ribeiro, M. T.; Singh, S.; and Guestrin, C. 2016. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135--1144. ACM.

Digital Library

[15]

Strauss, A., and Corbin, J. 1994. Grounded theory methodology. Handbook of qualitative research 17:273--85.

[16]

van Lent, M.; ; Carpenter, P.; McAlinden, R.; and Brobst, P. 2005. Increasing replayability with deliberative and reactive planning. In 1st Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE), 135--140.

Digital Library

[17]

van Lent, M.; Fisher, W.; and Mancuso, M. 2004. An explainable artificial intelligence system for small-unit tactical behavior. In Proceedings of the 16th conference on Innovative Applications of Artifical Intelligence.

Digital Library

[18]

Watkins, C., and Dayan, P. 1992. Q-learning. Machine Learning 8(3--4):279-292.

Digital Library

[19]

Weston, J.; Bordes, A.; Chopra, S.; Rush, A. M.; van Merriënboer, B.; Joulin, A.; and Mikolov, T. 2015. Towards ai-complete question answering: A set of prerequisite toy tasks. arXiv preprint arXiv:1502.05698.

[20]

Yosinski, J.; Clune, J.; Fuchs, T.; and Lipson, H. 2015. Understanding neural networks through deep visualization. In In ICMLWorkshop on Deep Learning. Citeseer.

[21]

Zeiler, M. D., and Fergus, R. 2014. Visualizing and understanding convolutional networks. In European conference on computer vision, 818--833. Springer.

Cited By

Luo LZhang GXu HYang YFang CLi QSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)End-to-end neuro-symbolic reinforcement learning with textual explanationsProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693432(33533-33557)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693432
Herrewijnen ENguyen DBex Fvan Deemter K(2024)Human-annotated rationales and explainable text classification: a surveyFrontiers in Artificial Intelligence10.3389/frai.2024.12609527Online publication date: 24-May-2024
https://doi.org/10.3389/frai.2024.1260952
Cao FHan SChung HLarson K(2024)PEACHProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/686(6207-6215)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/686
Show More Cited By

Index Terms

Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Natural language generation
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Automated rationale generation: a technique for explainable AI and its effects on human perceptions
IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

Automated rationale generation is an approach for real-time explanation generation whereby a computational model learns to translate an autonomous agent's internal state and action data representations into natural language. Training on human ...
Explainable Argumentation for Wellness Consultation
Explainable, Transparent Autonomous Agents and Multi-Agent Systems
Abstract
There has been a recent resurgence in the area of explainable artificial intelligence as researchers and practitioners seek to provide more transparency to their algorithms. Much of this research is focused on explicitly explaining decisions or ...
The Use and Misuse of Counterfactuals in Ethical Machine Learning
FAccT '21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

The use of counterfactuals for considerations of algorithmic fairness and explainability is gaining prominence within the machine learning community and industry. This paper argues for more caution with the use of counterfactuals when the facts to be ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AIES '18: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society

December 2018

406 pages

ISBN:9781450360128

DOI:10.1145/3278721

Program Chairs:
Jason Furman
Harvard University, USA
,
Gary Marchant
Arizona State University, USA
,
Huw Price
Cambridge University, UK
,
Francesca Rossi
IBM Research, USA & University of Padova, Italy

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 December 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Office of Naval Research

Conference

AIES '18

Sponsor:

SIGAI

AIES '18: AAAI/ACM Conference on AI, Ethics, and Society

February 2 - 3, 2018

LA, New Orleans, USA

Acceptance Rates

AIES '18 Paper Acceptance Rate 61 of 162 submissions, 38%;

Overall Acceptance Rate 61 of 162 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

62
Total Citations
View Citations
1,987
Total Downloads

Downloads (Last 12 months)448
Downloads (Last 6 weeks)41

Reflects downloads up to 16 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Luo LZhang GXu HYang YFang CLi QSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)End-to-end neuro-symbolic reinforcement learning with textual explanationsProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693432(33533-33557)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693432
Herrewijnen ENguyen DBex Fvan Deemter K(2024)Human-annotated rationales and explainable text classification: a surveyFrontiers in Artificial Intelligence10.3389/frai.2024.12609527Online publication date: 24-May-2024
https://doi.org/10.3389/frai.2024.1260952
Cao FHan SChung HLarson K(2024)PEACHProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/686(6207-6215)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/686
Tian YKummerfeld JLi TZhang T(2024)SQLucid: Grounding Natural Language Database Queries with Interactive ExplanationsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676368(1-20)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676368
Ning ZTian YZhang ZZhang TLi T(2024)Insights into Natural Language Database Query Errors: from Attention Misalignment to User Handling StrategiesACM Transactions on Interactive Intelligent Systems10.1145/365011414:4(1-32)Online publication date: 2-Mar-2024
https://dl.acm.org/doi/10.1145/3650114
Sanneman LTucker MShah J(2024)An Information Bottleneck Characterization of the Understanding-Workload Tradeoff in Human-Centered Explainable AIProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659032(2175-2198)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3659032
Buhl TVon Mammen S(2024)Surrogate Models of Self-Organising Systems2024 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)10.1109/ACSOS-C63493.2024.00029(47-52)Online publication date: 16-Sep-2024
https://doi.org/10.1109/ACSOS-C63493.2024.00029
Antamis TDrosou AVafeiadis TNizamis AIoannidis DTzovaras D(2024)Interpretability of deep neural networksNeurocomputing10.1016/j.neucom.2024.128204601:COnline publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.128204
Zheng CWang KGao SYu YWang ZTang Y(2024)Design of multi-modal feedback channel of human–robot cognitive interface for teleoperation in manufacturingJournal of Intelligent Manufacturing10.1007/s10845-024-02451-xOnline publication date: 9-Jul-2024
https://doi.org/10.1007/s10845-024-02451-x
Schwettmann SShaham TMaterzynska JChowdhury NLi SAndreas JBau DTorralba AOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)FINDProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3669430(75688-75715)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3669430
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten