research-article

Global Explanations of Neural Networks: Mapping the Landscape of Predictions

Authors:

Ceena Modarres,

John PaisleyAuthors Info & Claims

AIES '19: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society

Pages 279 - 287

https://doi.org/10.1145/3306618.3314230

Published: 27 January 2019 Publication History

Abstract

A barrier to the wider adoption of neural networks is their lack of interpretability. While local explanation methods exist for one prediction, most global attributions still reduce neural network decisions to a single set of features. In response, we present an approach for generating global attributions called GAM, which explains the landscape of neural network predictions across subpopulations. GAM augments global explanations with the proportion of samples that each attribution best explains and specifies which samples are described by each attribution. Global explanations also have tunable granularity to detect more or fewer subpopulations. We demonstrate that GAM's global explanations 1) yield the known feature importances of simulated data, 2) match feature weights of interpretable statistical models on real data, and 3) are intuitive to practitioners through user studies. With more transparent predictions, GAM can help ensure neural network decisions are generated for the right reasons.

References

[1]

Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. 2018. Towards better understanding of gradient-based attribution methods for Deep Neural Networks. In International Conference on Learning Representations. https://openreview.net/forum?id=Sy21R9JAW

[2]

Alexander Binder, Gré goire Montavon, Sebastian Bach, Klaus-Robert Mü ller, and Wojciech Samek. 2016. Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers. CoRR, Vol. abs/1604.00825 (2016). arxiv: 1604.00825 http://arxiv.org/abs/1604.00825

[3]

Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '16). ACM, New York, NY, USA, 785--794.

Digital Library

[4]

Anupam Datta, Shayak Sen, and Yair Zick. 2016. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In Security and Privacy (SP), 2016 IEEE Symposium on. IEEE, 598--617.

[5]

Dua Dheeru and Efi Karra Taniskidou. 2017. UCI Machine Learning Repository: Mushroom Data Set. https://archive.ics.uci.edu/ml/datasets/mushroom

[6]

Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017).

[7]

Gregory A. Fredricks and Roger B. Nelsen. 2007. On the relationship between Spearman's rho and Kendall's tau for pairs of continuous random variables. Journal of Statistical Planning and Inference, Vol. 137, 7 (2007), 2143 -- 2150.

[8]

Nicholas Frosst and Geoffrey Hinton. 2017. Distilling a neural network into a soft decision tree. arXiv preprint arXiv:1711.09784 (2017).

[9]

Youngwook Kim. 2018. Kaggle: Modeling on Mushroom Dataset. https://www.kaggle.com/kt66nf/quick-keras-modeling-on-mushroom-dataset/notebook

[10]

Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Kristof T. Schutt, Maximilian Alber, Sven Dahne, Dumitru Erhan, and Been Kim. 2018a. The (Un)reliability of saliency methods. (2018).

[11]

Pieter-Jan Kindermans, Kristof T. Schutt, Maximilian Alber, Klaus-Robert Muller, Dumitru Erhan, Been Kim, and Sven Dahne. 2018b. Learning how to explain neural networks: PatternNet and PatternAttribution. (2018). https://openreview.net/forum?id=Hkn7CBaTW

[12]

William R. Knight. 1966. A Computer Method for Calculating Kendall's Tau with Ungrouped Data. J. Amer. Statist. Assoc., Vol. 61, 314 (1966), 436--439. http://www.jstor.org/stable/2282833

[13]

Himabindu Lakkaraju, Stephen H Bach, and Jure Leskovec. 2016. Interpretable decision sets: A joint framework for description and prediction. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 1675--1684.

Digital Library

[14]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature, Vol. 521, 7553 (2015), 436.

[15]

Paul Lee and Philip Yu. 2012. Mixtures of weighted distance-based models for ranking data with applications in political studies., Vol. 56 (08 2012), 2486--2500.

Digital Library

[16]

Paul H. Lee and Philip L. H. Yu. 2010. Distance-based Tree Models for Ranking Data. Comput. Stat. Data Anal., Vol. 54, 6 (June 2010), 1672--1682.

Digital Library

[17]

Scott Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. CoRR, Vol. abs/1705.07874 (2017). arxiv: 1705.07874 http://arxiv.org/abs/1705.07874

Digital Library

[18]

Gré goire Montavon, Sebastian Bach, Alexander Binder, Wojciech Samek, and Klaus-Robert Mü ller. 2015. Explaining NonLinear Classification Decisions with Deep Taylor Decomposition. CoRR, Vol. abs/1512.02479 (2015). arxiv: 1512.02479 http://arxiv.org/abs/1512.02479

[19]

Hae-Sang Park and Chi-Hyuck Jun. 2009. A Simple and Fast Algorithm for K-medoids Clustering. Expert Syst. Appl., Vol. 36, 2 (March 2009), 3336--3341.

Digital Library

[20]

Marco Tú lio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. CoRR, Vol. abs/1602.04938 (2016). arxiv: 1602.04938 http://arxiv.org/abs/1602.04938

[21]

Peter J Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, Vol. 20 (1987), 53--65.

Digital Library

[22]

Grace S. Shieh, Zhidong Bai, and Wei-Yann Tsai. 2000. RANK TESTS FOR INDEPENDENCE - WITH A WEIGHTED CONTAMINATION ALTERNATIVE. Statistica Sinica, Vol. 10, 2 (2000), 577--593. http://www.jstor.org/stable/24306733

[23]

Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning Important Features Through Propagating Activation Differences. CoRR, Vol. abs/1704.02685 (2017). arxiv: 1704.02685 http://arxiv.org/abs/1704.02685

[24]

Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. CoRR, Vol. abs/1312.6034 (2013). arxiv: 1312.6034 http://arxiv.org/abs/1312.6034

[25]

Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic Attribution for Deep Networks. CoRR, Vol. abs/1703.01365 (2017). arxiv: 1703.01365 http://arxiv.org/abs/1703.01365

[26]

Weichao Xu, Yunhe Hou, Y.S. Hung, and Yuexian Zou. 2013. A comparative analysis of Spearman's rho and Kendall's tau in normal and contaminated normal models. Signal Processing, Vol. 93, 1 (2013), 261 -- 276.

Digital Library

[27]

Chengliang Yang, Anand Rangarajan, and Sanjay Ranka. 2018. Global Model Interpretation via Recursive Partitioning. arXiv preprint arXiv:1802.04253 (2018).

Cited By

Yang YWang JMao XLu WWang RZheng H(2024)Susceptibility Modeling and Potential Risk Analysis of Thermokarst Hazard in Qinghai–Tibet Plateau Permafrost Landscapes Using a New Interpretable Ensemble Learning MethodAtmosphere10.3390/atmos1507078815:7(788)Online publication date: 29-Jun-2024
https://doi.org/10.3390/atmos15070788
Sarridis IKoutlis CPapadopoulos SDiou CGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)FaceX: Understanding Face Attribute Classifiers through Summary Model ExplanationsProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658007(758-766)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658007
Sullivan E(2024)SIDEs: Separating Idealization from Deceptive 'Explanations' in xAIProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658999(1714-1724)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658999
Show More Cited By

Index Terms

Global Explanations of Neural Networks: Mapping the Landscape of Predictions
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
      1. Semantic networks
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification

Recommendations

Artificial neural networks: learning algorithms, performance evaluation, and applications
Global exponential stability of delayed Hopfield neural networks

In this paper, we discuss delayed Hopfield neural networks, investigating their global exponential stability. Sufficient conditions ensuring global exponential stability of delayed Hopfield neural networks are given.
Comparison of Stochastic Global Optimization Methods to Estimate Neural Network Weights

Training a neural network is a difficult optimization problem because of numerous local minima. Many global search algorithms have been used to train neural networks. However, local search algorithms are more efficient with computational resources, and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AIES '19: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society

January 2019

577 pages

ISBN:9781450363242

DOI:10.1145/3306618

General Chairs:
Vincent Conitzer
Duke University, USA
,
Gillian Hadfield
University of Toronto + Vector Institute, Canada; OpenAI, USA
,
Shannon Vallor
Santa Clara University, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
AAAI: American Association for Artificial Intelligence

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 January 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

AIES '19

Sponsor:

SIGAI
AAAI

AIES '19: AAAI/ACM Conference on AI, Ethics, and Society

January 27 - 28, 2019

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 61 of 162 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

61
Total Citations
View Citations
644
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)13

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang YWang JMao XLu WWang RZheng H(2024)Susceptibility Modeling and Potential Risk Analysis of Thermokarst Hazard in Qinghai–Tibet Plateau Permafrost Landscapes Using a New Interpretable Ensemble Learning MethodAtmosphere10.3390/atmos1507078815:7(788)Online publication date: 29-Jun-2024
https://doi.org/10.3390/atmos15070788
Sarridis IKoutlis CPapadopoulos SDiou CGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)FaceX: Understanding Face Attribute Classifiers through Summary Model ExplanationsProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658007(758-766)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658007
Sullivan E(2024)SIDEs: Separating Idealization from Deceptive 'Explanations' in xAIProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658999(1714-1724)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658999
Yang XLiu FLin G(2024)Neural Logic Vision Language ExplainerIEEE Transactions on Multimedia10.1109/TMM.2023.331027726(3331-3340)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3310277
Chaudhury SSadhukhan PSengupta K(2024)Explainable AI Using the Wasserstein DistanceIEEE Access10.1109/ACCESS.2024.336048412(18087-18102)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3360484
Yin SLi NXiao KSong XYin JWang C(2024)Mineral prospectivity mapping using knowledge embedding and explainable ensemble learning: A case study of the Keeryin ore concentration in Sichuan, ChinaOre Geology Reviews10.1016/j.oregeorev.2024.106017(106017)Online publication date: Apr-2024
https://doi.org/10.1016/j.oregeorev.2024.106017
Yan BDing WJin ZZhang LWang LDu MYang QHe Y(2024)Explainable machine learning-based prediction for aerodynamic interference of a low-rise building on a high-rise buildingJournal of Building Engineering10.1016/j.jobe.2023.10828582(108285)Online publication date: Apr-2024
https://doi.org/10.1016/j.jobe.2023.108285
Huang FMao DJiang SZhou CFan XZeng ZCatani FYu CChang ZHuang JJiang BLi Y(2024)Uncertainties in landslide susceptibility prediction modeling: A review on the incompleteness of landslide inventory and its influence rulesGeoscience Frontiers10.1016/j.gsf.2024.10188615:6(101886)Online publication date: Nov-2024
https://doi.org/10.1016/j.gsf.2024.101886
Liu XYang JZou DLi ZChen YCao X(2024)Utilizing DEM and interpretable ML algorithms to examine particle size distribution's role in small-strain shear modulus of gap-graded granular mixturesConstruction and Building Materials10.1016/j.conbuildmat.2024.136232428(136232)Online publication date: May-2024
https://doi.org/10.1016/j.conbuildmat.2024.136232
Chen HDu FYang XPan XQi X(2024)A novel causality analysis of brain tumor subregion-based automated segmentation networks on multi-sequence MRIsBiomedical Signal Processing and Control10.1016/j.bspc.2024.10646695(106466)Online publication date: Sep-2024
https://doi.org/10.1016/j.bspc.2024.106466
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents