review-article

Free access

Techniques for interpretable machine learning

Authors:

Mengnan Du,

Ninghao Liu,

Xia HuAuthors Info & Claims

Communications of the ACM, Volume 63, Issue 1

Pages 68 - 77

https://doi.org/10.1145/3359786

Published: 20 December 2019 Publication History

All formats PDF

Abstract

Uncovering the mysterious ways machine learning models make decisions.

References

[1]

Altmann, A., Toloşi, L., Sander, O. and Lengauer T. Permutation importance: A corrected feature importance measure. Bioinformatics 26, 10 (2010), 1340--1347.

Crossref

Google Scholar

[2]

Ancona, M., Ceolini, E., Oztireli, C. and Gross, M. Towards better understanding of gradient-based attribution methods for deep neural networks. In Proceedings of the Intern. Conf. Learning Representations, 2018.

Google Scholar

[3]

Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller K.-R. and Samek, W. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS One 10, 7 (2015), e0130140.

Crossref

Google Scholar

[4]

Bahdanau, D., Cho, K. and Bengio, Y. Neural machine translation by jointly learning to align and translate. In Proceedings of the Intern. Conf. Learning Representations, 2015.

Google Scholar

[5]

Bastani, O., Kim, C., and Bastani, H. Interpretability via model extraction. In Proceedings of the Fairness, Accountability, and Transparency in Machine Learning Workshop, 2017.

Google Scholar

[6]

Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M. and Elhadad, N. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the ACM SIGKDD Intern. Conf. Knowledge Discovery and Data Mining. ACM, 2015.

Digital Library

Google Scholar

[7]

Chen, T. and Guestrin, C. Xgboost: A scalable tree boosting system. In Proceedings of the ACM SIGKDD Intern. Conf. Knowledge Discovery and Data Mining. ACM, 2016.

Digital Library

Google Scholar

[8]

Dabkowski, P. and Gal, Y. Real time image saliency for black box classifiers. Advances in Neural Information Processing Systems (2017), 6970--6979.

Google Scholar

[9]

Dix, A. Human issues in the use of pattern recognition techniques. Neural Networks and Pattern Recognition in Human Computer Interaction (1992), 429--451.

Google Scholar

[10]

Doshi-Velez, F. and Kim, B. Towards a rigorous science of interpretable machine learning. 2017.

Google Scholar

[11]

Du, M., Liu, N., Song, Q. and Hu, X. Towards explanation of DNN-based prediction with guided feature inversion. In Proceedings of the ACM SIGKDD Intern. Conf. Knowledge Discovery and Data Mining, 2018.

Digital Library

Google Scholar

[12]

Du, M., Liu, N., Yang, F. and Hu, X. On attribution of recurrent neural network predictions via additive decomposition. In Proceedings of the WWW Conf., 2019.

Digital Library

Google Scholar

[13]

Fong, R. and Vedaldi, A. Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the Intern. Conf. Computer Vision, 2017.

Crossref

Google Scholar

[14]

Freitas, A.A. Comprehensible classification models: A position paper. ACM SIGKDD Explorations Newsletter, 2014.

Digital Library

Google Scholar

[15]

Goodfellow, I., Bengio, Y and Courville, A. Deep Learning, Vol.1. MIT Press, Cambridge, MA, 2016.

Digital Library

Google Scholar

[16]

Goodfellow, I.J., Shlens, J. and Szegedy, C. Explaining and harnessing adversarial examples. In Proceedings of the Intern. Conf. Learning Representations, 2015.

Google Scholar

[17]

Kádár, A., Chrupa-la, G., and Alishahi, A. Representation of linguistic form and function in recurrent neural networks. Computational Linguistics 43, 4 (2017), 761--780.

Digital Library

Google Scholar

[18]

Karpathy, A., Johnson, J., and Fei-Fei, L. Visualizing and understanding recurrent networks. In Proceedings of the ICLR Workshop, 2016.

Google Scholar

[19]

Liu, N., Du, M., and Hu, X. Representation interpretation with spatial encoding and multimodal analytics. In Proceedings of the ACM Intern. Conf. Web Search and Data Mining, 2019.

Digital Library

Google Scholar

[20]

Liu, N., Yang, H., and Hu, X. Adversarial detection with model interpretation. In Proceedings of the ACM SIGKDD Intern. Conf. Knowledge Discovery and Data Mining, 2018.

Digital Library

Google Scholar

[21]

McCullagh, P. and Nelder, J.A. Generalized Linear M, Vol. 37. CRC Press, 1989.

Crossref

Google Scholar

[22]

Miller, T. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence (2018).

Google Scholar

[23]

Molnar, C. Interpretable Machine Learning (2018); https://christophm.github.io/interpretable-ml-book/.

Google Scholar

[24]

Mudrakarta, P.K., Taly, A., Sundararajan, M. and Dhamdhere, K. Did the model understand the question? In Proceedings of the 56^th Annual Meeting of the Assoc. Computational Linguistics, 2018.

Crossref

Google Scholar

[25]

Nguyen, A., Dosovitskiy, A., Yosinski, J., Brox, T. and Clune, J. Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. Advances in Neural Information Processing Systems, 2016.

Digital Library

Google Scholar

[26]

Nguyen, A., Yosinski, J. and Clune, J. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE Conf. Computer Vision and Pattern Recognition, 2015.

Crossref

Google Scholar

[27]

Nguyen, A., Yosinski, J. and Clune, J. Multifaceted feature visualization: Uncovering the different types of features learned by each neuron in deep neural networks. In Proceedings of the ICLR Workshop, 2016.

Google Scholar

[28]

Peters, M.E. et al. Deep contextualized word representations. In Proceedings of the NAACL-HLT, 2018.

Google Scholar

[29]

Quinlan, J.R. Simplifying decision trees. Intern. J. Man-Machine Studies 27, 3 (1987), 221--234.

Digital Library

Google Scholar

[30]

Ribeiro, M.T., Singh, S. and Guestrin, C. Why should I trust you? Explaining the predictions of any classifier. In Proceedings of the ACM SIGKDD Intern. Conf. Knowledge Discovery and Data Mining, 2016.

Crossref

Google Scholar

[31]

Ribeiro, M.T., Singh, S. and Guestrin, C. Anchors: High-precision model-agnostic explanations. In Proceedings of the AAAI Conf. Artificial Intelligence, 2018.

Google Scholar

[32]

Sabour, S., Frosst, N. and Hinton, G.E. Dynamic routing between capsules. Advances in Neural Information Processing Systems, 2017.

Google Scholar

[33]

Simonyan, K., Vedaldi, A. and Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Proceedings of the ICLR Workshop, 2014.

Google Scholar

[34]

Springenberg, J.T., Dosovitskiy, A., Brox, T. and Riedmiller, M. Striving for simplicity: The all convolutional net. In Proceedings of the ICLR workshop, 2015.

Google Scholar

[35]

Tomsett, R., Braines, D., Harborne, D., Preece, A. and Chakraborty, S. Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. In Proceedings of the ICML Workshop on Human Interpretability in Machine Learning, 2018.

Google Scholar

[36]

Vandewiele, G., Janssens, G., Ongenae, O., and Van Hoecke, F.S. Genesim: Genetic extraction of a single, interpretable model. In Proceedings of the NIPS Workshop, 2016.

Google Scholar

[37]

Wachter, S., Mittelstadt, B. and Russell, C. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. 2017.

Google Scholar

[38]

Xu, K. et al. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the Intern. Conf. Machine Learning, 2015.

Google Scholar

[39]

Zhang, Q., Wu, Y.N. and Zhu, S.-C. Interpretable convolutional neural networks. In Proceedings of the IEEE Conf. Computer Vision and Pattern Recognition, 2018.

Crossref

Google Scholar

[40]

Zhou, B., Khosla, A., Lapedriza, A., Oliva, A. and Torralba, A. Object detectors emerge in deep scene CNNs. In Proceedings of the Intern. Conf. Learning Representations, 2015.

Google Scholar

Cited By

View all

Liu ILiu FZhong QMa FNi S(2024)Your blush gives you away: detecting hidden mental states with remote photoplethysmography and thermal imagingPeerJ Computer Science10.7717/peerj-cs.191210(e1912)Online publication date: 20-Mar-2024
https://doi.org/10.7717/peerj-cs.1912
Pappachan PMoslehpour MBansal RRahaman M(2024)Transparency and AccountabilityChallenges in Large Language Model Development and AI Ethics10.4018/979-8-3693-3860-5.ch006(178-211)Online publication date: 30-Aug-2024
https://doi.org/10.4018/979-8-3693-3860-5.ch006
Goar VYadav N(2024)Foundations of Machine LearningIntelligent Optimization Techniques for Business Analytics10.4018/979-8-3693-1598-9.ch002(25-48)Online publication date: 26-Apr-2024
https://doi.org/10.4018/979-8-3693-1598-9.ch002
Show More Cited By

Index Terms

Techniques for interpretable machine learning
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

Lifelong Machine Learning
Interpretable Machine Learning in Healthcare
BCB '18: Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics

This tutorial extensively covers the definitions, nuances, challenges, and requirements for the design of interpretable and explainable machine learning models and systems in healthcare. We discuss many uses in which interpretable machine learning ...
Machine Learning: The State of the Art

The two fundamental problems in machine learning (ML) are statistical analysis and algorithm design. The former tells us the principles of the mathematical models that we establish from the observation data. The latter defines the conditions on which ...

Comments

Information & Contributors

Information

Published In

Communications of the ACM Volume 63, Issue 1

January 2020

90 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/3377354

Editor:
Andrew A. Chien
Association for Computing Machinery, New York, NY

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 December 2019

Published in CACM Volume 63, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Review-article
Popular
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

714
Total Citations
View Citations
36,240
Total Downloads

Downloads (Last 12 months)3,524
Downloads (Last 6 weeks)470

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Liu ILiu FZhong QMa FNi S(2024)Your blush gives you away: detecting hidden mental states with remote photoplethysmography and thermal imagingPeerJ Computer Science10.7717/peerj-cs.191210(e1912)Online publication date: 20-Mar-2024
https://doi.org/10.7717/peerj-cs.1912
Pappachan PMoslehpour MBansal RRahaman M(2024)Transparency and AccountabilityChallenges in Large Language Model Development and AI Ethics10.4018/979-8-3693-3860-5.ch006(178-211)Online publication date: 30-Aug-2024
https://doi.org/10.4018/979-8-3693-3860-5.ch006
Goar VYadav N(2024)Foundations of Machine LearningIntelligent Optimization Techniques for Business Analytics10.4018/979-8-3693-1598-9.ch002(25-48)Online publication date: 26-Apr-2024
https://doi.org/10.4018/979-8-3693-1598-9.ch002
Jafarzadehfadaki MSisiopiku V(2024)Embracing Urban Micromobility: A Comparative Study of E-Scooter Adoption in Washington, D.C., Miami, and Los AngelesUrban Science10.3390/urbansci80200718:2(71)Online publication date: 18-Jun-2024
https://doi.org/10.3390/urbansci8020071
Ma CXu XZhou MHu TQi C(2024)A Deep Learning Approach for Chromium Detection and Characterization from Soil Hyperspectral DataToxics10.3390/toxics1205035712:5(357)Online publication date: 11-May-2024
https://doi.org/10.3390/toxics12050357
Bahameish MStockman TRequena Carrión J(2024)Strategies for Reliable Stress Recognition: A Machine Learning Approach Using Heart Rate Variability FeaturesSensors10.3390/s2410321024:10(3210)Online publication date: 18-May-2024
https://doi.org/10.3390/s24103210
Xie JWang ZYu ZDing YGuo B(2024)Prototype Learning for Medical Time Series Classification via Human–Machine CollaborationSensors10.3390/s2408265524:8(2655)Online publication date: 22-Apr-2024
https://doi.org/10.3390/s24082655
Zhuang HLiu XYan YLi BWu CLiu W(2024)Multiple Land-Use Simulations and Driving Factor Analysis by Integrating a Deep Cascade Forest Model and Cellular Automata: A Case Study in the Pearl River Delta, ChinaRemote Sensing10.3390/rs1615275016:15(2750)Online publication date: 27-Jul-2024
https://doi.org/10.3390/rs16152750
Al-Hussaini IWhite BVarmeziar AMehra NSanchez MLee JDeGroote NMiller TMitchell C(2024)An Interpretable Machine Learning Framework for Rare Disease: A Case Study to Stratify Infection Risk in Pediatric LeukemiaJournal of Clinical Medicine10.3390/jcm1306178813:6(1788)Online publication date: 20-Mar-2024
https://doi.org/10.3390/jcm13061788
Mallinger KBaeza-Yates R(2024)Responsible AI in Farming: A Multi-Criteria Framework for Sustainable Technology DesignApplied Sciences10.3390/app1401043714:1(437)Online publication date: 3-Jan-2024
https://doi.org/10.3390/app14010437
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Magazine Site

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Lifelong Machine Learning

Interpretable Machine Learning in Healthcare

Machine Learning: The State of the Art

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Digital Edition

Magazine Site

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations