research-article

ALEX: Active Learning based Enhancement of a Classification Model's EXplainability

Authors:

Debasis GangulyAuthors Info & Claims

CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

Pages 3309 - 3312

https://doi.org/10.1145/3340531.3417456

Published: 19 October 2020 Publication History

Abstract

An active learning (AL) algorithm seeks to construct an effective classifier with a minimal number of labeled examples in a bootstrapping manner. While standard AL heuristics, such as selecting those points for annotation for which a classification model yields least confident predictions, there has been no empirical investigation to see if these heuristics lead to models that are more interpretable to humans. In the era of data-driven learning, this is an important research direction to pursue. This paper describes our work-in-progress towards developing an AL selection function that in addition to model effectiveness also seeks to improve on the interpretability of a model during the bootstrapping steps. Concretely speaking, our proposed selection function trains an 'explainer' model in addition to the classifier model, and favours those instances where a different part of the data is used, on an average, to explain the predicted class. Initial experiments exhibited encouraging trends in showing that such a heuristic can lead to developing more effective and more explainable end-to-end data-driven classifiers.

Supplementary Material

MP4 File (3340531.3417456.mp4)

Presentation of "ALEX: Active Learning based Enhancement of a Classification Model's EXplainability"

Download
39.01 MB

References

[1]

Abien Fred Agarap. 2018. Deep Learning using Rectified Linear Units (ReLU). ArXiv abs/1803.08375 (2018).

[2]

Jianbo C., Le S., Martin W., and Michael J. 2018. Learning to Explain: An Information-Theoretic Perspective on Model Interpretation. In Proc. of ICML'18, Vol. 80. 882--891.

[3]

A. Fujii, K. Inui, T. Tokunaga, and H. Tanaka. 1998. Selective Sampling for Example-based Word Sense Disambiguation. CL 24, 4 (1998), 573--597.

[4]

Bhavya Ghai, Q. Vera Liao, Yunfeng Zhang, Rachel Bellamy, and Klaus Mueller. 2020. Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience. arXiv:2001.09219

[5]

Yann LeCun and Corinna Cortes. 2010. MNIST handwritten digit database. (2010).

[6]

David D. Lewis and Jason Catlett. 1994. Heterogenous Uncertainty Sampling for Supervised Learning. In Proc. of ICML'94. 148--156.

[7]

S. Lipovetsky and M. Conklin. 2001. Analysis of Regression in Game Theory Approach. Applied Stochastic Models 17 (10 2001), 319 -- 330.

[8]

Scott Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions.

[9]

Hieu T. Nguyen and Arnold Smeulders. 2004. Active Learning Using Pre- Clustering. In Proc. of ICML'04. 79.

[10]

Richard Phillips, Kyu Chang, and Sorelle Friedler. 2017. Interpretable Active Learning. (07 2017).

[11]

T. Ribeiro, S. Singh, and C. Guestrin. 2016. 'Why Should I Trust You'?: Explaining the Predictions of Any Classifier. In Proc. of KDD'16. 1135--1144.

[12]

T. Scheffer, C. Decomain, and S. Wrobel. 2001. Active Hidden Markov Models for Information Extraction. In Advances in Intelligent Data Analysis. 309--318.

[13]

Burr Settles. 2009. Active Learning Literature Survey. Computer Sciences Technical Report 1648. University of Wisconsin--Madison.

[14]

Burr Settles and Mark Craven. 2008. An Analysis of Active Learning Strategies for Sequence Labeling Tasks. In Proc. of EMNLP'08. 1070--1079.

[15]

Han X., Kashif R., and Roland V. 2017. Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. CoRR abs/1708.07747 (2017).

[16]

Zhilu Zhang and Mert Sabuncu. 2018. Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels.

Cited By

Flores CValenzuela AVerschae R(2024)Active Learning for Image Classification: A Comprehensive Analysis in AgricultureProceedings of Ninth International Congress on Information and Communication Technology10.1007/978-981-97-5441-0_49(607-616)Online publication date: 18-Dec-2024
https://doi.org/10.1007/978-981-97-5441-0_49
Chandra MGanguly DSaha TOunis IFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)'Choose your Data Wisely': Active Learning based Selection with Multi-Objective Optimisation for Mitigating StereotypesProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615261(3768-3772)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615261
Mannmeusel JRothfelder MKhoshrou S(2023)Speeding Things Up. Can Explainability Improve Human Learning?Explainable Artificial Intelligence10.1007/978-3-031-44064-9_4(66-84)Online publication date: 30-Oct-2023
https://doi.org/10.1007/978-3-031-44064-9_4
Show More Cited By

Index Terms

ALEX: Active Learning based Enhancement of a Classification Model's EXplainability
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Active learning

Recommendations

An active learning-based SVM multi-class classification model

Traditional multi-class classification models are based on labeled data and are not applicable to unlabeled data. To overcome this limitation, this paper presents a multi-class classification model that is based on active learning and support vector ...
Multi-Label Active Learning Algorithms for Image Classification: Overview and Future Promise

Image classification is a key task in image understanding, and multi-label image classification has become a popular topic in recent years. However, the success of multi-label image classification is closely related to the way of constructing a training ...
Active learning for text classification with reusability

We investigate the reusability problem in active learning for text classification.The reusability problem affects active learning systems for text classification.If the consumer classifier type is known, it should be used for the selector.Local and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

October 2020

3619 pages

ISBN:9781450368599

DOI:10.1145/3340531

General Chairs:
Mathieu d'Aquin
DSI, Insight, NUI Galway, Ireland
,
Stefan Dietze
GESIS, Cologne, Germany, Heinrich-Heine-University Düsseldorf, Germany, L3S Research Center, Germany
,
Program Chairs:
Claudia Hauff
TU Delft, The Netherlands
,
Edward Curry
DSI, Insight, NUI Galway, Ireland
,
Philippe Cudre Mauroux
eXascale, University of Fribourg, Switzerland

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '20

Sponsor:

CIKM '20: The 29th ACM International Conference on Information and Knowledge Management

October 19 - 23, 2020

Virtual Event, Ireland

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
205
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Flores CValenzuela AVerschae R(2024)Active Learning for Image Classification: A Comprehensive Analysis in AgricultureProceedings of Ninth International Congress on Information and Communication Technology10.1007/978-981-97-5441-0_49(607-616)Online publication date: 18-Dec-2024
https://doi.org/10.1007/978-981-97-5441-0_49
Chandra MGanguly DSaha TOunis IFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)'Choose your Data Wisely': Active Learning based Selection with Multi-Objective Optimisation for Mitigating StereotypesProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615261(3768-3772)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615261
Mannmeusel JRothfelder MKhoshrou S(2023)Speeding Things Up. Can Explainability Improve Human Learning?Explainable Artificial Intelligence10.1007/978-3-031-44064-9_4(66-84)Online publication date: 30-Oct-2023
https://doi.org/10.1007/978-3-031-44064-9_4
Jalali AHaslhofer BKriglstein SRauber A(2023)Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered AnalysisIntelligent Computing10.1007/978-3-031-37717-4_46(712-733)Online publication date: 1-Sep-2023
https://doi.org/10.1007/978-3-031-37717-4_46
Huang YTang YZhu XZhuang HCherubin L(2022)Physics-Coupled Spatio-Temporal Active Learning for Dynamical SystemsIEEE Access10.1109/ACCESS.2022.321454410(112909-112920)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3214544

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten