Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3593013.3594122acmotherconferencesArticle/Chapter ViewAbstractPublication PagesfacctConference Proceedingsconference-collections
research-article

Achieving Diversity in Counterfactual Explanations: a Review and Discussion

Published: 12 June 2023 Publication History

Abstract

In the field of Explainable Artificial Intelligence (XAI), counterfactual examples explain to a user the predictions of a trained decision model by indicating the modifications to be made to the instance so as to change its associated prediction. These counterfactual examples are generally defined as solutions to an optimization problem whose cost function combines several criteria that quantify desiderata for a good explanation meeting user needs. A large variety of such appropriate properties can be considered, as the user needs are generally unknown and differ from one user to another; their selection and formalization is difficult. To circumvent this issue, several approaches propose to generate, rather than a single one, a set of diverse counterfactual examples to explain a prediction. This paper proposes a review of the numerous, sometimes conflicting, definitions that have been proposed for this notion of diversity. It discusses their underlying principles as well as the hypotheses on the user needs they rely on and proposes to categorize them along several dimensions (explicit vs implicit, universe in which they are defined, level at which they apply), leading to the identification of further research challenges on this topic.

References

[1]
André Artelt and Barbara Hammer. 2019. On the computation of counterfactual explanations–A survey. arXiv:1911.07749 (2019).
[2]
André Artelt and Barbara Hammer. 2020. Convex Density Constraints for Computing Plausible Counterfactual Explanations. In Artificial Neural Networks and Machine Learning. 353–365.
[3]
Maximilian Becker, Nadia Burkart, Pascal Birnstill, and Jürgen Beyerer. 2021. A Step Towards Global Counterfactual Explanations: Approximating the Feature Space Through Hierarchical Division and Graph Search.Adv. Artif. Intell. Mach. Learn. 1, 2 (2021), 90–110.
[4]
Umang Bhatt, Isabel Chien, Muhammad Bilal Zafar, and Adrian Weller. 2021. DIVINE: Diverse Influential Training Points for Data Visualization and Model Refinement. arXiv preprint arXiv:2107.05978 (2021).
[5]
Clara Bove, Jonathan Aigrain, Marie-Jeanne Lesot, Charles Tijus, and Marcin Detyniecki. 2022. Contextualization and exploration of local feature importance explanations to improve understanding and satisfaction of non-expert users. In 27th International Conference on Intelligent User Interfaces, IUI. 807–819.
[6]
Clara Bove, Marie-Jeanne Lesot, Charles Tijus, and Marcin Detyniecki. 2023. Investigating the Intelligibility of Plural Counterfactual Examples for Non-Expert Users: an Explanation User Interface Proposition and User Study. In 28th International Conference on Intelligent User Interfaces, IUI.
[7]
Garvin Brod, Markus Werkle-Bergner, and Yee Lee Shing. 2013. The influence of prior knowledge on memory: a developmental cognitive neuroscience perspective. Frontiers in behavioral neuroscience 7 (2013), 139.
[8]
Ngoc Bui, Duy Nguyen, and Viet Nguyen. 2022. Counterfactual Plans under Distributional Ambiguity. International Conference on Learning Representations (2022).
[9]
Nadia Burkart and Marco F. Huber. 2021. A Survey on the Explainability of Supervised Machine Learning. Journal of Artificial Intelligence Research 70 (2021), 245–317.
[10]
Thomas Calvo, Gaspar Mayor, and Radko Mesiar (Eds.). 2002. Aggregation Operators: New Trends and Applications. Vol. 97. Springer.
[11]
Miguel Á Carreira-Perpiñán and Suryabhan Singh Hada. 2021. Counterfactual explanations for oblique decision trees: Exact, efficient algorithms. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 6903–6911.
[12]
Furui Cheng, Yao Ming, and Huamin Qu. 2020. Dece: Decision explorer with counterfactual explanations for machine learning models. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 1438–1447.
[13]
Dac Collaris, Leo M. Vink, and Jarke J. van Wijk. 2018. Instance-level explanations for fraud detection: a case study. In 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018).
[14]
Susanne Dandl, Christoph Molnar, Martin Binder, and Bernd Bischl. 2020. Multi-Objective Counterfactual Explanations. In Parallel Problem Solving from Nature – PPSN XVI. Springer International Publishing.
[15]
Marcin Detyniecki. 2001. Fundamentals on aggregation operators. Thesis manuscript, Université Pierre et Marie Curie (2001).
[16]
Finale Doshi-Velez and Been Kim. 2017. Towards A Rigorous Science of Interpretable Machine Learning. arXiv:1702.08608 (2017).
[17]
Michael Downs, Jonathan L. Chu, Yaniv Yacoby, Finale Doshi-Velez, and Pan WeiWei. 2020. CRUDS: Counterfactual Recourse Using Disentangled Subspaces. ICML Workshop on Human Interpretability in Machine Learning (2020), 1–23.
[18]
David Gentile, Greg A. Jamieson, and Birsen Domez. 2021. Evaluating human understanding in XAI systems. In ACM CHI XCXAI Workshop.
[19]
Michel Grabisch, Jean-Luc Marichal, Radko Mesiar, and Endre Pap. 2009. Aggregation Functions. Number 127 in Encyclopedia of Mathematics and its Applications. Cambridge Univ. Press.
[20]
Nina Grgic-Hlaca, Elissa M. Redmiles, Krishna P. Gummadi, and Adrian Weller. 2018. Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction. In Proceedings of the 2018 World Wide Web Conference(WWW ’18). 903–912. https://doi.org/10.1145/3178876.3186138
[21]
Riccardo Guidotti. 2022. Counterfactual explanations and how to find them: literature review and benchmarking. Data Mining and Knowledge Discovery (04 2022), 1–55.
[22]
Riccardo Guidotti, Anna Monreale, Fosca Giannotti, Dino Pedreschi, Salvatore Ruggieri, and Franco Turini. 2019. Factual and counterfactual explanations for black box decision making. IEEE Intelligent Systems 34, 6 (2019), 14–23.
[23]
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A Survey of Methods for Explaining Black Box Models. ACM Comput. Surv. 51, 5 (2018), 42 pages.
[24]
Suryabhan Singh Hada and Miguel Á Carreira-Perpiñán. 2021. Exploring counterfactual explanations for classification and regression trees. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 489–504.
[25]
Denis J. Hilton. 1996. Mental Models and Causal Explanation: Judgements of Probable Cause and Explanatory Relevance. Thinking & Reasoning 2, 4 (1996), 273–308. https://doi.org/10.1080/135467896394447
[26]
Adulam Jeyasothy, Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, and Marcin Detyniecki. 2022. Integrating Prior Knowledge in Post-hoc Explanations. In Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU’2022).
[27]
Kentaro Kanamori, Takuya Takagi, Ken Kobayashi, Yuichi Ike, Kento Uemura, and Hiroki Arimura. 2021. Ordered counterfactual explanation by mixed-integer linear optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 11564–11574.
[28]
Amir-Hossein Karimi, Gilles Barthe, Borja Balle, and Isabel Valera. 2020. Model-agnostic counterfactual explanations for consequential decisions. In International Conference on Artificial Intelligence and Statistics. PMLR, 895–905.
[29]
Amir-Hossein Karimi, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. 2023. A survey of algorithmic recourse: contrastive explanations and consequential recommendations. ACM Computing Surveys (CSUR) (2023).
[30]
Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. 2021. Algorithmic recourse: from counterfactual explanations to interventions. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 353–362.
[31]
Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. 2020. Interpreting Interpretability: Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376219
[32]
Michael T. Lash, Qihang Lin, Nick Street, Jennifer G. Robinson, and Jeffrey Ohlmann. 2017. Generalized Inverse Classification. In Proc. of the SIAM Int. Conf. on Data Mining. 162–170.
[33]
Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, and Marcin Detyniecki. 2019. Issues with post-hoc counterfactual explanations: a discussion. arXiv preprint arXiv:1906.04774 (2019).
[34]
Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2018. Comparison-based Inverse Classification for Interpretability in Machine Learning. In Proc. of Int. Conf. on Information Processing and Management of Uncertainty in Knowledge-Based Systems. Springer, 100–111.
[35]
Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2019. The dangers of post-hoc interpretability: unjustified counterfactual explanations. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2801–2807.
[36]
Dan Ley, Umang Bhatt, and Adrian Weller. 2022. Diverse, global and amortised counterfactual explanations for uncertainty estimates. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 7390–7398.
[37]
Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. 2021. Explainable AI: A Review of Machine Learning Interpretability Methods. Entropy 23, 1 (2021).
[38]
Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proc. of the 31st Int. Conf. on Neural Information Processing Systems. 4768–4777.
[39]
Divyat Mahajan, Chenhao Tan, and Amit Sharma. 2019. Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers. NeurIPS workshop (2019).
[40]
Raphael Mazzine and David Martens. 2021. A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data. Applied Science (2021).
[41]
Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence 267 (2019), 1–38.
[42]
Kiarash Mohammadi, Amir-Hossein Karimi, Gilles Barthe, and Isabel Valera. 2021. Scaling guarantees for nearest counterfactual explanations. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 177–187.
[43]
Christoph Molnar. 2022. Interpretable Machine Learning (2 ed.). https://christophm.github.io/interpretable-ml-book
[44]
Jonathan Moore, Nils Hammerla, and Chris Watkins. 2019. Explaining deep learning models with constrained adversarial examples. In Pacific Rim international conference on artificial intelligence. Springer, 43–56.
[45]
Ramaravind K. Mothilal, Amit Sharma, and Chenhao Tan. 2020. Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations. In Proc. of the 2020 Conf. on Fairness, Accountability, and Transparency. ACM.
[46]
Philip Naumann and Eirini Ntoutsi. 2021. Consequence-aware sequential counterfactual generation. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 682–698.
[47]
Guillermo Navas-Palencia. 2021. Optimal counterfactual explanations for scorecard modelling. arXiv preprint arXiv:2104.08619 (2021).
[48]
Duy Nguyen, Ngoc Bui, and Viet Anh Nguyen. 2023. Feasible Recourse Plan via Diverse Interpolation. In International Conference on Artificial Intelligence and Statistics. PMLR, 4679–4698.
[49]
Axel Parmentier and Thibaut Vidal. 2021. Optimal counterfactual explanations in tree ensembles. In International Conference on Machine Learning. PMLR, 8422–8431.
[50]
Martin Pawelczyk, Sascha Bielawski, Johannes van den Heuvel, Tobias Richter, and Gjergji Kasneci. 2021. CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms. NeuRIPS 2021 Track on Datasets and Benchmarks (2021).
[51]
Martin Pawelczyk, Klaus Broelemann, and Gjergji Kasneci. 2020. Learning model-agnostic counterfactual explanations for tabular data. In Proceedings of The Web Conference 2020. 3126–3132.
[52]
Martin Pawelczyk, Tobias Leemann, Asia Biega, and Gjergji Kasneci. 2022. On the Trade-Off between Actionable Explanations and the Right to be Forgotten. arXiv preprint arXiv:2208.14137 (2022).
[53]
Judea Pearl. 2009. Causality. Cambridge University Press (2009).
[54]
Rafael Poyiadzi, Kacper Sokol, Raul Santos-Rodriguez, Tijl De Bie, and Peter Flach. 2020. FACE: Feasible and Actionable Counterfactual Explanations. In Proc. of the AAAI/ACM Conf. on AI, Ethics, and Society.
[55]
Goutham Ramakrishnan, Yun Chan Lee, and Aws Albarghouthi. 2020. Synthesizing action sequences for modifying model decisions. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 5462–5469.
[56]
Peyman Rasouli and Ingrid Chieh Yu. 2022. CARE: Coherent actionable recourse based on sound counterfactual explanations. International Journal of Data Science and Analytics (2022), 1–26.
[57]
Kaivalya Rawal and Himabindu Lakkaraju. 2020. Beyond individualized recourse: Interpretable and interactive summaries of actionable recourses. Advances in Neural Information Processing Systems 33 (2020), 12187–12198.
[58]
Annabelle Redelmeier, Martin Jullum, Kjersti Aas, and Anders Løland. 2021. MCCE: Monte Carlo sampling of realistic counterfactual explanations. arXiv preprint arXiv:2111.09790 (2021).
[59]
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why should I trust you?" Explaining the predictions of any classifier. In Proc. of the 22nd ACM SIGKDD Int. Conf. on knowledge discovery and data mining. 1135–1144.
[60]
Pau Rodríguez, Massimo Caccia, Alexandre Lacoste, Lee Zamparo, Issam Laradji, Laurent Charlin, and David Vazquez. 2021. Beyond Trivial Counterfactual Explanations With Diverse Valuable Explanations. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 1056–1065.
[61]
Chris Russell. 2019. Efficient search for diverse coherent explanations. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 20–28.
[62]
Waddah Saeed and Christian Omlin. 2023. Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities. Knowledge-Based Systems 263 (2023), 110273.
[63]
Robert-Florian Samoilescu, Arnaud Van Looveren, and Janis Klaise. 2021. Model-agnostic and Scalable Counterfactual Explanations via Reinforcement Learning. arXiv preprint arXiv:2106.02597 (2021).
[64]
Maximilian Schleich, Zixuan Geng, Yihong Zhang, and Dan Suciu. 2021. GeCo: Quality Counterfactual Explanations in Real Time. Proc. VLDB Endow. 14, 9 (oct 2021), 1681–1693.
[65]
Shubham Sharma, Jette Henderson, and Joydeep Ghosh. 2020. CERTIFAI: A Common Framework to Provide Explanations and Analyse the Fairness and Robustness of Black-Box Models. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society(AIES ’20). Association for Computing Machinery, 166–172.
[66]
Dylan Slack, Anna Hilgard, Himabindu Lakkaraju, and Sameer Singh. 2021. Counterfactual explanations can be manipulated. Advances in neural information processing systems 34 (2021), 62–75.
[67]
Barry Smyth and Mark T Keane. 2022. A few good counterfactuals: generating interpretable, plausible and diverse counterfactual explanations. In International Conference on Case-Based Reasoning. Springer, 18–32.
[68]
Rand J Spiro, Paul J Feltovich, Richard L Coulson, and Daniel K Anderson. 1989. Multiple analogies for complex concepts: antidotes for analogy-induced misconception in advanced knowledge acquisition. In Similarity and analogical reasoning. 498–531.
[69]
Ramya Srinivasan and Ajay Chander. 2020. Explanation Perspectives from the Cognitive Sciences—A Survey. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20. 4812–4818.
[70]
Emily Sullivan and Philippe Verreault-Julien. 2022. From Explanation to Recommendation: Ethical Standards for Algorithmic Recourse. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 712–722.
[71]
Erico Tjoa and Cuntai Guan. 2020. A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI. IEEE Transactions on Neural Networks and Learning Systems (2020).
[72]
Stratis Tsirtsis and Manuel Gomez Rodriguez. 2020. Decisions, counterfactual explanations and strategic behavior. Advances in Neural Information Processing Systems 33 (2020), 16749–16760.
[73]
Berk Ustun, Alexander Spangher, and Yang Liu. 2019. Actionable Recourse in Linear Classification. In Proc. of the Conf. on Fairness, Accountability, and Transparency. Association for Computing Machinery, 10–19.
[74]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE.Journal of machine learning research 9, 11 (2008).
[75]
Arnaud Van Looveren and Janis Klaise. 2021. Interpretable Counterfactual Explanations Guided by Prototypes. In Proc. of European Conf. on Machine Learning.
[76]
Sahil Verma, John Dickerson, and Keegan Hines. 2020. Counterfactual explanations for machine learning: A review. arXiv preprint arXiv:2010.10596 (2020).
[77]
Sahil Verma, Keegan Hines, and John P Dickerson. 2021. Amortized Generation of Sequential Counterfactual Explanations for Black-box Models. arXiv preprint arXiv:2106.03962 (2021).
[78]
Tom Vermeire, Thibault Laugel, Xavier Renard, David Martens, and Marcin Detyniecki. 2022. How to choose an explainability method? towards a methodical implementation of XAI in practice. In Machine Learning and Principles and Practice of Knowledge Discovery in Databases: International Workshops of ECML PKDD 2021, Virtual Event, September 13-17, 2021, Proceedings, Part I. Springer, 521–533.
[79]
Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2018. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. Harvard journal of law & technology 31 (2018), 841–887.
[80]
Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y Lim. 2019. Designing theory-driven user-centric explainable AI. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–15.
[81]
Prateek Yadav, Peter Hase, and Mohit Bansal. 2021. Low-cost algorithmic recourse for users with uncertain cost functions. arXiv preprint arXiv:2111.01235 (2021).
[82]
Fan Yang, Sahan Suresh Alva, Jiahao Chen, and Xia Hu. 2021. Model-based counterfactual synthesizer for interpretation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1964–1974.
[83]
Wenzhuo Yang, Jia Li, Caiming Xiong, and Steven CH Hoi. 2022. MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation. arXiv preprint arXiv:2205.15540 (2022).

Cited By

View all
  • (2024)Automated Boundary Identification for Machine Learning ClassifiersProceedings of the 17th ACM/IEEE International Workshop on Search-Based and Fuzz Testing10.1145/3643659.3643927(1-8)Online publication date: 14-Apr-2024
  • (2024)Framework for Bias Detection in Machine Learning Models: A Fairness ApproachProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635731(1152-1154)Online publication date: 4-Mar-2024
  • (2024)Fast Diffusion-Based Counterfactuals for Shortcut Removal and GenerationComputer Vision – ECCV 202410.1007/978-3-031-73016-0_20(338-357)Online publication date: 26-Oct-2024
  • Show More Cited By

Index Terms

  1. Achieving Diversity in Counterfactual Explanations: a Review and Discussion

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency
      June 2023
      1929 pages
      ISBN:9798400701924
      DOI:10.1145/3593013
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 12 June 2023

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. XAI
      2. actionable recourse
      3. counterfactual explanations
      4. diversity.
      5. explainability
      6. interpretability
      7. review
      8. survey
      9. transparency

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      FAccT '23

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)251
      • Downloads (Last 6 weeks)23
      Reflects downloads up to 10 Nov 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Automated Boundary Identification for Machine Learning ClassifiersProceedings of the 17th ACM/IEEE International Workshop on Search-Based and Fuzz Testing10.1145/3643659.3643927(1-8)Online publication date: 14-Apr-2024
      • (2024)Framework for Bias Detection in Machine Learning Models: A Fairness ApproachProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635731(1152-1154)Online publication date: 4-Mar-2024
      • (2024)Fast Diffusion-Based Counterfactuals for Shortcut Removal and GenerationComputer Vision – ECCV 202410.1007/978-3-031-73016-0_20(338-357)Online publication date: 26-Oct-2024
      • (2024)Safe Data Resampling Method Based on Counterfactuals AnalysisArtificial Neural Networks and Machine Learning – ICANN 202410.1007/978-3-031-72332-2_13(178-193)Online publication date: 17-Sep-2024
      • (2024)Human-in-the-Loop Personalized Counterfactual RecourseExplainable Artificial Intelligence10.1007/978-3-031-63800-8_2(18-38)Online publication date: 10-Jul-2024

      View Options

      Get Access

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media