Abstract
In the field of explainable Artificial Intelligence (XAI), sequential counterfactual (SCF) examples are often used to alter the decision of a trained classifier by implementing a sequence of modifications to the input instance. Although certain test-time algorithms aim to optimize for each new instance individually, recently Reinforcement Learning (RL) methods have been proposed that seek to learn policies for discovering SCFs, thereby enhancing scalability. As is typical in RL, the formulation of the RL problem, including the specification of state space, actions, and rewards, can often be ambiguous. In this work, we identify shortcomings in existing methods that can result in policies with undesired properties, such as a bias towards specific actions. We propose to use the output probabilities of the classifier to create a more informative reward, to mitigate this effect.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brughmans, D., Leyman, P., Martens, D.: Nice: an algorithm for nearest instance counterfactual explanations. Data Mining Knowl. Discov. 1–39 (2023)
Cai, Y., Zimek, A., Ntoutsi, E.: Xproax-local explanations for text classification with progressive neighborhood approximation. In: 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA), pp. 1–10. IEEE (2021)
Chen, Z., Silvestri, F., Wang, J., Zhu, H., Ahn, H., Tolomei, G.: Relax: reinforcement learning agent explainer for arbitrary predictive models. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp. 252–261 (2022)
Dandl, S., Molnar, C., Binder, M., Bischl, B.: Multi-objective counterfactual explanations. In: Bäck, T., et al. (eds.) PPSN 2020. LNCS, vol. 12269, pp. 448–469. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58112-1_31
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. (CSUR) 51(5), 1–42 (2018)
Keane, M.T., Smyth, B.: Good counterfactuals and where to find them: a case-based technique for generating counterfactuals for explainable AI (XAI). In: Watson, I., Weber, R. (eds.) ICCBR 2020. LNCS (LNAI), vol. 12311, pp. 163–178. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58342-2_11
Kelly, M., Longjohn, R., Nottingham, K.: The UCI machine learning repository (2023). https://archive.ics.uci.edu
Kommiya Mothilal, R., Mahajan, D., Tan, C., Sharma, A.: Towards unifying feature attribution and counterfactual explanations: different means to the same end. In: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 652–663 (2021)
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)
Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Mahajan, D., Tan, C., Sharma, A.: Preserving causal constraints in counterfactual explanations for machine learning classifiers. arXiv preprint arXiv:1912.03277 (2019)
Naumann, P., Ntoutsi, E.: Consequence-aware sequential counterfactual generation. In: Oliver, N., Pérez-Cruz, F., Kramer, S., Read, J., Lozano, J.A. (eds.) ECML PKDD 2021. LNCS (LNAI), vol. 12976, pp. 682–698. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86520-7_42
Pawelczyk, M., Broelemann, K., Kasneci, G.: Learning model-agnostic counterfactual explanations for tabular data. In: Proceedings of the Web Conference 2020, pp. 3126–3132 (2020)
Ramakrishnan, G., Lee, Y.C., Albarghouthi, A.: Synthesizing action sequences for modifying model decisions. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 5462–5469 (2020)
Tsirtsis, S., De, A., Rodriguez, M.: Counterfactual explanations in sequential decision making under uncertainty. Adv. Neural. Inf. Process. Syst. 34, 30127–30139 (2021)
Verma, S., Hines, K.E., Dickerson, J.P.: Amortized generation of sequential algorithmic recourses for black-box models. In: AAAI Conference on Artificial Intelligence (2021)
Xiong, J., et al.: Parametrized deep q-networks learning: reinforcement learning with discrete-continuous hybrid action space. arXiv preprint arXiv:1810.06394 (2018)
Acknowledgements
Our work is Funded by the Deutsche Forschungsgemeinsschaft (DFG, German Research Foundation) - SFB1463 - 434502799. I further acknowledge the support by the European Union, Horizon Europe project MAMMOth under contract number 101070285.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Panagiotou, E., Ntoutsi, E. (2025). Learning Impartial Policies for Sequential Counterfactual Explanations Using Deep Reinforcement Learning. In: Meo, R., Silvestri, F. (eds) Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2023. Communications in Computer and Information Science, vol 2133. Springer, Cham. https://doi.org/10.1007/978-3-031-74630-7_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-74630-7_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-74629-1
Online ISBN: 978-3-031-74630-7
eBook Packages: Artificial Intelligence (R0)