-
(Predictable) Performance Bias in Unsupervised Anomaly Detection
Authors:
Felix Meissen,
Svenja Breuer,
Moritz Knolle,
Alena Buyx,
Ruth Müller,
Georgios Kaissis,
Benedikt Wiestler,
Daniel Rückert
Abstract:
Background: With the ever-increasing amount of medical imaging data, the demand for algorithms to assist clinicians has amplified. Unsupervised anomaly detection (UAD) models promise to aid in the crucial first step of disease detection. While previous studies have thoroughly explored fairness in supervised models in healthcare, for UAD, this has so far been unexplored.
Methods: In this study, w…
▽ More
Background: With the ever-increasing amount of medical imaging data, the demand for algorithms to assist clinicians has amplified. Unsupervised anomaly detection (UAD) models promise to aid in the crucial first step of disease detection. While previous studies have thoroughly explored fairness in supervised models in healthcare, for UAD, this has so far been unexplored.
Methods: In this study, we evaluated how dataset composition regarding subgroups manifests in disparate performance of UAD models along multiple protected variables on three large-scale publicly available chest X-ray datasets. Our experiments were validated using two state-of-the-art UAD models for medical images. Finally, we introduced a novel subgroup-AUROC (sAUROC) metric, which aids in quantifying fairness in machine learning.
Findings: Our experiments revealed empirical "fairness laws" (similar to "scaling laws" for Transformers) for training-dataset composition: Linear relationships between anomaly detection performance within a subpopulation and its representation in the training data. Our study further revealed performance disparities, even in the case of balanced training data, and compound effects that exacerbate the drop in performance for subjects associated with multiple adversely affected groups.
Interpretation: Our study quantified the disparate performance of UAD models against certain demographic subgroups. Importantly, we showed that this unfairness cannot be mitigated by balanced representation alone. Instead, the representation of some subgroups seems harder to learn by UAD models than that of others. The empirical fairness laws discovered in our study make disparate performance in UAD models easier to estimate and aid in determining the most desirable dataset composition.
△ Less
Submitted 25 September, 2023;
originally announced September 2023.
-
FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcare
Authors:
Karim Lekadir,
Aasa Feragen,
Abdul Joseph Fofanah,
Alejandro F Frangi,
Alena Buyx,
Anais Emelie,
Andrea Lara,
Antonio R Porras,
An-Wen Chan,
Arcadi Navarro,
Ben Glocker,
Benard O Botwe,
Bishesh Khanal,
Brigit Beger,
Carol C Wu,
Celia Cintas,
Curtis P Langlotz,
Daniel Rueckert,
Deogratias Mzurikwao,
Dimitrios I Fotiadis,
Doszhan Zhussupov,
Enzo Ferrante,
Erik Meijering,
Eva Weicken,
Fabio A González
, et al. (95 additional authors not shown)
Abstract:
Despite major advances in artificial intelligence (AI) for medicine and healthcare, the deployment and adoption of AI technologies remain limited in real-world clinical practice. In recent years, concerns have been raised about the technical, clinical, ethical and legal risks associated with medical AI. To increase real world adoption, it is essential that medical AI tools are trusted and accepted…
▽ More
Despite major advances in artificial intelligence (AI) for medicine and healthcare, the deployment and adoption of AI technologies remain limited in real-world clinical practice. In recent years, concerns have been raised about the technical, clinical, ethical and legal risks associated with medical AI. To increase real world adoption, it is essential that medical AI tools are trusted and accepted by patients, clinicians, health organisations and authorities. This work describes the FUTURE-AI guideline as the first international consensus framework for guiding the development and deployment of trustworthy AI tools in healthcare. The FUTURE-AI consortium was founded in 2021 and currently comprises 118 inter-disciplinary experts from 51 countries representing all continents, including AI scientists, clinicians, ethicists, and social scientists. Over a two-year period, the consortium defined guiding principles and best practices for trustworthy AI through an iterative process comprising an in-depth literature review, a modified Delphi survey, and online consensus meetings. The FUTURE-AI framework was established based on 6 guiding principles for trustworthy AI in healthcare, i.e. Fairness, Universality, Traceability, Usability, Robustness and Explainability. Through consensus, a set of 28 best practices were defined, addressing technical, clinical, legal and socio-ethical dimensions. The recommendations cover the entire lifecycle of medical AI, from design, development and validation to regulation, deployment, and monitoring. FUTURE-AI is a risk-informed, assumption-free guideline which provides a structured approach for constructing medical AI tools that will be trusted, deployed and adopted in real-world practice. Researchers are encouraged to take the recommendations into account in proof-of-concept stages to facilitate future translation towards clinical practice of medical AI.
△ Less
Submitted 8 July, 2024; v1 submitted 11 August, 2023;
originally announced September 2023.
-
The Localized Subtraction Approach For EEG and MEG Forward Modeling
Authors:
Malte B. Höltershinken,
Pia Lange,
Tim Erdbrügger,
Yvonne Buschermöhle,
Fabrice Wallois,
Alena Buyx,
Sampsa Pursiainen,
Johannes Vorwerk,
Christian Engwer,
Carsten H. Wolters
Abstract:
In FEM-based EEG and MEG source analysis, the subtraction approach has been proposed to simulate sensor measurements generated by neural activity. While this approach possesses a rigorous foundation and produces accurate results, its major downside is that it is computationally prohibitively expensive in practical applications. To overcome this, we developed a new approach, called the localized su…
▽ More
In FEM-based EEG and MEG source analysis, the subtraction approach has been proposed to simulate sensor measurements generated by neural activity. While this approach possesses a rigorous foundation and produces accurate results, its major downside is that it is computationally prohibitively expensive in practical applications. To overcome this, we developed a new approach, called the localized subtraction approach. This approach is designed to preserve the mathematical foundation of the subtraction approach, while also leading to sparse right-hand sides in the FEM formulation, making it efficiently computable. We achieve this by introducing a cut-off into the subtraction, restricting its influence to the immediate neighborhood of the source. In this work, this approach will be presented, analyzed, and compared to other state-of-the-art FEM right-hand side approaches. Furthermore, we discuss how to arrive at an efficient and stable implementation. We perform validation in multi-layer sphere models where analytical solutions exist. There, we demonstrate that the localized subtraction approach is vastly more efficient than the subtraction approach. Moreover, we find that for the EEG forward problem, the localized subtraction approach is less dependent on the global structure of the FEM mesh when compared to the subtraction approach. Additionally, we show the localized subtraction approach to rival, and in many cases even surpass, the other investigated approaches in terms of accuracy. For the MEG forward problem, we show the localized subtraction approach and the subtraction approach to produce highly accurate approximations of the volume currents close to the source. The localized subtraction approach thus reduces the computational cost of the subtraction approach to an extent that makes it usable in practical applications without sacrificing rigorousness and accuracy.
△ Less
Submitted 24 February, 2023;
originally announced February 2023.
-
CutFEM forward modeling for EEG source analysis
Authors:
Tim Erdbrügger,
Andreas Westhoff,
Malte Hoeltershinken,
Jan-Ole Radecke,
Yvonne Buschermoehle,
Alena Buyx,
Fabrice Wallois,
Sampsa Pursiainen,
Joachim Gross,
Rebekka Lencer,
Christian Engwer,
Carsten Wolters
Abstract:
Source analysis of Electroencephalography (EEG) data requires the computation of the scalp potential induced by current sources in the brain. This so-called EEG forward problem is based on an accurate estimation of the volume conduction effects in the human head, represented by a partial differential equation which can be solved using the finite element method (FEM). FEM offers flexibility when mo…
▽ More
Source analysis of Electroencephalography (EEG) data requires the computation of the scalp potential induced by current sources in the brain. This so-called EEG forward problem is based on an accurate estimation of the volume conduction effects in the human head, represented by a partial differential equation which can be solved using the finite element method (FEM). FEM offers flexibility when modeling anisotropic tissue conductivities but requires a volumetric discretization, a mesh, of the head domain. Structured hexahedral meshes are easy to create in an automatic fashion, while tetrahedral meshes are better suited to model curved geometries. Tetrahedral meshes thus offer better accuracy, but are more difficult to create. Methods: We introduce CutFEM for EEG forward simulations to integrate the strengths of hexahedra and tetrahedra. It belongs to the family of unfitted finite element methods, decoupling mesh and geometry representation. Following a description of the method, we will employ CutFEM in both controlled spherical scenarios and the reconstruction of somatosensory evoked potentials. Results: CutFEM outperforms competing FEM approaches with regard to numerical accuracy, memory consumption and computational speed while being able to mesh arbitrarily touching compartments. Conclusion: CutFEM balances numerical accuracy, computational efficiency and a smooth approximation of complex geometries that has previously not been available in FEM-based EEG forward modeling.
△ Less
Submitted 30 November, 2022;
originally announced November 2022.
