The use of algorithms to support prediction-based decision-making is becoming commonplace in a ra... more The use of algorithms to support prediction-based decision-making is becoming commonplace in a range of domains including health, criminal justice, education, social services, lending, and hiring. An assumption governing such decisions is that there is a property Y such that individual a should be allocated resource R by decision-maker D if a is Y. When there is uncertainty about whether a is Y, algorithms may provide valuable decision support by accurately predicting whether a is Y on the basis of known features of a. Based on recent work on statistical evidence in epistemology this article presents an argument against relying exclusively on algorithmic predictions to allocate resources when they provide purely statistical evidence that a is Y. The article then responds to the objection that any evidence that will increase the proportion of correct decisions should be accepted as the basis for allocations regardless of its epistemic deficiency. Finally, some important practical asp...
Machine learning algorithms are expected to improve referral decisions. In this article I discuss... more Machine learning algorithms are expected to improve referral decisions. In this article I discuss the legitimacy of deferring referral decisions in primary care to recommendations from such algorithms. The standard justification for introducing algorithmic decision procedures to make referral decisions is that they are more accurate than the available practitioners. The improvement in accuracy will ensure more efficient use of scarce health resources and improve patient care. In this article I introduce a proceduralist framework for discussing the legitimacy of algorithmic referral decisions and I argue that in the context of referral decisions the legitimacy of an algorithmic decision procedure can be fully accounted for in terms of the instrumental values of accuracy and fairness. I end by considering how my discussion of procedural algorithmic legitimacy relates to the debate on algorithmic fairness.
Several studies have documented that when presented with data from social media platforms machine... more Several studies have documented that when presented with data from social media platforms machine learning (ML) models can make accurate predictions about users, e.g., about whether they are likely to suffer health-related conditions such as depression, mental disorders, and risk of suicide. In a recent article, Ploug (Philos Technol 36:14, 2023) defends a right not to be subjected to AI profiling based on publicly available data. In this comment, I raise some questions in relation to Ploug’s argument that I think deserves further discussion.
2023 ACM Conference on Fairness, Accountability, and Transparency
Recent work on algorithmic fairness has largely focused on the fairness of discrete decisions, or... more Recent work on algorithmic fairness has largely focused on the fairness of discrete decisions, or classifications. While such decisions are often based on risk score models, the fairness of the risk models themselves has received considerably less attention. Risk models are of interest for a number of reasons, including the fact that they communicate uncertainty about the potential outcomes to users, thus representing a way to enable meaningful human oversight. Here, we address fairness desiderata for risk score models. We identify the provision of similar epistemic value to different groups as a key desideratum for risk score fairness, and we show how even fair risk scores can lead to unfair risk-based rankings. Further, we address how to assess the fairness of risk score models quantitatively, including a discussion of metric choices and meaningful statistical comparisons between groups. In this context, we also introduce a novel calibration error metric that is less sample size-biased than previously proposed metrics, enabling meaningful comparisons between groups of different sizes. We illustrate our methodology-which is widely applicable in many other settingsin two case studies, one in recidivism risk prediction, and one in risk of major depressive disorder (MDD) prediction. CCS CONCEPTS • Computing methodologies → Machine learning; • Applied computing → Health informatics; Law, social and behavioral sciences; • Social and professional topics → Computing / technology policy; User characteristics.
When is it justified to use opaque artificial intelligence (AI) output in medical decision-making... more When is it justified to use opaque artificial intelligence (AI) output in medical decision-making? Consideration of this question is of central importance for the responsible use of opaque machine learning (ML) models, which have been shown to produce accurate and reliable diagnoses, prognoses, and treatment suggestions in medicine. In this article, I discuss the merits of two answers to the question. According to the Explanation View, clinicians must have access to an explanation of why an output was produced. According to the Validation View, it is sufficient that the AI system has been validated using established standards for safety and reliability. I defend the Explanation View against two lines of criticism, and I argue that within the framework of evidence-based medicine mere validation seems insufficient for the use of AI output. I end by characterizing the epistemic responsibility of clinicians and point out how a mere AI output cannot in itself ground a practical conclusio...
Z-inspection® is a registered trademark. The content of this work is open access distributed unde... more Z-inspection® is a registered trademark. The content of this work is open access distributed under the terms and conditions of the Creative Commons (Attribution-NonCommercial-ShareAlike CC BY-NC-SA) license The views expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations.
This article's main contributions are twofold: 1) to demonstrate how to apply the general Europea... more This article's main contributions are twofold: 1) to demonstrate how to apply the general European Union's High-Level Expert Group's (EU HLEG) guidelines for trustworthy AI in practice for the domain of healthcare and 2) to investigate the research question of what does "trustworthy AI" mean at the time of the COVID-19 pandemic. To this end, we present the results of a post-hoc self-assessment to evaluate the trustworthiness of an AI system for predicting a multiregional score conveying the degree of lung compromise in COVID-19 patients, developed and verified by an interdisciplinary team with members from academia, public hospitals, and industry in time of
Advances in our scientific understanding and technological power in recent decades have dramatica... more Advances in our scientific understanding and technological power in recent decades have dramatically amplified our capacity to intentionally manipulate complex ecological and biological systems. An implication of this is that biological and ecological problems are increasingly understood and approached from an engineering perspective. In environmental contexts, this is exemplified in the pursuits of geoengineering, designer ecosystems, and conservation cloning. In human health contexts, it is exemplified in the development of synthetic biology, bionanotechnology, and human enhancement technologies. Designer Biology: The Ethics of Intensively Engineering Biological and Ecological Systems consists of thirteen chapters (twelve of them original to the collection) that address the ethical issues raised by technological intervention and design across a broad range of biological and ecological systems. Among the technologies addressed are geoengineering, human enhancement, sex selection, genetic modification, and synthetic biology. The aim of the collection is to advance and enrich our understanding of the ethical issues raised by these technologies, as well as to identify general lessons about the ethics of engineering complex biological and ecological systems that can be applied as new technologies and practices emerge. The insights that emerge will be especially valuable to students and scholars of environmental ethics, bioethics, or technology ethics.
With the increasing use of algorithms in high-stakes areas such as criminal justice and health ha... more With the increasing use of algorithms in high-stakes areas such as criminal justice and health has come a significant concern about the fairness of prediction-based decision procedures. In this article I argue that a prominent class of mathematically incompatible performance parity criteria can all be understood as applications of John Broome's account of fairness as the proportional satisfaction of claims. On this interpretation these criteria do not disagree on what it means for an algorithm to be fair. Rather they express different understandings of what grounds a claim to a good being allocated by an algorithmic decision procedure. I then argue that an important implication of the Broomean interpretation is that it strengthens the case for outcome-based criteria. Finally, I consider how a version of the levelling-down objection to performance parity criteria arises within the Broomean account.
This paper documents how an ethically aligned co-design methodology ensures trustworthiness in th... more This paper documents how an ethically aligned co-design methodology ensures trustworthiness in the early design phase of an artificial intelligence (AI) system component for healthcare. The system explains decisions made by deep learning networks analyzing images of skin lesions. The co-design of trustworthy AI developed here used a holistic approach rather than a static ethical checklist and required a multidisciplinary team of experts working with the AI designers and their managers. Ethical, legal, and technical issues potentially arising from the future use of the AI system were investigated. This paper is a first report on co-designing in the early design phase. Our results can also serve as guidance for other early-phase AI-similar tool developments.
Several authors have argued that contractualism faces a dilemma when it comes to justifying risks... more Several authors have argued that contractualism faces a dilemma when it comes to justifying risks generated by socially valuable activities. At the heart of the matter is the question of whether contractualists should adopt an ex post or an ex ante perspective when assessing whether an action or policy is justifiable to each person. In this paper I argue for the modest conclusion that ex post contractualism is a live option notwithstanding recent criticisms raised by proponents of the ex ante perspective. I then consider how an ex post contractualist can best respond to the problem that it seems to prohibit a range of intuitively permissible and socially valuable activities.
The British Journal for the Philosophy of Science, 2018
One reason for the popularity of Craver’s mutual manipulability (MM) account of constitutive rele... more One reason for the popularity of Craver’s mutual manipulability (MM) account of constitutive relevance is that it seems to make good sense of the experimental practices and constitutive reasoning in the life sciences. Two recent papers (Baumgartner and Gebharter [2016]; Baumgartner and Casini [2017]) propose a theoretical alternative to (MM) in light of several important conceptual objections. Their alternative approach, the no de-coupling (NDC) account, conceives of constitution as a dependence relation that once postulated provides the best explanation of the impossibility of breaking the common cause coupling of a macro-level mechanism and its micro-level components. This entails an abductive view of constitutive inference. Proponents of the NDC or abductive account recognize that their discussion leaves open a big question concerning the practical dimension of the notion of constitutive relevanssssce: Is it possible to faithfully reconstruct constitutional reasoning in science i...
According to Bedau and Triant decision-makers will be substantially ignorant about the consequenc... more According to Bedau and Triant decision-makers will be substantially ignorant about the consequences of their candidate choices when making decisions about synthetic biology. Bedau and Triant characterize decisions under such circumstances as decisions in the dark. They then argue that when making decisions in the dark, decision-makers should not appeal to the Precautionary Principle because this principle does not sufficiently accommodate important features of 'deciding and acting virtuously'. In the paper, I point out that it is unclear how to understand the virtue-based approach in the regulatory context of synthetic biology. I then argue that Daniel Steel's tripod interpretation of the Precautionary Principle provides a strong response to the problem of incoherence, which I suggest is at the heart of the criticism raised by Bedau and Triant. Finally, I raise a worry that incoherence problems may still crop up within the framework of Steel's account.
The use of algorithms to support prediction-based decision-making is becoming commonplace in a ra... more The use of algorithms to support prediction-based decision-making is becoming commonplace in a range of domains including health, criminal justice, education, social services, lending, and hiring. An assumption governing such decisions is that there is a property Y such that individual a should be allocated resource R by decision-maker D if a is Y. When there is uncertainty about whether a is Y, algorithms may provide valuable decision support by accurately predicting whether a is Y on the basis of known features of a. Based on recent work on statistical evidence in epistemology this article presents an argument against relying exclusively on algorithmic predictions to allocate resources when they provide purely statistical evidence that a is Y. The article then responds to the objection that any evidence that will increase the proportion of correct decisions should be accepted as the basis for allocations regardless of its epistemic deficiency. Finally, some important practical asp...
Machine learning algorithms are expected to improve referral decisions. In this article I discuss... more Machine learning algorithms are expected to improve referral decisions. In this article I discuss the legitimacy of deferring referral decisions in primary care to recommendations from such algorithms. The standard justification for introducing algorithmic decision procedures to make referral decisions is that they are more accurate than the available practitioners. The improvement in accuracy will ensure more efficient use of scarce health resources and improve patient care. In this article I introduce a proceduralist framework for discussing the legitimacy of algorithmic referral decisions and I argue that in the context of referral decisions the legitimacy of an algorithmic decision procedure can be fully accounted for in terms of the instrumental values of accuracy and fairness. I end by considering how my discussion of procedural algorithmic legitimacy relates to the debate on algorithmic fairness.
Several studies have documented that when presented with data from social media platforms machine... more Several studies have documented that when presented with data from social media platforms machine learning (ML) models can make accurate predictions about users, e.g., about whether they are likely to suffer health-related conditions such as depression, mental disorders, and risk of suicide. In a recent article, Ploug (Philos Technol 36:14, 2023) defends a right not to be subjected to AI profiling based on publicly available data. In this comment, I raise some questions in relation to Ploug’s argument that I think deserves further discussion.
2023 ACM Conference on Fairness, Accountability, and Transparency
Recent work on algorithmic fairness has largely focused on the fairness of discrete decisions, or... more Recent work on algorithmic fairness has largely focused on the fairness of discrete decisions, or classifications. While such decisions are often based on risk score models, the fairness of the risk models themselves has received considerably less attention. Risk models are of interest for a number of reasons, including the fact that they communicate uncertainty about the potential outcomes to users, thus representing a way to enable meaningful human oversight. Here, we address fairness desiderata for risk score models. We identify the provision of similar epistemic value to different groups as a key desideratum for risk score fairness, and we show how even fair risk scores can lead to unfair risk-based rankings. Further, we address how to assess the fairness of risk score models quantitatively, including a discussion of metric choices and meaningful statistical comparisons between groups. In this context, we also introduce a novel calibration error metric that is less sample size-biased than previously proposed metrics, enabling meaningful comparisons between groups of different sizes. We illustrate our methodology-which is widely applicable in many other settingsin two case studies, one in recidivism risk prediction, and one in risk of major depressive disorder (MDD) prediction. CCS CONCEPTS • Computing methodologies → Machine learning; • Applied computing → Health informatics; Law, social and behavioral sciences; • Social and professional topics → Computing / technology policy; User characteristics.
When is it justified to use opaque artificial intelligence (AI) output in medical decision-making... more When is it justified to use opaque artificial intelligence (AI) output in medical decision-making? Consideration of this question is of central importance for the responsible use of opaque machine learning (ML) models, which have been shown to produce accurate and reliable diagnoses, prognoses, and treatment suggestions in medicine. In this article, I discuss the merits of two answers to the question. According to the Explanation View, clinicians must have access to an explanation of why an output was produced. According to the Validation View, it is sufficient that the AI system has been validated using established standards for safety and reliability. I defend the Explanation View against two lines of criticism, and I argue that within the framework of evidence-based medicine mere validation seems insufficient for the use of AI output. I end by characterizing the epistemic responsibility of clinicians and point out how a mere AI output cannot in itself ground a practical conclusio...
Z-inspection® is a registered trademark. The content of this work is open access distributed unde... more Z-inspection® is a registered trademark. The content of this work is open access distributed under the terms and conditions of the Creative Commons (Attribution-NonCommercial-ShareAlike CC BY-NC-SA) license The views expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations.
This article's main contributions are twofold: 1) to demonstrate how to apply the general Europea... more This article's main contributions are twofold: 1) to demonstrate how to apply the general European Union's High-Level Expert Group's (EU HLEG) guidelines for trustworthy AI in practice for the domain of healthcare and 2) to investigate the research question of what does "trustworthy AI" mean at the time of the COVID-19 pandemic. To this end, we present the results of a post-hoc self-assessment to evaluate the trustworthiness of an AI system for predicting a multiregional score conveying the degree of lung compromise in COVID-19 patients, developed and verified by an interdisciplinary team with members from academia, public hospitals, and industry in time of
Advances in our scientific understanding and technological power in recent decades have dramatica... more Advances in our scientific understanding and technological power in recent decades have dramatically amplified our capacity to intentionally manipulate complex ecological and biological systems. An implication of this is that biological and ecological problems are increasingly understood and approached from an engineering perspective. In environmental contexts, this is exemplified in the pursuits of geoengineering, designer ecosystems, and conservation cloning. In human health contexts, it is exemplified in the development of synthetic biology, bionanotechnology, and human enhancement technologies. Designer Biology: The Ethics of Intensively Engineering Biological and Ecological Systems consists of thirteen chapters (twelve of them original to the collection) that address the ethical issues raised by technological intervention and design across a broad range of biological and ecological systems. Among the technologies addressed are geoengineering, human enhancement, sex selection, genetic modification, and synthetic biology. The aim of the collection is to advance and enrich our understanding of the ethical issues raised by these technologies, as well as to identify general lessons about the ethics of engineering complex biological and ecological systems that can be applied as new technologies and practices emerge. The insights that emerge will be especially valuable to students and scholars of environmental ethics, bioethics, or technology ethics.
With the increasing use of algorithms in high-stakes areas such as criminal justice and health ha... more With the increasing use of algorithms in high-stakes areas such as criminal justice and health has come a significant concern about the fairness of prediction-based decision procedures. In this article I argue that a prominent class of mathematically incompatible performance parity criteria can all be understood as applications of John Broome's account of fairness as the proportional satisfaction of claims. On this interpretation these criteria do not disagree on what it means for an algorithm to be fair. Rather they express different understandings of what grounds a claim to a good being allocated by an algorithmic decision procedure. I then argue that an important implication of the Broomean interpretation is that it strengthens the case for outcome-based criteria. Finally, I consider how a version of the levelling-down objection to performance parity criteria arises within the Broomean account.
This paper documents how an ethically aligned co-design methodology ensures trustworthiness in th... more This paper documents how an ethically aligned co-design methodology ensures trustworthiness in the early design phase of an artificial intelligence (AI) system component for healthcare. The system explains decisions made by deep learning networks analyzing images of skin lesions. The co-design of trustworthy AI developed here used a holistic approach rather than a static ethical checklist and required a multidisciplinary team of experts working with the AI designers and their managers. Ethical, legal, and technical issues potentially arising from the future use of the AI system were investigated. This paper is a first report on co-designing in the early design phase. Our results can also serve as guidance for other early-phase AI-similar tool developments.
Several authors have argued that contractualism faces a dilemma when it comes to justifying risks... more Several authors have argued that contractualism faces a dilemma when it comes to justifying risks generated by socially valuable activities. At the heart of the matter is the question of whether contractualists should adopt an ex post or an ex ante perspective when assessing whether an action or policy is justifiable to each person. In this paper I argue for the modest conclusion that ex post contractualism is a live option notwithstanding recent criticisms raised by proponents of the ex ante perspective. I then consider how an ex post contractualist can best respond to the problem that it seems to prohibit a range of intuitively permissible and socially valuable activities.
The British Journal for the Philosophy of Science, 2018
One reason for the popularity of Craver’s mutual manipulability (MM) account of constitutive rele... more One reason for the popularity of Craver’s mutual manipulability (MM) account of constitutive relevance is that it seems to make good sense of the experimental practices and constitutive reasoning in the life sciences. Two recent papers (Baumgartner and Gebharter [2016]; Baumgartner and Casini [2017]) propose a theoretical alternative to (MM) in light of several important conceptual objections. Their alternative approach, the no de-coupling (NDC) account, conceives of constitution as a dependence relation that once postulated provides the best explanation of the impossibility of breaking the common cause coupling of a macro-level mechanism and its micro-level components. This entails an abductive view of constitutive inference. Proponents of the NDC or abductive account recognize that their discussion leaves open a big question concerning the practical dimension of the notion of constitutive relevanssssce: Is it possible to faithfully reconstruct constitutional reasoning in science i...
According to Bedau and Triant decision-makers will be substantially ignorant about the consequenc... more According to Bedau and Triant decision-makers will be substantially ignorant about the consequences of their candidate choices when making decisions about synthetic biology. Bedau and Triant characterize decisions under such circumstances as decisions in the dark. They then argue that when making decisions in the dark, decision-makers should not appeal to the Precautionary Principle because this principle does not sufficiently accommodate important features of 'deciding and acting virtuously'. In the paper, I point out that it is unclear how to understand the virtue-based approach in the regulatory context of synthetic biology. I then argue that Daniel Steel's tripod interpretation of the Precautionary Principle provides a strong response to the problem of incoherence, which I suggest is at the heart of the criticism raised by Bedau and Triant. Finally, I raise a worry that incoherence problems may still crop up within the framework of Steel's account.
Uploads
Papers by Sune Holm