- Department of Philosophy, Logic and Scientific Method
The London School of Economics and Political Science
Houghton Street
London WC2A 2AE
United Kingdom
- Carnegie Mellon University, Philosophy, AlumnusUniversity of Groningen, Department of Theoretical Philosophy, Post-Doc, and 2 moreadd
- I am a faculty member in philosophy at the London School of Economics. My research focuses on the social epistemology of science.edit
Marine ecosystems are under increasing pressure from human activity, yet successful management relies on knowledge. The evidence-based policy (EBP) approach has been promoted on the grounds that it provides greater transparency and... more
Marine ecosystems are under increasing pressure from human activity, yet successful management relies on knowledge. The evidence-based policy (EBP) approach has been promoted on the grounds that it provides greater transparency and consistency by relying on 'high quality' information. However, EBP also creates epistemic responsibilities. Decision-making where limited or no empirical evidence exists, such as is often the case in marine systems, creates epistemic obligations for new information acquisition. We argue that philosophical approaches can inform the science-policy interface. Using marine biosecurity examples, we specifically examine the epistemic challenges in the acquisition and acceptance of evidence to inform policy, discussing epistemic due care and biases in consideration of evidence.
Research Interests:
Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to... more
Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings.
Research Interests:
What differentiates scientific research from non-scientific inquiry? Philosophers addressing this question have typically been inspired by the exalted social place and intellectual achievements of science. They have hence tended to point... more
What differentiates scientific research from non-scientific inquiry? Philosophers addressing this question have typically been inspired by the exalted social place and intellectual achievements of science. They have hence tended to point to some epistemic virtue or methodological feature of science that sets it apart. Our discussion on the other hand is motivated by the case of commercial research, which we argue is distinct from (and often epistemically inferior to) academic research. We consider a deflationary view in which science refers to whatever is regarded as epistemically successful, but find that this does not leave room for the important notion of scientific error and fails to capture distinctive social elements of science. This leads us to the view that a demarcation criterion should be a widely upheld social norm without immediate epistemic connotations. Our tentative answer is the communist norm, which calls on scientists to share their work widely for public scrutiny and evaluation.
Research Interests:
This note contains a corrective and a generalization of results by Borsboom et al. (2008), based on Heesen and Romeijn (2019). It highlights the relevance of insights from psychometrics beyond the context of psychological testing.
Research Interests:
This article investigates how the credit incentive to engage in questionable research practices (up to and including fraud) interacts with cumulative advantage, the process whereby high-status academics more easily increase their status... more
This article investigates how the credit incentive to engage in questionable research practices (up to and including fraud) interacts with cumulative advantage, the process whereby high-status academics more easily increase their status than low-status academics. I use a mathematical model to highlight two dynamics that have not yet received much attention. First, due to cumulative advantage, questionable research practices may pay off over the course of an academic career even if they are not attractive at the level of individual publications. Second, because of the role of bottleneck moments in academic careers, questionable research practices may be selected for even if they do not provide a benefit in expectation. I also observe that within the model, the most successful academics are the most likely to have benefited from fraud.
Research Interests:
Peer review is often taken to be the main form of quality control on academic research. Usually journals carry this out. However, parts of maths and physics appear to have a parallel, crowd-sourced model of peer review, where papers are... more
Peer review is often taken to be the main form of quality control on academic research. Usually journals carry this out. However, parts of maths and physics appear to have a parallel, crowd-sourced model of peer review, where papers are posted on the arXiv to be publicly discussed. In this paper we argue that crowd-sourced peer review is likely to do better than journal-solicited peer review at sorting papers by quality. Our argument rests on two key claims. First, crowd-sourced peer review will lead on average to more reviewers per paper than journal-solicited peer review. Second, due to the wisdom of the crowds, more reviewers will tend to make better judgments than fewer. We make the second claim precise by looking at the Condorcet Jury Theorem as well as two related jury theorems developed specifically to apply to peer review.
Research Interests:
Recent philosophical work has praised the reward structure of science , while recent empirical work has shown that many scientific results may not be reproducible. I argue that the reward structure of science incentivizes scientists to... more
Recent philosophical work has praised the reward structure of science , while recent empirical work has shown that many scientific results may not be reproducible. I argue that the reward structure of science incentivizes scientists to focus on speed and impact at the expense of the reproducibility of their work, thus contributing to the so-called reproducibility crisis. I use a rational choice model to identify a set of sufficient conditions for this problem to arise, and I argue that these conditions plausibly apply to a wide range of research situations. Currently proposed solutions will not fully address this problem. Philosophical commentators should temper their optimism about the reward structure of science.
Research Interests:
Peer reviewers at many funding agencies and scientific journals are asked to score submissions both on individual criteria and overall. The overall scores should be some kind of aggregate of the criteria scores. Carole Lee identifies this... more
Peer reviewers at many funding agencies and scientific journals are asked to score submissions both on individual criteria and overall. The overall scores should be some kind of aggregate of the criteria scores. Carole Lee identifies this as a potential locus for bias to enter the peer review process, which she calls commensuration bias. Here I view the aggregation of scores through the lens of social choice theory. I argue that in many situations, especially when reviewing grant proposals, it is impossible to avoid commensuration bias.
Research Interests:
There is a commonly made distinction between two types of scientists: risk-taking, trailblazing mavericks and detail-oriented followers. A number of recent papers have discussed the question what a desirable mixture of mavericks and... more
There is a commonly made distinction between two types of scientists: risk-taking, trailblazing mavericks and detail-oriented followers. A number of recent papers have discussed the question what a desirable mixture of mavericks and followers looks like. Answering this question is most useful if a scientific community can be steered toward such a desirable mixture. One attractive route is through credit incentives: manipulating rewards so that reward-seeking scientists are likely to form the desired mixture of their own accord. Here I argue that (even in theory) this idea is less straightforward than it may seem. Interpreting mavericks as scientists who prioritize rewards over speed and risk, I show in a deliberatively simple model that there is a fixed mixture which is not particularly likely to be desirable and which credit incentives cannot alter. I consider a way around this result, but this has some major drawbacks. I conclude that credit incentives are not as promising a way to create a desirable mixture of mavericks and followers as one might have thought.
Research Interests:
Pre-publication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the... more
Pre-publication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behavior of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing pre-publication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive or neutral. We conclude that on present evidence abolishing peer review weakly dominates the status quo.
Research Interests:
This paper offers a new angle on the common idea that the process of science does not support epistemic diversity. Under minimal assumptions on the nature of journal editing, we prove that editorial procedures, even when impartial in... more
This paper offers a new angle on the common idea that the process of science does not support epistemic diversity. Under minimal assumptions on the nature of journal editing, we prove that editorial procedures, even when impartial in themselves, disadvantage less prominent research programs. This purely statistical bias in article selection further skews existing differences in the success rate and hence attractiveness of research programs, and exacerbates the reputation difference between the programs. After a discussion of the modeling assumptions, the paper ends with a number of recommendations that may help promote scientific diversity through editorial decision making.
Research Interests:
I show that the social stratification of academic science can arise as a result of academics' preference for reading work of high epistemic value. This is consistent with a view on which academic superstars are highly competent academics,... more
I show that the social stratification of academic science can arise as a result of academics' preference for reading work of high epistemic value. This is consistent with a view on which academic superstars are highly competent academics, but also with a view on which superstars arise primarily due to luck. I argue that stratification is beneficial if most superstars are competent, but not if most superstars are lucky. I also argue that it is impossible to tell whether most superstars are in fact competent or lucky, or which group a given superstar belongs to, and hence whether stratification is overall beneficial.
Research Interests: Philosophy, Epistemology, Philosophy of Science, Social Networks, Epistemic Luck, and 10 moreFormal Epistemology, Social Epistemology, Bounded Rationality, Network Formation, Bayesian statistics, Citation Analysis, Economics of Superstars, Sequential Decision Theory, Bayesian epistemology, and Social Structure of Science
Should editors of scientific journals practice triple-anonymous reviewing? I consider two arguments in favor. The first says that insofar as editors' decisions are affected by information they would not have had under triple-anonymous... more
Should editors of scientific journals practice triple-anonymous reviewing? I consider two arguments in favor. The first says that insofar as editors' decisions are affected by information they would not have had under triple-anonymous review, an injustice is committed against certain authors. I show that even well-meaning editors would commit this wrong and I endorse this argument. The second argument says that insofar as editors' decisions are affected by information they would not have had under triple-anonymous review, it will negatively affect the quality of published papers. I distinguish between two kinds of biases that an editor might have. I show that one of them has a positive effect on quality and the other a negative one, and that the combined effect could be either positive or negative. Thus I do not endorse the second argument in general. However, I do endorse this argument for certain fields, for which I argue that the positive effect does not apply.
Research Interests:
Recent empirical work has shown that many scientific results may not be reproducible. By itself, this does not entail that there is a problem (or a "reproducibility crisis"). However, I argue that there is a problem: the reward structure... more
Recent empirical work has shown that many scientific results may not be reproducible. By itself, this does not entail that there is a problem (or a "reproducibility crisis"). However, I argue that there is a problem: the reward structure of science incentivizes scientists to focus on speed and impact at the expense of the reproducibility of their work. I illustrate this using a well-known failure of reproducibility: Fleischmann and Pons' work on cold fusion. I then use a rational choice model to identify a set of sufficient conditions for this problem to arise, and I argue that these conditions plausibly apply to a wide range of research situations. In the conclusion I consider possible solutions and implications for how Fleischmann and Pons' work should be evaluated.
Research Interests:
The communist norm requires that scientists widely share the results of their work. Where did this norm come from, and how does it persist? I argue on the basis of a game-theoretic model that rational credit-maximizing scientists will in... more
The communist norm requires that scientists widely share the results of their work. Where did this norm come from, and how does it persist? I argue on the basis of a game-theoretic model that rational credit-maximizing scientists will in many cases conform to the norm. This means that the origins and persistence of the communist norm can be explained even in the absence of a social contract or enforcement, contrary to recent work by Michael Strevens but adding to previous work emphasizing the benefits of the incentive structure created by the priority rule.
Research Interests:
In the second volume of the "Handbuch der physiologischen Optik", published in 1860, Helmholtz sets out a three-receptor theory of color vision using coterminal response curves, and shows that this theory can unify most phenomena of color... more
In the second volume of the "Handbuch der physiologischen Optik", published in 1860, Helmholtz sets out a three-receptor theory of color vision using coterminal response curves, and shows that this theory can unify most phenomena of color mixing known at the time. Maxwell had publicized the same theory five years earlier, but Helmholtz barely acknowledges this fact in the "Handbuch". Some historians have argued that this is because Helmholtz independently discovered the theory around the same time as Maxwell. This paper argues that this hypothesis is implausible. By writing what he did in the "Handbuch", Helmholtz (purposefully or not) influenced the field's perception of its own history. As a result, Helmholtz has received more recognition for his contributions to the field of color mixing than was his due, and Maxwell less.
Research Interests:
Social scientists use many different methods, and there are often substantial disagreements about which method is appropriate for a given research question. In response to this uncertainty about the relative merits of different methods,... more
Social scientists use many different methods, and there are often substantial disagreements about which method is appropriate for a given research question. In response to this uncertainty about the relative merits of different methods, W. E. B. Du Bois advocated for and applied `methodological triangulation'. This is to use multiple methods simultaneously in the belief that, where one is uncertain about the reliability of any given method, if multiple methods yield the same answer that answer is confirmed more strongly than it could have been by any single method. Against this, methodological purists believe that one should choose a single appropriate method and stick with it. Using tools from voting theory, we show Du Boisian methodological triangulation to be more likely to yield the correct answer than purism, assuming the scientist is subject to some degree of diffidence about the relative merits of the various methods. This holds even when in fact only one of the methods is appropriate for the given research question.
Research Interests:
This paper addresses the problem of judgment aggregation in science. How should scientists decide which propositions to assert in a collaborative document? We distinguish the question of what to write in a collaborative document from the... more
This paper addresses the problem of judgment aggregation in science. How should scientists decide which propositions to assert in a collaborative document? We distinguish the question of what to write in a collaborative document from the question of collective belief. We argue that recent objections to the application of the formal literature on judgment aggregation to the problem of judgment aggregation in science apply to the latter, not the former question. The formal literature has introduced various desiderata for an aggregation procedure. Proposition-wise majority voting emerges as a procedure that satisfies all desiderata which represent norms of science. An interesting consequence is that not all collaborating scientists need to endorse every proposition asserted in a collaborative document.
Research Interests:
In this paper we propose and analyze a game-theoretic model of the epistemology of peer disagreement. In this model, the peers' rationality is evaluated in terms of their probability of ending the disagreement with a true belief. We find... more
In this paper we propose and analyze a game-theoretic model of the epistemology of peer disagreement. In this model, the peers' rationality is evaluated in terms of their probability of ending the disagreement with a true belief. We find that different strategies---in particular, one based on the Steadfast View and one based on the Conciliatory View---are rational depending on the truth-sensitivity of the individuals involved in the disagreement. Interestingly, the Steadfast and the Conciliatory Views can even be rational simultaneously in some circumstances. We tentatively provide some reasons to favor the Conciliatory View in such cases. We argue that the game-theoretic perspective is a fruitful one in this debate, and this fruitfulness has not been exhausted by the present paper.
Research Interests:
A number of philosophers of science and statisticians have attempted to justify conclusions drawn from a finite sequence of evidence by appealing to results about what happens if the length of that sequence tends to infinity. If their... more
A number of philosophers of science and statisticians have attempted to justify conclusions drawn from a finite sequence of evidence by appealing to results about what happens if the length of that sequence tends to infinity. If their justifications are to be successful, they need to rely on the finite sequence being either indefinitely increasing or of a large size. These assumptions are often not met in practice. This paper analyzes a simple model of collecting evidence and finds that the practice of collecting only very small sets of evidence before taking a question to be settled is rationally justified. This shows that the appeal to long run results can be used neither to explain the success of actual scientific practice nor to give a rational reconstruction of that practice.
Research Interests:
***NB: A significantly revised version of this paper is published in Synthese as "Academic Superstars: Competent or Lucky?". It is available on my academia.edu page or (under open access) from the publisher.*** It is well-known that some... more
***NB: A significantly revised version of this paper is published in Synthese as "Academic Superstars: Competent or Lucky?". It is available on my academia.edu page or (under open access) from the publisher.***
It is well-known that some scientists are more prominent than others. But what makes one scientist more prominent than another? I propose a possible mechanism that produces differences in prominence: scientists' desire for information. In a model of a scientific community exchanging information, I show that this mechanism does indeed produce the kind of patterns of prominence that are actually observed. I discuss the implications of this result for three possible explanations of an individual scientist's prominence: an explanation based on scientific merit, an explanation based on epistemically irrelevant factors (e.g., gender bias or charisma), and an explanation based on epistemic luck. Depending on which of these explanations is correct one may draw different conclusions about a scientist based on prominence. I discuss policy recommendations that result from this, including suggestions about when it is appropriate to use measures of prominence (e.g., citation metrics) in giving out grants and awards.
It is well-known that some scientists are more prominent than others. But what makes one scientist more prominent than another? I propose a possible mechanism that produces differences in prominence: scientists' desire for information. In a model of a scientific community exchanging information, I show that this mechanism does indeed produce the kind of patterns of prominence that are actually observed. I discuss the implications of this result for three possible explanations of an individual scientist's prominence: an explanation based on scientific merit, an explanation based on epistemically irrelevant factors (e.g., gender bias or charisma), and an explanation based on epistemic luck. Depending on which of these explanations is correct one may draw different conclusions about a scientist based on prominence. I discuss policy recommendations that result from this, including suggestions about when it is appropriate to use measures of prominence (e.g., citation metrics) in giving out grants and awards.
Research Interests:
The rational price of the Pasadena and Altadena Game, introduced by Nover and Hajek (2004), has been the subject of considerable discussion. Easwaran (2008) has suggested that weak expectations - the value to which the average payoff s... more
The rational price of the Pasadena and Altadena Game, introduced by Nover and Hajek (2004), has been the subject of considerable discussion. Easwaran (2008) has suggested that weak expectations - the value to which the average payoff s converge in probability - can give the rational price of such games. We argue against the normative force of weak expectations in the standard framework. Furthermore, we propose to replace this framework by a bounded utility perspective: this shift renders the problem more realistic and accounts for the role of weak expectations. In particular, we demonstrate that in a bounded utility framework, all agents, even if they have di fferent value functions and disagree on the price of an individual game, will finally agree on the rational price of a repeated, averaged game. Thus, we explain the intuitive appeal of weak expectations, while avoiding both trivialization of the original paradox and the drawbacks of previous approaches.
Research Interests:
Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the... more
Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive or neutral. We conclude that on present evidence abolishing peer review weakly dominates the status quo.
Research Interests:
This paper offers a new angle on the common idea that the process of science does not support epistemic diversity. Under minimal assumptions on the nature of journal editing we prove that editorial procedures, despite being impartial in... more
This paper offers a new angle on the common idea that the process of science does not support epistemic diversity. Under minimal assumptions on the nature of journal editing we prove that editorial procedures, despite being impartial in themselves, disadvantage less prominent research programs. In particular, we show that the quality of editorial decisions, as measured by false positives and negatives, is lower for programs that on the whole deliver fewer good papers or perform worse in editor assessments. This purely statistical bias in article selection further skews the existing differences in the success rate and hence attractiveness of research programs, and exacerbates the reputation difference between the programs. Importantly, our results can be based on an assumption of real differences between the programs, but under certain circumstances our results also hold when the programs are identical in terms of the quality of their output. The paper ends with a number of recommendations that may help promote scientific diversity through editorial decision making.
Research Interests:
I show that the conflicting pressures to publish quickly and to publish accurately (which requires careful checking) allow for a rational tradeoff from the perspective of credit maximization. Under some plausible assumptions, the balance... more
I show that the conflicting pressures to publish quickly and to publish accurately (which requires careful checking) allow for a rational tradeoff from the perspective of credit maximization. Under some plausible assumptions, the balance that is optimal for the individual scientist will be suboptimal from a social perspective. In particular, I demonstrate in two models (one in which accuracy trades off against speed and one in which accuracy trades off against speed and impact) that scientists will tend to publish work of a lower accuracy level than is socially desirable. I illustrate the relevant phenomena using a case study of Fleischmann and Pons' research on cold fusion. I consider philosophical and practical implications of my results as well as consequences for how Fleischmann and Pons' work should be evaluated.
Research Interests:
The communist norm requires that scientists widely share the results of their work. Where did this norm come from, and how does it persist? Michael Strevens provides a partial answer to these questions by showing that scientists should be... more
The communist norm requires that scientists widely share the results of their work. Where did this norm come from, and how does it persist? Michael Strevens provides a partial answer to these questions by showing that scientists should be willing to sign a social contract that mandates sharing. However, he also argues that it is not in an individual credit-maximizing scientist's interest to follow this norm. This means that something in addition to scientists' interest in credit maximization is needed to explain the communist norm. I argue against Strevens that individual scientists can rationally conform to the communist norm, even in the absence of a social contract or other ways of socially enforcing the norm, by proving results to this effect in a game-theoretic model. This shows that the incentives provided to scientists through the priority rule are sufficient to explain both the origins and the persistence of the communist norm, adding to previous results emphasizing the benefits of the incentive structure created by the priority rule.
Research Interests:
Social scientists use many different methods, and there are often substantial disagreements about which method is appropriate for a given research question. A proponent of methodological triangulation believes that if multiple methods... more
Social scientists use many different methods, and there are often substantial disagreements about which method is appropriate for a given research question. A proponent of methodological triangulation believes that if multiple methods yield the same answer that answer is confirmed more strongly than it could have been by any single method. Methodological purists, on the other hand, believe that one should choose a single appropriate method and stick with it. Using formal tools from voting theory, we show that triangulation is more likely to lead to the correct answer than purism, assuming the scientist is subject to some degree of diffidence about the relative merits of the various methods. This is true even when in fact only one of the methods is appropriate for the given research question.
Research Interests:
In this paper we propose and analyze a game-theoretic model of the epistemology of peer disagreement. In this model, the peers' rationality is evaluated in terms of their probability of ending the disagreement with a true belief. We find... more
In this paper we propose and analyze a game-theoretic model of the epistemology of peer disagreement. In this model, the peers' rationality is evaluated in terms of their probability of ending the disagreement with a true belief. We find that different strategies---in particular, one based on the Steadfast View and one based on the Conciliatory View---are rational depending on the truth-sensitivity of the individuals involved in the disagreement. Interestingly, the Steadfast and the Conciliatory Views can even be rational simultaneously in some circumstances. We tentatively provide some reasons to favor the Conciliatory View in such cases. We argue that the game-theoretic perspective is a fruitful one in this debate, and this fruitfulness has not been exhausted by the present paper.
Research Interests:
Research Interests:
Research Interests:
The management game "How flow can you go?" is developed to convince decision makers of international logistic providers that their current planning methods of their transportation flows may be considerably... more
The management game "How flow can you go?" is developed to convince decision makers of international logistic providers that their current planning methods of their transportation flows may be considerably improved using OR-techniques. In fact, we have tested the game with several planners of several logistic providers and it turns out that the mathematical tool included in the management game