Philos Stud (2018) 175:831–858 https://doi.org/10.1007/s11098-017-0895-4 When journal editors play favorites Remco Heesen1 Published online: 25 March 2017 Ó The Author(s) 2017. This article is an open access publication Abstract Should editors of scientific journals practice triple-anonymous reviewing? I consider two arguments in favor. The first says that insofar as editors’ decisions are affected by information they would not have had under tripleanonymous review, an injustice is committed against certain authors. I show that even well-meaning editors would commit this wrong and I endorse this argument. The second argument says that insofar as editors’ decisions are affected by information they would not have had under triple-anonymous review, it will negatively affect the quality of published papers. I distinguish between two kinds of biases that an editor might have. I show that one of them has a positive effect on quality and the other a negative one, and that the combined effect could be either positive or negative. Thus I do not endorse the second argument in general. However, I do endorse this argument for certain fields, for which I argue that the positive effect does not apply. Keywords Feminist philosophy of science
Bias
Peer review
Social epistemology
Formal epistemology 1 Introduction Journal editors occupy an important position in the scientific landscape. By making the final decision on which papers get published in their journal and which papers do not, they have a significant influence on what work is given attention and what work is ignored in their field (Crane 1967). & Remco Heesen rdh51@cam.ac.uk 1 Faculty of Philosophy, University of Cambridge, Sidgwick Avenue, Cambridge CB3 9DA, UK 123 832 R. Heesen In this paper I investigate the following question: should the editor be informed about the identity of the author when she is deciding whether to publish a particular paper? Under a single- or double-anonymous reviewing procedure, the editor knows who the author of each submitted paper is.1 Under a triple-anonymous reviewing procedure, the author’s name and affiliation are hidden from the editor unless and until the paper is accepted for publication. So the question is: should journals practice triple-anonymous reviewing?2 Two kinds of arguments have been given in favor of triple-anonymous reviewing. One focuses on the treatment of the author by the editor. On this kind of argument, revealing identity information to the editor will lead the editor to (partially) base her judgment on irrelevant information. This is unfair to the author, and is thus bad. The second kind of argument highlights the effect on the journal and its readers. Again, the idea is that the editor will base her judgment on identity information if given the chance to do so. But now the further claim is that as a result the journal will accept worse papers. After all, if a decision to accept or reject a paper is influenced by the editor’s biases, this suggests that a departure has been made from a putative ‘‘objectively correct’’ decision. This harms the readers of the journal, and is thus bad.3 This paper assesses these arguments. I distinguish between two different ways the editor’s judgment may be affected if the author’s identity is revealed to her. First, the editor may treat authors she knows differently from authors she does not know, a phenomenon I will call connection bias. Second, the editor may treat authors differently based on some aspect of their identity (e.g., their gender), which I will call identity bias. I make the following three claims. My first claim is that connection bias actually benefits rather than harms the readers of the journal. This benefit is the result of a reduction in editorial uncertainty about the quality of submitted papers. I construct a model to show in a formally precise way how such a benefit might arise—surprisingly, no assumption that the scientists the editor knows are ‘‘better scientists’’ is required—and I cite empirical evidence that such a benefit indeed does arise. However, this benefit only applies in certain fields; I argue that mathematics and parts of the humanities are excluded (Sect. 2). My second claim is that whenever connection bias or identity bias affects an editorial decision, this constitutes an epistemic injustice in the sense of Fricker (2007) against the disadvantaged author. If the editor is to be (epistemically) just, she should prevent these biases from operating, which can be done through triple1 The difference is that under a single-anonymous procedure any reviewers who advise on the publishability of the paper are informed about the identity of the author, whereas under a doubleanonymous procedure the reviewers are not told who the author is. The identity of the reviewers is kept hidden from the author regardless of whether a single-, double-, or triple-anonymous procedure is used. 2 The relevant procedures are often called single-, double-, and triple-blind reviewing. I avoid this terminology as it has been criticized for being ableist (Tremain 2017, introduction). 3 Hence, I distinguish between the effects of triple-anonymous reviewing on the author and on the readers of the journal. This reflects a growing understanding that in order to study the social epistemology of science, what is good for an individual inquirer must be distinguished from what is good for the wider scientific community (Kitcher 1993; Strevens 2003; Mayo-Wilson et al. 2011). 123 When journal editors play favorites 833 anonymous reviewing. So I endorse an argument of the first of the two kinds I identified above: triple-anonymous reviewing is preferable because not doing so is unfair to authors (Sects. 3, 4). My third claim is that whether editorial biases harm the journal and its readers depends on a number of factors. Connection bias benefits readers, whereas identity bias harms them. Whether there is an overall benefit or harm depends on the strength of the editor’s identity bias, the relative sizes of the different groups, and other factors, as I illustrate using the model. As a result I do not in general endorse the second kind of argument, that triple-anonymous reviewing is preferable because readers of the journal are harmed otherwise. However, I do endorse this argument for fields like mathematics, where I claim that the benefits of connection bias do not apply (Sect. 5). Zollman (2009) has studied the effects of different editorial policies on the number of papers published and the selection criteria for publication, but he does not focus specifically on the editor’s decisions. Economists have studied models in which editorial decisions play an important role (Ellison 2002; Faria 2005; Besancenot et al. 2012), but they have not been concerned with biases the editor may be subject to. Other economists have done empirical work investigating the differences between papers with and without an author-editor connection (Laband and Piette 1994; Medoff 2003; Smith and Dombrowski 1998, more on this later), but they do not provide a model that can explain these differences. This paper thus fills a gap in the literature. I compare double- and triple-anonymous reviewing as opposed to single- and double-anonymous reviewing. The latter comparison has been studied extensively, see Blank (1991) for a prominent empirical study and Snodgrass (2006) and Lee et al. (2013, especially pp. 10–11) for literature reviews. In contrast, I know of almost no empirical or theoretical work directly comparing double- and tripleanonymous reviewing (one exception is Lee and Schunn 2010, p. 7). While I focus on comparing double- and triple-anonymous review, some of what I say may carry over to the context of comparing single- and double-anonymous review. In Sect. 5 I comment briefly on the extent to which the formal model I present applies in the context of comparing single- and double-anonymous review. However, I leave it to the reader to judge to what extent the arguments I make on the basis of the model carry over. 2 A model of connection bias As mentioned, journal editors have a certain measure of power in a scientific community because they decide which papers get published.4 An editor could use this power to the benefit of her friends or colleagues, or to promote certain subfields 4 Different journals may have different policies, such as one in which associate editors make the final decision for papers in their (sub)field. Here, I simply define ‘‘the editor’’ to be whomever makes the final decision whether to publish a particular paper. 123 834 R. Heesen or methodologies over others. This phenomenon has been called editorial favoritism. Bailey et al. (2008a, b) find that academics believe editorial favoritism to be fairly prevalent, with a nonnegligible percentage claiming to have perceived it firsthand. Hull (1988, chapter 9) finds a limited degree of favoritism in his study of reviewing practices at the journal Systematic Zoology. And Laband (1985) and Piette and Ross (1992) find that papers whose author has a connection to the journal editor are allocated more journal pages than papers by authors without such a connection.5 In this paper, I refer to the phenomenon that editors are more likely to accept papers from authors they know than papers from authors they do not know as connection bias. Academics tend to disapprove of this behavior (Sherrell et al. 1989; Bailey et al. 2008a, b). In both studies by Bailey et al., in which subjects were asked to rate the seriousness of various potentially problematic behaviors by editors and reviewers, this disapproval was shown to be part of a general and strong disapproval of ‘‘selfish or cliquish acts’’ in the peer review process.6 Thus it appears that the reason academics disapprove of connection bias is that it shows the editor acting on private interests, whereas disinterestedness is the norm in science (Merton 1942). On the other hand, there is some evidence that connection bias improves the overall quality of accepted papers (Laband and Piette 1994; Medoff 2003; Smith and Dombrowski 1998). Does this mean scientists are misguided in their disapproval? In this section, I use a formal model to show that editors may display connection bias even if their only goal is to accept the best papers, and that this may improve quality, consistent with Laband and Piette’s, Medoff’s, and Smith and Dombrowski’s findings. Note that in this section I discuss connection bias only. Subsequent sections discuss identity bias. Consider a simplified scientific community. Each scientist produces a paper and submits it to the community’s only journal which has one editor. Some papers are more suitable for publication than others. I assume that this suitability can be measured on a single numerical scale. For convenience I call this the quality of the paper. However, I remain neutral on how this notion should be interpreted, e.g., as an objective measure of the epistemic value of the paper, or as the number of times the paper would be cited in future papers if it was published, or as the average 5 Here, page allocation is used as a proxy for journal editors’ willingness to push the paper. The more obvious variable to use here would be whether or not the paper is accepted for publication. Unfortunately, there are no empirical studies which measure the influence of author-editor relationships on acceptance decisions directly. Presumably this is because information about rejected papers is usually not available. 6 This evidence conflicts to some extent with other survey findings. If connection bias was a serious worry for working scientists, one would expect them to rank knowing the editor and the composition of the editorial board more generally among the most important factors in deciding where to submit their papers. But Ziobrowski and Gibler (2000) find that this is not the case (these factors are ranked twelfth and sixteenth in a list of sixteen potentially relevant factors in their survey). In a similar survey by Mackie (1998, chapter 4), twenty percent of authors indicated that knowing the editor and/or her preferences is an important consideration in deciding where to submit a paper. 123 When journal editors play favorites 835 subjective value each member of the scientific community would assign to it if they read it.7 Crucially, the editor does not know the quality of the paper at the time it is submitted. This section aims to show how uncertainty about quality can lead to connection bias. To make this point, I assume that the editor cares only about quality, i.e., she makes an estimate of the quality of a paper and publishes those and only those papers whose quality estimate is high. Let qi be the quality of the paper submitted by scientist i. qi is modeled as a random variable to reflect uncertainty about quality. Since some scientists are more likely to produce high quality papers than others, the mean li of this random variable may be different for each scientist. I assume that quality follows a normal distribution with fixed variance: qi j li  Nðli ; r2in Þ (read: ‘‘qi given li follows a normal distribution with mean li and variance r2in ’’; the subscript in indicates that this is the variance in the quality of individual papers by the same author). The assumptions of normality and fixed variance are made primarily to keep the mathematics simple. Below I make similar assumptions on the distribution of average quality in the scientific community and the distribution of reviewers’ estimates of the quality of a paper. The results below likely hold under many different distributional assumptions.8 If the editor knows scientist i, she has some prior information on the average quality of scientist i’s work. This is reflected in the model by assuming that the editor knows the value of li . In contrast, the editor is uncertain about the average quality of the work of scientists she does not know. All she knows is the distribution of average quality in the larger scientific community, which I also assume to be normal: li  Nðl; r2sc Þ. Note that I assume the scientific community to be homogeneous: average paper quality follows the same distribution in the two groups of scientists (those known to the editor and those not known to the editor). If I assumed instead that scientists known to the editor write better papers on average the results would be qualitatively similar to those I present below. If scientists known to the editor write worse papers on average this would affect my results. However, since most journal editors are relatively central figures in their field (Crane 1967), this seems implausible for most cases. The editor’s prior for the quality of a paper submitted by some scientist i reflects this difference in information. If she knows the scientist she knows the value of li , and so her prior is pðqi j li Þ  Nðli ; r2in Þ. If the editor does not know scientist i she is uncertain about li . Integrating out this uncertainty yields a prior pðqi Þ  Nðl; r2in þ r2sc Þ for the quality of scientist i’s paper. When the editor receives a paper she sends it out for review. The reviewer provides an estimate ri of the paper’s quality which is again a random variable. I assume that the reviewer’s report is unbiased, i.e., its mean is the actual quality qi of 7 See Bright (2017) for more on potential difficulties with the notion of quality. 8 This claim is made precise and proved in Heesen and Romeijn (2017). 123 836 R. Heesen the paper. Once again I use a normal distribution to reflect uncertainty: ri j qi  Nðqi ; r2rv Þ.9 The editor uses the information from the reviewer’s report to update her beliefs. I assume that she does this by conditioning on ri . Thus, her posterior for the quality of scientist i’s paper is pðqi j ri Þ if she does not know the author, and pðqi j ri ; li Þ if she does. The posterior distributions are themselves normal distributions whose mean is a weighted average of ri and the prior mean (see Proposition 5 in ‘‘Appendix’’). I write lU i for the mean of the posterior distribution if the editor does not know scientist i and lKi if she does. I assume that the editor publishes any paper whose (posterior) expected quality is above some threshold q . So a paper written by a scientist unknown to the editor is  published if lU i [ q and a paper written by a scientist known to the editor is K  published if li [ q . Other standards could be used: risk-averse standards might require high (greater than 50%) confidence that the paper is above some threshold. For the qualitative results presented here this makes no difference (see Proposition 7 in the Appendix). The first theorem establishes the existence of connection bias in the model (refer to the Appendix for all proofs). It says that the editor is more likely to publish a paper written by an arbitrary author she knows than a paper written by an arbitrary author she does not know, whenever q [ l (for any positive value of r2sc and r2rv ). The condition amounts to a requirement that the journal’s acceptance rate is less than 50%. This is true of most reputable journals in most fields (physics being a notable exception). Theorem 1 (Connection Bias) If q [ l, r2sc [ 0, and r2rv [ 0, the acceptance probability for authors known to the editor is higher than the acceptance probability  for authors unknown to the editor, i.e., PrðlKi [ q Þ [ PrðlU i [ q Þ: Theorem 1 shows that in my model any journal with an acceptance rate lower than 50% will be seen to display connection bias. Thus I have established the surprising result that an editor who cares only about the quality of the papers she publishes may end up publishing more papers by her friends and colleagues than by scientists unknown to her, even if her friends and colleagues are not, as a group, better scientists than average.10 9 The reviewer’s report could reflect the opinion of a single reviewer, or the averaged opinion of multiple reviewers. The editor could even act as a reviewer herself, in which case the report reflects her findings which she has to incorporate in her overall beliefs about the quality of the paper. The assumption I make in the text covers these scenarios, as long as a given journal is fairly consistent in the number of reviewers used. Some journals may use different numbers of reviewers for different papers (potentially affecting the variance if more reviewers give more accurate information than fewer) or employ reviewers in different roles (e.g., one reviewer to assess technical aspects of the paper and one reviewer to assess non-technical aspects). My model does not apply to journals where these differences correlate with the existence or absence of a connection between editor and author. 10 The model presented in this section is formally similar to that of Miller (1994) and Borsboom et al. (2008). They assume genuine differences in average quality between groups, so a result like Theorem 1 is true but unsurprising in their models. 123 When journal editors play favorites 837 Why does this surprising result hold? The distribution of the posterior mean lU i has lower variance than the distribution of lKi (see Proposition 6 in the Appendix). That is, the variance of lU i is lower in an ‘‘objective’’ sense: this is not a claim about the editor’s subjective uncertainty about her judgment. This is because lU i is a weighted average of l and ri , keeping it relatively close to the overall mean l compared to lKi , which is a weighted average of li and ri (which tend to differ from l in the same direction). Note that the result assumes that scientists known to the editor and scientists unknown to the editor are held to the same ‘‘standard’’ (the threshold q ). Alternatively, the editor might enforce equal acceptance rates for the two groups. This would be formally equivalent to raising the threshold for known scientists (or lowering the threshold for unknown scientists). Theorem 1 describes a subjective effect: an editor who uses information about the average quality of papers produced by scientists she knows will believe that scientists she knows produce on average more papers that meet her quality threshold. Does this translate into an objective effect? In order to answer this question I compare the average quality of accepted papers, or more formally, the expected value of the quality of a paper, conditional on meeting the publication threshold, given that the author is either known to the editor or not. Theorem 2 (Positive Effect of Connection Bias) If r2sc [ 0, and r2rv [ 0, the average quality of accepted papers from authors known to the editor is higher than the average quality of accepted papers from authors unknown to the editor, i.e.,  E½qi j lKi [ q Š [ E½qi j lU i [ q Š. The editor’s knowledge of the average quality of papers written by scientists she knows makes it such that among those scientists relatively many whose papers are accepted have relatively high average quality. Since this correlates with paper quality the average quality of accepted papers in this group is relatively high, yielding Theorem 2. The theorem shows that the editor can use the extra information she has about scientists she knows to improve the average quality of the papers published in her journal. The surprising result, then, is that the editor’s connection bias actually benefits rather than harms the readers of the journal. In other words, the editor can use her connections to ‘‘identify and capture high-quality papers’’, as Laband and Piette (1994) suggest. To what extent does this show that the connection bias observed in reality is the result of editors capturing high-quality papers, as opposed to editors using their position of power to help their friends? At this point the model yields an empirical prediction. If connection bias is (primarily) due to capturing high-quality papers, the quality of papers by authors the editor knows should be higher than average, as shown in the model. If, on the other hand, connection bias is (primarily) a result of the editor accepting for publication papers written by authors she knows even though they do not meet the quality standards of the journal, then the quality of papers by authors the editor knows should be lower than average. 123 838 R. Heesen If subsequent citations are a good indication of the quality11 of a paper, a simple regression can test whether accepted papers written by authors with an author-editor connection have higher or lower average quality than papers without such a connection. This empirical test has been carried out a number of times, and the results favor the hypothesis that editors use their connections to improve the quality of published papers (Laband and Piette 1994; Smith and Dombrowski 1998; Medoff 2003).12 Note that in the above (qualitative) results, nothing depends on the sizes of the variances r2in , r2sc , and r2rv . The values of the variances do matter when the acceptance rate and average quality of papers are compared quantitatively. For example, reducing r2rv (making the reviewer’s report more accurate) reduces the differences in the acceptance rate and average quality of papers. Note also that the results depend on the assumption that r2sc and r2rv are positive. What is the significance of these assumptions? If r2rv ¼ 0, i.e., if there is no variance in the reviewer’s report, the reviewer reports the quality of the paper with perfect accuracy. In this case the ‘‘extra information’’ the editor has about authors she knows is not needed, and so there is no difference in acceptance rate or average quality based on whether the editor knows the author. But it seems unrealistic to expect reviewer’s reports to be this accurate. If r2sc ¼ 0 there is either no difference in the average quality of papers produced by different authors, or learning the identity of the author does not tell the editor anything about the expected quality of that scientist’s work. In this case there is no value to the editor (with regard to determining the quality of the submitted paper) in learning the identity of the author. So here there is also no difference in acceptance rate or average quality based on whether the editor knows the author. Under what circumstances should the identity of the author be expected to tell the editor something useful about the quality of a submitted paper? This seems to be most obviously the case in the lab sciences. The identity of the author, and hence the lab at which the experiments were performed, can increase or decrease the editor’s confidence that the experiments were performed correctly, including all the little checks and details that are impossible to describe in a paper. In such cases, ‘‘ the reader must rely on the author’s (and perhaps referee’s) testimony that the author really performed the experiment exactly as claimed, and that it worked out as reported’’ (Easwaran 2009, p. 359). 11 Recall that I have remained neutral on how the notion of quality should be interpreted. If quality is simply defined as ‘‘the number of citations this paper would get if it were published’’ the connection between quality and citations is obvious. Even on other interpretations of quality, citations have frequently been viewed as a good proxy measure (Cole and Cole 1967, 1968; Medoff 2003). This practice has been defended by Cole and Cole (1971) and Clark (1957, chapter 3), and criticized by Lindsey (1989) and Heesen (forthcoming). 12 Laband and Piette and Medoff focus on economics journals and Smith and Dombrowski on accounting journals. Further research would be valuable to see whether these results generalize, especially to the natural sciences and the humanities. Note also that these results do not rule out the possibility that editors use their power to help their friends: they merely suggest that on balance editors’ use of connections has a positive effect on citations. 123 When journal editors play favorites 839 But in other fields, in particular mathematics and those parts of the humanities that focus on abstract arguments, there is no need to rely on the author’s reputation. This is because in these fields the paper itself is the contribution, so it is possible to judge papers in isolation of how or by whom they were created (Easwaran 2009). And in fact there exists a norm that this is how they should be judged: ‘‘Papers will rely only on premises that the competent reader can be assumed to antecedently believe, and only make inferences that the competent reader would be expected to accept on her own consideration.’’ (Easwaran 2009, p. 354). Arguably then, the epistemic advantage conferred by revealing identity information about the author to the editor applies only in certain fields. The relevant fields are those where part of the information in the paper is conferred on the authority of testimony. In mathematics and parts of the humanities, where a careful reading of a paper itself constitutes a reproduction of its argument, there is no relevant information to be learned from the identity of the author (i.e., r2sc ¼ 0). Or at least the publishing norms in these fields suggest that their members believe this to be the case. 3 Connection bias as an epistemic injustice The previous section discussed a formal model of editorial uncertainty about paper quality. I first established the existence of connection bias in this model. Then I showed that connection bias benefits the readers of the journal, insofar as readers care about the quality of accepted papers. Despite this benefit to readers, I claim that connection bias is unfair to authors. In this section I argue this claim by appealing to the concept of epistemic injustice, as developed by Fricker (2007). The type of epistemic justice that is relevant here is testimonial injustice. Fricker (2007, pp. 17–23) defines a testimonial injustice as a case where a speaker suffers a credibility deficit for which the hearer is ethically and epistemically culpable. Testimonial injustices may arise in various ways. Fricker is particularly interested in what she calls ‘‘the central case of testimonial injustice’’ (Fricker 2007, p. 28). This kind of injustice results from a negative identity-prejudicial stereotype, which is defined as follows: A widely held disparaging association between a social group and one or more attributes, where this association embodies a generalization that displays some (typically, epistemically culpable) resistance to counter-evidence owing to an ethically bad affective investment. (Fricker 2007, p. 35) Because the stereotype is widely held, it produces systematic testimonial injustice: the relevant social group will suffer a credibility deficit in many different social spheres. It is clear that connection bias is not an instance of the central case of testimonial injustice. This would require some negative stereotype associated with scientists unknown to the editor (as a group) which does not normally exist. So I set the central case aside (I return to it in Sect. 4) and focus on the question whether connection bias can produce (non-central cases of) testimonial injustice. 123 840 R. Heesen How are individual scientists affected by the differential acceptance rates established in Sect. 2? For scientist i, the probability of acceptance given the average quality of her papers li denotes the long-run average proportion of her papers that will be accepted (assuming she submits all her papers to the journal). Theorem 3 (Acceptance Rate for Individual Authors) Assume r2sc [ 0 and r2rv [ 0 . The acceptance rate for author i (with average quality li ) is higher if the editor knows her if and only if li exceeds a weighted average of l and q :      Pr lKi [ q jli  Pr lU i [ q jli iff li  r2in r2sc l þ q : r2in þ r2sc r2in þ r2sc The strict version is true as well, i.e., if the editor knows scientist i she is strictly better off if and only if li strictly exceeds the weighted average. Note that regardless of the values of the variances, any scientist whose average quality exceeds the threshold value (li  q ) benefits from connection bias. Conversely, a scientist of below average quality (li  l) is actually worse off if the editor knows her.13 Consider what this theorem says for a particular scientist i who is unknown to the editor and whose average quality li strictly exceeds the weighted average. Some of her papers are rejected even though they would have been accepted if the editor knew her. In Fricker’s terminology, scientist i suffers from a credibility deficit: fewer of her papers are considered credible (i.e., publishable) by the editor than would have been considered credible if the editor knew her. Is this credibility deficit suffered by scientist i ethically and epistemically culpable on the part of the editor? On the one hand, the editor is simply making maximal use of the information available to her. It just so happens that she has more information about scientists she knows than about others. But that is hardly the editor’s fault. Is it incumbent upon her to get to know the work of every scientist who submits a paper? This may well be too much to ask. But an alternative option is to remove all information about the authors of submitted papers. This can be done by using a triple-anonymous reviewing procedure, in which the editor is prevented from using information about scientists she knows in her evaluation. I conclude that the editor is ethically and epistemically culpable for credibility deficits suffered by scientists unknown to the editor whose average quality exceeds the weighted average specified in Theorem 3, and hence testimonial injustices are committed against such authors when a double-anonymous reviewing procedure is used. A similar epistemic injustice occurs for scientists known to the editor whose average quality is below the weighted average, as such authors would prefer that the editor not use information she has about their average quality. It is worth noting explicitly which scientists are better or worse off in terms of acceptance rates if a triple-anonymous procedure is introduced. If the acceptance 13 These claims assume that q [ l. Note also that only a minority of authors benefits from connection bias, as half of all authors satisfy li  l. 123 When journal editors play favorites 841 threshold q is held constant,14 nothing changes for scientists unknown to the editor. Scientists known to the editor will see their acceptance rate go down if their average quality exceeds the weighted average specified in Theorem 3, and up otherwise. The overall acceptance rate of the journal will go down (by Theorem 1). So the group that I based my argument on (unknown scientists of high average quality) is not necessarily made better off by switching to triple-anonymous reviewing. The argument for triple-anonymous reviewing given in this section is not about benefiting one group of scientists or harming another: rather, it is about fairness. Under a triple-anonymous procedure, at least all scientists are treated equally: any scientist who writes a paper of a given quality has the same chance of seeing that paper accepted. Whereas under a double-anonymous procedure, scientists are treated unfairly in that their acceptance rates may differ based only on an epistemically irrelevant characteristic (knowing the editor). I conclude that while journal readers may benefit from connection bias, it involves unfair treatment of authors. Because this unfair treatment takes the form of an epistemic injustice, which involves both ethically and epistemically culpable behavior, connection bias has both an epistemic benefit (to readers) and a cost (to the author). It would be a misinterpretation of my analysis, then, to conclude that connection bias is epistemically good but ethically bad. 4 Identity bias as an epistemic injustice So far, I have assumed that connection bias is the only bias journal editors display. The literature on implicit bias suggests further biases: ‘‘[i]f submissions are not anonymous to the editor, then the evidence suggests that women’s work will probably be judged more negatively than men’s work of the same quality’’ (Saul 2013, p. 45). Evidence for this claim is given by Wennerås and Wold (1997), Valian (1999, chapter 11), Steinpreis et al. (1999), Budden et al. (2008), and MossRacusin et al. (2012).15 So women scientists are at a disadvantage simply because of their gender identity. Similar biases exist based on other irrelevant aspects of scientists’ identity, such as race or sexual orientation (see Lee et al. 2013 for a critical survey of various biases in the peer review system). As Crandall (1982, p. 208) puts it: ‘‘The editorial process has tended to be run as an informal, 14 Things are slightly more subtle if the overall acceptance rate of the journal is held constant instead. The threshold will go down, say to q \q , and hence all scientists unknown to the editor will see their   U acceptance rates go up, as PrðlU i [ q j li Þ [ Prðli [ q j li Þ for all values of li . The acceptance rate for known scientists must correspondingly go down, but the effect on an individual known scientist i  depends on li . In particular, PrðlKi [ q j li Þ  PrðlU i [ q j li Þ iff li  r2in r2in r2 r2 r2 þ r2sc þ r2rv  ðq l þ 2 sc 2 q þ 2 in 2 in 2 r2rv þ rsc rin þ rsc rin þ rsc q Þ: 15 These citations show that the work of women in academia is undervalued in various ways. None of them focus on editor evaluations, but they support Saul’s claim unless it is assumed that journal editors as a group are significantly less biased than other academics. 123 842 R. Heesen old-boy network which has excluded minorities, women, younger researchers, and those from lower-prestige institutions’’.16 I use identity bias to refer to these kinds of biases. I now complicate the model of Sect. 2 to include identity bias. I then argue that allowing the editor’s decisions to be influenced by identity bias is unfair to authors, analogous to the argument of the previous section. I incorporate identity bias in the model by assuming the editor consistently undervalues members of one group (and overvalues the others). More precisely, she believes the average quality of papers produced by any scientist i from the group she is biased against to be lower than it really is by some constant quantity e. Conversely, she raises the average quality of papers written by any scientist not belonging to this group by d.17 So the editor has a different prior for the two groups; I use pA to denote her prior for the quality of papers written by scientists she is biased against, and pF for her prior for scientists she is biased in favor of. As before, the editor may know a given scientist or not. So there are now four groups. If scientist i is known to the editor and belongs to the stigmatized group the editor’s prior distribution on the quality of scientist i’s paper is pA ðqi j li Þ  Nðli e; r2in Þ. If scientist i is known to the editor but is not in the stigmatized group the prior is pF ðqi j li Þ  Nðli þ d; r2in Þ. If scientist i is not known to the editor and is in the stigmatized group the prior is pA ðqi Þ  Nðl e; r2in þ r2sc Þ. And if scientist i is not known to the editor and not in the stigmatized group the prior is pF ðqi Þ  Nðl þ d; r2in þ r2sc Þ.18 After the reviewer’s report comes in the editor updates her beliefs about the quality of the paper. This yields posterior distributions pA ðqi j ri ; li Þ, pF ðqi j ri ; li Þ, KF UA UF pA ðqi j ri Þ, and pF ðqi j ri Þ, with posterior means lKA i , li , li , and li , respectively. As before, the paper is published if the posterior mean exceeds the threshold q . This yields the unsurprising result that the editor is less likely to publish papers by scientists she is biased against. 16 The latter case is arguably different from the others, as academic affiliation is not as clearly irrelevant as gender or race: many would argue it is a valid signal of quality. I am inclined to think bias based on academic affiliation involves epistemic injustice, but I leave arguing this point in detail to future work. 17 This is a simplifying assumption: one could imagine having biases against multiple groups of different strengths, biases whose strength has some random variation, or biases which intersect in various ways (Collins and Chepp 2013; Bright et al. 2016). However, the assumption in the main text suffices for my purposes. It should be fairly straightforward to extend my results to more complicated cases like the ones just described. 18 Note that I assume that the editor displays bias against scientists in the stigmatized group regardless of whether she knows them or not. Under a reviewing procedure that is not triple-anonymous, the editor learns at least the name and affiliation of any scientist who submits a paper. This information is usually sufficient to determine with reasonable certainty the scientist’s gender. So at least for gender bias it seems reasonable to expect the editor to display bias even against scientists she does not know. Conversely, because negative identity-prejudicial stereotypes can work unconsciously, it does not seem reasonable to expect that the editor can withhold her bias from scientists she knows. 123 When journal editors play favorites 843 Theorem 4 (Identity Bias) If e [ 0, d [ 0,19 r2sc [ 0, and r2rv [ 0, the acceptance probability for authors the editor is biased against is lower than the acceptance probability for authors the editor is biased in favor of (keeping fixed whether or not the editor knows the author). That is,    KF     UF      Pr lKA and Pr lUA i [ q \ Pr li [ q i [ q \ Pr li [ q : Theorem 4 establishes the existence of identity bias in the model: authors that the editor is biased against are less likely to see their paper accepted than other authors. Any time a paper is rejected because of identity bias (i.e., the paper would have been accepted if the relevant part of the author’s identity had been different, all else being equal), a testimonial injustice occurs. Testimonial injustices resulting from identity bias can be instances of the central case of testimonial injustice, in which the credibility deficit results from a negative identity-prejudicial stereotype. The evidence suggests that negative identityprejudicial stereotypes affect the way people (not just men) judge women’s work, even when one does not consciously believe in these stereotypes. Moreover, those who think highly of their ability to judge work objectively and/or are primed with objectivity are affected more rather than less (Uhlmann and Cohen 2007; Stewart and Payne 2008, p. 1333). Similar claims plausibly hold for biases based on race or sexual orientation. So both connection bias and identity bias are responsible for injustices against authors. This is one way to spell out the claim that it is unfair to authors when journal editors do not use a triple-anonymous reviewing procedure. This constitutes the first kind of argument for triple-anonymous reviewing which I mentioned in the introduction, and which I endorse based on these considerations. 5 The tradeoff between connection bias and identity bias The second kind of argument I mentioned in the introduction claims that failing to use triple-anonymous reviewing harms the journal and its readers, because it would lower the average quality of accepted papers. In Sect. 2 I argued that connection bias actually has the opposite effect: it increases average quality. Identity bias complicates the picture, as it generally lowers the average quality of accepted papers. This raises the question whether the combined effect of connection bias and identity bias is positive or negative. In this section I show that there is no general answer to this question. I compare the average quality of accepted papers under a procedure subject to connection bias and identity bias to that under a triple-anonymous reviewing procedure. Under this procedure, the editor’s prior distribution for the quality of any 19 While the assumption that e and d are both positive is sensible given the intended interpretation, it is not required from a mathematical perspective: e þ d [ 0 suffices for this theorem. See the proof in the Appendix. 123 844 R. Heesen submitted paper is pðqi Þ  Nðl; r2in þ r2sc Þ, i.e., the prior for unknown authors from Sect. 2. Hence the posterior is pðqi j ri Þ with mean lU i , the probability of acceptance U   is Prðli [ q Þ and the average quality of accepted papers is E½qi j lU i [ q Š. As a result, the editor displays neither connection bias nor identity bias. In contrast, the double-anonymous reviewing procedure is subject to connection bias and identity bias. The overall probability that a paper is accepted under this procedure depends on the relative sizes of the four groups. I use pKA to denote the fraction of scientists known to the editor that she is biased against, pKF for the fraction known to the editor that she is biased in favor of, pUA for unknown scientists biased against, and pUF for unknown scientists biased in favor of (pKA þ pKF þ pUA þ pUF ¼ 1). Let Ai denote the event that scientist i’s paper is accepted under the doubleanonymous procedure. The overall probability of acceptance is       PrðAi Þ ¼ pKA Pr lKA þ pKF Pr lKF i [q i [q       þ pUA Pr lUA þ pUF Pr lUF i [q i [q ; and the average quality of accepted papers is E½qi j Ai Š.20 In the remainder of this section I assume that the editor’s biases are such that she believes the average quality of all submitted papers to be equal to the overall average l. In other words, her bias against women21 is canceled out on average by her bias in favor of men, weighted by the relative sizes of those groups: ðpKA þ pUA Þe ¼ ðpKF þ pUF Þd. Given the other parameter values, this fixes the value of d. This is a kind of commensurability requirement for the two procedures because it guarantees that the editor perceives the average quality of submitted papers to be l regardless of which reviewing procedure is used. As far as I can tell there are no interesting general conditions on the parameters that determine whether the double-anonymous procedure or the triple-anonymous procedure will lead to a higher average quality of accepted papers. The question I explore next, using some numerical examples, is how biased the editor needs to be for the epistemic costs of her identity bias to outweigh the epistemic benefits resulting from connection bias. In order to generate numerical data values have to be chosen for the parameters. First I set l ¼ 0 and q ¼ 2 . Since quality is an interval scale in this model, these choices are arbitrary. For the variances r2in (of the quality of individual papers), r2sc (of the average quality of authors), and r2rv (of the accuracy of the reviewer’s report), I choose a ‘‘small’’ and a ‘‘large’’ value (1 and 4 respectively). For the sizes of the four groups, I assume that the percentage of women among scientists the editor knows is equal to the percentage of women among scientists the editor does not know. I consider two cases for the editor’s identity bias: either half 20 Expressions for PrðAi Þ and E½qi j Ai Š using only the parameter values and standard functions are given in Proposition 14 in the Appendix. These expressions are used to generate the numerical results below. 21 For ease of exposition, in the remainder of this section I assume that the specific form of identity bias the editor displays is gender bias against women. 123 When journal editors play favorites 845 of all authors are women or women are a 30% minority.22 Similarly, I consider the case in which the editor knows half of all scientists submitting papers, and the case in which the editor knows 30% of them. As a result, there are 32 possible settings of the parameters (23 choices for the variances times 22 choices for the group sizes). It follows from Theorem 2 that when e ¼ 0 the double-anonymous procedure helps rather than harms the readers of the journal by increasing average quality relative to the triple-anonymous procedure. If e is positive but relatively small, this remains true, but when e is relatively big, the double-anonymous procedure harms the readers. This is because the average quality of published papers under the double-anonymous procedure decreases continuously as e increases. The interesting question, then, is where the turning point lies. How big does the editor’s bias need to be in order for the negative effects of identity bias on quality to cancel out the positive effects of connection bias? I determine the value of e for which the average quality of published papers under the double-anonymous procedure and the triple-anonymous procedure is the same. Figure 1 reports these numbers. I plot them against the acceptance rate that the triple-anonymous procedure would have for those values of the parameters. The bias e is measured in ‘‘quality points’’ (for reference: since l ¼ 0 and q ¼ 2, a paper needs to be two quality points above average to be accepted). The variances determine the acceptance rate of the triple-anonymous procedure. The eight possible settings correspond to six acceptance rates: 0.72, 4.16, 11.51, 16.36, 19.32, and 22.66%. The four different settings for the group sizes are indicated through the different shapes of the data points in Fig. 1. X’es indicate all groups are of equal size (pKA ¼ pKF ¼ pUA ¼ pUF ¼ 0:25), circles indicate women are a minority, pluses indicate authors known to the editor are a minority, and diamonds indicate both women and known authors are a minority. Since quality points do not have a clear interpretation outside the context of the model, I use the values of e shown in Fig. 1 to calculate the average rate of acceptance of papers authored by women and the average rate of acceptance of papers authored by men.23 The difference between these numbers gives an indication of the size of the editor’s bias: it measures (in percentage points, abbreviated pp) how many more papers the editor accepts from men, compared to women. 22 Bruner and O’Connor (2017) note that certain dynamics in academic life can lead to identity bias against groups as a result of the mere fact that they are a minority. Here I consider both the case where women are a minority (and are possibly stigmatized as a result of being a minority, as Bruner and O’Connor suggest) and the case where they are not (and so the negative identity-prejudicial stereotype has some other source). 23 These are calculated without regard for whether the editor knows the author or not. In particular, the rates of acceptance for women and men are respectively             þ pUA Pr lUA þ pUF Pr lUF pKA Pr lKA pKF Pr lKF i [q i [q i [q i [q and : pKA þ pUA pKF þ pUF 123 846 R. Heesen Fig. 1 The minimum size of the editor’s bias such that the quality costs of the double-anonymous procedure outweigh its benefits (measured in ‘‘quality points’’), in 32 cases, plotted as a function of the acceptance rate of the corresponding triple-anonymous procedure Fig. 2 The minimum size of the editor’s bias such that the quality costs of the double-anonymous procedure outweigh its benefits (given as a percentage point difference in acceptance rates) These differences are reported in Fig. 2. Even with this small sample of 32 cases, a large variation of results can be observed. I illustrate this by looking at two cases in detail. First, suppose that r2in ¼ r2sc ¼ 1 and r2rv ¼ 4, so there is relatively little variation in the quality of individual papers and in the average quality of authors but relatively high variation in reviewer estimates of quality. Then the triple-anonymous procedure has an acceptance rate as low as 0.72%. If the groups are all of equal size then under the double-anonymous procedure the acceptance rate for men needs to be as much as 2.66 pp higher than the acceptance rate for women, in order for the average quality under the two procedures to be equal. Clearly a 2.66 pp bias is very large for a journal that only accepts less than 1% of papers. If the bias is any less than that there is no harm to the readers in using the double-anonymous procedure. 123 When journal editors play favorites 847 Second, suppose that r2in ¼ r2sc ¼ 4 and r2rv ¼ 1, so the variation in quality of both papers and authors is relatively high but reviewers’ estimates are relatively accurate. Then the triple-anonymous procedure has an acceptance rate of 22.66%. If, moreover, the editor knows relatively few authors then the quality costs of the double-anonymous procedure outweigh its benefits whenever the acceptance rate for men is more than 2.23 pp higher than the acceptance rate for women. For a journal accepting about 23% of papers that means that even if the gender bias of the editor is relatively mild the journal’s readers are harmed if the double-anonymous procedure is used. Based on these results, and the fact that the parameter values are unlikely to be known in practice, it is unclear whether the double-anonymous procedure or the triple-anonymous procedure will lead to a higher average quality of published papers for any particular journal.24 So in general it is not clear that an argument that the double-anonymous procedure harms the journal’s readers can be made. At the same time, a general argument that the double-anonymous procedure helps the readers is not available either. Given this, I am inclined to recommend a tripleanonymous procedure for all journals because not doing so is unfair to authors. One might be tempted to draw a different policy recommendation from this paper: use triple-anonymous review to prevent the negative effects of identity bias on quality, but provide the editor with the author’s h-index or some other citation index to benefit from the reduced uncertainty associated with knowing an author’s average quality. I do not endorse this suggestion for at least two reasons. First, it is unfair to authors as discussed in Sect. 3. Second, depending on one’s interpretation of quality, it may be difficult or impossible to infer author quality from citations (Lindsey 1989; Heesen forthcoming; Bright 2017). I have argued in this section that the net effect of connection bias and identity bias on quality is unclear. But I argued in Sect. 2 that the positive effect of connection bias only exists in certain fields. In fields where papers rely partially on the author’s testimony there is value in knowing the identity of the author. But in other fields such as mathematics and parts of the humanities testimony is not taken to play a role—the paper itself constitutes the contribution to the field—and so arguably there is no value in knowing the identity of the author. In those fields, then, there is no quality benefit from connection bias, but there is still a quality cost from identity bias. So here the strongest case for the tripleanonymous procedure emerges, as the double-anonymous procedure is both unfair to authors and harms readers. I have focused on evaluating triple-anonymous review, in particular in contrast to double-anonymous review. In many fields, particularly in the natural sciences, single-anonymous review is the norm, and so the more pertinent question is whether they should switch to double-anonymous review. Can the present model be used or adapted to address this question? 24 Note that the evidence collected by Laband and Piette (1994) does not help settle this question, as they do not directly compare the triple-anonymous and the double-anonymous procedure. Their evidence supports a positive effect of connection bias, but not a verdict on the overall effect of triple-anonymizing on quality. 123 848 R. Heesen Analyzing a model in which both the editor and one or more reviewers display connection bias and/or identity bias is beyond the scope of this paper. Here I only discuss one relatively simple scenario: the case in which the editor does not display identity bias but the reviewer does. Suppose the reviewer is biased against one group, reducing reviewer estimates of paper quality by e if the author belongs to that group and raising estimates by d otherwise. If the editor knows the reviewer is biased, she can take the reviewer’s bias into account. In particular, if she knows which group the reviewer is biased against and the size of the bias, learning the biased reviewer estimate is equivalent to learning what the unbiased reviewer estimate would have been, and so a rational unbiased editor simply updates on the unbiased reviewer estimate. In this case reviewer bias has no effect on acceptance decisions at all. If the editor does not know the reviewer is biased, she may (naively) treat the biased reviewer estimate as an unbiased estimate. In this case the analysis is very similar to the one given above. A close analogue of Theorem 4 holds. The only difference is that the effect of the variances is flipped. High values of r2in and r2sc increase the consequences of the reviewer’s bias, while high values of r2rv reduce it. This is the reverse of what happens in the version of the model I analyzed above (cf. Proposition 12 in the Appendix). 6 Conclusion In this paper I have considered two types of arguments for triple-anonymous review: one based on fairness considerations from the perspective of the author and one based on the consequences for the readers of the journal. I have argued that the double-anonymous procedure introduces differential treatment of scientific authors. In particular, editors are more likely to publish papers by authors they know (connection bias, Theorem 1) and less likely to publish papers by authors they apply negative identity-prejudicial stereotypes to (identity bias, Theorem 4). Whenever a paper is rejected as a result of one of these biases an epistemic injustice (in the sense of Fricker 2007) is committed against the author. This is a fairness-based argument in favor of triple-anonymizing. From the readers’ perspective the story is more mixed, as connection bias has a positive effect on the quality of published papers and identity bias a negative one. Whether the readers are better off under the triple-anonymous procedure then depends on how these effects trade off, which is highly context-dependent. This yields a more nuanced view than that suggested by either Laband and Piette (1994), who focus only on connection bias, or by an argument for triple-anonymizing which focuses only on identity bias. However, in mathematics and parts of the humanities there is arguably no positive quality effect from connection bias, as knowing about an author’s other work is not taken to be relevant (Easwaran 2009). So here the negative effect of identity bias is the only relevant consideration from the readers’ perspective. In this situation, considerations concerning fairness for the author and considerations 123 When journal editors play favorites 849 concerning the consequences for the readers point in the same direction: in favor of triple-anonymous review. Acknowledgements Thanks to Kevin Zollman, Michael Strevens, Stephan Hartmann, Teddy Seidenfeld, Cailin O’Connor, Liam Bright, Shahar Avin, Jan-Willem Romeijn, an anonymous reviewer, and audiences at meetings of the Philosophy of Science Association in Atlanta and the Société de Philosophie des Sciences in Lausanne for valuable comments and discussion. This work was partially supported by the National Science Foundation under Grant SES 1254291 and by an Early Career Fellowship from the Leverhulme Trust and the Isaac Newton Trust. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Appendix: Acceptance rates and average quality The following properties of the normal distribution will be useful (see, e.g., Johnson et al. 1994, chapter 13, section 3). Let X  Nðm; s2 Þ. Then the moment-generating function of X is given by   1 E½expftX gŠ ¼ exp mt þ s2 t2 : ð1Þ 2 Let Y ¼ aX þ b (with a 6¼ 0). Then   Y  N am þ b; a2 s2 : ð2Þ In particular, X s m  Nð0; 1Þ has a standard normal distribution, with density function / and distribution function (or cumulative density function) U. Proposition 5   2 pðqi jri Þ  N lU ; r i pjr where lU i ¼ lKi and   pðqi jri ; li Þ  N lKi ; r2pjrl ; r2in þ r2sc r2rv r þ l; i r2in þ r2sc þ r2rv r2in þ r2sc þ r2rv r2 r2 ¼ 2 in 2 ri þ 2 rv 2 li ; rin þ rrv rin þ rrv r2pjr ¼ r2pjrl ¼  1 1 þ 2 2 2 rin þ rsc rrv  1 1 þ r2in r2rv 1 ; 1 : See DeGroot (2004, section 9.5, or any other textbook that covers Bayesian statistics) for a proof of Proposition 5. Note that r2pjr [ r2pjrl whenever r2sc [ 0 and 123 850 R. Heesen r2rv [ 0. Proposition 6 2 K 2 lU i  Nðl; rU Þ and li  Nðl; rK Þ , where r2U ¼ ðr2in þ r2sc Þ2 r2in þ r2sc þ r2rv and r2K ¼ r4in þ r2sc ðr2in þ r2rv Þ : r2in þ r2rv Moreover, if r2sc [ 0 and r2rv [ 0 , then r2U \r2K : Proof Since ri j qi  Nðqi ; r2rv Þ, qi j li  Nðli ; r2in Þ, and li  Nðl; r2sc Þ, it follows that ri j li  Nðli ; r2in þ r2rv Þ and ri  Nðl; r2in þ r2sc þ r2rv Þ. U Since l is a constant, lU i is a linear transformation of ri . By Eq. 2 li is normally distributed with mean l and variance r2U . For determining the distribution of lKi it is helpful first to define   r2in r4in Xi ¼ lKi li Þ. Then Xi j li  N 0; r2 þr li ¼ r2 þr by Eq. 2. Now I find the 2 ðri 2 in rv in rv distribution of lKi by using the moment-generating function and the law of total expectation. E½expftlKi gŠ ¼ E½E½expftXi þ tli g j li ŠŠ ¼ E½expftli gE½expftXi g j li ŠŠ   1 r4in 2 ¼ exp 0t þ 2 t E½expftli gŠ 2 rin þ r2rv   1 r4in þ r2sc ðr2in þ r2rv Þ 2 t : ¼ exp lt þ 2 r2in þ r2rv This establishes the distribution of lKi . Finally, note that r2U ¼ ðr2in þ r2sc Þ2 ðr2in þ r2rv Þ ðr2in þ r2sc þ r2rv Þðr2in þ r2rv Þ and r2K ¼ ðr2in þ r2sc Þ2 ðr2in þ r2rv Þ þ r2sc r4rv : ðr2in þ r2sc þ r2rv Þðr2in þ r2rv Þ So r2U \r2K whenever r2sc [ 0 and r2rv [ 0 (and r2U ¼ r2K otherwise, assuming either r2in [ 0 or r2rv [ 0). h Theorem 1   2 2 PrðlKi [ q Þ [ PrðlU i [ q Þ if q [ l; rsc [ 0 , and rrv [ 0. It follows from Proposition 6 that     q l Pr lKi [ q ¼ 1 U and rK Proof    Pr lU ¼1 i [q U   q l : rU Since U is (strictly) increasing in its argument, and rK [ rU by Proposition 6, the theorem follows immediately. h If the editor accepts papers only if her posterior confidence that qi [ q is at least a (with 1=2  a\1; the main text considers only the case a ¼ 1=2), a similar result holds. Let za be the number such that Uðza Þ ¼ a. 123 When journal editors play favorites 851 Proposition 7 Let r2sc [ 0 and r2rv [ 0: If a  1=2 and q þ za rpjr [ l (so the acceptance rate for unknown scientists is less than 50%), then25  K  U l q l q [ za [ Pr i [ za : Pr i rpjrl rpjr Proof By Proposition 6, lxi  Nðl; r2x Þ both for x ¼ U and x ¼ K. So   K za rpjrl þ q l l q Pr i ; [ za ¼ 1 U rK rpjrl  U  za rpjr þ q l l q Pr i [ za ¼ 1 U : rpjr rU The result follows because za  0, rK [ rU , and rpjr [ rpjrl . h Proposition 8   U U E½qi j lU i [ q Š ¼ E½li j li [ q Š and Proof E½qi j lKi [ q Š ¼ E½lKi j lKi [ q Š: Since lU i is simply an (invertible) transformation of ri ,   U 2  q j r  N l ; r qi j lU i i i i pjr : The distribution of qi j lKi is found using the moment-generating function and the law of total expectation: E½expftqi g j lKi Š ¼ E½E½expftqi g j li ; lKi Š j lKi Š ¼ E½E½expftqi g j li ; ri Š j lKi Š     1 2 2 1 2 2 K K K ¼ E exp li t þ rpjrl t j li ¼ exp li t þ rpjrl t ; 2 2 where the second equality follows because, if li is given, lKi is simply an invertible transformation of ri . So:   qi j lKi  qi j ri ; li  N lKi ; r2pjrl : Now the law of total expectation can be used to establish (for x ¼ U; K) that E½qi j lxi [ q Š ¼ E½E½qi j lxi Š j lxi [ q Š ¼ E½lxi j lxi [ q Š: Let X  Nðm; s2 Þ. Then X j X [ a follows a left-truncated normal distribution, with left-truncation point a. According to, e.g., Johnson et al. (1994, chapter 13, section 10.1), the mean of this distribution can be expressed as 25 To see that these are the correct acceptance rates, note that a paper by a scientist i unknown to the editor is accepted if the editor’s posterior satisfies Prðqi [ q j ri Þ [ a which is equivalent to 1 U q Þ=rpjr [ za . Analogous reasoning Uððq lU i Þ=rpjr Þ [ a by Proposition 5. This is equivalent to ðli applies to known scientists. 123 852 R. Heesen E½X j X [ aŠ ¼ m þ sR Here R is defined for all x 2 R by26 RðxÞ ¼ 1 a s m : ð3Þ /ðxÞ : UðxÞ It follows from the definitions that RðxÞ [ 0 for all x 2 R and that R0 ðxÞ ¼ RðxÞ2 Proposition 9 ð4Þ xRðxÞ: 2 (Gordon 1941) For all x [ 0; RðxÞ\ x xþ1 : Proposition 10 If X  Nðm; s2 Þ and Y  Nðm; r2 Þ with r [ s [ 0 then E½Y j Y [ aŠ [ E½X j X [ aŠ: Proof It suffices to show that the derivative oso E½X j X [ aŠ is positive for all s [ 0. Differentiating Eq. (3) (using Eq. (4)) yields  a m a m a m2 o a m2 E½X j X [ aŠ ¼ R þ1 R : os s s s s Since Rða s mÞ [ 0, o os E½X a j X [ aŠ [ 0 if and only if m2 a m a m R þ1 [ 0: s s s This is true whenever a s m  0 because then both terms in the sum are positive. Proposition 9 guarantees that it is true whenever a s m [ 0. h Theorem 2 Proof  2 2 E½qi j lKi [ q Š [ E½qi j lU i [ q Š whenever rsc [ 0; and rrv [ 0. By Proposition 8,  U U  E½qi j lU i [ q Š ¼ E½li j li [ q Š and E½qi j lKi [ q Š ¼ E½lKi j lKi [ q Š: 2 K 2 By Proposition 6, lU i  Nðl; rU Þ and li  Nðl; rK Þ, with rU \rK . Hence the conditions of Proposition 10 are satisfied, and the result follows. h Proposition 11  K li j li  N li ; r4in ; r2in þ r2rv ! 2 2 2 2 2 2 2 2 r r ðr þ r þ r Þ ðr þ r Þ in rv sc in rv l; in 2 sc l þ : lU i j li  N r2in þ r2sc þ r2rv i r2in þ r2sc þ r2rv ðrin þ r2sc þ r2rv Þ2 Proof Since li is given and hence behaves like a constant, both lKi and lU i are h simply linear transformations of ri , so both results follow from Eq. 2. 26 R is the inverse of what is known in the literature (e.g., Gordon 1941) as Mills’ ratio. 123 When journal editors play favorites Theorem 3 853 Given r2sc [ 0 and r2rv [ 0;      Pr lKi [ q j li  Pr lU i [ q j li , li  Proof r2in Assume r2sc [ 0 and r2rv [ 0. Then27 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi   r2in þ r2rv  ðq Pr lKi [ q j li ¼ 1 U r2in    Pr lU i [ q j li ¼ 1 U r2in r2 l þ 2 sc 2 q : 2 rin þ rsc þ rsc ! li Þ ; ðr2in þ r2sc Þðq l Þ þ r2rv ðq ffi pi ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðr2in þ r2sc Þ r2in þ r2rv  So PrðlKi [ q j li Þ  PrðlU i [ q j li Þ if and only if pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi r2in þ r2rv  ðr2in þ r2sc Þðq li Þ þ r2rv ðq lÞ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðq  r2in ðr2in þ r2sc Þ r2in þ r2rv lÞ ! : li Þ: Some algebra yields the result. h Proposition 12 where   2 pA ðqi j ri ; li Þ  N lKA ; r i pjrl ;   2 pA ðqi j ri Þ  N lUA i ; rpjr ; K lKA i ¼ li lUA ¼ lU i i e
r2rv ; r2in þ r2rv r2in e
r2rv ; þ r2sc þ r2rv   2 ; r pF ðqi j ri ; li Þ  N lKF i pjrl ;   2 pF ðqi j ri Þ  N lUF i ; rpjr ; ¼ lKi þ lKF i lUF ¼ lU i i þ d
r2rv ; r2in þ r2rv r2in d
r2rv : þ r2sc þ r2rv For a proof I refer once again to DeGroot (2004, section 9.5). Proposition 13  KA li  N l  UA li  N l e
r2rv ; r2 ; r2in þ r2rv K e
r2rv ; r2 ; r2in þ r2sc þ r2rv U  d
r2  N l þ 2 rv2 ; r2K ; rin þ rrv  d
r2rv ; r2 :  N l þ lUF i r2in þ r2sc þ r2rv U lKF i 27 The expression for PrðlKi [ q j li Þ and the remainder of this proof assume that r2in [ 0. If r2in ¼ 0  then the desired probability is one if li [ q and zero otherwise. Since 0\ PrðlU i [ q j li Þ\1, the result follows. 123 854 R. Heesen KF K Proof Since lKA i and li are simply li shifted by a constant they follow the same K distribution as li except that its mean is shifted by the same constant. Similarly lUA i and lUF are just lU h i i shifted by a constant. For notational convenience, I introduce qKA , qKF , qUA , and qUF , defined by qKA ¼ q þ qUA ¼ q þ e
r2rv ; þ r2rv r2in r2in e
r2rv ; þ r2sc þ r2rv qKF ¼ q qUF ¼ q d
r2rv ; þ r2rv r2in r2in d
r2rv : þ r2sc þ r2rv Theorem 4 If e þ d [ 0; r2sc [ 0, and r2rv [ 0;    KF     UF      Pr lKA and Pr lUA i [ q \ Pr li [ q i [ q \ Pr li [ q : Proof For the first inequality, note that  KA  q l  [ q Pr lKA ¼ 1 U \1 i rK  U  KF q l rK    ¼ Pr lKF i [q : The equalities follow from the distributions of the posterior means established in Proposition 13. The inequality follows from the fact that U is strictly increasing in its argument. By the same reasoning,  UA  UF    UA  q l q l   Pr li [ q ¼ 1 U \1 U ¼ Pr lUF i [q : rU rU Proposition 14  KA   KF q l q l U þ pKF 1 U rK rK   UA   UF q l q l þ pUA 1 U þ pUF 1 U rU rU   KA  KF rK q l q l pKA / þ pKF / E½qi j Ai Š ¼ l þ rK rK PrðAi Þ   UA  UF rU q l q l þ pUA / þ pUF / : rU rU PrðAi Þ  PrðAi Þ ¼ pKA 1 : Proof The expression for PrðAi Þ follows immediately from the distributions of the posterior means established in Proposition 13. To get an expression for E½qi j Ai Š, consider first the average quality of scientist i’s paper given that it is accepted and given that scientist i is in the group of scientists known to the editor that the editor is biased against: 123 When journal editors play favorites 855  KA   q l  K K KA ; E qi j lKA [ q j l [ q R ¼ E l ¼ l þ r K i i i rK  K KA and where the first equality uses the fact that lKA i [ q is equivalent to li [ q then applies Proposition 8, and the second equality uses Eq. 3. Similarly,  KF  q l KF  ; E qi j li [ q ¼ l þ rK R rK  UA  q l  E qi j lUA ¼ l þ rU R ; i [q rU  UF  q l  : E qi j lUF ¼ l þ rU R i [q rU The average quality of accepted papers E½qi j Ai Š is a weighted sum of these expectations. The weights are given by the proportion of accepted papers that are written by a scientist in that particular group. For example, authors known to the  editor that she is biased against form a pKA PrðlKA i [ q Þ= PrðAi Þ proportion of accepted papers. Hence    1  KA  E½qi j Ai Š ¼ pKA Pr lKA i [ q E qi j li [ q PrðAi Þ    1  KF  pKF Pr lKF þ i [ q E qi j li [ q PrðAi Þ    1  UA  þ pUA Pr lUA i [ q E qi j li [ q PrðAi Þ    1  UF  pUF Pr lUF þ i [ q E qi j li [ q PrðAi Þ   KA  KF rK q l q l pKA / ¼lþ þ pKF / rK rK PrðAi Þ   UA  UF rU q l q l pUA / þ þ pUF / : rU rU PrðAi Þ References Bailey, C. D., Hermanson, D. R., & Louwers, T. J. (2008a). An examination of the peer review process in accounting journals. Journal of Accounting Education, 26(2), 55–72. doi:10.1016/j.jaccedu.2008.04. 001. http://www.sciencedirect.com/science/article/pii/S0748575108000201. ISSN 0748-5751 Bailey, C. D., Hermanson, D. R., & Tompkins, J. G. (2008b). The peer review process in finance journals. Journal of Financial Education, 34:1–27. http://www.jstor.org/stable/41948838. ISSN 0093-3961. Besancenot, D., Huynh, K. V., & Faria, J. R. (2012). Search and research: The influence of editorial boards on journals’ quality. Theory and Decision, 73(4), 687–702. doi:10.1007/s11238-012-9314-7. ISSN 0040-5833. 123 856 R. Heesen Blank, R. M. (1991) The effects of double-blind versus single-blind reviewing: Experimental evidence from The American Economic Review. The American Economic Review, 81(5), 1041–1067. http:// www.jstor.org/stable/2006906. ISSN 00028282. Borsboom, D., Romeijn, J.-W., & Wicherts, J. M. (2008). Measurement invariance versus selection invariance: Is fair selection possible? Psychological Methods, 13(2), 75–98. doi:10.1037/1082989X.13.2.75. Bright, L. K. (2017). Against candidate quality. Manuscript. http://www.liamkofibright.com/uploads/4/8/ 9/8/48985425/acq-share.pdf. Bright, L. K., Malinsky, D., & Thompson, M. (2016). Causally interpreting intersectionality theory. Philosophy of Science, 83(1), 60–81. http://www.jstor.org/stable/10.1086/684173. ISSN 00318248. Bruner, J., & O’Connor, C. (2017). Power, bargaining, and collaboration. In T. Boyer-Kassem, C. MayoWilson, & M. Weisberg (eds.), Scientific collaboration and collective knowledge. Oxford University Press, Oxford. http://philpapers.org/rec/BRUPBA-2. Budden, A. E., Tregenza, T., Aarssen, L. W., Koricheva, J., Leimu, R., & Lortie, C. J. (2008). Doubleblind review favours increased representation of female authors. Trends in Ecology & Evolution, http://www.sciencedirect.com/science/article/pii/ 23(1):4–6. doi:10.1016/j.tree.2007.07.008. S0169534707002704. ISSN 0169-5347. Clark, K. E. (1957). America’s psychologists: A survey of a growing profession. Washington: American Psychological Association. Cole, S., & Cole, J. R. (1967). Scientific output and recognition: A study in the operation of the reward system in science. American Sociological Review, 32(3), 377–390. http://www.jstor.org/stable/ 2091085. ISSN 00031224. Cole, S., & Cole, J. R. (1968). Visibility and the structural bases of awareness of scientific research. American Sociological Review, 33(3), 397–413. http://www.jstor.org/stable/2091914. ISSN 00031224. Cole, J. R., & Cole, S. (1971). Measuring the quality of sociological research: Problems in the use of the ‘‘Science Citation Index’’. The American Sociologist, 6(1), 23–29. http://www.jstor.org/stable/ 27701705. ISSN 00031232. Collins, P. H., & Chepp, V. (2013) Intersectionality. In G. Waylen, K. Celis, J. Kantola, & S. L. Weldon (eds), The Oxford handbook of gender and politics (chapter 2, pp. 57–87). Oxford University Press, Oxford. ISBN 0199751455. Crandall, R. (1982). Editorial responsibilities in manuscript review. Behavioral and Brain Sciences, 5, 207–208. doi:10.1017/S0140525X00011316. http://journals.cambridge.org/article_S0140525X000 11316. ISSN 1469-1825. Crane, D. (1967). The gatekeepers of science: Some factors affecting the selection of articles for scientific journals. The American Sociologist, 2(4), 195–201. http://www.jstor.org/stable/27701277. ISSN 00031232. DeGroot, M. H. (2004). Optimal statistical decisions. New Jersey: Wiley. Easwaran, K. (2009). Probabilistic proofs and transferability. Philosophia Mathematica, 17(3), 341–362. doi:10.1093/philmat/nkn032. http://philmat.oxfordjournals.org/content/17/3/341.abstract. Ellison, G. (2002). Evolving standards for academic publishing: A q-r theory. Journal of Political Economy, 110(5), 994–1034. http://www.jstor.org/stable/10.1086/341871. ISSN 00223808. Faria, J. R. (2005). The game academics play: Editors versus authors. Bulletin of Economic Research, 57(1), 1–12. doi:10.1111/j.1467-8586.2005.00212.x. ISSN 1467-8586. Fricker, M. (2007). Epistemic injustice: Power and the ethics of knowing. Oxford: Oxford University Press. Gordon, R. D. (1941). Values of Mills’ ratio of area to bounding ordinate and of the normal probability integral for large values of the argument. The Annals of Mathematical Statistics, 12(3), 364–366. http://www.jstor.org/stable/2235868. ISSN 00034851. Heesen, R. (forthcoming). Academic superstars: Competent or lucky? Synthese. doi:10.1007/s11229-0161146-5. ISSN 1573-0964. Heesen, R., & Romeijn, J. -W. (2017). Epistemic diversity and editor decisions: A statistical Matthew effect. In preparation. Hull, D. L. (1988). Science as a process: An evolutionary account of the social and conceptual development of science. Chicago: University of Chicago Press. Johnson, N. L., Kotz, S., & Balakrishnan., N. (1994). Continuous univariate distributions (2nd ed., Vol. 1). New York: Wiley. 123 When journal editors play favorites 857 Kitcher, P. (1993). The advancement of science: Science without legend, objectivity without illusions. Oxford: Oxford University Press. Laband, D. N. (1985). Publishing favoritism: A critique of department rankings based on quantitative publishing performance. Southern Economic Journal, 52(2), 510–515. http://www.jstor.org/stable/ 1059636. ISSN 00384038. Laband, D. N., & Piette, M. J. (1994). Favoritism versus search for good papers: Empirical evidence regarding the behavior of journal editors. Journal of Political Economy, 102(1), 194–203. http:// www.jstor.org/stable/2138799. ISSN 00223808. Lee C. J., & Schunn, C. D. (2010). Philosophy journal practices and opportunities for bias. American Philosophical Association Newsletter on Feminism and Philosophy, 10(1), 5–10. http://www. apaonline.org/?feminism_newsletter. ISSN 2155-9708. Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17. doi:10.1002/asi.22784. ISSN 1532-2890. Lindsey, D. (1989). Using citation counts as a measure of quality in science: Measuring what’s measurable rather than what’s valid. Scientometrics, 15(3–4), 189–203. doi:10.1007/BF02017198. ISSN 0138-9130. Mackie, C. D. (1998). Canonizing economic theory: How theories and ideas are selected in economics. New York: M. E. Sharpe. Mayo-Wilson, C., Zollman, K. J. S., & Danks, D. (2011). The independence thesis: When individual and social epistemology diverge. Philosophy of Science, 78(4), 653–677. http://www.jstor.org/stable/10. 1086/661777. ISSN 00318248. Medoff, M. H. (2003). Editorial favoritism in economics? Southern Economic Journal, 70(2), 425–434. http://www.jstor.org/stable/3648979. ISSN 00384038. Merton, R. K. (1942). A note on science and democracy. Journal of Legal and Political Sociology, 1(1–2), 115–126. Miller, E. M. (1994). The relevance of group membership for personnel selection: A demonstration using Bayes’ theorem. The Journal of Social, Political, and Economic Studies, 19(3), 323–358. ISSN 0278-839X. Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., & Handelsman, J. (2012). Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109(41), 16474–16479. doi:10.1073/pnas.1211286109. http://www.pnas.org/content/109/ 41/16474.abstract. Piette, M. J., & Ross, K. L. (1992). A study of the publication of scholarly output in economics journals. Eastern Economic Journal, 18(4), 429–436. http://www.jstor.org/stable/40325474. ISSN 00945056. Saul, J. (2013). Implicit bias, stereotype threat, and women in philosophy. In K. Hutchison & F. Jenkins (Eds.), Women in philosophy: What needs to change? (pp. 39–60). Oxford: Oxford University Press. chapter 2. Sherrell, D. L., Hair, J. F, Jr., & Griffin, M. (1989). Marketing academicians’ perceptions of ethical research and publishing behavior. Journal of the Academy of Marketing Science, 17(4), 315–324. doi:10.1007/BF02726642. ISSN 0092-0703. Smith, K. J., & Dombrowski, R. F. (1998). An examination of the relationship between author-editor connections and subsequent citations of auditing research articles. Journal of Accounting Education, 16(3–4), 497–506. doi:10.1016/S0748-5751(98)00019-0. http://www.sciencedirect.com/science/ article/pii/S0748575198000190. ISSN 0748-5751. Snodgrass, R. (2006). Single- versus double-blind reviewing: An analysis of the literature. ACM SIGMOD Record, 35(3), 8–21. doi:10.1145/1168092.1168094. ISSN 0163-5808. Steinpreis, R. E., Anders, K. A., & Ritzke, D. (1999). The impact of gender on the review of the curricula vitae of job applicants and tenure candidates: A national empirical study. Sex Roles, 41(7–8), 509–528. doi:10.1023/A:1018839203698. ISSN 0360-0025. Stewart, B. D., & Payne, B. K. (2008). Bringing automatic stereotyping under control: Implementation intentions as efficient means of thought control. Personality and Social Psychology Bulletin, 34(10), 1332–1345. doi:10.1177/0146167208321269. http://psp.sagepub.com/content/34/10/1332.abstract. Strevens, M. (2003). The role of the priority rule in science. The Journal of Philosophy, 100(2), 55–79. http://www.jstor.org/stable/3655792. ISSN 0022362X. Tremain, S. (2017). Foucault and feminist philosophy of disability. Ann Arbor: University of Michigan Press. 123 858 R. Heesen Uhlmann, E. L., & Cohen, G. L. (2007). ‘‘I think it, therefore it’s true’’: Effects of self-perceived objectivity on hiring discrimination. Organizational Behavior and Human Decision Processes, 104(2), 207–223. doi:10.1016/j.obhdp.2007.07.001. http://www.sciencedirect.com/science/article/ pii/S0749597807000611. ISSN 0749-5978. Valian, V. (1999). Why so slow? The advancement of women. Cambridge: MIT Press. Wennerås, C., & Wold, A. (1997). Nepotism and sexism in peer-review. Nature, 387(6631), 341–343. doi:10.1038/387341a0. ISSN 0028-0836. Ziobrowski, A. J., & Gibler, K. M. (2000). Factors academic real estate authors consider when choosing where to submit a manuscript for publication. Journal of Real Estate Practice and Education, 3(1), 43–54. http://ares.metapress.com/content/1762151051KM2227. ISSN 1521-4842. Zollman, K. J. S. (2009). Optimal publishing strategies. Episteme, 6, 185–199. doi:10.3366/ E174236000900063X. http://journals.cambridge.org/article_S1742360000001283. ISSN 1750-0117. 123