Author's response to reviews Title: Analysis of decisions made in meta-analyses of depression screening and the risk of "hunch" bias: a case study Authors: Felicity A Goodyear-Smith (f.goodyear-smith@auckland.ac.nz) Mieke L van Driel (m.vandriel@uq.edu.au) Chris Del Mar (cdelmar@bond.edu.au) Bruce Arroll (b.arroll@auckland.acnz) Version: 3 Date: 9 April 2012 Author's response to reviews: Dear Editor Thank you for these reviewers’ helpful comments which we have addressed in the revised version of our paper. Prof Song refers to the fact that prior beliefs being a reason for the observed discrepancies between the two reviews is possible but not proven. We concur with this and emphasis this point by adding that the concept of ‘hunch bias’ is a hypothesis of ours. Regarding the question about the consequences of discrepant conclusions from systematic reviews, different conclusions can result in opposing recommendations (for example, to screen or not to screen). This can have important consequences which might be reflected in clinical guidelines, as is indeed the situation in our case, where the US guidelines recommended screening but the UK ones recommended not screening. We recommend that guideline writers and health policy makers should check all available systematic reviews to ensure such discrepancies do not exist. Where contradicting reviews are found they should address these discrepancies and justify any stand they take, not make a subjective decision to suit. This is where prior hunch disclosure would be of assistance. We have added this to the discussion. Prof Song asks “Specifically, are you able to make a judgement about which of the two systematic reviews is more believable?” and “A couple of further sensitivity analyses could be conducted, by including/excluding both the Lewis study and the Wells study.” These are interesting questions but beyond the scope of this paper. Our aim was not to determine the most accurate of the systematic reviews but to demonstrate the impact of selection or exclusion of data on the overall conclusions and recommendations. Prof Song asks the important question “It is interesting to know whether the two systematic reviews have cited each other’s earlier publication and discussed the differences in conclusions.” In fact they have cited each other but this does not appear to have prevented the discrepancies we have identified in their reviews. We have included this in the discussion. Dr Hazem also suggests that to address the issue of “identical systematic reviews with conflicting results and evidence… authors of systematic reviews should compare their work to previous review/meta-analyses and point out whether there was a conflict in one outcome or the other, one recommendation or the other and then attempt to explain it” and that this “should be a requirement for the discussion section for systematic reviews.” We agree that this recommendation could be made more explicit in the PRISMA checklist (“Conclusion #26: Provide a general interpretation of the results in the context of other evidence”) but could be in addition to a priori declaration of the “hunch” of the authors at the onset of the review. We have emphasized this in the discussion. Dr Hazem mentions that the group of authors sharing a same prior conviction is not very likely, however we argue that both individuals and groups may hold prior beliefs. Regarding Dr Hazem’s suggestion that “N-of-1 randomzed controlled trials are the highest level of evidence”, we agree that a single N-of 1-trial is a strong study design. However, a systematic review or meta-analysis of RCTs or N-of 1-trials trumps this as a higher quality of evidence (OCEBM Levels of Evidence Working Group*. "The Oxford 2011 Levels of Evidence". Oxford Centre for Evidence-Based Medicine. http://www.cebm.net/index.aspx?o=5653). We have added N-of-1 trials to the first sentence of the background. We have corrected the minor errors identified by Dr Hazem and have made changes to ensure that the paper conforms to the journal style.